Archives of the TeradataForum

Message Posted: Fri, 23 Oct 2004 @ 00:01:50 GMT

	<Prev	Next>		<<First	<Prev	Next>	Last>>

You are correct on CRC at the disk sector level. The software checksum feature is not intended to enhance that capability. But know that the disk level CRC is generated inside the disk drive on the write of data, and checked and then stripped on the read. So the data on the way to and fro the disk drive is not protected by the disk CRC.

The TERADATA checksum, as I mentioned in a previous reply, can detect corruption introduced along the path between the file system to the disk. One real world example is the hardware FIFO (first in first out) device. These are often used for speed matching between different interconnects (like Fibre to PCI for example). Even if a FIFO were to pass through ECC, what if the FIFO gets stuck and repeats the same word a few times? The ECC on each word still looks good, but the data is corrupt.

Oh, and by the way, you CAN have checksum on spool files. The feature allows the specification of checksum levels for each of the following system class objects.

System Tables
System Journal Tables
System Logging Tables
Permanent Journal Tables
Temporary Tables

The technology trends are such that calculating simple checksums (not CRC's) in software is reasonable with current hardware and will just fade into the noise in the future. And the technology trends in storage are such that we want more checking: increasing complexity, SAN, NAS, iSCSI, etc.

Checksums are not for everyone. Like any feature I would recommend evaluating it to determine its value in your warehouse. We use them internally now in our test labs. Data integrity failures are infrequent, but they do happen. It comes down to evaluating a data loss risk vs. performance overhead vs. business impact tradeoff.

Mark

	<Prev	Next>		<<First	<Prev	Next>	Last>>