Archives of the TeradataForum

Message Posted: Thu, 09 Nov 2006 @ 14:59:30 GMT

	<Prev	Next>		<<First	<Prev	Next>	Last>>

David:

It is good that you now have a solution for someone picking a very poor separator.

However, consider these things:

1. You will have to do this same preparation everytime you need a new script or copy and paste

2. VARTEXT means that all data coming in needs to be converted to the data type in the table (even CHAR and VARCHAR needs to be padded or length bytes set)

3. Separators must be located using a character by character scan of the incoming data, realize that it is the utility and CPU on the host computer locating and converting the data, not Teradata (increased time and increase CPU utilization)

4. therefore separators are NEVER as fast as fixed location and native data types being loaded

For example: I have a 359K record load file that I have loaded with Fastload on a single CPU / 2 AMP system:

Using vartext it takes 8 minutes to load that data. Using fixed length, correct type data takes about 24 seconds. That is about 20 times slower. Something to consider if your loads are large or planning to get larger.

As your data load volumes increase in frequency or volume, your data loads are going to slow exponentally. Pretty soon Fastload will have a new name at your site: Half-Fastload and of course, MLoad is never as fast as Fastload. Life is filled with trade-offs. I want you to make an informed decision and not a knee-jerk reaction for a decision on moving forward to a successful multi-terabyte implementation of your warehouse.

Regards,

Michael Larkins
Certified Teradata Master
Certified Teradata SQL Instructor

	<Prev	Next>		<<First	<Prev	Next>	Last>>