Archives of the TeradataForum

Message Posted: Sat, 31 Jan 2009 @ 18:17:38 GMT

	<Prev	Next>		<<First	<Prev	Next>	Last>>

Hello Craig:

You are not giving us much to work with in your description. However, since you are talking about parsing and varchar, I am going out on a limb and thinking that you are using a delimited file. If this is true, you have no alternative but to use varchar for all fields and therefore are forced to live with the parsing in order to find the delimiter, where ever it might be within the record.

Remember, none of this processing has anything to do with the Teradata database engine. It must all be done on the host computer and as you have noticed, the power of that computer is going to dictate the speed of that operation. So Teradata spends more time waiting for the data to arrive and then at Phase II, Teradata kicks in and does its parallel loading activity across the AMP processors.

Native mode data is ALWAYS the fastest because there is no parsing and there is no conversion from VARCHAR to anything else. As slow as the parsing is, many times it is the conversion from one data type to another that eats the lunch of that poor little host computer - double whammy!

The easiest solution for you is to move the fastload job to a more powerful host computer. Otherwise, you are going to need a different way to build the file so that it is in a Fastload format (2-byte record VLI, followed by each smallint, date, etc fixed length or VARCHAR (2-byte vli+data) field and lastly a carriage return for all systems other than a mainframe). The other alternative if you are getting your data from another database would be to use OLE DB instead of a flat file.

Hope this provides a little food for thought,

Michael Larkins
Certified Teradata Master
Certified Teradata SQL Instructor

	<Prev	Next>		<<First	<Prev	Next>	Last>>