Archives of the TeradataForum

Message Posted: Fri, 07 May 2004 @ 23:31:13 GMT

	<Prev	Next>		<<First	<Prev	Next>	Last>>

Our project is currently converting an existing Oracle Data Warehouse to a Teradata solution. This piece of the project is run on a UNIX platform, using bourne shell scripting. We have encountered two problems using MLoad with our current feeds. I am wondering how others handle these situations and what are the best practices?

1.) We have a comma delimited feed. This feed contains embedded commas within a field. Oracle SQL*Loader can handle imbedded delimiters in certain situations.

     Oracle Syntax:
         FIELDS TERMINATED BY "," OPTIONALLY ENCLOSED BY "'"

     Data Record Example:
         D202Y4WK,May 05 15:47:22 2004,4A,B904EC,'SelfScan:RWM,',ST

The 'SelfScan:RWM,' would load without incident in Oracle.

Does Teradata have a method to handle embedded delimiters? If not what is the best practice? Having the layout changed at the source is the most logical, but this may be difficult.

2) We use MLoad to load data feeds. Many of our feeds contain garbage data. An example would be a data field in the MLoad script is defined as .FIELD DISK * VARCHAR(1);. The feed is another comma-delimited file. The field in the record is five characters long. The MLoad process abends before the acquisition step. No data is loaded and the MLoad process informs you of the error encountered.

Does Teradata have a work around to handle these situations where the incoming record is longer than the expected length? In our Oracle environment, a reject file is created and the rest of the file is loaded. Does the data feed have to conform exactly to the MLoad layout? We could create special awk scripts to validate the data, but this would be highly undesirable. We have over 120 tables loaded through this process and each table would have to have a customized script. Does anyone have a different solution?

	<Prev	Next>		<<First	<Prev	Next>	Last>>