Archives of the TeradataForum
Message Posted: Wed, 01 Sep 2004 @ 18:27:22 GMT
| Subj: | | Re: FAST EXPORT Replace function |
| |
| From: | | Victor Sokovin |
| | Thank you every one for your input. I think its our Data center standard to use pipe delimeted flat file. I will try to discuss with my
team and try to come up with some other delimeter. The pipe with in the data that I was talking about did not put by FastExport. Instead it exists
in the data, which actually should have been cleansed before arriving to our teradata box. We are trying to trace that back and see why is that
happening. With Fast Export I have seen many unneccessary characters putting in the flat file. I had to trim and cast each data field and use mode
Record to get a clean data. Our option to choose Fast Export is because of its speed. I am not very familiar with BTEQ. Can anyone enlighten me on
the speed issues and clean data issues between FastExport and BTEQ. These scripts are going to pull about 30 mil rows of volume with a rowsize of
250 | |
I think these two facts - delimiter already in the data (very rare event; I doubt anybody could guess about that from the original
posting) and the relatively high volumes - make BTEQ recommendation irrelevant. BTEQ is essentially a reporting tool and it would probably take
hours to generate the "report" with 30 m records. FastExport will take you what, 15 min or so?
I think you should indeed stick with FastExport and try to sort out the quality of data you receive from the source. If the delimiter has been
agreed upon they should follow the rules of the game and get the pipe out the data before it gets loaded into the table.
If that is not an option, you'll have to cleanse the data yourself. "Replace" UDF can be downloaded for free from the NCR site that another
participant listed here. I personally never used that UDF so I won't be able to make a recommendation based on experience. But it's free so why
not give it a try? As I said before, I don't know whether one can use UDF directly in the SELECT in FastExport or one has to do the cleansing on
the table level before starting the export.
Regards,
Victor
| 
