Archives of the TeradataForum
Message Posted: Mon, 24 Feb 2003 @ 21:00:50 GMT
If you have a lot of duplicates (>50) and if you load duplicates into a set table it will slow down the load a lot without a Unique Index to use for a duplicate row check. If you have a Unique Index then it will slow down a little.
If you use a Multi-set table the load will not slow down as long as you throw away the duplicate rows (Automatic in Fastload) (Multiload has an option to ignore duplicate rows, otherwise this may be slow due to the row by row insert into the UV table)
Note; Since, Fastload automatically throws duplicates away, even a multi-set table will not store these if you use the Fastload Utility.
This process works well and in parallel if you design it correctly. However, the check needs to be done, somewhere. Duplicate rows in the final table are almost (haven't found a reason yet) never the correct thing to do.
Multiset tables are ok from a performance standpoint, but you need to know the data is clean.
|Copyright 2016 - All Rights Reserved|
|Last Modified: 28 Jun 2020|