Archives of the TeradataForum

Message Posted: Thu, 19 Jul 2012 @ 09:28:18 GMT

	<Prev	Next>		<<First	<Prev	Next>	Last>>

Hi,

Possible cause is 'duplicate row check'. This process is likely to slow down as you add more rows to the target table. This is a problem when you have too many rows per PI data value (well, technically PI rowhash) AND the table is SET AND there are no unique indexes on the target table. This would not show up as skewed data.

What happens is this:

- as the process starts, there are relatively few rows for any given PI rowhash value.

- duplicate row checking happens, but with only a few rows to check it doesn't appear to slow you down

- as you add more data to the target table, the number of rows with same PI rowhash grows, and you can now 'see' the processing slow down.

Another thing to check, is your ** target ** table skewed? This will just make it worse.

If the duplicate row check looks like the culprit then a couple of things come to mind

- add a unique index

- change the table to multiset (providing of course that your data model/processing can handle this)

Note that both of the above 'solutions' require you to cancel the existing processing.

Cheers,

Dave

Ward Analytics Ltd - Information in motion (www.ward-analytics.com)

	<Prev	Next>		<<First	<Prev	Next>	Last>>