Archives of the TeradataForum
Message Posted: Tue, 04 May 2004 @ 22:40:30 GMT
Subj: | | Re: ETL: Push vs Pull |
|
From: | | Bruce.Grant |
One problem I've seen with the PULL strategy occurs when the source database is being changed frequently: since the source DB developers don't
"own" the extraction process, they may not think about the impact of their changes on it and may not communicate change information to the DW
side.
When the source database side maintains the extraction process, it is more likely to stay in sync with source database changes.
(At least, this has been my experience with a dozen Oracle-to-Teradata extraction processes for our customer, some "PUSH" and some
"PULL").
An approach which is working well for us is a "PUSH" extraction which automatically generates control files describing the data files (record
count, field names and formats). The DW loading process can check these against expected values and immediately catch any unanticipated format
changes.
Bruce Grant
Bull Services
|