Archives of the TeradataForum

Message Posted: Wed, 19 Jun 2002 @ 02:20:31 GMT

	<Prev	Next>		<<First	<Prev	Next>	Last>>

This seems to be a problem with the tool!

My experience is more mixed. If you get a high volume high performance tool (which Informatica really is not!) you want to get some transforms done outside of Teradata and others need to be done inside Teradata. Overall this is a difficult balancing act (cf. below).

I am not sure whether Informatica supports the generation of fastload/load/fastexport scripts; other tools do which further reduces the development cycle (and improves TCO by reducing maintenance effort in the wake of change.)

Cheers

Thomas

Some Considerations:

Fundamentally NCR does suggest to do all the transforms in the warehouse. This is a very reasonable proposition for two reasons. One, most ETL tools are not made for high performance/high volume processing. Secondly NCR does have a self interest in funding database expansion. (Most transforms in a database do not show linear scalability which soon results in additional nodes.) Personally I try to move ETL away from the TD system for several reasons.

- batch windows get smaller as the value of the warehouse grows

- at the same time data volumes and business logic complexity grows

- I purchased the database for the user and not for the ETL process

- a separate ETL server is cheaper to expand than a Teradata system

- there are high performance tools that transform extremely fast, moving more data than I have typically seen

- these tools offer linear (or better in rare cases!) scalability

- operationally a separate ETL server is a bliss if the organization is in catchup mode

With this said I do a mix approach and try to figure out the cost of moving data out off the TD system versus doing it on the TD system. Various business requirements need to be considered to come to a final conclusion of what should be done on the warehouse and what can be processed externally. Unfortunately this will not give you people the black/white clear cut answer most are looking for.

Thomas Buehlmann

	<Prev	Next>		<<First	<Prev	Next>	Last>>