Archives of the TeradataForum

Message Posted: Mon, 20 May 2003 @ 00:10:11 GMT

	<Prev	Next>		<<First	<Prev	Next>	Last>>

Situation

At my current client we use AbInitio as an ETL tool. Depending on the sub project and its need more or less use of bteq (in AbInitio RunSQL) is made. We run bteq in btet mode. (This is typically used if the data volume is large and already in Teradata.)

My question

For the purpose of creating metrics we have to figure out how many records got inserted (INSERT INTO SELECT FROM), updated (UPDATE) or deleted (DELETE). To some extent we are interested in information pertaining to DROP table (success/failure). The original approach the client took, was to emit the same query twice, once for the purpose of metrics (using a count instead of really doing the update/insert/delete) followed by the "real" query. I would prefer not doing this and am looking into alternatives.

Here what I can see as a solution:

   A) write a user exit for bteq
   + relatively straight forward
   + good metrics collection
   + no need to redo activities
   + information can be captured as the query process processes, potential
   to redirect resulting figures
   - not easy to associate query to result in that we have the plain query
   text but not context (which part of ETL ran this?)

   B) log file scraping
   + metrics and context in one file
   - requires developers to adhere to conventions in order to support the
   scraping process (this is tricky....)
   - scraping is inherently unsafe
   - can only be done after the fact

I wonder what the audiences experience and approach to this requirement is.

Thanks

Thomas

Thomas Buehlmann
Accenture
Phoenix, AZ 85016

	<Prev	Next>		<<First	<Prev	Next>	Last>>