Archives of the TeradataForum
Message Posted: Mon, 20 May 2003 @ 00:10:11 GMT
At my current client we use AbInitio as an ETL tool. Depending on the sub project and its need more or less use of bteq (in AbInitio RunSQL) is made. We run bteq in btet mode. (This is typically used if the data volume is large and already in Teradata.)
For the purpose of creating metrics we have to figure out how many records got inserted (INSERT INTO SELECT FROM), updated (UPDATE) or deleted (DELETE). To some extent we are interested in information pertaining to DROP table (success/failure). The original approach the client took, was to emit the same query twice, once for the purpose of metrics (using a count instead of really doing the update/insert/delete) followed by the "real" query. I would prefer not doing this and am looking into alternatives.
Here what I can see as a solution:
A) write a user exit for bteq + relatively straight forward + good metrics collection + no need to redo activities + information can be captured as the query process processes, potential to redirect resulting figures - not easy to associate query to result in that we have the plain query text but not context (which part of ETL ran this?) B) log file scraping + metrics and context in one file - requires developers to adhere to conventions in order to support the scraping process (this is tricky....) - scraping is inherently unsafe - can only be done after the fact
I wonder what the audiences experience and approach to this requirement is.
|Copyright 2016 - All Rights Reserved|
|Last Modified: 28 Jun 2020|