Archives of the TeradataForum
Message Posted: Fri, 07 Sep 2001 @ 14:21:32 GMT
Subj: | | Re: Data in DWH --- Issues |
|
From: | | Jim Downey |
Amer,
It doesn't matter who is responsible. You have the problem. Obviously it is not a problem for the OLTP or they would have fixed it
before passing it to you. It's kind of a cop out from the vendor to say it is not their responsibility to clean the data. I have talked
with enough people to know that the data is never as clean as you want it to be.
There are several vendors who do address cleanup. Two products I have used are Finalist and Sagent. We have a very similar situation in
that our OLTP stores name and address in an 8 line free form format. We load this data into an Address Table. We then build a Scrubbed
Address table with the same key as the address table. Address keys that are not in the Scrubbed Address Table are passed through the
scrubbing process until they become scrubbed. Some will require manual scrubbing. This method allows us to present both the address as
stored in the OLTP (so the business can see how bad they really are and perhaps effect a change) or a scrubbed address that they can
use.
I hope this helps.
Jim
|