Archives of the TeradataForum
Message Posted: Thu, 07 Feb 2008 @ 11:13:45 GMT
| Subj: | | Re: Reduce loading time |
| |
| From: | | Victor Sokovin |
Omer,
| | - We are reading the same data more than once. That is the preferred strategy from the folks in TD. But I don't think that is taking too
much time because the querying part is not bad, it's the loading part. We are not reloading every table during each run. Also, in some of the jobs
we are using the table being loaded as a source table as well. That takes a loooooooooooooooooooooooooonnnnng time but the requirement is such
that we cannot change it. Any ideas on how can that be fixed/changed? | |
I don't understand this part. What do you mean by "We are reading the same data more than once. That is the preferred strategy from the folks
in TD."?
| | 1. We have only UPIs defined on the tables. No other indexes at all. | |
That is interesting and very unusual. You either have a very special data model or the UPIs are artificial. I tend to think (pessimism shows)
it is the latter and that makes me think of another recent thread on this subject:
www.teradataforum.com/teradata/20070105_090738.htm
Do you see any similarities with your case like long lists of columns composing each UPI?
| | 1. We have a staging area and we load data using Multiload and Fastload. We are using Informatica for transforming the data. | |
Which methods does Informatica use to populate data from the staging area to the end tables?
| | 2. This option would require changing the entire job-scheduling. | |
That's right. Moreover, it also requires a complete rethinking of the business use of the database. Long learning curve but once you've done it
you'll never want to go back to the "batch mentality". Worth trying, I should say.
Victor
| 
