 |
 |
Archives of the TeradataForum
Message Posted: Wed, 06 Feb 2008 @ 14:47:39 GMT
| Subj: | | Re: Reduce loading time |
| |
| From: | | Victor Sokovin |
| | 1. We have only MLOADS in our entire process. This process has around 4000 individual jobs (actually each job is an Informatica session). I
have switched to using TPUMP for some of the smaller tables (like < 1000 rows) but it looks like TPUMP is actually degrading
performance. | |
You may need to reduce the number of jobs. Each time you run a job has toinitialize the utility, create and drop temporary tables, flush the
logs etc. These operations are very time consuming.
It is an easy advice to give but you will probably need to redesign the entire application. If you go for it, consider the following
improvements:
1. Introduce a staging area and load large chunks of data there (FastLoad, MultiLoad), then transform the data and populate the user
tables. The latter can often be done by simple SQL, not by utilities, but utilities can also be considered.
2. Introduce continuous data flows based on some queueing mechanisms such as supported by WebSphere.
Regards,
Victor
| |
