Home Page for the TeradataForum
 

Archives of the TeradataForum

Message Posted: Wed, 06 Feb 2008 @ 14:47:39 GMT


     
  <Prev Next>   <<First <Prev Next> Last>>  


Subj:   Re: Reduce loading time
 
From:   Victor Sokovin

  1. We have only MLOADS in our entire process. This process has around 4000 individual jobs (actually each job is an Informatica session). I have switched to using TPUMP for some of the smaller tables (like < 1000 rows) but it looks like TPUMP is actually degrading performance.  


You may need to reduce the number of jobs. Each time you run a job has toinitialize the utility, create and drop temporary tables, flush the logs etc. These operations are very time consuming.

It is an easy advice to give but you will probably need to redesign the entire application. If you go for it, consider the following improvements:

1. Introduce a staging area and load large chunks of data there (FastLoad, MultiLoad), then transform the data and populate the user tables. The latter can often be done by simple SQL, not by utilities, but utilities can also be considered.

2. Introduce continuous data flows based on some queueing mechanisms such as supported by WebSphere.


Regards,

Victor



     
  <Prev Next>   <<First <Prev Next> Last>>  
 
 
 
 
 
 
 
 
  
  Top Home Privacy Feedback  
 
 
Copyright for the TeradataForum (TDATA-L), Manta BlueSky    
Copyright 2016 - All Rights Reserved    
Last Modified: 15 Jun 2023