Home Page for the TeradataForum

Archives of the TeradataForum

Message Posted: Fri, 20 Jul 2007 @ 13:42:48 GMT

  <Prev Next>   <<First <Prev Next> Last>>  

Subj:   Re: What is IS STAGED
From:   Victor Sokovin

  What I thought is if we use pipe line the data gets loaded as soon as it enters pipe line but fo each and every record it need to travel through network and load the data but when we enable IS STAGED, it will write to a flat file and only once this flat file need to be traveled and write the data. So what I felt was enabling IS STAGED was better( Please correct if I am wrong). But there are some cases where I came across like when this is staged is enabled than only the session success fully gets complete else it fails. So I think this IS STAGE is not just for performance sake ..right.  

  So I want to know what is that diffrece, I am just not able to figure it out.  

In terms of data travelling through the network both methods are pretty similar. It's the same number of bytes going from A to B. When you load the file (IS_STAGED on) the feed of data into the network should be more even though, whereas the pipe can get empty at some moments (when the export process feeding it experiences delays) so you will see peaks in network usage and some periods with no data at all. It depends again on what hardware you have at your disposal. If your networks are fast and I/O devices on the ETL and Teradata servers are of high quality you probably are better off utilizing them at full capacity.

There is also a job management perspective here. Yes, you can win a few seconds with pipes but once you consider using them you might start getting those famous error messages informing you of "broken pipes". In those cases you will have to make sure that you know how to restart your processes correctly. Yet another consideration to make.

You also mention that some jobs are only successful when you use the IS_STAGED method. This is true. When you ask ML to use pipes it has to use extra software called "access module" (client OS specific plug-in). As any additional software it has its share of restrictions and bugs. ETL tools are often unaware of them and cannot handle all requests properly. This also imposes practical limitations on the usage of pipes.



  <Prev Next>   <<First <Prev Next> Last>>  
  Top Home Privacy Feedback  
Copyright for the TeradataForum (TDATA-L), Manta BlueSky    
Copyright 2016 - All Rights Reserved    
Last Modified: 15 Jun 2023