Archives of the TeradataForum
Message Posted: Fri, 20 Jul 2007 @ 13:42:48 GMT
| Subj: | | Re: What is IS STAGED |
| |
| From: | | Victor Sokovin |
| | What I thought is if we use pipe line the data gets loaded as soon as it enters pipe line but fo each and every record it need to travel
through network and load the data but when we enable IS STAGED, it will write to a flat file and only once this flat file need to be traveled and
write the data. So what I felt was enabling IS STAGED was better( Please correct if I am wrong). But there are some cases where I came across
like when this is staged is enabled than only the session success fully gets complete else it fails. So I think this IS STAGE is not just for
performance sake ..right. | |
| | So I want to know what is that diffrece, I am just not able to figure it out. | |
In terms of data travelling through the network both methods are pretty similar. It's the same number of bytes going from A to B. When you load
the file (IS_STAGED on) the feed of data into the network should be more even though, whereas the pipe can get empty at some moments (when the
export process feeding it experiences delays) so you will see peaks in network usage and some periods with no data at all. It depends again on
what hardware you have at your disposal. If your networks are fast and I/O devices on the ETL and Teradata servers are of high quality you
probably are better off utilizing them at full capacity.
There is also a job management perspective here. Yes, you can win a few seconds with pipes but once you consider using them you might start
getting those famous error messages informing you of "broken pipes". In those cases you will have to make sure that you know how to restart your
processes correctly. Yet another consideration to make.
You also mention that some jobs are only successful when you use the IS_STAGED method. This is true. When you ask ML to use pipes it has to use
extra software called "access module" (client OS specific plug-in). As any additional software it has its share of restrictions and bugs. ETL
tools are often unaware of them and cannot handle all requests properly. This also imposes practical limitations on the usage of pipes.
Regards,
Victor
| 
