Archives of the TeradataForum
Message Posted: Sat, 22 Feb 2003 @ 17:05:35 GMT
| Subj: || || Re: Can it be any faster |
| From: || || rmhsmd |
Consider Teradata, the LAN, the Fastload client platform CPU, and the source speed. Any of these may be the effective bottleneck for the
acquisition phase. It is often possible to make it faster--the approach depends on what the bottleneck is.
Teradata performance is relatively well documented in the Capacity Planning Reports. Depending on the hardware Teradata has been
measured from 5-8 MB/Second per node for 100 and 500 byte records for the acquisition phase.
fwiw, in the FDL acquisition phase each session is a direct connection to a different AMP (PEs uninvolved). The client FDL process
builds and sends 64K buffers to these--thus this is balanced across the AMPs. Each amp computes each row's hash and does buffered
redistribution to its target amp. Teradata and the client connection are scalable. pre-Teradata Warehouse Builder Fastload client is a
single process that reads a single input using asynch non-blocking I/O on the Teradata sessions. Additional client SMP exploitation is
possible via enabling 'parallel CLI' (the TCP/IP send/receive is done in separate threads that interchange info with Fastload via shared
memory). TWB can scale the processing on an SMP client and read multiple input streams as well.