Archives of the TeradataForum

Message Posted: Wed, 28 May 2008 @ 14:11:49 GMT

	<Prev	Next>		<<First	<Prev	Next>	Last>>

<-- Anonymously Posted: Wed, 28 May 2008 08:55 -->

NumValues and NumRows encounter numeric overflow.

When the query is EXPLAINed, it generates a 3704: (�0A�X) is not a valid Teradata token.

Attributes such as ModeFreq, NumNulls are returning zero. (So they might overflow if they encountered nonzero or null values?? )

Leaving 4 AS PadBytes in the code does not appear to contribute to the overflow condition.

GSC has this info.

Here is the (view based) code that generates the numeric overflow.

     HASHBUCKET(SUBSTR(STATS, 49+Offset+7, 1)
             || SUBSTR(STATS, 49+Offset+6, 1) (BYTE(4))) / TD12 AS NumValuesw1,

     HASHBUCKET(SUBSTR(STATS, 49+Offset+5, 1)
             || SUBSTR(STATS, 49+Offset+4, 1) (BYTE(4))) / TD12 AS NumValuesw2,

     HASHBUCKET(SUBSTR(STATS, 49+Offset+3, 1)
             || SUBSTR(STATS, 49+Offset+2, 1) (BYTE(4))) / TD12 AS NumValuesw3,

     HASHBUCKET(SUBSTR(STATS, 49+Offset+1, 1)
             || SUBSTR(STATS, 49+Offset+0, 1) (BYTE(4))) / TD12 AS NumValuesw4,


     CASE WHEN SUBSTR(STATS, 49+Offset+0, 8) = '00'xb THEN 0

     ELSE
       (-1**(NumValuesw1 / 32768))
       * (2**((NumValuesw1/16 MOD 2048) - 1023))
       * (1 + ((NumValuesw1 MOD 16) * 2**-4) + (NumValuesw2 * 2**-20)
            + (NumValuesw3 * 2**-36) + (NumValuesw4 * 2**-52))

     END AS NumValues,

     HASHBUCKET(SUBSTR(STATS, 57+Offset+7, 1)
             || SUBSTR(STATS, 57+Offset+6, 1) (BYTE(4))) / TD12 AS NumRowsw1,

     HASHBUCKET(SUBSTR(STATS, 57+Offset+5, 1)
             || SUBSTR(STATS, 57+Offset+4, 1) (BYTE(4))) / TD12 AS NumRowsw2,

     HASHBUCKET(SUBSTR(STATS, 57+Offset+3, 1)
             || SUBSTR(STATS, 57+Offset+2, 1) (BYTE(4))) / TD12 AS NumRowsw3,

     HASHBUCKET(SUBSTR(STATS, 57+Offset+1, 1)
             || SUBSTR(STATS, 57+Offset+0, 1) (BYTE(4))) / TD12 AS NumRowsw4,


     CASE WHEN SUBSTR(STATS, 57+Offset+0, 8) = '00'xb THEN 0

     ELSE
       (-1**(NumRowsw1 / 32768))
       * (2**((NumRowsw1/16 MOD 2048) - 1023))
       * (1 + ((NumRowsw1 MOD 16) * 2**-4) + (NumRowsw2 * 2**-20)
            + (NumRowsw3 * 2**-36) + (NumRowsw4 * 2**-52))

     END AS NumRows

	<Prev	Next>		<<First	<Prev	Next>	Last>>