Archives of the TeradataForum
Message Posted: Tue, 14 Sep 2004 @ 15:32:59 GMT
Subj: | | Re: Interesting Calculations If U spend time ! |
|
From: | | paul.burns |
you wrote:
| "Can anyone explain why we get an error when I do CAST with more than 6 decimal points for the above calculation? | |
SELECT 6*393.2600*1.57000*(CAST(10 AS DECIMAL(18,7))/60.00) Res - Gives an error"
The obvious answer would be in the data types. The 'first' number of the expression seems to determine the data type for the expression.
Hence this being the first part SELECT 6*393.2600*1.57000 gives a result that is store as a 'smaller' data type the the second part *(CAST(10 AS
DECIMAL(18,7))/60.00). Then the result of the two part is outside the range of the first data type.
You could change the first part to SELECT 6.00*393.26*1.57 and still maintain the decimal places. But that still does not give a accurate
result. The result should be 393.26 * 1.57 = 617.4182.
Try this:
SELECT (6(float))*393.2600*1.57000 * ((10 (float)) / 60 ) res
Paul
|