Home Page for the TeradataForum
 

Archives of the TeradataForum

Message Posted: Mon, 05 May 2003 @ 19:21:22 GMT


     
  <Prev Next>   <<First <Prev Next> Last>>  


Subj:   Re: Why it is rounding the last digit?
 
From:   Chuck McDevitt

IEEE 64-bit floats (8 bytes) have 15-17 decimal digits of accuracy.

IEEE 32-bit floats (4 bytes) only have around 5-6 digits of accuracy.



     
  <Prev Next>   <<First <Prev Next> Last>>  
 
 
 
 
 
 
 
 
  
  Top Home Privacy Feedback  
 
 
Copyright for the TeradataForum (TDATA-L), Manta BlueSky    
Copyright 2016 - All Rights Reserved    
Last Modified: 15 Jun 2023