Home Page for the TeradataForum
 

Archives of the TeradataForum

Message Posted: Tue, 09 Sep 2003 @ 13:17:49 GMT


     
  <Prev Next>   <<First <Prev
Next>
Last>>
 


Subj:   Re: Round 5 up (override "RoundHalfwayMagUp" set to FALSE)
 
From:   Geoffrey Rommel

  We seek suggestions for accomplishing "round 5 up" for a specific project....  


First suggestion: buy a round of stiff drinks for everybody. Unfortunately, the techniques for rounding depend on the data types you're working with. If your inputs and outputs are floating-point, the floating-point unit may produce results that surprise you; in such a case, you will probably have to use a value for "one-half" that is slightly larger than 0.5 in order to make things come out predictably. (This technique is used in the Perl module Math::Round, available from CPAN.)

If your inputs and outputs are packed decimal or some other fixed-point form, the best approach is probably this:

intermediate = input * 10000; /* Multiply by the appropriate power of 10 */

rounded = floor(intermediate + 0.5); /* The usual "round up" formula */

final = rounded / 10000; /* Divide by the appropriate power of 10 */


  + the answer 3956.00005 needs to round to 3956.0001(presently rounds to 395.6000)  


  + the answer 10061.00045 needs to round to 10061.0005 (presently rounds to 1006.1004)  


These examples are puzzling. Is the present rounding mistakenly dividing things by 10?



     
  <Prev Next>   <<First <Prev
Next>
Last>>
 
 
 
 
 
 
 
 
 
  
  Top Home Privacy Feedback  
 
 
Copyright for the TeradataForum (TDATA-L), Manta BlueSky    
Copyright 2016 - All Rights Reserved    
Last Modified: 15 Jun 2023