Home Page for the TeradataForum

Archives of the TeradataForum

Message Posted: Fri, 27 May 2005 @ 21:33:03 GMT

  <Prev Next>   <<First <Prev Next> Last>>  

Subj:   Re: Distance based on latitude and longitude
From:   Victor Sokovin

  Let me retract my previous question. The variance between the two techniques is caused by different values for "Earth Radius". Once I fixed that, the results are identical!  

Perhaps for your example they are but in general the formulae might give different results. "Law of Cosines for Spherical Trigonometry" (that's what the second formula in your posting is), despite its simpler look and mathematical exactness, is not recommended for *numerical* calculations.

At the risk of bothering the forum with the details I'll quote just a part from the GIS FAQ (the full version can be found on comp.inofsystems.gis):

"An UNRELIABLE way to calculate distance on a spherical Earth is the

Law of Cosines for Spherical Trigonometry


     a = sin(lat1) * sin(lat2)
     b = cos(lat1) * cos(lat2) * cos(lon2 - lon1)
     c = arccos(a + b)
     d = R * c

Although this formula is mathematically exact, it is unreliable for small distances because the inverse cosine is ill-conditioned. Sinnott (in the article cited above) offers the following table to illustrate the point:

     cos (5 degrees) = 0.996194698
     cos (1 degree) = 0.999847695
     cos (1 minute) = 0.9999999577
     cos (1 second) = 0.9999999999882
     cos (0.05 sec) = 0.999999999999971

A computer carrying seven significant figures cannot distinguish the cosines of any distances smaller than about one minute of arc. The function min(1,(a + b)) could replace (a + b) as the argument for the inverse cosine to guard against possible round-off errors, but doing so would be to "polish a cannonball". "



  <Prev Next>   <<First <Prev Next> Last>>  
  Top Home Privacy Feedback  
Copyright for the TeradataForum (TDATA-L), Manta BlueSky    
Copyright 2016 - All Rights Reserved    
Last Modified: 15 Jun 2023