Ticket #234 (closed defect: fixed)

Opened 2 years ago

Last modified 22 months ago

Code: fixed_xte_fact.m_nVal==1 if FIXED_MATH

Reported by: max Owned by: jwharington@…
Priority: normal Milestone: XCSoar 6.0.x
Component: Other Version:
Keywords: Cc:

Description

The resolution of the "fixed" type is too small to represent the constant "fixed_xte_fact" (Earth.cpp), which is calculated by the formula:

1.0 / (fixed_rad_to_deg * fixed_earth_r)

or:

1.0 / 365031411

The number 365031411 needs 28 bits to be stored, which happens to be exactly the accuracy of the "fixed" data type. The division algorithm seems to round this up, resulting in a very inaccurate result "m_nVal==1". Let's compare:

1.0 / 365031411 = 0.73537632080654014 * (2-28)
our result = 1.0 * (2
-28)

This is an error of more than 35%, and is not even remotely usable for real calculations.

My new division algorithm (which is more than 3x faster) always rounds down, and its result is "0".

Change History

comment:1 Changed 2 years ago by Turbo

  • Summary changed from fixed_xte_fact.m_nVal==1 if FIXED_MATH to Code: fixed_xte_fact.m_nVal==1 if FIXED_MATH

Is there a reason why it is not just #defined?!

comment:2 Changed 22 months ago by max

  • Status changed from new to closed
  • Resolution set to fixed

Fixed in 409f3c6a87dda26891d802645ab6be78c1bb48e7

Note: See TracTickets for help on using tickets.