Opened 12 years ago

# Calculate should use an appropriate number of significant figures for trigonometric functions

Reported by: Owned by: greenfeld rwh Unspecified by Maintainer Unspecified Calculate 0.92.x Minor walter, lucyd OLPC New

### Description

Calculate should use an appropriate number of significant figures for trigonometric functions. Right now it provides errors in the noise where it should provide a zero answer.

This potentially could confuse students if none of their other figures are that minuscule. Seen in Calculate-36 in OLPC 11.2.0 os13.

Examples seen on an XO-1.5 (using degrees; radians are similar):

• sin(180) = 1.22460*10-16
• cos(90) = 6.12303*10-17
• tan(90) = 1.63317*1016 {technically I believe this is undefined, but Google's calculator makes the same mistake for this one entry}.

### comment:1 Changed 11 years ago by godiard

This ticket can be related to #2332

### comment:2 Changed 10 years ago by lucyd

I propose the following idea for solving this issue.

• Maintain a window of digits in the current computed value after decimal point
• Check if any non-zero digit(significant figure) occurs in this window
• If yes then go with the current format
• If no then approximate the value to zero

This approach is for values between 0 and 1. Similarly we can check in a window of digits before the decimal point for approximating the value to infinity. All other cases follow the existing format.