Opened 13 years ago

Last modified 11 years ago

#2700 assigned defect

Calculate significant digit calculations may be exponentializing Integer numbers one digit early

Reported by: greenfeld Owned by: lionaneesh
Priority: Unspecified by Maintainer Milestone: Unspecified
Component: Calculate Version: 0.92.x
Severity: Minor Keywords: easy-hack
Cc: Distribution/OS: OLPC
Bug Status: Assigned

Description

  1. Set Calculate to only show 6 digits in the result.
  2. Calculate the result of "1/3", which should be "0.333333" {6 significant digits}.
  3. Calculate the result of "1e4", which should appear as "10000" {5 significant digits, so to speak}
  4. "1000.119" results in "1000.11" {6 digits shown, although an interesting truncation.}
  5. Calculate the result of "1e5". This appears as "1x105", not "100000", which has 6 digits in it. Is this correct, or are we refusing to show an extra digit for integer numbers where we would do so for decimal numbers?

Again seen in Calculate 36.

Attachments (1)

0001-Fix-significant-digit-issues-with-calculate.patch (821 bytes) - added by lionaneesh 11 years ago.

Download all attachments as: .zip

Change History (3)

comment:1 Changed 12 years ago by godiard

  • Keywords easy-hack added

comment:2 Changed 11 years ago by lionaneesh

  • Bug Status changed from Unconfirmed to Assigned
  • Owner changed from rwh to lionaneesh
  • Status changed from new to assigned
Note: See TracTickets for help on using tickets.