Bug 139306 - UI: numerical string in formula bar (input line) and cell editing truncated (to 20 digits)
Summary: UI: numerical string in formula bar (input line) and cell editing truncated (...
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
7.1.0.0.alpha0+
Hardware: All All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: target:24.2.0 target:7.6.0.2
Keywords:
Depends on:
Blocks: Calc-Formula-Bar
  Show dependency treegraph
 
Reported: 2020-12-29 17:00 UTC by b.
Modified: 2023-07-21 11:19 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description b. 2020-12-29 17:00:58 UTC
Description:
keying in '6,74316406210E-11' in a cell produces appr. display, but the input line shows '0,00000000006743164062' omitting the last digit '1', 

affecting plenty other figures in that range as well, only down from x,yE-15 the format for input line and cell edit change to scientific format and present all digits for editing, 

(assumption: somehow related to the number format limited to 16 significant digits and 4 fake zeroes, but in the past it was a 'relative' restriction and figures as '6,7431640621000E-12' could be handeled and edited starting with the 'real value', now it's an 'absolute' limit and everything behind the 20th decimal digit is chopped for the input line, for cell edit and e.g. copy and paste to other versions of calc, pasting to e.g. notepad works with more digits)

Steps to Reproduce:
1. key '6,74316406210E-11' into a cell, 
2. hit enter
3. observe input line showing truncated / rounded value, 
4. start editing the cell with 'F2', 
5. observe presented value for edit being truncated / rounded as well, 

Actual Results:
truncated / rounded string used, 

Expected Results:
full string or 'E-format'-string with all digits presented and editable, 


Reproducible: Always


User Profile Reset: Yes



Additional Info:
Version: 7.2.0.0.alpha0+ (x64)
Build ID: 35e471bb4d1388cf5afcdcee214cf5111edf44e3
CPU threads: 8; OS: Windows 6.1 Service Pack 1 Build 7601; UI render: default; VCL: win
Locale: de-DE (de_DE); UI: en-US
Calc:
Comment 1 Jean-Baptiste Faure 2021-08-15 11:54:11 UTC
The last digit is not lost. You can see it if you use a scientific format with enough digits. If you paste
6,74316406210E-11 in A1
and
6,74316406200E-11 in A2
then
A3 = A1-A2
you get 9,9999E-22 = 1.E-21

The problem is that the last digits in A1 or A2 are not editable in scientific form. You can't copy and modify the content of A1 to fill A2.

Setting as NEW for that problem.

Best regards. JBF
Comment 2 b. 2021-08-15 15:54:07 UTC
@JBF: hmmm ... may be you missed the point ... e.g. 
            '6.74316406210123000000E-11' is shown as a cells value, but 
'0.00000000006743164062' is shown in the formula bar, and the same is offered for editing when i press F2. 
it's more than a problem of the last 'hidden digit', and not a problem of too few decimals for display. it's the step 'edit' in the cell and the formatting for display in the formula bar. 
reg. b.
Comment 3 Vladislav Tarakanov 2023-06-29 16:37:22 UTC
This error is still reproducible. Check with: 
Version: 24.2.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: 160ea29f0003812ab1d4e1f6689c608c879ef509
CPU threads: 12; OS: Linux 6.1; UI render: default; VCL: gtk3
Locale: ru-RU (ru_RU.UTF-8); UI: en-US
Calc: CL threaded

It seems that we have found a solution to this problem. To do this, when casting a double value to a string, it was necessary to add an additional count of characters after the decimal point (now 20, in local build the error is corrected at 36)

The line that was changed has the note "we don't want to allocate/deallocate 2GB just to fill it with trailing '0' characters". We would like to first discuss with other developers the correctness of such changes in the code
Comment 4 Mike Kaganski 2023-06-29 17:38:46 UTC
(In reply to Vladislav Tarakanov from comment #3)

Yes, you found the correct line. The question is: what change do you suggest? Simply increasing the value from 20 to 36 (why 36? we only show up to 15 significant decimals) is not reasonable, because the next would be a report about a number with 20 leading zeroes instead of 10. Do you have a reasonable algorithm to find the needed count of total output decimals?
Comment 5 Vladislav Tarakanov 2023-06-29 17:58:41 UTC
(In reply to Mike Kaganski from comment #4)
> (In reply to Vladislav Tarakanov from comment #3)
> 
> Yes, you found the correct line. The question is: what change do you
> suggest? Simply increasing the value from 20 to 36 (why 36? we only show up
> to 15 significant decimals) is not reasonable, because the next would be a
> report about a number with 20 leading zeroes instead of 10. Do you have a
> reasonable algorithm to find the needed count of total output decimals?

We got the number 36 from the sum of two values: the count of characters displayed before 'E' in cells (22) and the maximum count of the degree, after which 'E' will also appear in the edit line ('E-14'). The maximum can be 14 zeros + 22 numbers after the decimal point.
Comment 6 Mike Kaganski 2023-06-29 18:17:31 UTC
(In reply to Vladislav Tarakanov from comment #5)
> We got the number 36 from the sum of two values: the count of characters
> displayed before 'E' in cells (22) and the maximum count of the degree,
> after which 'E' will also appear in the edit line ('E-14'). The maximum can
> be 14 zeros + 22 numbers after the decimal point.

First of all, the function is not specific to cells. It is a generic function, and may be used anywhere, e.g. in file format output, or in Basic.

Now what is "22"? what is "14"? I don't quite understand it. In Calc, we only allow up to 15 significant decimals shown, even if you increase decimal count to 20 (I *suspect* that 22 might be like "a zero plus a decimal separator plus 20 decimals").
Comment 7 Vladislav Tarakanov 2023-06-29 18:46:30 UTC
Maybe I really miscalculated.

14: E-14 is the last degree value after which a number from 0.123456 starts to be displayed as 0.654321E-15 in the edit line. When E-14 is entered, the output line will display the value normally, with 13 zeros before numbers.

22: Counted again, 15 numbers after the decimal point are displayed + 5 trailing zeros behind them. This is the count of characters that can be displayed after the zeros behind the decimal separator.

Then it turns out that up to 28 characters after the separator can be displayed: 13 zeros and 15 digits
Comment 8 Mike Kaganski 2023-06-29 19:05:16 UTC
(In reply to Vladislav Tarakanov from comment #7)

The "E-14" comes from the implementation of rtl_math_StringFormat_Automatic (see its logic in the same function), where it tries to choose the "optimal" representation of a number. See how it uses the number magnitude to switch between rtl_math_StringFormat_E (when the magnitude is <= -15 or >= 15), and rtl_math_StringFormat_F (when the magnitude is inside the bounds). So - is it this calculation that you intend to improve? Note that this will not help in cases when rtl_math_StringFormat_F is chosen explicitly, in which case, the switching will not happen, and any number, including E-300, will still be output using leading zeroes (and indeed, significands will be cut off).

I would recommend you to look at the "nOrigDigits", which shows the count of significant digits in the decimal representation (without any leading/trailing zeroes). You could consider it in combination with magnitude in case of the _F format; but note that you must then pay attention to how the buffer size is calculated, to avoid buffer overflow.
Comment 9 Vladislav Tarakanov 2023-06-30 08:30:45 UTC
Looked at the code again. As a result, the question arose, is it necessary to do this rounding at all (std::clamp<sal_Int32>(nDecPlaces, -20, 20))? We looked at the values that nDecPlaces can get before this rounding:

1. (fValue >= fB4) && eFormat != rtl_math_StringFormat_F : [0; 16]

2. (eFormat == rtl_math_StringFormat_Automatic || eFormat == rtl_math_StringFormat_F) && aParts.exponent >= 0 && fValue < 0x1p53 : [-15; 15]

3. eFormat = rtl_math_StringFormat_Automatic: [1; 14+|nExp|]

4. eFormat = rtl_math_StringFormat_G: [0; 14+|nExp|]

That is, the largest value of nDecPlaces depends only on the number of zeros after the separator
Comment 10 Mike Kaganski 2023-06-30 08:58:47 UTC
(In reply to Vladislav Tarakanov from comment #9)

The list you provided does not cover everything - e.g., _F outside of the limits described in #2. Can you provide a complete list of possibilities?
Comment 11 Amin Irgaliev 2023-06-30 09:18:53 UTC
(In reply to Mike Kaganski from comment #10)
> (In reply to Vladislav Tarakanov from comment #9)
> 
> The list you provided does not cover everything - e.g., _F outside of the
> limits described in #2. Can you provide a complete list of possibilities?

If you mean the assignment details in point 2 (comment 9), then here is its source code:

if (nDecPlaces == rtl_math_DecimalPlaces_Max)
    nDecPlaces = 0;
else
    nDecPlaces = std::clamp<sal_Int32>(nDecPlaces, -15, 15)

if (bEraseTrailingDecZeros && nDecPlaces > 0)
    nDecPlaces = 0;

The end result is a range of [-15;15].
Comment 12 Mike Kaganski 2023-06-30 09:37:43 UTC
(In reply to Amin Irgaliev from comment #11)

I mean, that in comment 9, you did not provide the whole set of possible situations, and thus, did not prove that there will never be a too large nDecPlaces. Namely, in your list, I do not see, what can nDecPlaces be, when e.g. eFormat == rtl_math_StringFormat_F (as in #2), but aParts.exponent < 0 or fValue >= 0x1p53
Comment 13 Amin Irgaliev 2023-06-30 10:46:37 UTC
(In reply to Mike Kaganski from comment #12)
> (In reply to Amin Irgaliev from comment #11)
> 
> I mean, that in comment 9, you did not provide the whole set of possible
> situations, and thus, did not prove that there will never be a too large
> nDecPlaces. Namely, in your list, I do not see, what can nDecPlaces be, when
> e.g. eFormat == rtl_math_StringFormat_F (as in #2), but aParts.exponent < 0
> or fValue >= 0x1p53

Yes, indeed, there is no such check in the code now. We considered what cases there might be, added additional checks and set clamp to [-20; 20]. But we removed line with default clamping after all checks. 

1) else if (eFormat == rtl_math_StringFormat_E || eFormat == rtl_math_StringFormat_E1 || eFormat == rtl_math_StringFormat_E2 || eFormat == rtl_math_StringFormat_FORCE_EQUAL_SIZE)
   {
       nDecPlaces = std::clamp<sal_Int32>(nDecPlaces, -20, 20);
   }
after if ((fValue >= fB4) && eFormat != rtl_math_StringFormat_F)

2) if (eFormat == rtl_math_StringFormat_F && (aParts.exponent < 0 || fValue >= 0x1p53))
   {
       nDecPlaces = std::clamp<sal_Int32>(nDecPlaces, -20, 20);
   }
Comment 14 Mike Kaganski 2023-06-30 11:21:43 UTC
(In reply to Amin Irgaliev from comment #13)

Please create a gerrit change; and in it, please provide comments with estimations which number range can be for every case, and why. This way, you can say that "in this, this, and this case, we can avoid clamping".
Comment 15 Commit Notification 2023-07-20 22:52:02 UTC
Vladislav Tarakanov committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/3f96f9cdd542bcc05cdd531da3c35bc6f0c2986b

tdf#139306 Incorrect nDecPlaces value after clamping

It will be available in 24.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 16 Commit Notification 2023-07-21 11:19:16 UTC
Vladislav Tarakanov committed a patch related to this issue.
It has been pushed to "libreoffice-7-6":

https://git.libreoffice.org/core/commit/ea5dbb6e3dd2fdb84cf68d95174c183aae34488c

tdf#139306 Incorrect nDecPlaces value after clamping

It will be available in 7.6.0.2.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.