Bug 136272 - Corrupted calc file: invalid number format condition [<= 1.79769313486232E+308 ]
Summary: Corrupted calc file: invalid number format condition [<= 1.79769313486232E+308 ]
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
5.1 all versions
Hardware: All All
: medium normal
Assignee: Eike Rathke
URL:
Whiteboard: target:7.2.0 target:7.1.0.0.beta2 tar...
Keywords:
Depends on:
Blocks:
 
Reported: 2020-08-29 20:06 UTC by dave
Modified: 2020-12-21 12:44 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:
Regression By:


Attachments
Corrupted LibreCalc fle (8.89 KB, application/vnd.oasis.opendocument.spreadsheet)
2020-08-29 20:07 UTC, dave
Details
This file takes about 5 minutes to open with LibreCalc. (40.45 KB, application/vnd.oasis.opendocument.spreadsheet)
2020-11-04 23:17 UTC, dave
Details
This file opens in the normal / expected time (about 1 second). (21.99 KB, application/vnd.oasis.opendocument.spreadsheet)
2020-11-04 23:20 UTC, dave
Details

Note You need to log in before you can comment on or make changes to this bug.
Description dave 2020-08-29 20:06:47 UTC
Description:
FILEOPEN / FILESAVE  A Calc file takes a long time to open (5 - 10 minutes).  Copied and pasted data (190 rows by 12 columns) from the corrupt file to a new file and it worked fine for a while but now does the same thing.  LibreCalc shows no error message.  I removed all of the data from the spreadsheet and saved the file.  It still takes a while to open for being an empty file.  But, doesn't take 5 minutes to open.  Downloaded Gnumeric and opened the file with with it.  Gnumeric shows the error "General ODF error Corrupted file: invalid number format condition [<= 1.79769313486232E+308 ]"  and then opens the file.  Saving the file with Gnumeric fixes the issue.  The file opens in seconds now with LibreCalc, but it will most likely become corrupted again as copying the data to a totally new spreadsheet didn't fix the issue.  I will attach a copy of the corrupted file with the data removed.

Steps to Reproduce:
1. Open the corrupted Calc file with LibreCalc.  It takes about 17 seconds (more than 5 minutes when it has data).
2. Compare this with opening an empty Calc file with LibreCalc (takes about 5 secons)
3. Open the corrupted Calc file with Gnumeric and see the error message.

Actual Results:
File takes a long time (5 - 10 minutes) to open (with 190 rows by 12 columns) of data.

Expected Results:
Should take less than 5 seconds to load even with 180 rows by 12 columns of data.


Reproducible: Always


User Profile Reset: Yes


OpenGL enabled: Yes

Additional Info:
Version: 7.0.0.3 (x86)
Build ID: 8061b3e9204bef6b321a21033174034a5e2ea88e
CPU threads: 16; OS: Windows 10.0 Build 19041; UI render: Skia/Vulkan; VCL: win
Locale: en-US (en_US); UI: en-US
Calc: threaded
Comment 1 dave 2020-08-29 20:07:59 UTC
Created attachment 164858 [details]
Corrupted LibreCalc fle
Comment 2 Janson Van 2020-09-03 22:05:48 UTC Comment hidden (no-value)
Comment 3 Eike Rathke 2020-11-04 21:38:52 UTC
Problem seems to be that 1.79769313486232E+308 is the double max value 1.7976931348623157e+308 rounded to 15 significant digits, which when read in and converted from string is not representable as a double value, hence does not form a valid condition.

Though the attached file does not expose a performance problem.
However, the double max value representation needs to be fixed.
Comment 4 dave 2020-11-04 22:17:23 UTC
Thank you for your response.  Hopefully, that will fix the issue.

The file I attached probably doesn't seem to expose a performance problem. However, with a file that has more data (only 190 rows by 12 columns) it takes 15 - 20 minutes for the file to open.  This is on an 8-core CPU and takes this long to open whether opening the file across a network or from the local hard disk.  Gnumeric opens the file in less than 5 seconds.  Once opened, fixed (by Gnumeric) and saved, Libreoffice is able to open the file in a reasonable amount of time (less than 5 seconds).

Perhaps filling the file with more data would amplify the performance issue.  Or, compare the time to open the file I attached with opening a completely empty file (which the file I attach is basically - as there is no data in any of the cells).

Another possibility is for me to upload a file that is populated with more data.  Perhaps that would show how incredibly slow it is (no exaggeration - about 15 minutes) to open the file.  If you want me to upload a file with more data, let me know.

Thanks again.  Let me know if you have any further questions.
Comment 5 dave 2020-11-04 22:22:44 UTC
One more thing.  These corrupted files take 15 minutes to open on multiple computers.  Since it's slow opening on multiple computers, it doesn't appear to be a problem related to a specific computer or configuration.  It's a data file issue.  And, opening, fixing and saving the file with Gnumeric fixes the problem on multiple computers.

I can check multiple platforms if needed.  Typically, we use the Windows version of LibreCalc, but we have access to Linux and can see if it exhibits the same slow opening issue.

Let me know if that's needed.
Comment 6 dave 2020-11-04 23:17:02 UTC
Created attachment 167009 [details]
This file takes about 5 minutes to open with LibreCalc.

This file contains more data than the original file I uploaded.  When this file is opened with LibreCalc, it takes about 5 minutes to open and doesn't show any errors.  When this file is opened with Gnumeric, it opens very quickly (less than a second) and shows the invalid number format condition error.  It's odd that this file takes so long to load since it's only 41,416 bytes.  If this file is opened with Gnumeric and (after getting the error) then saved, the file will open very quickly (less than a second).  I will upload a copy of the fixed (with Gnumeric) version of this file for comparison.
Comment 7 dave 2020-11-04 23:20:48 UTC
Created attachment 167010 [details]
This file opens in the normal / expected time (about 1 second).

This is the fixed version of the file I previously uploaded (167009).  It is an exact copy of file 167009 that has been opened and fixed with Gnumeric and then saved.  As you can see, after Gnumeric fixed the file, it is smaller (22,513 bytes instead of 41,416 bytes) and takes less than a second to load.
Comment 8 dave 2020-11-04 23:23:25 UTC
I tried opening the corrupted files with the Linux version of LibreCalc.  The files open normally (less than a second) with Linux.  So, this issue seems to be isolated to the Windows environment.  I did not try saving the file with Linux to see if that fixes the issue (like Gnumeric fixes).

If you need me to try that, let me know.
Comment 9 Eike Rathke 2020-11-24 14:42:43 UTC
Apart from the double max value string representation being off limits to be read in again, regarding any performance slowdown this all doesn't make sense. The value occurs only once in the document and is discarded anyway because it is a condition for a text number subformat which it is not needed in the number format or UI, but in ODF subformat style maps can't be stored without a condition; when read in it results in the MM/DD/YY;@ format it was before. (in fact the entire explicit text subformat isn't needed at all and MM/DD/YY would be sufficient but that's how Excel stores these so probably the document originated there).

However, I'll add a fix for the string representation, but it will likely not change anything for the Windows slowness. Not running Windows myself I can't check.
Comment 10 Commit Notification 2020-11-25 14:48:24 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/ee2cc952eeb5385ee37485c822d7ad7abd8fb989

Related: tdf#136272 accept 1.79769313486232e+308 as DBL_MAX

It will be available in 7.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 11 Ming Hua 2020-11-25 16:18:59 UTC
Can not reproduce with 6.4.7 on Windows 10:
Version: 6.4.7.2 (x64)
Build ID: 639b8ac485750d5696d7590a72ef1b496725cfb5
CPU threads: 2; OS: Windows 10.0 Build 18363; UI render: default; VCL: win; 
Locale: zh-CN (zh_CN); UI-Language: en-US
Calc: threaded

Both sample ODS files (attachment 164858 [details] and attachment 167009 [details]) opens in 1-2 seconds for me.  However the first one looks empty.  The second one looks normal, 12 rows and 180 cols of data, no error messages.
Comment 12 Commit Notification 2020-11-25 16:46:38 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "libreoffice-7-1":

https://git.libreoffice.org/core/commit/711e69e333faaf0570ddf2cb1fc721bc59cc61bc

Related: tdf#136272 accept 1.79769313486232e+308 as DBL_MAX

It will be available in 7.1.0.0.beta2.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 13 Commit Notification 2020-11-26 09:02:42 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "libreoffice-7-0":

https://git.libreoffice.org/core/commit/28521056a22719dffa0814883d06a7c9531b02ad

Related: tdf#136272 accept 1.79769313486232e+308 as DBL_MAX

It will be available in 7.0.5.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 14 Xisco Faulí 2020-11-26 09:52:37 UTC
The three files open within 1-2 seconds for me in

Version: 7.0.3.1 (x86)
Build ID: d7547858d014d4cf69878db179d326fc3483e082
CPU threads: 2; OS: Windows 6.1 Service Pack 1 Build 7601; UI render: Skia/Raster; VCL: win
Locale: es-ES (es_ES); Interfaz: es-ES
Calc: threaded

I can't reproduce the "General ODF error Corrupted file: invalid number format condition [<= 1.79769313486232E+308 ]" error either...
Comment 15 Xisco Faulí 2020-11-26 10:04:04 UTC
Meanwhile, I found bug 138507 while checking this issue in master

@Eike, in which file is it possible to reproduce the 1.79769313486232e+308 error ?
Comment 16 Eike Rathke 2020-11-26 14:19:38 UTC
@Xisco, it happens when you open the file in Gnumeric. We need to write a not (or less) rounded value 1.7976931348623157e+308 instead.
Comment 17 Commit Notification 2020-11-27 01:44:43 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/4713c19e2b79341dc27e66d4c6449497db1e73d8

Resolves: tdf#136272 convert DBL_MAX to 1.7976931348623157e+308

It will be available in 7.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 19 Commit Notification 2020-11-27 02:41:15 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "libreoffice-7-1":

https://git.libreoffice.org/core/commit/013807f152a2c869b863658cd2409344ffb72e39

Resolves: tdf#136272 convert DBL_MAX to 1.7976931348623157e+308

It will be available in 7.1.0.0.beta2.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 20 Commit Notification 2020-11-27 16:25:22 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/1a0f9a9c56e7b7952b813b3efd34f9be6c853957

Consistently use RTL_CONSTASCII_LENGTH(), tdf#136272 follow-up

It will be available in 7.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 21 Commit Notification 2020-11-28 11:27:20 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/87191b03e79e909a8ace3bcac35cfeea7f0d34ea

Unit tests for DBL_MAX to string conversion, tdf#136272

It will be available in 7.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 22 Commit Notification 2020-11-28 11:51:38 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "libreoffice-7-1":

https://git.libreoffice.org/core/commit/8ee421198ed2ffdabcde76778a3b10bd19a27766

Consistently use RTL_CONSTASCII_LENGTH(), tdf#136272 follow-up

It will be available in 7.1.0.0.beta2.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 23 Commit Notification 2020-12-08 14:21:40 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "libreoffice-7-0-4":

https://git.libreoffice.org/core/commit/699c1115788145e7e02fc376aead663e974e3524

Related: tdf#136272 accept 1.79769313486232e+308 as DBL_MAX

It will be available in 7.0.4.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 24 Commit Notification 2020-12-09 18:56:41 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "libreoffice-7-0-4":

https://git.libreoffice.org/core/commit/8a4430197982247d58c512fad1c0aea17578f057

Resolves: tdf#136272 convert DBL_MAX to 1.7976931348623157e+308

It will be available in 7.0.4.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 25 Commit Notification 2020-12-10 14:21:24 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "libreoffice-7-0":

https://git.libreoffice.org/core/commit/4c2c161018165b7484233547fea1511c84b3ffe3

Resolves: tdf#136272 convert DBL_MAX to 1.7976931348623157e+308

It will be available in 7.0.5.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.