Bug 142264 - charts lost by calc export to xlsx
Summary: charts lost by calc export to xlsx
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
7.1.0.3 release
Hardware: All All
: high normal
Assignee: Mike Kaganski
URL:
Whiteboard: target:7.3.0 target:7.2.1 target:7.1.6
Keywords: bibisected, bisected, regression
Depends on:
Blocks: OOXML-Chart
  Show dependency treegraph
 
Reported: 2021-05-13 19:55 UTC by paulystefan
Modified: 2021-09-10 00:33 UTC (History)
6 users (show)

See Also:
Crash report or crash signature:


Attachments
calc source file (6.60 MB, application/vnd.oasis.opendocument.spreadsheet)
2021-05-13 19:55 UTC, paulystefan
Details
xslx file export from version 7.13.2, charts in AT to RO lost (5.14 MB, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)
2021-05-13 20:01 UTC, paulystefan
Details
xlsx export with LO 7.0.6.2 (5.71 MB, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)
2021-05-13 20:06 UTC, paulystefan
Details
smaller example with 3,4,5 sheets, 3 ok, 4 and 5 chart loss in xlsx (1.17 MB, application/x-zip-compressed)
2021-05-14 10:15 UTC, paulystefan
Details

Note You need to log in before you can comment on or make changes to this bug.
Description paulystefan 2021-05-13 19:55:58 UTC
Created attachment 171975 [details]
calc source file

Version: 7.1.3.2 (x64) / LibreOffice Community
Build ID: 47f78053abe362b9384784d31a6e56f8511eb1c1
CPU threads: 8; OS: Windows 10.0 Build 19042; UI render: Skia/Raster; VCL: win
Locale: en-US (de_DE); UI: de-DE
Calc: CL

save of file with charts to xlsx.

charts are lost in xlsx

in 7.0.6 the save in xslx is successful.
Comment 1 paulystefan 2021-05-13 20:01:52 UTC
Created attachment 171976 [details]
xslx file export from version 7.13.2, charts in AT to RO lost

only sheets SE to uk have charts, before all are lost
Comment 2 paulystefan 2021-05-13 20:06:53 UTC
Created attachment 171977 [details]
xlsx export with LO 7.0.6.2

load 7.1.3.2 calc file into 7.0.6.2

make a save in new ods-file

and then save to xlsx 

all charts are available and ok.
Comment 3 m.a.riosv 2021-05-13 20:21:03 UTC
I can't repro with:
Version: 7.1.3.2 (x64) / LibreOffice Community
Build ID: 47f78053abe362b9384784d31a6e56f8511eb1c1
CPU threads: 4; OS: Windows 10.0 Build 21376; UI render: Skia/Raster; VCL: win
Locale: es-ES (es_ES); UI: en-US Calc: CL

Saving calc source file as xlsx, preserve the chart opening with calc or excel.

Please test with a clean profile, Menu/Help/Restart in Safe Mode
Comment 4 NISZ LibreOffice Team 2021-05-14 08:37:27 UTC
Confirming with 

Version: 7.2.0.0.alpha0+ (x64) / LibreOffice Community
Build ID: 6c8ca02c5935a800cff70f3c173319b454b63c41
CPU threads: 4; OS: Windows 10.0 Build 18363; UI render: default; VCL: win
Locale: hu-HU (hu_HU); UI: en-US
Calc: CL

The ods file contains 202 charts (according to the Navigator), after save and reload 20 is remaining of those.

Also in 7.1, but not yet in 

Version: 7.0.0.3 (x64)
Build ID: 8061b3e9204bef6b321a21033174034a5e2ea88e
CPU szálak: 4; OS: Windows 10.0 Build 18363; Felületmegjelenítés: alapértelmezett; VCL: win
Locale: hu-HU (hu_HU); UI: hu-HU
Calc: threaded

All 202 charts were exported to xlsx.
Comment 5 paulystefan 2021-05-14 09:22:03 UTC Comment hidden (obsolete)
Comment 6 paulystefan 2021-05-14 10:15:48 UTC
Created attachment 171993 [details]
smaller example with 3,4,5 sheets, 3 ok, 4 and 5 chart loss in xlsx

this is a smaller example with 3,4,5 sheets. 
3 ok in xlsx, all 18 charts there
4 sheets: only 20 charts, 4 charts loss of 24 in xlsx
5 sheets: only 20 charts, 10 charts loss of 30 in xlsx

all done in new parallel installation

Version: 7.1.3.2 (x64) / LibreOffice Community
Build ID: 47f78053abe362b9384784d31a6e56f8511eb1c1
CPU threads: 8; OS: Windows 10.0 Build 19042; UI render: Skia/Raster; VCL: win
Locale: de-DE (de_DE); UI: de-DE
Calc: CL
Comment 7 NISZ LibreOffice Team 2021-05-14 17:51:08 UTC
Thanks for the simplified example, bibisected with 7.1-win to:

https://git.libreoffice.org/core/+/574eec9036c5f185b3572ba1e0ca9d111eb361dc

author	Mike Kaganski <mike.kaganski@collabora.com>	Sat Sep 12 17:22:23 2020 +0300
committer	Mike Kaganski <mike.kaganski@collabora.com>	Tue Sep 15 06:09:04 2020 +0200

tdf#77007: chart must honor its parent's IsEnableSetModified

Adding CC to: Mike Kaganski
Comment 8 paulystefan 2021-06-03 21:05:35 UTC Comment hidden (obsolete)
Comment 9 paulystefan 2021-08-06 01:31:45 UTC Comment hidden (obsolete)
Comment 10 Timur 2021-08-17 10:28:20 UTC
I set High because it's marked regression and it's a high time to be fixed in branch 7.1 where it originated.
Comment 11 Mike Kaganski 2021-08-18 12:18:38 UTC
The number of OLE objects ending up saved to XLSX appears to depend on 'org.openoffice.Office.Common/Cache/DrawingEngine/OLE_Objects' value. The default value is 20.

Trying to investigate.
Comment 12 Mike Kaganski 2021-08-18 21:44:25 UTC
https://gerrit.libreoffice.org/c/core/+/120688

This is not actually a true regression, just a pre-existing bug that was only masked by a side effect of charts being incorrectly marked modified on open.

This is all about export filter not loading properly the OLE objects that may get unloaded by the memory management in LO. The export reused the obsoleted references to unloaded objects, which couldn't be exported correctly; and I even suspect that existing "IsValidObject" testing chart objects validity in sc/source/filter/xcl97/xcl97rec.cxx might be a hack to "workaround" this actual filter bug.

I am thankful to paulystefan for this bug and the nice reproducers, that eventually allowed me to notice the fixed number of object kept in the result.

I am suspecting now that there might be a similar problem elsewhere in our code, that could be responsible for many "I lost images in a document" complaints that were impossible to investigate, because there were never a reliable reproducer. Now I suppose that adding a substantial number of different images, while reducing GraphicMemoryLimit (or TotalCacheSize) might allow to find a good reproducing scenario.
Comment 13 Commit Notification 2021-08-19 04:42:20 UTC
Mike Kaganski committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/420e834007ca654db9803030726edb32c3ba5710

tdf#142264: make sure to load potentially unloaded objects when saving

It will be available in 7.3.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 14 Timur 2021-08-19 05:05:30 UTC
thanks Mike for the explanation and fix, adding Xisco to see that
Comment 15 Commit Notification 2021-08-19 10:27:17 UTC
Mike Kaganski committed a patch related to this issue.
It has been pushed to "libreoffice-7-2":

https://git.libreoffice.org/core/commit/4b9836e37410f36093951080f057f8061a681e70

tdf#142264: make sure to load potentially unloaded objects when saving

It will be available in 7.2.1.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 16 Commit Notification 2021-08-19 10:27:33 UTC
Mike Kaganski committed a patch related to this issue.
It has been pushed to "libreoffice-7-1":

https://git.libreoffice.org/core/commit/b8b3abc864e81c6ea76043a65ce234ada8651341

tdf#142264: make sure to load potentially unloaded objects when saving

It will be available in 7.1.7.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 17 Commit Notification 2021-09-01 08:14:56 UTC
Mike Kaganski committed a patch related to this issue.
It has been pushed to "libreoffice-7-1-6":

https://git.libreoffice.org/core/commit/93b789b6183cbda68dd076f94af525cbe8d39a80

tdf#142264: make sure to load potentially unloaded objects when saving

It will be available in 7.1.6.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 18 stragu 2021-09-10 00:33:07 UTC
Verified as fixed with attachment 171993 [details] and LO:

Version: 7.1.6.2 / LibreOffice Community
Build ID: 0e133318fcee89abacd6a7d077e292f1145735c3
CPU threads: 8; OS: Linux 5.4; UI render: default; VCL: gtk3
Locale: en-AU (en_AU.UTF-8); UI: en-US
Calc: threaded