Bug 91567 - FILEOPEN: Selection Lists added to XLSX imported files (not always.. problem in loading)
Summary: FILEOPEN: Selection Lists added to XLSX imported files (not always.. problem ...
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
4.1.6.2 release
Hardware: Other All
: medium normal
Assignee: Szymon Kłos
URL:
Whiteboard: target:6.0.0
Keywords: bibisected, bisected, filter:xlsx, regression
: 107641 (view as bug list)
Depends on:
Blocks: XLSX
  Show dependency treegraph
 
Reported: 2015-05-24 07:28 UTC by Luke
Modified: 2017-08-09 13:05 UTC (History)
8 users (show)

See Also:
Crash report or crash signature:


Attachments
Screenshot of Calc 5.1 vs Excel (267.47 KB, image/png)
2015-05-24 07:40 UTC, Luke
Details
Same bug in another file (387.02 KB, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)
2015-09-15 02:19 UTC, Luke
Details
Poof that this bug is Random (154.27 KB, image/png)
2016-06-14 01:26 UTC, Luke
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Luke 2015-05-24 07:28:18 UTC

    
Comment 1 Luke 2015-05-24 07:40:16 UTC
Created attachment 115948 [details]
Screenshot of Calc 5.1 vs Excel

Recent builds of Calc are adding erroneous Selection List dropdowns to some imported XLSX files. 

Steps to reproduce:
1. Open attachment 81710 [details] in a recent build of Calc 
2. Open attachment 81710 [details] in Excel or Calc 4.4
3. Compare Sheet: "London Olympic stats"
Comment 2 m.a.riosv 2015-05-24 10:52:38 UTC
Win7x64Ultimate

Not reproducible with:
Version 4.0.6.2 Build ID: 2e2573268451a50806fcd60ae2d9fe01dd0ce24

Reproducible with all version until master since:
Version 4.1.6.2 Build ID: 40ff705089295be5be0aae9b15123f687c05b0a
Comment 3 Matthew Francis 2015-08-14 09:21:13 UTC
This seems to have begun at the below commit.
Not Cc'ing the author by his request.

commit 512a14438c4098ca2e30d0b10b6e18506c8475f1
Author: Markus Mohrhard <markus.mohrhard@googlemail.com>
Date:   Sat Nov 2 17:04:34 2013 +0100

    remember column style during XLSX import, fdo#70315
    
    WE overwrote the column style that was set earlier with the default
    style.
    
    Change-Id: I4c02e1cdd3e72f27d75b8153e2af45b5658ff456
Comment 4 Luke 2015-09-15 02:19:53 UTC
Created attachment 118718 [details]
Same bug in another file

When reviewing the Results of the 2015 /r/Linux Distribution Survey, I came across the same bug in the "HateIndex" Sheet
Comment 5 Robinson Tryon (qubit) 2015-12-10 01:18:38 UTC Comment hidden (obsolete)
Comment 6 Luke 2016-06-11 16:47:15 UTC
This is a strange bug. I don't know if a memory corruption issue or time-out issue, but if you load the bugs docs enough times, occasionally they will render properly. Maybe 1/20 times you'll get the correct results on load.
Comment 7 Luke 2016-06-14 01:26:10 UTC
Created attachment 125644 [details]
Poof that this bug is Random

First attempt to open the file, it correctly imported. Next 5 successive attempts all resulted in incorrect imports.
Comment 8 Cor Nouws 2016-09-07 08:11:51 UTC
Indeed one in every 3 to 5 times it loads fine for me

Version: 5.3.0.0.alpha0+
Build ID: 9f0827d5824f1418bd3e26bb3ac9458179898e9c
CPU Threads: 4; OS Version: Linux 4.4; UI Render: default; 
TinderBox: Linux-rpm_deb-x86_64@70-TDF, Branch:master, Time: 2016-09-06_07:30:01
Locale: nl-NL (nl_NL.UTF-8); Calc: group
Comment 9 Xisco Faulí 2016-09-26 15:13:40 UTC
Adding Cc: to Markus Mohrhard
Comment 10 Markus Mohrhard 2016-10-16 11:28:02 UTC
This happens if the formatting is set through some other way than the normal formatting. In the case of the bug document this happens through the table. That table has its own formatting that is applied before the formatting from the cells is applied.

Based on this observation my fix seems to be a bit naive and we need to store the column styles in a map. And instead of taking the first ScPatternAttr in a column we need to query that map for the default column format.
Comment 11 Markus Mohrhard 2016-10-16 11:41:00 UTC
Also no idea why this should be a perf issue.
Comment 12 Luke 2016-10-25 22:10:26 UTC
Markus,
perf issue? Performance? I don't think so. It is sometimes non reproducible. Could it be a race condition/timing issue? Just now, first try of a recent build, I could not reproduce it again. 2nd try I could.

Did you look at both test documents? Are they caused by the same issue?

Thanks for looking into this!
Comment 13 Markus Mohrhard 2016-10-25 22:38:30 UTC
(In reply to Luke from comment #12)
> Markus,
> perf issue? Performance? I don't think so. It is sometimes non reproducible.
> Could it be a race condition/timing issue? Just now, first try of a recent
> build, I could not reproduce it again. 2nd try I could.
> 
> Did you look at both test documents? Are they caused by the same issue?
> 
> Thanks for looking into this!

The screenshot that is used as proof is from an auto-filter whereas the problem is related to table styles.
Comment 14 Aron Budea 2017-05-05 21:39:53 UTC
*** Bug 107641 has been marked as a duplicate of this bug. ***
Comment 15 Szymon Kłos 2017-08-02 10:48:05 UTC
@markus.mohrhard As I don't see any activity since a year ago, I will assign this bug to me (I've started working on it).

I also can confirm that issue is related to styles: When I removed style application from the code, additional buttons weren't shown (original ones were present).
Comment 16 Commit Notification 2017-08-04 08:41:41 UTC
Szymon Kłos committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=bca55386e187e2cb4ab277a597a420f15c6cebcf

tdf#91567 Apply pattern without AutoFilter only

It will be available in 6.0.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 17 Luke 2017-08-08 03:20:45 UTC
Verified FIXED in Version: 6.0.0.0.alpha0+ (x64)
Build ID: 88179e3de8865ea07d5017ca0723afd10ad44ba7

Tested attachment 81710 [details], attachment 118718 [details], and attachment 133078 [details]. All 3 were fixed by Szymon's patch. 

Thank you Szymon Kłos!
Comment 18 Cor Nouws 2017-08-09 13:05:02 UTC
thanks Szymon!