Bug 44969 - FILEOPEN: Can't open XML files bigger than 1.1 Mb
Summary: FILEOPEN: Can't open XML files bigger than 1.1 Mb
Status: CLOSED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
3.5.0 Beta3
Hardware: All All
: high normal
Assignee: Peter Jentsch
URL:
Whiteboard: (target:3.7.0) BSA bibisected35 bibis...
Keywords: regression
: 38492 51869 (view as bug list)
Depends on:
Blocks:
 
Reported: 2012-01-19 22:41 UTC by Ruslan Fatakhov
Modified: 2022-01-17 11:43 UTC (History)
10 users (show)

See Also:
Crash report or crash signature:


Attachments
It is report from oracle sistem (1.37 MB, text/xml)
2012-01-25 00:16 UTC, Ruslan Fatakhov
Details
File for test (1.37 MB, application/xml)
2012-02-07 20:41 UTC, Ruslan Fatakhov
Details
File for test (18.63 KB, application/x-7z-compressed)
2012-02-07 20:43 UTC, Ruslan Fatakhov
Details
console output when trying to open the file (only appears at the end, after canceling and waiting) (123.80 KB, text/x-log)
2012-02-22 04:54 UTC, Christian Lohmaier
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ruslan Fatakhov 2012-01-19 22:41:21 UTC
Problem description: 
Not open xml file size more 1.1 mb
Steps to reproduce:
1. Open xml file size 1.4, 3.3 mb
2. ....
3. ....

Current behavior:
Not open xml file size more 1.1 mb
Expected behavior:
Should open
Platform (if different from the browser): 
Ubuntu 11.10, Firefox/9.0.1
              
Browser: Mozilla/5.0 (Ubuntu; X11; Linux i686; rv:9.0.1) Gecko/20100101 Firefox/9.0.1
Comment 1 Florian Reisinger 2012-01-24 22:58:53 UTC
Please attach such a .xml file for proper testing...
Comment 2 Ruslan Fatakhov 2012-01-25 00:16:17 UTC
Created attachment 56116 [details]
It is report from oracle sistem
Comment 3 Ruslan Fatakhov 2012-02-06 21:42:37 UTC
This problem arises in 3.4.4 and in 3.4.5. In 3.3 opens not bad, Openoffice 3.3 opens well.
Comment 4 Ruslan Fatakhov 2012-02-07 20:41:31 UTC
Created attachment 56733 [details]
File for test

Hello, file for test
Comment 5 Ruslan Fatakhov 2012-02-07 20:43:37 UTC
Created attachment 56734 [details]
File for test

Hello, i put it in arhiv.
Comment 6 Pedro 2012-02-20 09:42:00 UTC
I can confirm that the attached XML file opens correctly in LibreOffice 3.3.4 but not with 3.4.5 or 3.5.0 under Windows XP Pro x86 SP3

It opens as a blank document in 3.4.5 and doesn't open at all under 3.5.0: it just reports a "General Error. General input/output error."
Comment 7 Ruslan Fatakhov 2012-02-20 20:46:08 UTC
Hello! Have you been able to test the opening MS Exell XML file and solve the problem? In the final version 3.5 this problem persists.
(In reply to comment #1)
> Please attach such a .xml file for proper testing...
Comment 8 Ruslan Fatakhov 2012-02-20 20:49:12 UTC
Hello! When you solve this problem, since I needed to work? Because of this I have to work in Openoffice 3.3 

(In reply to comment #6)
> I can confirm that the attached XML file opens correctly in LibreOffice 3.3.4
> but not with 3.4.5 or 3.5.0 under Windows XP Pro x86 SP3
> 
> It opens as a blank document in 3.4.5 and doesn't open at all under 3.5.0: it
> just reports a "General Error. General input/output error."
Comment 9 Florian Reisinger 2012-02-20 23:09:05 UTC
I can confirm this issue with LibO 3,5 final.
I will set the Importance to high, might help.....
OS: Win7 x64 SP 1
Comment 10 Pedro 2012-02-21 04:31:16 UTC
Hi Ruslan

> Hello! When you solve this problem, since I needed to work? Because of this I
> have to work in Openoffice 3.3 

I am not a developer. I just help in triaging and checking reported bugs. Posting additional comments without new information will not increase the chance that it is fixed ;)

I suggest that you use LibreOffice version 3.3.4 instead.
Comment 11 Pedro 2012-02-21 04:34:55 UTC
Additional information: the problem seems indeed to be related to the file size. 

Breaking the table in two (i.e. not changing headers and keeping first 1000 lines and not changing headers and keeping bottom 1000 lines) results in two files with ~700kB. 

Both open correctly in LO 3.5.0 (although quite slowly) so this seems to prove that there is no problem in the file structure but only on it's length.
Comment 12 Christian Lohmaier 2012-02-21 14:55:09 UTC
reassigning.
Comment 13 Muthu 2012-02-21 22:59:57 UTC
Something sure is broken...takes a lot of time in xslt, while it loads real quick on 3.3

CC'ing kohei and petr in case they have some info there?
Comment 14 Muthu 2012-02-21 23:01:48 UTC
ah...forgot to cc eike...cc'ing now..
Comment 15 Christian Lohmaier 2012-02-22 04:54:37 UTC
Created attachment 57452 [details]
console output when trying to open the file (only appears at the end, after canceling and waiting)

The essence is this:

runtime error: file file:///opt/libreoffice3.5/share/xslt/common/measure_conversion.xsl line 145 element param
xsltApplyXSLTTemplate: A potential infinite template recursion was detected.
You can adjust xsltMaxDepth (--maxdepth) in order to raise the maximum number of nested template calls and variables/params (currently set to 3000).
Comment 16 Ruslan Fatakhov 2012-02-22 19:44:08 UTC
Hi, what do I need to do?


(In reply to comment #15)
> Created attachment 57452 [details]
> console output when trying to open the file (only appears at the end, after
> canceling and waiting)
> 
> The essence is this:
> 
> runtime error: file
> file:///opt/libreoffice3.5/share/xslt/common/measure_conversion.xsl line 145
> element param
> xsltApplyXSLTTemplate: A potential infinite template recursion was detected.
> You can adjust xsltMaxDepth (--maxdepth) in order to raise the maximum number
> of nested template calls and variables/params (currently set to 3000).
Comment 17 Björn Michaelsen 2012-03-01 09:43:10 UTC
Regression does appear in oldest version of bibisect-3.5.tar.lzma and must be older.
Comment 18 Muthu 2012-03-06 00:08:19 UTC
@bjorn: it(the bug) is present even in 3.4, i guess (wrt the previous comments). But, I have tested 3.3 and the bug isn't there.
Comment 19 Markus Mohrhard 2012-04-03 19:07:39 UTC
I think it might be related to c76c986e17194b0f678ba81a9c49a31bcf206607 from Peter so I added him into CC.
Comment 20 Peter Jentsch 2012-04-03 23:29:32 UTC
We changed the default XSLT transformer from Savon9/Java to libxslt in LO 3.4. Apparently, libxslt's settings for maximum recursion depth differ from those in saxon. 

I'll have a look into this later today. As a workaround, you might try to save the file to another format (Excel 97), and then open it in LO.
Comment 21 Muthu 2012-04-04 06:24:03 UTC
@markus: Thanks!
@peter: I have reassigned this bug to you...I guess you are the best person to solve this. Thank you so much! [ Don't hesitate to reassign it again, if my assumptions are wrong ;) ]
Comment 22 Peter Jentsch 2012-04-04 15:27:18 UTC
Hi, I'm currently stuck debugging this, because my Mac build crashed with a EXC_BAD_ACCESS before libxslt goes dumping it's call stack because the maximum recursion limit has been reached. 

In theory, it would be possible to just raise that to a larger number (preferable one that matches the setting in saxon more closely), but I'll only be able to verify that when my linux build is in a debuggable state again. 

I'll have a look at ways to limit recursion depth in the xslt templates and raise a hand if I get stuck.
Comment 23 Peter Jentsch 2012-04-09 05:39:11 UTC
Just a note about the current state of analysis: 

the MS 2003 XML import filters heavily rely on recursion over all rows and columns of the imported file in some template rules. Saxon 8/9 for Java contain optimizations to handle tail recursion calls in template rules. libxml and Xalan don't optimize tail recursion and fail importing larger files sooner or later. 

It would be possible to avoid the recursion by doing two transformation passes, but I won't get to working on this before end of april. As a quick hack it would be possible extend the maximum allowed recursion depth for libxslt in the import filter code. But that would only push the limit a little and not really solve the problem.
Comment 24 Markus Mohrhard 2012-08-03 10:16:36 UTC
*** Bug 51869 has been marked as a duplicate of this bug. ***
Comment 25 Markus Mohrhard 2012-08-14 14:01:31 UTC
*** Bug 38492 has been marked as a duplicate of this bug. ***
Comment 26 Jean-Baptiste Faure 2012-08-19 20:53:41 UTC Comment hidden (obsolete)
Comment 27 Peter Jentsch 2012-09-16 09:46:16 UTC
Fixed by those commits on master:

fix endless recursion with some characters in spreadsheetml headers/footers:

http://cgit.freedesktop.org/libreoffice/core/commit/?id=8b950e8213c25212e6656a3e0da3ff6f470dcbfe

optimize font-decl template for libxslt (using exslt functions):
http://cgit.freedesktop.org/libreoffice/core/commit/?id=a2a10b59876951b6493419713e9054ceabd3d6cc

build and deliver internal libexslt. Use LIBEXSLT_LIBS is system xslt is used:
http://cgit.freedesktop.org/libreoffice/core/commit/?id=1c467763f4ca4bc1caaa3111f0ed85f388e6fe01

register exslt functions for libxslt filter:
http://cgit.freedesktop.org/libreoffice/core/commit/?id=eadb83f281b596e441a82798660f1a27c177b2c6

add for exslt:set:distinct template:
http://cgit.freedesktop.org/libreoffice/core/commit/?id=b5107faa150aab3c5480708219fc8d392a97f718


fix a problem when handling style named for conditional formatting:
http://cgit.freedesktop.org/libreoffice/core/commit/?id=9f29890d4e4fa916d46eeae081ef6e04eb1bfe81

fixed problem with template recursion in spreadsheetml import:
http://cgit.freedesktop.org/libreoffice/core/commit/?id=3420be984986bcff03d6d127b913fc07372fe89f

optimized handling of ConditionalFormatting elements:
http://cgit.freedesktop.org/libreoffice/core/commit/?id=8fdef3e8d8ead3903795df87cbf66256691542b1


Commits have not yet been backported to 3-6 and 3-5 branches, to which they should also be applied.
Comment 28 David Tardon 2012-09-25 11:15:31 UTC
Also:

add build support for libexslt:
http://cgit.freedesktop.org/libreoffice/core/commit/?id=ab03e87741f25d3a5532a75c3dc59b5556a2bb24
Comment 29 Rainer Bielefeld Retired 2012-10-21 06:34:38 UTC
Can someone please add target info if the complete fix is already available for a version before 3.7.0?

Test Document 2012-02-07 20:41 UTC still crashes with 3.6.3.1, so it seems fix is not available for 3.6? But here we have a conglomerate of problems,  "Can't open" is not a very clear description ...
Comment 30 Ruslan Fatakhov 2012-11-08 05:55:48 UTC
Hi all, Would it fixed in LibO 3.7?
Comment 31 Caolán McNamara 2012-11-23 12:01:41 UTC
yes, it works in 3.7/4.0
Comment 32 Watodis Smit 2022-01-15 10:00:45 UTC Comment hidden (spam)