Bug Hunting Session
Bug 88139 - math formulas imported incorrectly from file saved in 3.6.7.2
Summary: math formulas imported incorrectly from file saved in 3.6.7.2
Status: RESOLVED WORKSFORME
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Formula Editor (show other bugs)
Version:
(earliest affected)
4.3.3.2 release
Hardware: x86-64 (AMD64) Linux (All)
: medium major
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-01-07 07:51 UTC by Yury
Modified: 2015-12-14 20:59 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
doc with simple formula saved in 3.6.7.2 (10.54 KB, application/vnd.oasis.opendocument.text)
2015-01-07 07:52 UTC, Yury
Details
rendering in 3.6.7.2 (3.22 KB, image/png)
2015-01-07 07:55 UTC, Yury
Details
rendering in 4.3.5.2 (4.39 KB, image/png)
2015-01-07 07:58 UTC, Yury
Details
frame settings as shown for the incorrectly rendered formula (4.3.5.2) (38.76 KB, image/png)
2015-01-07 13:51 UTC, Yury
Details
sample with formulas in Times New Roman (15.00 KB, application/vnd.oasis.opendocument.text)
2015-01-16 09:41 UTC, Yury
Details
sample with times everywhere (14.95 KB, application/vnd.oasis.opendocument.text)
2015-01-16 11:38 UTC, Yury
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Yury 2015-01-07 07:51:54 UTC
Math formulas in document created in 3.6.7.2 writer (linux/amd64) are rendered distorted in 4.3.5.2 writer, if document is open and then formula is open (so, re-rendered).

Attached simple showcase doc saved in 3.6.7.2 and its rendering output in 3.6.7.2.

The source of distortion is somehow embedded in the imported formula object, because I can, e.g., re-input the formula source text or copy-paste-open the whole formula object, with 100% same result. Formula object created anew in 4.3.5.2 is rendered okay.

Font faces and sizes are identical in both 3.6.7.2 and 4.3.5.2 installations.

Attached rendering output of the showcase doc open in 4.3.5.2 with the 1st line containing imported and re-rendered object, and same object copied-pasted-re-rendered; the 2nd line contains input and formula object entered in 4.3.5.2 session.
Comment 1 Yury 2015-01-07 07:52:37 UTC
Created attachment 111888 [details]
doc with simple formula saved in 3.6.7.2
Comment 2 Yury 2015-01-07 07:55:08 UTC
Created attachment 111889 [details]
rendering in 3.6.7.2
Comment 3 Yury 2015-01-07 07:58:13 UTC
Created attachment 111890 [details]
rendering in 4.3.5.2

1st line - imported and re-rendered content and same content copied-pasted-re-rendered
2nd line - newly input content (same formula mathtype source text)
Comment 4 Yury 2015-01-07 08:54:05 UTC
While poking around I've found that the immediate cause of bad render is the `vertical position` setting of the imported formula frame, which is somehow offset by (in this case) 0.26in `from bottom` w/r to the `base line`.

Newly input formula gets this setting (correctly) zeroed and referring to `Character`, according to what is set in the `Formula` frame style.

Re-applying the frame style does not help. Also, I cannot directly edit the setting in question, as the frame `Position` settings (both horisontal and vertical) are grayed out.

Would somebody point out the relevant object and property hierarchy (if possible, for python interface) for me to throw together a (python) script which would iterate through the imported formulas and workaround the issue (until it gets fixed in the source)? My work is effectively blocked now (because of https://bugs.freedesktop.org/show_bug.cgi?id=88140 and this)

BTW, the UI click handling is somewhat crippled in its capability to select formula's *frame* (possibly, frames of other object types, too) for *editing* (I'm filing a ticket on this).
Comment 5 Yury 2015-01-07 13:50:24 UTC
That is how relevant tags look in the contents.xml.

Incorrectly rendered formula:

<draw:frame draw:style-name="fr1" draw:name="Object1" text:anchor-type="as-char" svg:y="-0.2638in" svg:width="0.4193in" svg:height="0.2717in" draw:z-index="0">

Correctly rendered formula:

<draw:frame draw:style-name="fr2" draw:name="Object3" text:anchor-type="as-char" svg:y="-0.2634in" svg:width="0.5075in" svg:height="0.4098in" draw:z-index="1">

In the 1st case, height is set incorrectly, and anchor interpreted incorrectly.

Attached pic with frame position dialog for the incorrectly rendered formula.
Comment 6 Yury 2015-01-07 13:51:09 UTC
Created attachment 111911 [details]
frame settings as shown for the incorrectly rendered formula (4.3.5.2)
Comment 7 Yury 2015-01-07 17:39:45 UTC
The latest 4.2.8 has this problem, too. The latest 4.4.0 does not have this problem.
Can the formula handling module be backported or something? 4.4.0 can't be safely used as it has problems with saving of the big docs.
Comment 8 Yury 2015-01-08 05:28:38 UTC
What "more actually" happens there is formulas' frames get their vertical size reduced. Which, in turn, makes rendering use glyphs squished vertically.

I see two problems here:

1) Unsolicited vert. size change on formulas' frames when importing docs saved in 3.* (and interpretation of it shown in frame `position` subdialog)

2) Dependence of rendering process on that frame parameter. After all, there is font size set in formula editor, what has the formula's frame height to do with anything? If operator sets absolute height too small, let output be clipped. The way this works right now is absurd.
Comment 9 tommy27 2015-01-11 08:59:11 UTC
just an information... what about the 4.0.x, 4.1.x and 4.2.x releases in respect of that file created in 3.6.7.2?

is the bad rendering specific to the 4.3.x branch or the incompatibility with 3.6.x affects all the 4.x series?
Comment 10 tommy27 2015-01-11 09:11:53 UTC
I also want to add that the bug is not reproducible under Windows 8.1 x64
the test file looks the same in 3.6.7.2 and 4.3.5.2
Comment 11 Yury 2015-01-11 09:43:50 UTC
(In reply to tommy27 from comment #9)
> just an information... what about the 4.0.x, 4.1.x and 4.2.x releases in
> respect of that file created in 3.6.7.2?
> 
> is the bad rendering specific to the 4.3.x branch or the incompatibility
> with 3.6.x affects all the 4.x series?

I do not have any 4.1.* here. As to the other series, here's a small table of what I'm witnessing on my system w/r to that big file:

         loads  formulas  saves
4.0.2.1   N     -         -
4.2.8.2   Y     resized   Y
4.3.5.2   Y     resized   Y
4.4.0.1   Y     okay      N

I might add that all 4.* variants consume ~3G of virtual memory on having this file open. I'm not sure this is tolerable.
Comment 12 Yury 2015-01-11 09:49:06 UTC
(In reply to tommy27 from comment #10)
> I also want to add that the bug is not reproducible under Windows 8.1 x64
> the test file looks the same in 3.6.7.2 and 4.3.5.2

Please try to open the formula for editing. Just on open it looks the same here too, as re-rendering isn't yet done then.
Comment 13 Yury 2015-01-11 10:06:06 UTC
(In reply to tommy27 from comment #9)

> is the bad rendering specific to the 4.3.x branch or the incompatibility
> with 3.6.x affects all the 4.x series?

Just to be precise, this isn't "bad render" as such, that is, it's not erroneous.

The problem is frame height is for some reason changed (reduced) on load, and the formula renderer in series 4.2, 4.3 forms its output w/r to the containing frame height, which is, padon me, absurd design decision!

In 3.* and in 4.0.2.1 the formula frame had manual resize blocked.

I can't use 4.4 yet, as it crashes on save of this file (any sufficiently big file, possibly?). The rendering of the "corrupted height" formulas returns them to normal state there, so I have some hope restored (possibly, for 4.4.0.2?).

However, from what I was able to notice on short use, the renderer's functioning in 4.4 also has some pecularities.
Comment 14 Yury 2015-01-11 12:58:38 UTC
Update of the table of what I'm witnessing on my system w/r to that big file:

         loads  formulas  saves
4.0.2.1   N     -         -
4.2.8.2   Y     resized   Y
4.3.5.2   Y     resized   Y
4.4.0.1   Y     okay      N
4.4.0.2   Y     okay      Y/N

Y/N in the last line denotes the fact that 4.4.0.2 fails near the end of saving the big file not every time, like 4.4.0.1, but like once in two saves.

I don't know what to say. All this is rather hurting. I'll have to try the apache variant, which is "only" at 4.1.1 currently.
Comment 15 Yury 2015-01-11 14:35:32 UTC
LO        loads  formulas  saves
3.6.7.2   N     -         -   
4.0.2.1   N     -         -
4.2.8.2   Y     resized   Y
4.3.5.2   Y     resized   Y
4.4.0.1   Y     okay      N
4.4.0.2   Y     okay      Y/N
AOO 4.1.1 N     -         -

Effectively this situation should mean dispensing with LO/AOO for any kind of serious/critical work... :(
Comment 16 Robinson Tryon (qubit) 2015-01-14 21:44:37 UTC
/me wades through tons of comments...

(..with a lot of useful content, though :-)

(In reply to Yury from comment #14)
> Update of the table of what I'm witnessing on my system w/r to that big file:
> 
>          loads  formulas  saves
> 4.0.2.1   N     -         -
> 4.2.8.2   Y     resized   Y
> 4.3.5.2   Y     resized   Y
> 4.4.0.1   Y     okay      N
> 4.4.0.2   Y     okay      Y/N
> 
> Y/N in the last line denotes the fact that 4.4.0.2 fails near the end of
> saving the big file not every time, like 4.4.0.1, but like once in two saves.

Okay, so from a practical standpoint of getting you back up and running, it sounds like if we can avoid the crashing in 4.4 or even 4.5, then you should be set.

> that big file:

Which big file? Is that one of the attachments?
Comment 17 Yury 2015-01-15 05:42:39 UTC
(In reply to Robinson Tryon (qubit) from comment #16)
...
> Okay, so from a practical standpoint of getting you back up and running, it
> sounds like if we can avoid the crashing in 4.4 or even 4.5, then you should
> be set.

Well, if you'd ask ME, I'd prefer 4.4... :) But in fact, guys, take your time and fix this thoroughly.

I'm ready to stick with 4.3 and just re-insert all those 100s of formulas (copy source, insert formula, insert source). Excepting the effort duplication, this has predictably positive outcome at least. :)

> > that big file:
> 
> Which big file? Is that one of the attachments?

That's the https://bugs.freedesktop.org/show_bug.cgi?id=88140, which started all this. I would've happily continued using the 3.6.7.2, if not for that. (As it happened, even 4.2 wouldn't prove sufficient.)
Comment 18 Robinson Tryon (qubit) 2015-01-15 07:52:41 UTC
(In reply to Yury from comment #17)
> (In reply to Robinson Tryon (qubit) from comment #16)
> > Which big file? Is that one of the attachments?
> 
> That's the https://bugs.freedesktop.org/show_bug.cgi?id=88140, which started
> all this.

tl;dr -- There's no attachment on bug 88140, because it's a 4MB document that you can't share, right?

I appreciate that you've been doing a lot of legwork here. For QA to make forward progress we're going to need some clean repro steps and a file we can provide to the developers. If there's something we can repro with the ODTs attached to this bug, let's do it and get something triaged, even if it's just a small part of a bigger problem.
Comment 19 Yury 2015-01-15 09:17:28 UTC
(In reply to Robinson Tryon (qubit) from comment #18)
> (In reply to Yury from comment #17)
> > (In reply to Robinson Tryon (qubit) from comment #16)
> > > Which big file? Is that one of the attachments?
> > 
> > That's the https://bugs.freedesktop.org/show_bug.cgi?id=88140, which started
> > all this.
> 
> tl;dr -- There's no attachment on bug 88140, because it's a 4MB document
> that you can't share, right?
> 
> I appreciate that you've been doing a lot of legwork here. For QA to make
> forward progress we're going to need some clean repro steps and a file we
> can provide to the developers. If there's something we can repro with the
> ODTs attached to this bug, let's do it and get something triaged, even if
> it's just a small part of a bigger problem.

This issue (#88139) has demo ODT attached already (the 1st attachment).

The formula in this ODT is rendered incorrectly (frame height changed etc.) in: 4.2.8.2, 4.3.5.2. It is rendered correctly (identical to 3.6.7.2) in: AOO 4.1.1. It is rendered generally correctly (there are some changes w/r to 3.6.7.2 in how {} sequences are treated) in: 4.4.0.

That other issue (#88140, big file) has currently no demo at all, as I'd rather obfuscate the formulas before showing it.
Comment 20 Robinson Tryon (qubit) 2015-01-15 21:23:07 UTC
(In reply to Yury from comment #19)
> This issue (#88139) has demo ODT attached already (the 1st attachment). (attachment 111888 [details])

Pretty awesome attachment number, eh? :-)

> 
> The formula in this ODT is rendered incorrectly (frame height changed etc.)
> in: 4.2.8.2, 4.3.5.2.

Okay, I'll test some versions out on Ubuntu 14.04.

LO 3.6.7.2: Opened up; looks somewhat like the example rendering (attachment 111889 [details])

I see you're using an 'XITS' font that doesn't seem to be distributed with LibreOffice. Can you reproduce the problem with a different font, e.g. Liberation Sans?   (If not, this might just be a font problem)

Status -> NEEDINFO
Comment 21 Yury 2015-01-16 09:41:14 UTC
Created attachment 112333 [details]
sample with formulas in Times New Roman

Well, how about 'Times New Roman' from MS webfonts pack? I made two formulas, one with "symbols" (cdot), and one without (doc attached). Both formulas are affected if opened in 4.3.5.2.
Comment 22 Robinson Tryon (qubit) 2015-01-16 09:51:41 UTC
(In reply to Yury from comment #21)
> Created attachment 112333 [details]
> sample with formulas in Times New Roman
> 
> Well, how about 'Times New Roman' from MS webfonts pack? I made two
> formulas, one with "symbols" (cdot), and one without (doc attached). Both
> formulas are affected if opened in 4.3.5.2.

When I open attachment 112333 [details], I still see XITS set as the font. Is this the right test file?
Comment 23 Yury 2015-01-16 11:38:53 UTC
Created attachment 112336 [details]
sample with times everywhere

The text part font doesn't matter, formula font does. To put everything beyond doubt, here's the file with Times everywhere.
Comment 24 Buovjaga 2015-01-16 12:33:03 UTC
(In reply to Yury from comment #23)
> Created attachment 112336 [details]
> sample with times everywhere
> 
> The text part font doesn't matter, formula font does. To put everything
> beyond doubt, here's the file with Times everywhere.

Could reproduce with 4.3.3 on Linux by clicking in the formula and then back to the document (I installed Times New Roman first). The formulas shrunk vertically a bit, the left one more than the right one.

No repro on other tested versions.

I guess we could request a backport of whatever it was that fixed this?

Ubuntu 14.10 64-bit Version: 4.5.0.0.alpha0+
Build ID: 7201fa0dddd7dd0352f69fd2b2b64efcb361ccad
TinderBox: Linux-rpm_deb-x86_64@46-TDF-dbg, Branch:master, Time: 2015-01-11_23:28:55

and

Version: 4.3.3.2
Build ID: 430m0(Build:2)

Win 7 64-bit, LibO Version: 4.4.0.2
Build ID: a3603970151a6ae2596acd62b70112f4d376b99

and Version: 4.5.0.0.alpha0+
Build ID: b3b4bbaf6cbd2226b659fea7d6ae473ccf84e9dd
TinderBox: Win-x86@39, Branch:master, Time: 2015-01-12_06:13:44
Comment 25 Buovjaga 2015-01-16 13:45:03 UTC
Adding bibisectRequest in the hopes that the good commit will be found and the fix backported to 4.3.
Comment 26 Yury 2015-02-04 06:08:37 UTC
Was something actually backported into 4.3.6.1? There seems to be no such issue in 4.3.6.1, and math rendering looks very much like in 4.4 series, first sign of which is lots (almost too much) of extra vertical space in the "sqrt x" and "x over y" renders.
Comment 27 Robinson Tryon (qubit) 2015-02-04 07:01:39 UTC
Comment on attachment 111911 [details]
frame settings as shown for the incorrectly rendered formula (4.3.5.2)

fix mimetype
Comment 28 Robinson Tryon (qubit) 2015-02-04 07:02:32 UTC
Comment on attachment 112333 [details]
sample with formulas in Times New Roman

fix mimetype
Comment 29 Robinson Tryon (qubit) 2015-02-04 07:03:16 UTC
Comment on attachment 112336 [details]
sample with times everywhere

fix mimetype
Comment 30 Robinson Tryon (qubit) 2015-02-04 07:30:26 UTC
Comment on attachment 111888 [details]
doc with simple formula saved in 3.6.7.2

fix mimetype
Comment 31 Yury 2015-02-14 06:16:29 UTC
In 4.3.6.2, there are new pecularities. Double clicking every formula in document (not editing, just opening for editing and closing) brought the correct content proportions back.

However, there was rather lots of added vertical spacing in every frame, even for one-letter formulas; also in `A over B` type of constructs (I've seen this kind of rendering before that, in 4.4.0 series). Re-opening the formula which had been already processed as described wasn't changing anything. Unfortunately, I do not rightly remember, whether I did this re-clicking in 4.3.6.1 or in 4.3.6.2.

Today, I've reopened in 4.3.6.2 some of the formulas, which had been processed before (correctly proportioned content, with extra vertical spacing). I see new re-sizing happens now, with the extra vertical spacing removed (proportions of content remain correct). Which in itself is good, but somewhat unexpected and, in fact, disturbing.

The question is, and I ask it here intentionally: is this an intended behaviour, being a result of corrections in source code, or should I prepare myself for a new surprises in formulas processing?
Comment 32 Robinson Tryon (qubit) 2015-12-14 05:32:37 UTC
Migrating Whiteboard tags to Keywords: (bibisectRequest)
[NinjaEdit]
Comment 33 Joel Madero 2015-12-14 20:59:29 UTC
From reading the comments it seems like this issue was resolved in 4.4 - the bibisectRequest was for a backport which is no longer useful (4.3 is way past EOL).

Closing this as WORKSFORME (we don't know what fixed it) and removing bibisectRequest.

I noticed additional issues were listed at the end - those should be separated into new bug reports.