Bug Hunting Session
Bug 75376 - Documents with fields have weird characters
Summary: Documents with fields have weird characters
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
4.2.1.1 release
Hardware: x86 (IA32) Windows (All)
: medium normal
Assignee: Michael Stahl (CIB)
URL:
Whiteboard: target:4.3.0 target:4.2.4
Keywords: regression
Depends on:
Blocks: mab4.2
  Show dependency treegraph
 
Reported: 2014-02-22 18:21 UTC by Charles
Modified: 2014-04-15 19:58 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:


Attachments
Screenshot showing the weird overlapping problem (64.47 KB, image/png)
2014-02-22 18:21 UTC, Charles
Details
Document exhibiting the problem with 4.2.1.1 (30.80 KB, application/vnd.oasis.opendocument.text)
2014-02-22 18:22 UTC, Charles
Details
document renders correctly on LODev 4.2.3.0+ build (100.86 KB, image/jpeg)
2014-02-22 20:07 UTC, V Stuart Foote
Details
problem doesn't show (98.56 KB, image/png)
2014-02-23 17:59 UTC, paolo_debortoli
Details
problem shows, see attachment (44.61 KB, image/png)
2014-02-23 18:29 UTC, Luuk
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Charles 2014-02-22 18:21:21 UTC
Created attachment 94575 [details]
Screenshot showing the weird overlapping problem

After updating to 4.2.1.1, documents with fields show weird characters overlapping the field and text immediately following the field.

See attached screenshot and sample document.

Downgrading to 4.2.0, or 4.1.5 (also earlier versions like 4.0.4 and 4.0.6) shows the documents normally (without the strange characters).
Comment 1 Charles 2014-02-22 18:22:50 UTC
Created attachment 94576 [details]
Document exhibiting the problem with 4.2.1.1

This document exhibits the weird overlapping characters with 4.2.1.1, but looks perfectly fine with all earlier versions up to and including 4.2.0.
Comment 2 Charles 2014-02-22 19:57:39 UTC
Reports from the list suggest this may be a Windows only problem.

I'm running Windows 7 64 bit, and experienced this on 4 different systems, downgrading all of them solved it.
Comment 3 V Stuart Foote 2014-02-22 20:07:26 UTC
Created attachment 94582 [details]
document renders correctly on LODev 4.2.3.0+ build

Can confirm corrupt text on Windows 7 sp1, 64-bit with
Version: 4.2.1.1
Build ID: d7dbbd7842e6a58b0f521599204e827654e1fb8b

However document renders correctly in current build of master
Version: 4.3.0.0.alpha0+
Build ID: 637353bb46d3c7d9537e47e4e003aef78a0c0ab3
TinderBox: Win-x86@39, Branch:master, Time: 2014-02-19_01:39:31

as well as in a TB-42 build of 4.2.3.0+
Version: 4.2.3.0.0+
Build ID: 5fd90cdd1fdb20ab7f6a2b67c384f0994f09a86b
TinderBox: Win-x86@42, Branch:libreoffice-4-2, Time: 2014-02-21_22:38:15
Comment 4 V Stuart Foote 2014-02-22 20:07:59 UTC
resolved with 4.2.3.0+ so setting works for me.
Comment 5 Charles 2014-02-22 20:18:25 UTC
Your response is confusing.

'Works on 4.2.3+' does NOT equate to 'Works for me on 4.2.1.1', which is what this bug is.

You in fact confirmed the bug.

Are you saying the fix will be in the next release - ie, 4.2.2 (NOT 4.2.3)?
Comment 6 V Stuart Foote 2014-02-23 01:54:45 UTC
@Charles,

It is a valid issue with 4.2.1.1--I could reproduce your result.

But what point is there to keeping this open?  The 4.2.1 branch will receive no further development. And the issue is not present in the code that will be released as 4.2.2--nor in the releases prior to 4.2.1.1. It is an annoying corner case that will not likely be tracked to a specific cause.
Comment 7 Charles 2014-02-23 17:48:48 UTC
@Stuart

I agree, but you seem to be missing my two main points as to why I reopened the bug...

1. This is a critical BLOCKER for the CURRENT RELEASE, version (4.2.1), and

2. When you closed the bug, you did so saying that is was fixed on the 4.3+ branch,m which to me says the fix will NOT be in the next release versions (4.2.2).

Setting 'works for me' is contrary to your confirmation of the bug.

It does NOT work for you with the build the bug is reported for.

Setting RESOLVED > FIXED in 4.2.2 branch.

In my opinion, an emergency new release (4.2.1.2?) should be issued whenever something like this is encountered (in this case only for the Windows version), otherwise lots more people will stumble into this mess until the next release is posted.
Comment 8 paolo_debortoli 2014-02-23 17:56:45 UTC
it doesn't show the problem on my system  (LO 4.2.0.4 italian on linux ubuntu).
Comment 9 paolo_debortoli 2014-02-23 17:59:36 UTC
Created attachment 94607 [details]
problem doesn't show
Comment 10 Charles 2014-02-23 18:18:05 UTC
@paulo

No offense, but why would you bother reporting that you don't see the bug on an EARLIER VERSION THAT THE BUG REPORTED SPECIFICALLY SAID DOES NOT EXHIBIT THE PROBLEM???
Comment 11 Luuk 2014-02-23 18:29:48 UTC
Created attachment 94614 [details]
problem shows, see attachment
Comment 12 V Stuart Foote 2014-02-23 18:42:17 UTC
@Charles,
(In reply to comment #7)
> In my opinion, an emergency new release (4.2.1.2?) should be issued whenever
> something like this is encountered (in this case only for the Windows
> version), otherwise lots more people will stumble into this mess until the
> next release is posted.

Charles, again it is a corner case. Most certainly is NOT a "CRITICAL" case. We don't even know where the document came from. And considering that it opens correctly in 4.2.0.4 and earlier, in current 4.2.2 (aka 4.2.3+ in TB42) and master, as well as in AOO and MS Office 2010 following "correction".  

YES...it is an issue in 4.2.1.1 release--but that is the extent of it.

There was just no point in keeping the issue open as can be seen by the random postings--it is confusing.

The ESC has decided there will be no 4.2.1.2 final release, the next will be 4.2.2rc and final(s) there.  Period!

That is why I resolved it WORKSFORME, you could have done the same because we do not know what (if anything) caused the problem--or what commit has fixed it with subsequent builds.
Comment 13 Charles 2014-02-23 18:56:14 UTC
@Stuart

Please understand I am not trying to be difficult, and I am not trying to offend you, I am trying to point out that all of the confusion in the comments in this bug are of your own making.

1. As far as I can see and my testing shows, it is *any* document that
   has *any* fields in it, so, no, it is *not* a 'corner case', and

2. Again you miss my point, that you made two mistakes when you closed
   this bug:

 a) closing as WORKS FOR ME was simply plain wrong, because it
    did/does *not* work in the reported version (4.2.1), which
    yourself confirmed - so, you should have chosen FIXED as
    the result, not WORKS FOR ME, and

 b) you should have said it was fixed in 4.2+, NOT 4.3+

The comments are confusing because YOU MADE CONFUSING COMMENTS and selected an INCORRECT RESULT when you closed the bug.
Comment 14 Luuk 2014-02-23 19:55:34 UTC
If i turn off 'Field Shadings' (CTRL+F8, or menu:View/Field shadings)
 then the doc look OK.
Comment 15 Pedro 2014-03-01 12:02:45 UTC
(In reply to comment #6)
> @Charles,
> 
> It is a valid issue with 4.2.1.1--I could reproduce your result.
> 
> But what point is there to keeping this open?  The 4.2.1 branch will receive
> no further development. And the issue is not present in the code that will
> be released as 4.2.2--nor in the releases prior to 4.2.1.1. It is an
> annoying corner case that will not likely be tracked to a specific cause.

The Bug is still present in 4.2.2.1 under Windows XP x86.
Comment 16 V Stuart Foote 2014-03-01 18:26:39 UTC
Not a blocker, but it is a regression, lowering priority but adding to 4.2 MAB for review.

Affecting Windows builds only where it does appear in Windows but not Linux release of Version: 4.2.2.1
Build ID: 3be8cda0bddd8e430d8cda1ebfd581265cca5a0f

Nor does it show in
Version: 4.2.3.0.0+
Build ID: 90aad4ad6aa814710ce7553cb196392da949e9a8
TinderBox: Win-x86@42, Branch:libreoffice-4-2, Time: 2014-03-01_05:36:21

nor any builds of master (including Christian's TB47).

As noted, the <Ctrl>+F8 toggle of field shading, turns the errant characters on and off. The errant text can not be selected/copied, suggesting it is an issue with the rendering used to apply the field shading for these dynamic fields. I unpacked the .odt and searched but could not figure out the XML for assigning shading. I do see the correct text for each field entry in the content.xml.

Looking at the resulting document, the two date fields --shown as Date(Fixed) when toggled with <Ctrl>+F9-- are correctly formed/rendered, as is the text in the 'Date needed by' and 'GL Account#' fields. Problem appears in the formatting of the fields holding the other inserted text--'Check amount', Needed for', 'Check to', 'Address', and 'Requester'. Each of these shows a field shading width of 1 character and also the same fixed errant text: "ÀxéS"

@Charles, can you provide additional detail of how you built the form. And how the fields are populated. Can you attach the form template?

I'm unable to capture the actual byte code for the displayed text, but suspect it is a somehow corrupted paint command for rendering the field shading. It is interesting in that the resulting text object is persistent enough to be treated as misspelled text, although it can not be further selected or acted upon (with spell checker F7 frame).

To me what is odd, is that this mishandling of field shade rendering manifests in the TDF release builds for Windows only, but is not apparent in any of the TinderBox builds. So is this some sort of MSVC++ Windows build GDI issue?
Comment 17 Charles 2014-03-03 11:25:21 UTC
(In reply to comment #16)
> Not a blocker, but it is a regression, lowering priority but adding to 4.2
> MAB for review.

Ummm... it is a blocker for everyone on Windows who uses Templates with form fields on them.

> @Charles, can you provide additional detail of how you built the form. And
> how the fields are populated. Can you attach the form template?

We created these years ago, but it affects every one that I opened, which was a lot...

They were very simple forms, standard fields, nothing weird or tricky done with them.

I have long since uninstalled all versions of 4.2.2 - but did you simply try creating a new template with the same fields?

Maybe I can find the time later today to create a new form/template with the same fields in 4.1.5, then try opening that on a PC with 4.2.2 installed and see if the problem persists. Then I'll try creating the same template with 4.2.2, and see if still persists...

Or you could do that easily enough if you already have multiple versions available...?

> To me what is odd, is that this mishandling of field shade rendering
> manifests in the TDF release builds for Windows only, but is not apparent in
> any of the TinderBox builds. So is this some sort of MSVC++ Windows build
> GDI issue?

Dunno - any other info I can provide as to versions of anything installed on my systems?
Comment 18 Charles 2014-03-03 11:27:59 UTC
(In reply to comment #17)
> I have long since uninstalled all versions of 4.2.2 - but did you simply try
> creating a new template with the same fields?
> 
> Maybe I can find the time later today to create a new form/template with the
> same fields in 4.1.5, then try opening that on a PC with 4.2.2 installed and
> see if the problem persists. Then I'll try creating the same template with
> 4.2.2, and see if still persists...

Sorry - meant 4.2.1 above, not 4.2.2...
Comment 19 Pedro 2014-03-15 16:15:20 UTC
Regression is still present in LO 4.2.3.1 under Windows XP
Comment 20 Björn Michaelsen 2014-03-18 00:53:43 UTC
Likely cause of this regression is:
commit c2b5521921b806ff7b04cdacebde3834d2aafd4b
Author: Oliver-Rainer Wittmann <orw@apache.org>
Date:   Mon Nov 18 11:29:24 2013 +0000

    Resolves: #i33737# enable in-place editing of Input Fields     
    (cherry picked from commit c2afeb1c3f11e8f420b59f3786eb8626c99ff59
Comment 21 Michael Stahl (CIB) 2014-04-08 21:55:13 UTC
release builds:
4.2.0.3: buggy
4.2.0.4: not buggy
4.2.3.3: buggy

there were very few changes between 4.2.0.3 and 4.2.0.4,
and one of them is:

commit 37376041d4c30588fc577c20ec91a14e819d439a
Author:     Michael Meeks <michael.meeks@collabora.com>
AuthorDate: Mon Jan 27 10:26:48 2014 +0000

    Disable LTO for now - suspected cause of fdo#73464.

... which turned out to be not true, but maybe LTO is
responsible for this bug?  incidentally the above commit
was reverted for 4.2.1.1, which is the version this bug
is filed against.

do we have any tinderbox that builds with MSVC 2010 and LTO?
Comment 22 Michael Stahl (CIB) 2014-04-10 10:10:02 UTC
works:
http://dev-builds.libreoffice.org/daily/master/Win-x86@47-TDF/2014-04-07_11.14.34/

buggy:
http://dev-builds.libreoffice.org/daily/master/Win-x86@47-TDF/2014-04-09_12.09.54/
(which is special build with --disable-lto removed from config)

Stuart reports in comment #3 that libreoffice-4-2 builds from TB-42 work;
Thorsten says that tinderbox has ENABLE_LTO=TRUE and MSVC2012

=> only MSVC 2010 with ENABLE_LTO results in this bug
Comment 23 Commit Notification 2014-04-10 11:02:44 UTC
Michael Stahl committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=0d8e3a145901ab0124d40d33a50e2de28dc0c8ab

fdo#75376: configure: disable LTO by default for MSVC too



The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 24 Charles 2014-04-10 11:20:36 UTC
Thanks for fixing this Michael!

But...

So this means that the fix won't show up in official builds until 4.3?

Too bad... this means that we cannot use 4.2 in our office, so will have to wait until at leqast 4.3.3 before a major upgrade.

That really sucks.

Is this really such an invasive change that it cannot be backported?
Comment 25 Commit Notification 2014-04-10 14:14:33 UTC
Michael Stahl committed a patch related to this issue.
It has been pushed to "libreoffice-4-2":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=b6a0eacbc36b9db10f12a246606d8e2f1c30fef9&h=libreoffice-4-2

fdo#75376: configure: disable LTO by default for MSVC too


It will be available in LibreOffice 4.2.4.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 26 V Stuart Foote 2014-04-11 06:42:59 UTC
On Windows 7 sp1, 64-bit

For testing Michael S's patches, kind of need for Cloph's TB-47 to spin up a build of 4.2.4 branch in release configuration using MSVC 2010.

Working with master and MSVC 2010 from Cloph's TB-47 build from the 9th,
Version: 4.3.0.0.alpha0+
Build ID: 20fb1bfc72e626251b435bcff2339e1e425c7130
TinderBox: Win-x86@47-TDF, Branch:MASTER, Time: 2014-04-09_12:09:54
the malformed field shading exists, and can still be worked around with <CTRL>+F8 to turn off field shading.

And now Cloph's TB-47 build from the 10th is rendering field shading correctly with the http://cgit.freedesktop.org/libreoffice/core/commit/?id=0d8e3a145901ab0124d40d33a50e2de28dc0c8ab patch in place.
Version: 4.3.0.0.alpha0+
Build ID: 83c888bdb0a6c9795cebfc53fa74d6da8bb692b2
TinderBox: Win-x86@47-TDF, Branch:MASTER, Time: 2014-04-10_20:33:35

Issue as described was present in the RC3 and is in the Final release for 4.2.3 built with MSVC 2010. I have not seen it in TB-42 builds of 4.2 that use MSVC 2012. Re-verified that against a couple of recent TB-42 builds.