Bug 127471 - Copied calc diagram in gdi format looks ok under linux, but the fonts looks weird under windows.
Summary: Copied calc diagram in gdi format looks ok under linux, but the fonts looks w...
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: LibreOffice (show other bugs)
Version:
(earliest affected)
Inherited From OOo
Hardware: All All
: medium normal
Assignee: Armin Le Grand
URL:
Whiteboard: target:7.0.2 target:6.4.7 target:7.0....
Keywords:
Depends on:
Blocks:
 
Reported: 2019-09-10 11:39 UTC by Ilhan Yesil
Modified: 2021-03-12 08:33 UTC (History)
7 users (show)

See Also:
Crash report or crash signature:


Attachments
The calc document (49.89 KB, application/vnd.oasis.opendocument.spreadsheet)
2019-09-10 11:41 UTC, Ilhan Yesil
Details
Picture showing that font is ok under Linux (12.41 KB, image/jpeg)
2019-09-10 11:42 UTC, Ilhan Yesil
Details
Picture showing that font is weird under Windows (12.78 KB, image/jpeg)
2019-09-10 11:43 UTC, Ilhan Yesil
Details
Test file to check the system-dependent variations of MetaFontAction (8.87 KB, application/vnd.oasis.opendocument.graphics)
2021-01-27 17:33 UTC, Armin Le Grand
Details
unx versikon of exported testfile selection (1.79 KB, image/emf)
2021-01-29 12:15 UTC, Armin Le Grand
Details
win version of exported testfile selection (1.79 KB, image/emf)
2021-01-29 12:15 UTC, Armin Le Grand
Details
The StarViewMetafile extracted from the odt created as described (43.96 KB, application/octet-stream)
2021-01-29 16:17 UTC, Armin Le Grand
Details
Testfile to check with different Fonts (16.69 KB, application/vnd.oasis.opendocument.graphics)
2021-02-11 15:47 UTC, Armin Le Grand
Details
Testfile to check with different Scales (15.95 KB, application/vnd.oasis.opendocument.graphics)
2021-02-11 15:47 UTC, Armin Le Grand
Details
Testfile to check with different Sizes (14.62 KB, application/vnd.oasis.opendocument.graphics)
2021-02-11 15:48 UTC, Armin Le Grand
Details
All needed files to check and test, using old/new office before/after change (55.12 KB, application/zip)
2021-02-17 08:56 UTC, Armin Le Grand
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ilhan Yesil 2019-09-10 11:39:37 UTC
Description:
Under Linux a diagram in the attached calc document is copied in gdi format into a Textdocument. Still under Linux, if the Textdocument is closed and reopened, the gdi picture looks as expected. But if the Textdocument is opened under Windows, the Fonts looks very weird.

Steps to Reproduce:
1. Under Linux, open attached ods document.
2. Select diagram and copy it.
3. Create a new text document and paste the copied diagram in gdi format via menu -> "Edit->Paste Special".
4. Save the text document, close it and reopen it.
5. Copy the text document to a Windows machine and open it there.

Actual Results:
Under Linux after step 4, the diagram looks ok.
Under Windows after step 5, the Fonts looks weird.

See also attached documents.

Expected Results:
The gdi picture shouldn't show disformed Fonts, the text should be readible.


Reproducible: Always


User Profile Reset: No



Additional Info:
Comment 1 Ilhan Yesil 2019-09-10 11:41:35 UTC
Created attachment 154077 [details]
The calc document
Comment 2 Ilhan Yesil 2019-09-10 11:42:53 UTC
Created attachment 154078 [details]
Picture showing that font is ok under Linux
Comment 3 Ilhan Yesil 2019-09-10 11:43:28 UTC
Created attachment 154079 [details]
Picture showing that font is weird under Windows
Comment 4 Ilhan Yesil 2019-09-10 11:45:16 UTC
Self confirming according to https://wiki.documentfoundation.org/QA/Guidelines_for_public_and_private_sector_deployments. Issue has originally been reported by another user in our internal issue tracker.
Comment 5 Ilhan Yesil 2019-09-10 12:11:21 UTC
I've stripped out the original document from information, till there are three simple series of data. There must be settings in the diagram, which causes this problem, as if I create a new diagram using the same three simple data-series, the gdi picture looks ok under Linux and Windows.
Comment 6 Ilhan Yesil 2019-09-10 12:34:23 UTC
in drawinglayer/source/primitive2d/textlayoutdevice.cxx
in function: vcl::Font getVclFontFromFontAttribute

if I comment out the lines:

 #ifdef _WIN32
             // for WIN32 systems, correct the FontWidth if FontScaling is used
             if(bFontIsScaled && nHeight > 0)
             {
                 const FontMetric aUnscaledFontMetric(Application::GetDefaultDevice()->GetFontMetric(aRetval));

                 if(aUnscaledFontMetric.GetAverageFontWidth() > 0)
                 {
                    const double fScaleFactor(static_cast<double>(nWidth) / static_cast<double>(nHeight));
                    const sal_uInt32 nScaledWidth(basegfx::fround(static_cast<double>(aUnscaledFontMetric.GetAverageFontWidth()) * fScaleFactor));
                    aRetval.SetAverageFontWidth(nScaledWidth);
                 }
             }
 #endif

then the output under Windows is no longer wrong. 

I haven't see side effects so far, if I comment out these lines. Maybe this is an approach to solve this, somehow it has to do with the scale factor of fonts under windows.
Comment 7 Khaled Hosny 2019-09-11 19:21:48 UTC
Does enabling/disabling OpenGL on Windows affect this in anyway?
Comment 8 Ilhan Yesil 2019-09-12 07:30:12 UTC
In my tests OpenGL was enabled, I've disabled it now. OpenGL has no effect.
Comment 9 Xisco Faulí 2019-09-26 09:41:21 UTC
Reproduced back to 

LibreOffice 3.3.0 
OOO330m19 (Build:6)
tag libreoffice-3.3.0.4
Comment 10 Xisco Faulí 2020-02-17 10:50:44 UTC
Dear Ilhan Yesil,
This bug has been in ASSIGNED status for more than 3 months without any
activity. Resetting it to NEW.
Please assign it back to yourself if you're still working on this.
Comment 11 Commit Notification 2020-08-19 08:34:24 UTC
Samuel Mehrbrodt committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/8891a2fc2a4bf86add68691b7ac167a07a8add84

tdf#127471 Remove font width scaling hack

It will be available in 7.1.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 12 Commit Notification 2020-08-19 14:22:01 UTC
Samuel Mehrbrodt committed a patch related to this issue.
It has been pushed to "libreoffice-7-0":

https://git.libreoffice.org/core/commit/d935040e8f586ccb9dcf7fef30d72715a9f0ac98

tdf#127471 Remove font width scaling hack

It will be available in 7.0.2.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 13 Commit Notification 2020-09-04 08:57:57 UTC
Samuel Mehrbrodt committed a patch related to this issue.
It has been pushed to "libreoffice-6-4":

https://git.libreoffice.org/core/commit/b9306c4f721d3833296af144ac07dc2a064d1975

tdf#127471 Remove font width scaling hack

It will be available in 6.4.7.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 14 V Stuart Foote 2020-10-16 12:53:38 UTC
causes breakage of bug 136891
Comment 15 Samuel Mehrbrodt (allotropia) 2020-10-28 13:23:49 UTC
Going to revert this on 7.0: https://gerrit.libreoffice.org/c/core/+/104948
Comment 16 Xisco Faulí 2020-10-28 13:27:38 UTC
(In reply to Samuel Mehrbrodt (CIB) from comment #15)
> Going to revert this on 7.0: https://gerrit.libreoffice.org/c/core/+/104948

I would revert in in master, backport it to 7.0 and then, if you want to fix it, follow the same path, fix it in master and backport it. just my 2 cents
Comment 17 Commit Notification 2020-10-29 09:11:55 UTC
Samuel Mehrbrodt committed a patch related to this issue.
It has been pushed to "libreoffice-7-0":

https://git.libreoffice.org/core/commit/d578bd6582131e52c1752f16f21ba2164ccdc970

Revert "tdf#127471 Remove font width scaling hack"

It will be available in 7.0.4.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 18 Commit Notification 2020-10-29 09:15:07 UTC
Samuel Mehrbrodt committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/f39e4b6b6f8aa8b4af22b6eb30a52e98cd5a6455

Revert "tdf#127471 Remove font width scaling hack"

It will be available in 7.1.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 19 Commit Notification 2020-10-29 11:23:05 UTC
Stephan Bergmann committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/5a08034df97dd768b481fe999e81e80027774a5c

Revert "Revert "tdf#127471 Remove font width scaling hack""

It will be available in 7.1.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 20 Aron Budea 2020-11-12 06:20:44 UTC
Another revert attempt: https://gerrit.libreoffice.org/c/core/+/105620
Now including revert of Stephan's a879b15d59618e73797ad779666f72cd040ff99a ("-Werror,-Wunused-variable (clang-cl)") following the original commit.
Comment 21 Xisco Faulí 2020-11-12 09:02:20 UTC
so the commit is reverted in 7.0 but not in master
Comment 22 Commit Notification 2020-11-12 20:44:48 UTC
Aron Budea committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/5db34391ebf7a51d26196f3fe28f3d9327ea2eb6

Revert "tdf#127471 Remove font width scaling hack"

It will be available in 7.1.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 23 Aron Budea 2020-11-12 20:46:41 UTC
(In reply to Xisco Faulí from comment #21)
> so the commit is reverted in 7.0 but not in master
Now it is. Reopening the bug report, since we're back at square one.
Comment 24 Armin Le Grand 2021-01-27 13:13:23 UTC
Have some ideas/thoughts stretching over that whole processing - sigh, grepping...
Comment 25 Armin Le Grand 2021-01-27 14:51:41 UTC
Here comes the BG info: VCL tries to be AFAP to have the same API on all systems, but there are a view rare exceptions. One *is* the horizontal font scaling that is involved here. Next problem is that MTF is *not* as I say so often a VectorGraphicFormat, but more a recording of executed paint commands. Together with 1st point this leads to *different* versions of MTF for the same content. BTW this is not the case in Primitives, I took thorough care to not have system-dependent stuff there...
These diffs are the reason for the code Samuel tried to comment out. There are some more places that have to react to that exception.
1st step will be to have a more simple reproducable case, should be not too complicated. To fix I will have to deeply analyze the diffs in MTF created from Win and Linux (maybe Mac too) and decide if/how this is avoidable/detectable (also need to take care of existing MTFs as always - sigh)..
Comment 26 Armin Le Grand 2021-01-27 17:33:05 UTC
Created attachment 169193 [details]
Test file to check the system-dependent variations of MetaFontAction
Comment 27 Armin Le Grand 2021-01-27 17:37:35 UTC
Added a test file where text is scaled 150%, 100% and 50%. Export to Win and Linux shows e.g. for 1st (150%)
-> aSerializer.writeSize(rImplFont.maAverageFontSize);
Win: 552/635 -> 0.869 -> inverse: 1.15
Lin: 952/635 -> 1.4992..
So the system dependent part here is that for Win to paint using Win API the inverse relation width/height is used. Question will now be *if* we can detect for written metafiles if win or linux was written and it would be possible to correct that at load time...
Comment 28 Armin Le Grand 2021-01-29 12:15:00 UTC
Created attachment 169267 [details]
unx versikon of exported testfile selection
Comment 29 Armin Le Grand 2021-01-29 12:15:30 UTC
Created attachment 169268 [details]
win version of exported testfile selection
Comment 30 Armin Le Grand 2021-01-29 12:21:12 UTC
Did the needed deep-dive debugging through both systems (win/unx). In both cases, loaded test file scaledtext, select all, export as EMF. Strange fact: On win, 'Selection' is not activated in export dialog by default ?!?
Added results as scaledext_unx/scaledext_win accordingly to this task. Both have the same size (had hoped that e.g. different fonts would lead to diffs which may have been used to separate, but no) of 1828 bytes.
Opening win file on unx and vice-versa shows the expected error.
Now doing a binary compare and (probably) a deep-debug on both systems to see if *some* possibility to find out if EMF was created on which system is *somehow* possible - that would be easiest to be able to fix this ... but I'm not very confident...
Comment 31 Armin Le Grand 2021-01-29 12:31:23 UTC
Correction: File sizes are different,
scaledext_unx is 1920b
scaledext_win is 1828b
chances getting better, deep-checking for diffs...
Comment 32 Armin Le Grand 2021-01-29 14:15:04 UTC
Diff on mnRecordCunt 26:
EMR_SETVIEWPORTEXTEX w/h unx: 86/86, win 96/96 (maybe an error in writing DPI that we can use?)
Comment 33 Armin Le Grand 2021-01-29 14:26:52 UTC
Diff on mnRecordCunt 20:
EMR_EXTCREATEFONTINDIRECTW of course, this is where the wrong lfWidth is (952 unx, 551 win), but no other diff to detect change.
Saw a 'hack' at MtfTools::ImplMap from old times:

        // !!! HACK: we now always set the width to zero because the OS width is interpreted differently;
        // must later be made portable in SV (KA 1996-02-08)

may have to do with this problem and may show age of it.
Comment 34 Armin Le Grand 2021-01-29 14:42:32 UTC
Diff on mnRecordCunt 16:
EMR_EXTTEXTOUTW the text action itself. Small diffs in coordinates, but nothing to allow guessing on OS of creator. Checked through text action creations, no obvoious diff.
Comment 35 Armin Le Grand 2021-01-29 14:46:26 UTC
2nd text is not scaled, still same seek pos in stream...
Comment 36 Armin Le Grand 2021-01-29 14:50:37 UTC
3rd text scaled, still same seek pos (1680) in EMR_EXTTEXTOUTW action.
Comment 37 Armin Le Grand 2021-01-29 14:57:58 UTC
Both read same bytelength (1828) from included ReadEnhWMF stream (see xemfparser.cxx:130). No read errors, used. Size is adapted, slight diffs.

Thus only usable hint maybe from comment 33: EMR_EXTCREATEFONTINDIRECTW
Comment 38 Armin Le Grand 2021-01-29 15:25:26 UTC
Argh, correction: Of course EMR_SETVIEWPORTEXTEX from comment 32!
Comment 39 Armin Le Grand 2021-01-29 15:33:49 UTC
Trying wmf now - interestingly, also emf is created embedded, see

vcllo.dll!EMFWriter::WriteEMF(const GDIMetaFile & rMtf) Line 272 (c:\lo\work01\vcl\source\filter\wmf\emfwr.cxx:272)
vcllo.dll!WMFWriter::WriteEmbeddedEMF(const GDIMetaFile & rMTF) Line 1826 (c:\lo\work01\vcl\source\filter\wmf\wmfwr.cxx:1826)
vcllo.dll!WMFWriter::WriteWMF(const GDIMetaFile & rMTF, SvStream & rTargetStream, const FilterConfigItem * pFConfigItem, bool bPlaceable) Line 1746 (c:\lo\work01\vcl\source\filter\wmf\wmfwr.cxx:1746)
vcllo.dll!ConvertGDIMetaFileToWMF(const GDIMetaFile & rMTF, SvStream & rTargetStream, const FilterConfigItem * pConfigItem, bool bPlaceable) Line 83
Comment 40 Armin Le Grand 2021-01-29 15:38:01 UTC
Import uses embedded emf anyways, so emf stays in question to being fixed.
Another question is if we also need to fix/adapt wmf itself for older files that have no embedded emf, in the sense of trying to detect and correct, but also in future sense of securing (new version of MTFAction containing a bool or so, BTW not possible on emf due to being normed and not under our control...)
Comment 41 Armin Le Grand 2021-01-29 15:48:56 UTC
Indeed, current master on Ubuntu writes (86, 86) as EMR_SETVIEWPORTEXTEX. Does someone know if *all* linux'es do that...?
Repeated test with unx/win versions of wmf, same effect on both systems due to using embedded emf. Do we need to take care of 'old' wmf which contain no emf?
Comment 42 Armin Le Grand 2021-01-29 16:17:18 UTC
Created attachment 169277 [details]
The StarViewMetafile extracted from the odt created as described
Comment 43 Armin Le Grand 2021-01-29 16:18:40 UTC
Checked the SVM directly extracted from the *.odt created as described, also added as attachment. This has no embeddings and the same problem on win. Sl also need a solution for pure old SVMs...
Comment 44 Armin Le Grand 2021-02-01 11:34:49 UTC
Created simplified svm files for scaledtext file. There is *no* entry for exporting to svm in the 'Export' dialog (with 'selection'), so had to convert to metafile, open in 7zip, extract from Picture folder.
Win and Unx versions same size, checking deeper for detectable diffs (low chance)
Comment 45 Armin Le Grand 2021-02-01 11:43:20 UTC
Using hd and diff shows:
10c10
< 00000090  b8 03 00 00 7b 02 00 00  ff ff 00 00 02 00 05 00  |....{...........|
---
> 00000090  28 02 00 00 7b 02 00 00  ff ff 00 00 02 00 05 00  |(...{...........|
17c17
< 00000100  01 00 00 4f 03 00 00 60  05 00 00 6b 06 00 00 04  |...O...`...k....|
---
> 00000100  01 00 00 4f 03 00 00 60  05 00 00 66 06 00 00 04  |...O...`...f....|
61c61
< 000003c0  6e 73 00 00 3d 01 00 00  7b 02 00 00 ff ff 00 00  |ns..=...{.......|
---
> 000003c0  6e 73 00 00 b8 00 00 00  7b 02 00 00 ff ff 00 00  |ns......{.......|
68c68
< 00000430  00 00 00 6a 00 00 00 1c  01 00 00 cd 01 00 00 26  |...j...........&|
---
> 00000430  00 00 00 6a 00 00 00 17  01 00 00 c5 01 00 00 1e  |...j............|

four diffs, 8 bytes diff, more than expected esp. last diff has two spots, checking import if/what that is...
Comment 46 Armin Le Grand 2021-02-01 12:14:42 UTC
I see a chance using the also imported DXArray. These are the same on win/unx (except last value), so possible strategy:

- remember last loaded font action in metafile
- on text action, if last font and it's a scaled font, check with temp local created dxarray and if there would be a noticeable diff to loaded one
- if yes, it might be valuable to adapt the font scaling and compare if that is more plausible

Only possible caveat are cases where single letters are by purpose made much wider/taller than it would be expected - not very plausible...

Question is if this might be possible for wmf/emf, too - do these have a DXArray saved...? Need to check...
This should eventually be accompanied with extending the font metafile entry (new version) to add a bool when the font is scaled that marks if scale is win-specific.

Alternative: correct the font action at save time to always use the unx def for scaling (that's the more plausible one anyways, win is the exception). Then correction at load always needed, but no new version of font action needed.

Huge alternative: Change Vcl Font to always use unx font scale define and *only* convert to win format in sys-dependent part (where font is set win-specific in system layer). That would mean to find all exception handlings that do something different on win for font scaling currently, too, what is not easy and potentially dangerous. OTOH tat would avoid errors in the future on that level - no one would have to know that font scaling is sys-dependent - this *will* lead to errors in the future. I am pretty sure that each change/feature in this area did lead to errors on the 'other' system in the past.

Have to think about these alternatives and further check their doability (e.g. if emf/wmf has DXArray...)...
Comment 47 Armin Le Grand 2021-02-01 13:56:30 UTC
Let's check if we can correct the value. From drawinglayer/source/primitive2d/textlayoutdevice.cxx we know what FontWidth is set to:

Unx:
    const sal_uInt32 nHeight(basegfx::fround(fabs(fFontScaleY)));
    const sal_uInt32 nWidth(basegfx::fround(fabs(fFontScaleX)));
    const bool bFontIsScaled(nHeight != nWidth);
    vcl::Font aRetval(rFontAttribute.getFamilyName(), rFontAttribute.getStyleName(),
                      Size(bFontIsScaled ? std::max<sal_uInt32>(nWidth, 1) : 0, nHeight));
so, just FontSize(nW, nH) goes into it and is loaded again.

Win:
    vcl::Font aRetval(rFontAttribute.getFamilyName(), rFontAttribute.getStyleName(),
                      Size(0, nHeight));

    if (bFontIsScaled && nHeight > 0)
    {
        const FontMetric aUnscaledFontMetric(
            Application::GetDefaultDevice()->GetFontMetric(aRetval));

        if (aUnscaledFontMetric.GetAverageFontWidth() > 0)
        {
            const double fScaleFactor(static_cast<double>(nWidth) / static_cast<double>(nHeight));
            const sal_uInt32 nScaledWidth(basegfx::fround(
                static_cast<double>(aUnscaledFontMetric.GetAverageFontWidth()) * fScaleFactor));
            aRetval.SetAverageFontWidth(nScaledWidth);
        }
    }
Thus, UFM -> Application::GetDefaultDevice()->GetFontMetric(aRetval).GetAverageFontWidth() is added as factor, so we have UFM * (nW/nH). So we need to get 
  FontSize(UFM * (nW/nH), nH)
To be able to reconstruct nW at all, we need to know UFM which does not get loaded and is not in the file. Thus I will have to check what
  Application::GetDefaultDevice()->GetFontMetric(aRetval).GetAverageFontWidth()
will be on unx systems. It *will* probably be available on Win systems, but correction would need to take place in both possible directions on both systems. And this *needs* to be done at a Font with no set FontWidth (see win part above).
Checking that...
Comment 48 Armin Le Grand 2021-02-01 14:51:41 UTC
Checked, but unfortunately

        Font aUnscaledFont(rFont);
        aUnscaledFont.SetAverageFontWidth(0);
        const FontMetric aUnscaledFontMetric(Application::GetDefaultDevice()->GetFontMetric(aUnscaledFont));
        const tools::Long nAFW(aUnscaledFontMetric.GetAverageFontWidth());

gets nW == nH, I get the same value as FontHeight on Linux,
Getting the orig value (963 for the example while nH == 635) on Windows.

This means I could correct (if I can reliably detect) loading a unx metafile on win to win-specific stuff, but not when loading a win metafile on unx.

When loading a win metafile with scaled font on unx the information is lost and cannot be recreated - AFAICS.

So I would guess trying to correct old files at load may be possible when loading a unx-created mtf with scaled fonts on win, but - since the other direction is not possible and the detection is an open question (based on DXArray?) is too expensive.

So I will now concentrate on avoiding the error for new created files.
Comment 49 Armin Le Grand 2021-02-01 16:47:31 UTC
Thought a while about possibilities. We have old office versions win/unx, old file versions win/unx, new office versions win/unx and new file versions win/unx. This gives a 4x4 matrix of possible exchanges.

First thought about writing the better unx like definition in new offices to new files always (also from win), no change to format (no increase of VersionCompat).
New office win loading: will always assume unx
 old win: adapt is wrong, error (!)
 new win: adapt, ok
 old unx: adapt, ok
 new unx: adapt, ok
New office unx loading: will always assume unx
 old win: error (as now)
 new win: ok
 old unx: ok
 new unx: ok
Old office win loading:
 old win: ok
 new win: is now unx, error (!)
 old unx: error (as now)
 new unx: error
Old office unx loading:
 old win: error (as now)
 new win: is now unx, ok
 old unx: ok
 new unx: ok
The (!) mark the problems: Exchange between old/new win version would be bad. I think this is hardly acceptable. Someone updating it's win office version would find unexpected errors in his files.

Alternative: Extend VersionCompat, write 2nd font scaling value, I think best would be to always write unx version there. Expand win version, copy on unx. Thus: If no extension, old format, not detectable, same errors as now.
If extension and equal, unx wrote these. If different, win wrote these. Table:
New office win loading:
 old win: ok
 new win: detectable, ok
 old unx: not detectable/distinguishable from win version, error (as now)
 new unx: detectable, ok
New office unx loading:
 old win: error (as now)
 new win: detectable, ok
 old unx: ok
 new unx: detectable, ok
Old office win loading:
 old win: ok
 new win: ok
 old unx: error (as now)
 new unx: error (stays)
Old office unx loading:
 old win: error (as now)
 new win: error (stays)
 old unx: ok
 new unx: ok
I prefer this solution, but will have to think about it some more. Both versions have six error cases, but 2nd version has more 'as now' cases and will not 'destroy' compatibility for someone just updating it's executable.
Comment 50 Armin Le Grand 2021-02-02 13:30:20 UTC
Implementing cross-system...
Comment 51 Armin Le Grand 2021-02-02 14:58:00 UTC
Did some 1st checks, looks promising. Re-investigated what values would be needed, but indeed the unx-def (better: non-windows def) is to be used.
Reason is simple: Only on Windows the conversion to/from pre-scaled windows FontWidth in vcl::Font::Width is possible. This again is due to getting only there a useful value when calling GetAverageFontWidth().
On Win this really returns a average font width, less than height, more than zero, about 2/3rd of height.
On other systems you just get same value as height. This is okay when reading at quite some comments that W==H means no scaling.
Thus we have two defs for no scaling on two systems, and - maybe to be able to handle it unifornmely at all - a third one: If W==zero we have no scale on all systems.
Need to do more experiments, but looks food. Full test also needed (on all systems).

Note: This fixes general SVM exchange problems between systems. Due to this being used indirectly by the described but, that one will be fixed (checked that), but also potentially many more occurrences.

Note that when doing the 1st steps in orig description with Win and Unx exchanged, the error will also be there (and also be fixed).

Also note that WMF/EMF have potentially the same problem and would also need fixes IMHO. Time and options to do that would need further evaluation.

Also note that a general rework to always have the one (non-win) def for font scale in vcl::Font and used everywhere and only use the sys-dep parts in the sys-dep layers is what should be done. That will even need more eval (and time) if this should be done, but would fix all mentioned cases more or less.
Comment 52 Armin Le Grand 2021-02-02 17:03:44 UTC
Autotests done on all systems, added 1st version to gerrit (see https://gerrit.libreoffice.org/c/core/+/110330)
Comment 53 Commit Notification 2021-02-03 16:07:34 UTC
Armin Le Grand (Allotropia) committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/40b56cd8da8c38582dc4660b486993d1b4711535

tdf#127471 improve SVM FontScaling im/export

It will be available in 7.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 54 Armin Le Grand 2021-02-04 11:19:17 UTC
Same problem potentially in WMF/EMF, checking...
Comment 55 Armin Le Grand 2021-02-04 11:37:35 UTC
Checked, same error. Will keep this task open.
Comment 56 Armin Le Grand 2021-02-05 14:59:44 UTC
Note for BP to somewhere
40b56cd8da8c38582dc4660b486993d1b4711535 (fix) *and*
78886bc1a66fa867893cecf00919f644faa2088d (missing includes)
are needed - thanks to Mike for 2nd one!
Comment 57 Thorsten Behrens (allotropia) 2021-02-07 23:28:56 UTC
Related then is https://bugs.documentfoundation.org/show_bug.cgi?id=103859 - and I wonder if some pruning of possibly duplicate bugs is in order once the EMF/WMF aspects from there are solved?
Comment 58 Armin Le Grand 2021-02-08 14:13:52 UTC
Made decision to also attach EMF/WMF stuff and maybe detect error in old files (see 2nd table in Comment 49)...
Comment 59 Armin Le Grand 2021-02-08 17:20:22 UTC
WMF: Writes at vcl/source/filter/wmf/emfwr.cxx:465, reads at emfio/source/reader/emfreader.cxx:1741. EMF does the same. Looking deeper where WinMtfFontStyle is created (height is negative in LOGFONTW), also some mapping/scaling is done in CreateObjectIndexed/MtfTools::ImplMap.
Comment 60 Armin Le Grand 2021-02-08 17:58:08 UTC
Looks as if I can - for now - only correct to always using non-win font scaling, detect if old win format has to happen later...
Comment 61 Armin Le Grand 2021-02-09 09:48:32 UTC
Checking EMF/WMF docu to see what/if I can somewhere in the font stuff place a marker to allow detecting the difference to old files...
Comment 62 Armin Le Grand 2021-02-09 10:13:53 UTC
What we write is a Font Object using EXTCREATEFONTINDIRECTW, but extended by EXTLOGFONTW. We only read EXTCREATEFONTINDIRECTW, so there might be some possibilities to add data. Checking if/where lte least hack-like possibililty might exist - need to deep-dive MS docu...
Comment 63 Armin Le Grand 2021-02-09 12:32:22 UTC
During my coffee break I though about all that EMF/WMF and got to a completely different POV: I *cannot* use the non-win notation as std notation there, these *are* Win-formats. So, for EMF/WMF I have to go the opposite direction: Need to use the win-format as std format.
This is *very* problematic - it means to prepare/correct at save/load on (non-win) unx systems to win notation, but except on Win a value for AverageFontWidth is *not* available. This is (see above) pre-multiplied in Font::Size.Width() and needed for adaption.
I will do some experiments with non-win written EMF/WMF on win-apps other tan LO, this should show the error. If this is correct it will be even more important to find a way to work around this and to fix it.
Already thinking about ways how (and if) to 'create' that AverageFontWidth value on non-win systems...
Comment 64 Armin Le Grand 2021-02-09 14:24:56 UTC
tried with quite some apps on Win - usually, the examples written by win version of LO look better - so prove. Except: Gimp - the linux version is better - seems as if gimp has the same bug...?
Comment 65 Armin Le Grand 2021-02-09 14:25:56 UTC
Will now experiment/evaluate how to calculate a 'good' Win-like AverageFontWidth on Linux/non-win...
Comment 66 Armin Le Grand 2021-02-09 15:04:48 UTC
Checked where the AverageFontWidth comes from, Win is:
tmAveCharWidth at TEXTMETRICW structure (wingdi.h), used in vcl\win\gdi\salfont.cxx. This is ten used to fill data in GetFontMetric() calls.
Comment 67 Armin Le Grand 2021-02-09 15:07:49 UTC
On Linux we have FreetypeFont::GetFontMetric in vcl/unx/generic/glyphs/freetype_glyphcache.cxx that copies value from FreetypeFont in vcl/unx/generic/glyphs/freetype_glyphcache.cxx. There it gets inited by FreetypeFont::FreetypeFont using
    mnWidth = rFSD.mnWidth;
    if( !mnWidth )
        mnWidth = rFSD.mnHeight;
which leads to W==H. Looking at classes involved I can not find *anything* similar to AverageFontWidth at all. Checking docu on web...
Comment 68 Armin Le Grand 2021-02-11 10:00:52 UTC
Found some interesting links, excerps:
-- MS Docu:
The average width of characters in the font (generally defined as the width of the letter x ). This value does not include the overhang required for bold or italic characters.
-- Dev having same problem:
pDC->SelectObject(pFont);

pDC->GetTextMetrics(&tmNew);

size = pDC->GetTextExtent(_T("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"),52);
tmNew.tmAveCharWidth = (size.cx/26+1)/2;
--
Will experiment in code - on win, add testcode doing this and compare to tmAveCharWidth to see how to get as close to orig value reliably as I can
Comment 69 Armin Le Grand 2021-02-11 11:19:19 UTC
Testing with width of 'x' gives 15% derivation on win, so not really correct. Investigating further...
Comment 70 Armin Le Grand 2021-02-11 11:53:46 UTC
Testing 2nd method on win gives better result, 0,08% deviation.
Checked both methods on linux, I get the same results for both cases (and of course no useful result from FontMetric directly). Not to bad. Font Height is also slightly different on linux, derivates about 1,2%.
Need to check with more fonts on both systems...
Comment 71 Armin Le Grand 2021-02-11 15:47:03 UTC
Created attachment 169683 [details]
Testfile to check with different Fonts
Comment 72 Armin Le Grand 2021-02-11 15:47:40 UTC
Created attachment 169684 [details]
Testfile to check with different Scales
Comment 73 Armin Le Grand 2021-02-11 15:48:05 UTC
Created attachment 169685 [details]
Testfile to check with different Sizes
Comment 74 Armin Le Grand 2021-02-11 15:50:22 UTC
Need to check with more examples to find best calculation method, added test files with different fonts, scales and sizes. From each one, select all, export..., selection, export twice, to *.emf and *.wmf, on Windows and Linux. To check then load on Windows and compare to real AvgFontWidth (only available on Win)
Comment 75 Armin Le Grand 2021-02-11 16:34:04 UTC
diffonts: 80% close to value, 20% have deviations of up to 26%, but usually smaller. Second method is nearly always better.

diffscale: No effect - good, scale has no influence

diffsizes: Small effect (under 1%) - also good

Thus I'll give 2nd method a try...
Comment 76 Armin Le Grand 2021-02-12 15:07:28 UTC
Decided to do some more experiments/tests to maybe get closer to the original value. Also trying to find more info somewhere..
Comment 77 Armin Le Grand 2021-02-12 16:56:40 UTC
Tried various combinations...
Best is a loop creating a artificial string from ascii 32 to 127, I then tried which character is the best for each font, result was "+z*4E wZ-\"cat%", so nothing clear (and *not* 'x'). Using that string gave mixed results, so no solution either.
Best now is the artificial string, thinking further about it...
Comment 78 Armin Le Grand 2021-02-15 09:48:15 UTC
Will now do a full matrix test with the fonts and all chars, lets see which will be best suited for calculation
Comment 79 Armin Le Grand 2021-02-15 10:48:31 UTC
diffonts_old_unx on win with 32 measurements gives the lowest deviations for the first ten chars:
(97 'a', 3.1907319859348364)
(112 'p', 3.4003442128876005)
(117 'u', 3.4617083271467317)
(110 'n', 3.4739973308338934)
(76 'L', 3.4757888677422857)
(52 '4', 3.5169020625612513)
(103 'g', 3.5614644666862247)
(100 'd', 3.5854005513490552)
(122 'z', 3.6068763419352967)
(48 '0', 3.6165617411726911)...
This combination is 12 times more bad than 32-127 version, but also 20 times better.
With using 1st five it is also good.
Using just 'a' ist 6 times worse, but 10 times better already, compared with 'x' it's the same.
Looks as if 32-127 ersion is just best, so I'll go with it - can be buffered anyways if needed, it's just used at emfio...
Comment 80 Armin Le Grand 2021-02-15 16:38:19 UTC
Added code to convert to Win-like notation on linux EMF/WMF write and back-convert at load. Thus for EMF/WMF:
New office win loading: will always assume win
 old win: ok
 new win: ok
 old unx: error (as before)
 new unx: ok
New office unx loading: will always assume win
 old win: ok
 new win: ok
 old unx: error (!)
 new unx: ok
Old office win loading:
 old win: ok
 new win: ok
 old unx: error (as before)
 new unx: ok
Old office unx loading:
 old win: error (as before)
 new win: error (as before)
 old unx: ok
 new unx: error (!)
Cannot do changes on old offices win/unx, of course.
Thinking about how new offices might detect old unx written...
Comment 81 Armin Le Grand 2021-02-16 09:18:20 UTC
For new office loading old file I could add a flag to newly written files. No flag -> old file. Problem is to detect unx-written ones then.
Of course no chance to change old offices. Also makes no sense to write definitely wrong (non-win-style) new files, anyways it cannot be forseen that a file will be loaded by old office at any point in time.
I will make a deep check between old file written linux/win to see if there is any chance to detect that case...
Comment 82 Armin Le Grand 2021-02-16 10:32:30 UTC
May take action at emfio\source\reader\mtftools.cxx MtfTools::DrawText before mpGDIMetaFile->AddAction. From the cases in comment80 (new office, old unx):

New office win loading: will always assume win
 old unx: error (as before)
No adaption, in Font will/should be too big FontSize

New office unx loading: will always assume win
 old unx: error (!)
Already adapted by code in WinMtfFontStyle::WinMtfFontStyle, assuming that source is win, but is unx -> will be double too large. Maybe possible to remember in principle wrong but in this case correct, un-corrected size somewhere

Due to correction in 2nd case will need to detect and handle *two* cases here.
Continue debugging...
Comment 83 Armin Le Grand 2021-02-16 10:34:43 UTC
This gets complex, better already put 1st correction code in - preparing commit
Comment 84 Armin Le Grand 2021-02-16 14:17:31 UTC
Due to needing avgFontWidth in all places on all systems I'll prep done changes to have this on a central place
Comment 85 Armin Le Grand 2021-02-16 15:35:15 UTC
Lotzs of sys-dep stuff, need to run checks on win and linux...
Comment 86 Commit Notification 2021-02-17 08:31:38 UTC
Armin Le Grand (Allotropia) committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/9d161857f1d4afcb772b477455797a2da0e47a9b

tdf#127471 correct EMF/WMF im/export for scaled font

It will be available in 7.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 87 Armin Le Grand 2021-02-17 08:56:52 UTC
Created attachment 169805 [details]
All needed files to check and test, using old/new office before/after change
Comment 88 Armin Le Grand 2021-02-17 15:11:53 UTC
Continuing on comment 82 stuff: Check how to possibly detect old EMF/WMF files and correct them...
Comment 89 Armin Le Grand 2021-02-17 15:27:53 UTC
Best chances probably are at emfio\source\reader\emfreader.cxx.
In EMR_EXTCREATEFONTINDIRECTW/CreateObjectIndexed I could remember last imported font, it's pos in metafile, too
In EMR_EXTTEXTOUTA/EMR_EXTTEXTOUTW at DrawText I could use the DXArray and text (and Font) to check if scaling is plausible or would be more plausible with adapted version. For both systems in different situations, and only when FontScaling is used. Will test...
Comment 90 Armin Le Grand 2021-02-18 11:07:51 UTC
Checked import of win_old scaled text, maximum derivation between expected and imported text length is 7.9081632653061229%, so the detection needs to be somewhat diffuse.
Same result for win_new (7.9081632653061229)
For unx_new we get 38.966900702106308% which is larger as expected - this is already corrected...
For unx_old (the error case) we get a max deviation 58.784334049011619%. This will be the base for unsharp detection. Needt to also evaluate a min deviation for that case...
For unx_old the min is stable around 40% except for diffonts_old_unx, goes down to need to check there.
Goes from around 40% for most fonts down to 31% for one, 17% for another, down tio 2.4288910194950386% for Webdings - understandable, chars have quite all the same size there.
So I'll also have to take Scaled/Unscaled AverageFontWidth into account. Checking that next.
Comment 91 Armin Le Grand 2021-02-19 18:06:35 UTC
Have a 1st working version on linux, checked with all new and old files (checking only error case ist *not* sufficient, also need to make sure new or old working stuff is not killed).
Need to cross-check on Windows ASAP due to needing to do different corrections there...
Comment 92 Armin Le Grand 2021-02-22 11:39:31 UTC
Cross-check on Win ok, thinking about doing this always for scaled text or adding a mechanism that 1st checks for viability of imported old unx-style emf/wmf and only then corrects all ScaledText(s) equally...
Comment 93 Armin Le Grand 2021-02-23 09:21:07 UTC
Re-evaluated and moved more to tooling in refactored version, need to take into account that there may be various DrawText actions for each W_META_CREATEFONTINDIRECT (in WMF read, similar in EMF read).
Need to take a 2nd look at scaledtext_diffonts examples, some fonts are not too good. The others scaledtext_diffscales and scaledtext_diffsizes look already pretty good.
Comment 94 Armin Le Grand 2021-02-23 10:26:26 UTC
New form as tooling works better, also on win. Also is a needed preparation, will be much easier adaptable *if* we - one day - will refactor these EMF/WMF imports to directly create primitives. Doing last checks/build tests on both systems...
Comment 95 Commit Notification 2021-02-23 19:43:06 UTC
Armin Le Grand (Allotropia) committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/3d33e4ce3987ea17e73a72e84f7f0df7af8101a6

tdf#127471 Detect&Correct EMF/WMF with wrong FontScale

It will be available in 7.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 96 Armin Le Grand 2021-02-24 20:41:26 UTC
Mike Kaganski added 16901e4ccad95d55d4ae0fa2d6f6103b4177592e which fixes two problems - thanks!
Comment 97 Commit Notification 2021-03-09 09:20:53 UTC
Armin Le Grand (Allotropia) committed a patch related to this issue.
It has been pushed to "libreoffice-7-1":

https://git.libreoffice.org/core/commit/a5cecd2ea50dc930cfc0d90be69c08ddd144d3db

tdf#127471 improve SVM FontScaling im/export

It will be available in 7.1.2.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 98 Commit Notification 2021-03-10 08:42:12 UTC
Armin Le Grand (Allotropia) committed a patch related to this issue.
It has been pushed to "libreoffice-7-1":

https://git.libreoffice.org/core/commit/125efab4d8573fda1882a542f6fb669c7fc12b56

tdf#127471 correct EMF/WMF im/export for scaled font

It will be available in 7.1.2.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 99 Roman Kuznetsov 2021-03-10 15:14:37 UTC
Armin, is this bug fixed? If yes, then change status to FIXED please

ps: I checked in

Version: 7.2.0.0.alpha0+ (x64) / LibreOffice Community
Build ID: 9d8accf03984a64a4105826e55b221962628eb93
CPU threads: 4; OS: Windows 6.1 Service Pack 1 Build 7601; UI render: Skia/Raster; VCL: win
Locale: ru-RU (ru_RU); UI: en-US
Calc: CL

font on the chart in Writer look much better than in Ilhan's attach for Windows.
Comment 100 Armin Le Grand 2021-03-10 16:08:00 UTC
@Roman: pretty much, backporting to libreoffice-7-1, then will set to fixed
Comment 101 Commit Notification 2021-03-11 20:10:14 UTC
Armin Le Grand (Allotropia) committed a patch related to this issue.
It has been pushed to "libreoffice-7-1":

https://git.libreoffice.org/core/commit/593e4f5b92619d9d44dc3cb79f8b5088ef8276af

tdf#127471 Detect&Correct EMF/WMF with wrong FontScale

It will be available in 7.1.3.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 102 Armin Le Grand 2021-03-12 08:33:34 UTC
Done. Open to add comment to RelNotes