Bug 158950 - Paste as Rich Text Format loses character color and paragraph alignment from styles
Summary: Paste as Rich Text Format loses character color and paragraph alignment from ...
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
24.8.0.0 alpha0+
Hardware: All All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: target:24.8.0 target:24.2.1 target:7.6.5
Keywords: bibisected, bisected, regression
Depends on:
Blocks: Paste
  Show dependency treegraph
 
Reported: 2023-12-31 17:14 UTC by Regina Henschel
Modified: 2024-01-17 19:39 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
Source for copy&paste (15.04 KB, application/vnd.oasis.opendocument.text)
2023-12-31 17:14 UTC, Regina Henschel
Details
Source file after copy (17.99 KB, application/vnd.oasis.opendocument.text)
2024-01-01 03:09 UTC, m_a_riosv
Details
Screencast (2.39 MB, video/mp4)
2024-01-01 10:15 UTC, Regina Henschel
Details
File for FILEOPEN (2.55 KB, application/msword)
2024-01-12 19:00 UTC, Regina Henschel
Details
font color in paragraph style (41.44 KB, application/msword)
2024-01-15 14:06 UTC, Regina Henschel
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Regina Henschel 2023-12-31 17:14:53 UTC
Created attachment 191674 [details]
Source for copy&paste

Open attached document. Mark the complete text and copy it to clipboard.

Set cursor in a new paragraph after the existing text. Use Paste special and insert the clipboard as "Rich Text Format". Notice that a font color which is specified in a style is lost and the paragraph alignment specified in the style is lost.

For comparison paste the clipboard to a document in Word using its "Paste special" and "Rich Text Format". Font color and paragraph alignment are correct in Word.

Thus this is an import error in LibreOffice.
Comment 1 m_a_riosv 2024-01-01 03:09:33 UTC
Created attachment 191679 [details]
Source file after copy

Attached the file after copy.
No repro:
Version: 24.8.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: 0f6f5048d223731aa52b768a77244d0208711391
CPU threads: 16; OS: Windows 10.0 Build 22631; UI render: Skia/Vulkan; VCL: win
Locale: es-ES (es_ES); UI: en-US Calc: CL threaded
Comment 2 Regina Henschel 2024-01-01 10:15:53 UTC
Created attachment 191682 [details]
Screencast

I see the problem in Version: 7.6.4.1 (X86_64) / LibreOffice Community
Build ID: e19e193f88cd6c0525a17fb7a176ed8e6a3e2aa1
CPU threads: 32; OS: Windows 10.0 Build 22631; UI render: Skia/Vulkan; VCL: win
Locale: de-DE (de_DE); UI: en-US
Calc: threaded

and Version: 24.8.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: 4871de96cb5e31e5ab06cf97e02e09e0e04a4de8
CPU threads: 32; OS: Windows 10.0 Build 22631; UI render: default; VCL: win
Locale: de-DE (de_DE); UI: en-US
Calc: threaded
Comment 3 raal 2024-01-01 10:27:33 UTC
I can confirm with Version: 24.8.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: 71c28942fbc7f36e5bcd46c5a6cdfbb3fcbcd6a0
CPU threads: 4; OS: Linux 6.2; UI render: default; VCL: gtk3
Locale: cs-CZ (cs_CZ.UTF-8); UI: en-US
Calc: threaded
HTML format works.

Works in Version 4.1.0.0.alpha0+ (Build ID: efca6f15609322f62a35619619a6d5fe5c9bd5a) / regression.
Comment 4 raal 2024-01-01 10:41:55 UTC
This seems to have begun at the below commit in bibisect repository/OS linux-64-7.6.
Adding Cc: to Tomaž Vajngerl ; Could you possibly take a look at this one?
Thanks
 5a616cecbc70fe071aad39707500f412a200cd2d is the first bad commit
commit 5a616cecbc70fe071aad39707500f412a200cd2d
Author: Jenkins Build User <tdf@pollux.tdf>
Date:   Fri Jan 13 01:53:16 2023 +0100

    source 197e5f81213d14fdcbff40edf73385ecd4cd9815

144783: introduce {Char,Fill}ColorThemeReference which uses XThemeColor | https://gerrit.libreoffice.org/c/core/+/144783
Comment 5 Regina Henschel 2024-01-12 19:00:27 UTC
Created attachment 191900 [details]
File for FILEOPEN

It is not only a problem with copy&paste inside our document. I see the wrong character color too when opening an .rtf file.

Open the attached document that was created using Word. It should have the word "style" in red. When you set the cursor into the word and look in the Stylist, you see, that the word has assigned the character style "myRed". But somewhere this is overwritten with a direct formatting in black.
Comment 6 raal 2024-01-13 07:33:56 UTC
(In reply to Regina Henschel from comment #5)
> Created attachment 191900 [details]
> File for FILEOPEN
> 
> It is not only a problem with copy&paste inside our document. I see the
> wrong character color too when opening an .rtf file.
> 

Confirm with the commit mentioned above. Before the commit the color is red, after the commit the color is black.
Comment 7 Regina Henschel 2024-01-13 14:21:56 UTC
The wrong color comes in this way:

After the red color is set in case NS_ooxml::LN_EG_RPrBase_color in DomainMapper::sprmWithProps( Sprm& rSprm, const PropertyMapPtr& rContext ) this case is used again.

In this second use go into 
    pProperties->resolve(*pThemeColorHandler);
You will see that m_aAttributes and m_aSprems are empty and thus nothings happens in "resolve" method. So the default value of 0 of mnColor is used in
    m_pImpl->GetTopContext()->Insert(PROP_CHAR_COLOR, uno::Any(pThemeColorHandler->mnColor));
resulting in black text.

For testing I have set the default value of mnColor to -1 and have changed the call to
    if (pThemeColorHandler->mnColor != -1)
        m_pImpl->GetTopContext()->Insert(PROP_CHAR_COLOR, uno::Any(pThemeColorHandler->mnColor));
Then the color becomes red.

The previous version had not used Insert(PROP_CHAR_COLOR,...) at all in the case NS_ooxml::LN_EG_RPrBase_color, but had only set the PROP_CHAR_THEME_FOO attributes. Thus it was not visible, that the case NS_ooxml::LN_EG_RPrBase_color is used a second time. A seeming solution would be the above mentioned use of color value -1. But the real question is, why the case is reached a second time with neither a value in m_aAttributes nor in m_aSprms.
Comment 8 Regina Henschel 2024-01-15 14:06:03 UTC
Created attachment 191954 [details]
font color in paragraph style

The attached document has a font color in paragraph style. It shows the same problem, that the font color is black instead of the color of the style.

I have investigated further with these results (addressed to developers):

The problem starts in the method
static void cloneAndDeduplicateSprm(std::pair<Id, RTFValue::Pointer_t> const& rSprm, RTFSprms& ret, Id nStyleType, RTFSprms* pDirect = nullptr)
in https://opengrok.libreoffice.org/xref/core/writerfilter/source/rtftok/rtfsprm.cxx?r=75f6a86a#283
This method essentially looks whether parameter 'rSprm' is already contained in parameter 'ret' and removes it from 'ret' in such case.
It is called from RTFSprms::cloneAndDeduplicate(..).

rSprm is a std::pair<Id, RTFValue::Pointer_t>
ret is essentially a std::vector< std::pair<Id, RTFValue::Pointer_t> >.
The cloneAndDeduplicateSprm method first compares '.first' components, the 'Id'. If a matching item is found then pValue points to the .second component from it.
In a second step the method compares the '.second' components. If they are equal, the item is removed from 'ret'.

In case of the example document and color (id 92551, usable in VS as condition for breakpoint), you see that the items rSprm.second and *pValue are equal, when you manually compare them. But the comparison 'rSprm.second->equals(*pValue)' returns false.

Let's step into that comparison and see why it fails.

The 'equals' method is RTFValue::equals() from
https://opengrok.libreoffice.org/xref/core/writerfilter/source/rtftok/rtfvalue.cxx?r=f6fa9c45#164
'*this' and parameter rOther are the same {m_nValue=0 m_sValue=empty m_pAttributes={Params: 1} ...}.
So the conditions if (m_nValue != rOther.m_nValue) and if (m_sValue != rOther.m_sValue) are wrong and we reach
if (!m_pAttributes->equals(rOther))

In the example document the vector this->m_pAttributes has one item
(91603, {m_nValue=5154606 m_sValue=empty m_pAttributes={Params: 0}...})
rOther is the RTFValue object {m_nValue=0 m_sValue=empty m_pAttributes={Params: 1} ... } from *pValue above. 

I wonder why here the sub-item m_pAttributes is compared with the total RTFValue.

Let's step into that comparison.

This 'equals' method is a RTFSprms::equals() from
https://opengrok.libreoffice.org/xref/core/writerfilter/source/rtftok/rtfsprm.cxx?r=75f6a86a#440

That method uses a std:all_of, which iterates over all items in m_pSprms.
The things which are combined in the conjunction are the return values of a lambda-function.

The lambda-function compares its parameter raPair with the capture rOther. Actual values of the parameter raPair are the items in vector *m_pAttributes, rOther is the same in all comparisons.

The comparison is done in raPair.second->equals(rOther). For the example document we get
raPair.second={m_nValue=5154606 m_sValue=empty m_pAttributes={Params: 0}...} 
rOther={m_nValue=0 m_sValue=empty m_pAttributes={Params: 1} ... }

This 'equals' method is a RTFValue::equals.
It returns immediately 'false' because m_nValue=5154606 and rOther.m_nValue=0, thus different.

The same mechanism was in the version without lambda-function. (BTW, the old version is much easier to understand.)

I wonder about this RTFSprms::equals() method. It compares a vector of (Id, RTFValue) items with a single RTFValue? I would expect that its input parameter is a vector of (Id, RTFValues) and the iteration is parallel about both vectors.

If I disable the comparison
        if (!m_pAttributes->equals(rOther))
            return false;
in RTFValue::equals(const RTFValue& rOther), then the example document loads with the correct colors.
Comment 9 Tomaz Vajngerl 2024-01-16 12:09:59 UTC
Thanks, I'll take a look
Comment 10 Regina Henschel 2024-01-16 12:50:49 UTC
(In reply to Tomaz Vajngerl from comment #9)
> Thanks, I'll take a look

Noel proposes a fix in https://gerrit.libreoffice.org/c/core/+/162164
Comment 11 Commit Notification 2024-01-17 08:15:21 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/f6a559ccacaab875d1aa1da70e3011592158e9d4

tdf#158950 Paste as RTF loses char color and paragraph alignment from styles

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 12 Commit Notification 2024-01-17 10:05:39 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "libreoffice-24-2":

https://git.libreoffice.org/core/commit/b7219afbb13b923376ec5055209c92b087cc2c53

tdf#158950 Paste as RTF loses char color and paragraph alignment from styles

It will be available in 24.2.1.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 13 Commit Notification 2024-01-17 11:55:57 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "libreoffice-7-6":

https://git.libreoffice.org/core/commit/0b0b829420c8ce0195c913bf1e3a006e5dce2357

tdf#158950 Paste as RTF loses char color and paragraph alignment from styles

It will be available in 7.6.5.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 14 Commit Notification 2024-01-17 15:12:29 UTC
Xisco Fauli committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/f4647575a1e80169ba6f363fff72087d22165780

tdf#158950: sw_rtfexport6: Add unittest

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 15 Xisco Faulí 2024-01-17 19:39:28 UTC
I believe we can close this issue as FIXED.
@Noel, thanks for fixing it!