Description: When I try to configure 🟙 as a bullet character in LibreOffice, this fails. The character picker seems to be restricted to BMP characters for this. (that is, any character that would fit in 2 bits UTF-16). Steps to Reproduce: 1. Create a list 2. Try to change the character of the list to 🟙 (1f7d9, needs e.g. Noto Sans Symbols 2 font) Actual Results: Random different characters are used instead, and/or the font doesn't change correctly Expected Results: Bullet character changes to 🟙 Reproducible: Always User Profile Reset: No Additional Info: Version: 25.2.3.2 (X86_64) / LibreOffice Community Build ID: 520(Build:2) CPU threads: 8; OS: Linux 6.6; UI render: default; VCL: gtk3 Locale: en-US (en_US.UTF-8); UI: en-US 25.2.3-2 Calc: threaded
Confirmed. Choosing a glyph to use as bullet from Unicode SMP fails and is visually corrupted. Using the 'Unordered' list tab of the 'Bullets and Numbering' dialog. Unable to complete assignment of a glyph drawn from Unicode SMP for use as bullet in an unordered list. Seems some sort of multibyte limitation defining a bullet, the glyphs are available to document and the LibreOffice Special Character dialog (SCD) can handle. Font chosen makes no difference, the SCD charmap--launched from the 'Change Bullet' button [1][2]. The pick from the charmap is made, and added to list of recents on the SCD. So it is not a selection issue, but the actual glyph from SMP is being corrupted in updating the Unordered list style preview and as rendered to document. =-testing-= Version: 25.8.0.0.alpha0+ (X86_64) / LibreOffice Community Build ID: 3ab82581e5a5abc03c7ff06eac76a0b2fa0c9c34 CPU threads: 28; OS: Windows 11 X86_64 (build 26100); UI render: Skia/Vulkan; VCL: win Locale: en-US (en_US); UI: en-US Calc: CL threaded =-ref-= https://gerrit.libreoffice.org/c/core/+/163735 https://gerrit.libreoffice.org/c/core/+/167660
Created attachment 200703 [details] A Screenshot Of The Replacment Symbol In "Bullets and Numbering" I see the "�" symbol when I attempt to apply one in libreoffice-writer-25.2.3.2-1.fc42. Is that what you're referring to?
That's not a guaranteed outcome, I have seen various wrong options. I guess the bullet character just uses a 16 bit char variable?
Yep, I was right. sal_Unicode is used here, which is 16 bit. This is the result of using a mostly USC-2 internal system that sometimes behaves as UTF-16, but not always. I'm pretty sure I have seen more bugs like this and they mostly stick around because the storage format seems to rely on 16 bit characters as well? Is there any official policy on how Unicode should work in LibreOffice? At this point it's a mashup of UTF-8 on the borders, UTF-16 internally, with issues, and sometimes individual codepoints...
It looks to me more that the problem is that the chosen font isn’t being respected rather than a problem with Unicode encoding. For example if you follow these steps then it works fine: * Start with a fresh configuration by deleting instdir/user/registrymodifications.xcu * Make sure you have the Noto Sans Symbols 2 font installed https://fonts.google.com/noto/specimen/Noto+Sans+Symbols+2 * Create a new Writer document * Click on the unordered bullet button on the “Toggle Unordered List” button on the toolbar (or press Shift+F12) * Click Format -> Bullets and Numbering… from the menu * Select the “Customize” from the list on the left * Click on the “Select…” button next to the “Character:” label * Choose Noto Sans Symbols 2 for the font * Type “1f7d9” in the hexadecimal box to get the “NINE POINTED WHITE STAR” character. * Click ok on that dialog and the one underneath it. You should now have a working bullet list with the special symbol. Otherwise you can do these steps and it won’t work: * Start with a fresh configuration by deleting instdir/user/ * Create a new Writer document * Click on the unordered bullet button on the “Toggle Unordered List” button on the toolbar (or press Shift+F12) * Click Format -> Bullets and Numbering… from the menu * Select the “Unordered” tab (although it’s probably already selected) * Click the second list style and then the first one (for some reason the button below is disabled if you don’t do this) * Click the “Change Bullet” button * Choose Noto Sans Symbols 2 for the font * Type “1f7d9” in the hexadecimal box to get the “NINE POINTED WHITE STAR” character. * Click ok on that dialog and the one underneath it. Now you should get a rectangle where the star should be. I think the problem is just that it hasn’t changed the font for the style. I would think that editing the “Bullets” character style to change the font would fix the problem but it doesn’t seem to, unless I’ve misunderstood how that is supposed to work. You can see that cBullet in SvxNumberFormat is already sal_UCS4 which is 32-bit. https://cgit.freedesktop.org/libreoffice/core/tree/include/editeng/numitem.hxx#n117
Yeah, 90% of the code is already correct, I think I found a possible minimal fix. I think there are one or two spots in the character picker that get it wrong and cause this issue.
It's not purely a font issue though, there is one line that pushes the character through a 16 bit variable and corrupts the character.
Yes, you’re right, I stand corrected. I think I can see the line you are talking about. The two sets of steps in my comment above end up creating the SvxCharacterMap dialog in two different places. One of the them stores the result in a sal_Unicode and the other in a sal_UCS4 which explains the different behaviour. No AI needed to fix this one I think :)
Yep, a very minimal fix would just substitute that sal_Unicode, and some corrections in places where it's invoked.
Neil Roberts committed a patch related to this issue. It has been pushed to "master": https://git.libreoffice.org/core/commit/4888f6f3f59785d0e661b58060bd8458f866bf10 tdf#166488 Fix local variable used to store char from SvxCharacterMap It will be available in 26.2.0. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Neil Roberts committed a patch related to this issue. It has been pushed to "master": https://git.libreoffice.org/core/commit/b476a78f409a43f67d7b46ebf99fec7bb226bb8f tdf#166488 Add a UITest It will be available in 26.2.0. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Neil Roberts committed a patch related to this issue. It has been pushed to "libreoffice-25-8": https://git.libreoffice.org/core/commit/6f37e13706f5b0239862255929299d463fb1050b tdf#166488 Fix local variable used to store char from SvxCharacterMap It will be available in 25.8.4. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.