Bug 92940 - many Emojis do not work
Summary: many Emojis do not work
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: graphics stack (show other bugs)
Version:
(earliest affected)
5.0.0.0.alpha0+ Master
Hardware: Other Windows (All)
: high normal
Assignee: Not Assigned
URL:
Whiteboard: target:5.3.0
Keywords: implementationError
: 100212 (view as bug list)
Depends on:
Blocks:
 
Reported: 2015-07-26 12:32 UTC by tommy27
Modified: 2016-11-19 08:41 UTC (History)
9 users (show)

See Also:
Crash report or crash signature:


Attachments
autocorrect replacement table with emoji (10.09 KB, image/png)
2015-07-26 12:32 UTC, tommy27
Details

Note You need to log in before you can comment on or make changes to this bug.
Description tommy27 2015-07-26 12:32:31 UTC
Created attachment 117448 [details]
autocorrect replacement table with emoji

I started noticing this as soon as the new feature was implemented ( see https://goo.gl/LA5XzK ) and I thought was a temporary issue but now I'm still seeing it even in latest builds

as you may see in the screenshot some emojis are correctly displayed (i.e.  :airplane:) while many others (i.e.  :alien) have a rectangle instead of their symbol.

the problem is not limited to the autocorrect replacement table but affects editing as well...

if you type :airplane: it is correctly replaced by that symbol, whilst :alien: is replaced by a blank space.

my tests were performed under Win8.1 x64 using:

LibO 5.0.0.4
Build ID: cf112dc905650fb985306a7a03d2fe3fcc6c978f
Versione locale: en-US (it_IT)

and

LibO 5.1.0.0.alpha1+
Build ID: 8cfdd81b70ef37927b40497ffd10034f28335034
TinderBox: Win-x86@39, Branch:master, Time: 2015-07-24_02:47:18
Locale: en-US (it_IT)
Comment 1 László Németh 2015-07-27 11:59:06 UTC
It seems, LibreOffice 5 cannot show non-BMP Unicode characters (eg. these characters are missing from the Insert Special Characters dialog window, too, for example choosing Segoe UI Emoji or Deja VU Sans).

Installing LibreOffice 4.4 on the same Windows 8.1, that has no problem with non-BMP characters, only LibreOffice 5.0.

If I right remember, this problem can depend from the start of LibreOffice 5, so some Windows code page settings could be the problem.

Sample text: ⌨ (non-BMP: U+2328), 📷 (BMP: U+1F4F7)
Comment 2 László Németh 2015-07-27 12:53:36 UTC
> Sample text: ⌨ (non-BMP: U+2328), 📷 (BMP: U+1F4F7)

Sorry, ⌨ is a BMP character (under 64k characters, U+2328), 📷 is a non-BMP one (>64k, U+1F4F7)
Comment 3 Horst 2015-09-12 21:24:01 UTC
Can not confirm the correct behavior in 4.4.5 on Win7 home x64.
Is same as with 5.0.1.

When I installed 5.0.1 it worked once for me and then never again??????
Comment 4 tommy27 2015-09-23 18:59:05 UTC
many emojis still don't work using LibO 5.1.0.0.alpha1+
Build ID: 0ca9a3523cb2ccf97d7d13fdf4616af0dbe173d9
TinderBox: Win-x86@62-merge-TDF, Branch:MASTER, Time: 2015-09-23_05:16:23
Locale: it-IT (it_IT)

this new 5.0.x feature never worked fine.
it's an implementation error
Comment 5 László Németh 2015-09-25 16:14:51 UTC
Checking it on Windows 10 with a fresh LibreOffice profile using LibreOffice 5.0.2, I cannot reproduce the BMP-only problem. Emoji replacement works without any problem. 

Check the followings to get a working emoji support on your operating system:

1. On Linux, you need to install Symbola font (or other fonts with emoji support). On recent Windowses, Emoji support is default, on older Windowses, it's possible to update for it (see MS document on "An update for the Segoe UI symbol font in Windows 7 and in Windows Server 2008 R2 is available").

2. First check the BMP character replacements, eg. with :alpha: in an English language documents. If you had problems, check the list in the AutoCorrect dialog. If the (English language) replacement list don't contain the emoji replacements, remove the copy of the old acor*dat from your profile (IMPORTANT NOTE: don't delete it, if you don't want to lose your own custom replacements.):

For example, moving the copy of the acor_en-US.dat to ~/tmp under Linux:

mv .config/libreoffice.orig/4/user/autocorr/acor_en-US.dat ~/tmp

After restarting LibreOffice, check the list in that dialog window, you will see the replacements (using LibreOffice 5.0).

Recent Linux and Windows versions show also the Emoji characters in the dialog window (they have extended Unicode UI fonts).

3. Try non-BMP character replacements. On Linux, in the case of using some simple text fonts (for example, old Windows fonts or Liberation), you don't need to set explicitly Symbola in the text to get non-BMP characters, otherwise it needs to set Symbola in character formatting of the text. Also on Windows, I didn't need to specify Segoe UI Emoji.
Comment 6 tommy27 2015-10-03 08:18:26 UTC
I still have issues with many emojis

retested under Win8.1 x64 using LibO 5.1.0.0.alpha1+ (x64)
Build ID: 8c7ba16ba4bdedb4354f342b20d5a5de8a132b48
TinderBox: Win-x86_64@62-TDF, Branch:MASTER, Time: 2015-09-24_21:53:09
Locale: it-IT (it_IT)

for example :alpha: and :airplane: are correctly replaced by their corresponding symbols while many other like :alien: and :phone: are replaced by nothing.

I wonder if this is a Windows only bug or if I need to install some special font in order to visualize all the emojis
Comment 7 Horst 2015-10-03 15:53:46 UTC
I made some testing since my last comment.
Mostly on Win7 Home 64bit.
The following LO versions do NOT show the replacement table correctly (non-BMP not displayed:
LO4.4.5.2 (Standard install), rest admin install 5.0.1.2x64, 5.0.2.2x86, 4.3.1.2
This old installs show it correctly, a few missing:
4.1.3.1, 3.5.7.2.

When I use them in a test file they show up sometimes all, some or 2-3 missing.

The display in a document might be related to bug#57999. If I use a new document or an Arial only doc as 1. file then (Font list is WYSIWYG) the Emojis show up in the same session. Restarting LO and using Emoji test file mostly a no show or partial show.
Comment 8 tommy27 2015-10-05 09:18:42 UTC
tested on another Win7x64 machine using LibO 5.1.0.0.alpha1+
Build ID: 25de5cfa43b2b1cb7d7214470acc7719839e13fe
TinderBox: Win-x86@62-merge-TDF, Branch:MASTER, Time: 2015-10-01_08:49:54
Locale: it-IT (it_IT)

bug is still there and many emojis are not shown in the replacement table
Comment 9 tommy27 2015-10-31 13:29:38 UTC
bug still reproducible under Win8.1 x64 using LibO  5.1.0.0.alpha1+ (x64)
Build ID: f173e148d8f2d0600fd50bee05e536932d529643
TinderBox: Win-x86_64@62-TDF, Branch:MASTER, Time: 2015-10-30_22:26:03
Locale: it-IT (it_IT)
Comment 10 Robinson Tryon (qubit) 2015-12-10 10:05:10 UTC Comment hidden (obsolete)
Comment 11 tommy27 2016-03-19 07:17:49 UTC
bug still present in:
LibO 5.2.0.0.alpha0+
Build ID: 6eb7cd38e348e8a9d6498bfc2d41e91725eb34aa
CPU Threads: 4; OS Version: Windows 6.29; UI Render: default; 
TinderBox: Win-x86@39, Branch:master, Time: 2016-03-16_12:53:35
Locale: it-IT (it_IT)
Comment 12 V Stuart Foote 2016-03-20 19:24:54 UTC
@tommy27, *

On Windows 10 Pro 64-bit en-US with
Version: 5.2.0.0.alpha0+
Build ID: 1d5767c6e464b914812867aac5c3ccd0745dd1ea
CPU Threads: 8; OS Version: Windows 6.19; UI Render: default (and GL); 
TinderBox: Win-x86@42, Branch:master, Time: 2016-03-20_00:02:24
Locale: en-US (en_US)

Continue to see this, but beleive there are two aspects to mishandling of the :<emoji>: substitutions:

1. method of launch

2. support for non-BMP codepoints by language in use


I think they are both related to incorrect font handling and need for fallback font selection both with and without OpenGL (even the new DirectWrite/GDI+ based layout on MS Windows). Affect by method of launch shows up as in bug 71603, where launching form the StartCenter does not "enable" functional font handling.

But you do get functional font behavior if using CLI (e.g. "swriter.exe %USERPROFILE%\documents\testDocument.odt") or using the OS file association and launch from Windows Explorer.

And at that point when launched, fallback and emoji autocorrection ends up "enabled". Which can be confirmed by checking the list of defined "emoji" in  Tools -> AutoCorrect -> AutoCorrect Options dialog.  If "enabled" and working, it will show a character for each :<emoji>: substitution.  But the glyphs all look to be drawn from system font--so Seoge UI on Windows Vista, 8, 8.1 and 10.

If not working--many of the :<emoji>: entries will show a place holder character.

But even when "enabled" in the AutoCorrect Options list, there is another issue in font handling that has been preventing display on the document canvas of emoji glyphs in the selected font. Either a placeholder character is rendered, or the cell that should hold the glyph gets no character and adjacent characters collapse over the space.

Happens even for paragraphs using fonts that shouldn't need fallback for emoji glyps that are defined for SMP codepoints (i.e. > U+FFFF). 

Really seems they both are related issues in font handling for deciding that a font has codepoint coverage and its glyphs should be used for the given language of a document or a paragraph.
Comment 13 V Stuart Foote 2016-06-04 18:18:41 UTC
*** Bug 100212 has been marked as a duplicate of this bug. ***
Comment 15 ⁨خالد حسني⁩ 2016-11-02 16:05:33 UTC
(In reply to V Stuart Foote from comment #14)
> @Khaled, a backport to 5.2 feasible?

Since this have been broken for several release already, I don't think there is an urgency that warrant a backport, and I'd rather wait for the fix to get more testing.
Comment 16 V Stuart Foote 2016-11-16 06:50:43 UTC
*** Bug 102271 has been marked as a duplicate of this bug. ***
Comment 17 tommy27 2016-11-19 08:41:19 UTC
being the original reporter I confirm it's fixed in LibO 5.3.0.0.alpha1+
Build ID: 8d613870b2cd2e3e4396b4fa97dbd8080fda8f52
CPU Threads: 4; OS Version: Windows 6.2; UI Render: default; Layout Engine: new; 
TinderBox: Win-x86@62-merge-TDF, Branch:MASTER, Time: 2016-11-18_22:58:29
Locale: it-IT (it_IT); Calc: group