Bug 38159 - Better full text justification with auto character scaling and paragraph level adjustment
Summary: Better full text justification with auto character scaling and paragraph leve...
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
Inherited From OOo
Hardware: All All
: medium enhancement
Assignee: Not Assigned
URL:
Whiteboard: vote
Keywords:
: 150670 154355 (view as bug list)
Depends on:
Blocks: Font-Rendering
  Show dependency treegraph
 
Reported: 2011-06-10 08:59 UTC by Joe Hudson
Modified: 2024-02-27 10:19 UTC (History)
24 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Joe Hudson 2011-06-10 08:59:30 UTC
When formatting a serious piece of text with full justification, the current algorithm seems to work on a line by line basis. The result is often large unsightly gaps between words. This is not acceptable for professional work, e.g. typesetting a book.

Currently if someone wants to produce a book (for example) in LibreOffice with full justification, they need to go through the text, line by line, tweaking the word and character spacing to produce a more pleasing result.

There is a solution to this problem: use the same kind of paragraph level justification algorithm used by Scribus, WordPerfect, Latex, Indesign or Quarkexpress. This approach produces far superior results, with much more even word spacing, and would make jobs such as typesetting books much more efficient and enjoyable while using LibreOffice.

(Why not just use Scribus or Latex? Because LibreOffice has some great usability features that they don't.)

So, the request is for a better full text justification algorithm. It's something that the Openoffice community have also been requesting for almost 10 years now (http://openoffice.org/bugzilla/show_bug.cgi?id=3243) Would be awesome to finally have it.

I have no experience developing for LibreOffice, but if someone was willing to help bring me up to speed, I'd be willing to help make this valuable feature a reality.
Comment 1 Björn Michaelsen 2011-12-23 12:29:14 UTC Comment hidden (obsolete)
Comment 2 Joe Hudson 2012-04-01 19:36:52 UTC
Yes, unfortunately still persists in 3.5
It would make such a great difference for professional document formatting to have this feature. Perhaps someone will make a GSoC project for it? ...
Comment 3 dingodog 2012-05-16 13:38:21 UTC
this a really needed feature, specially with the great improvements to LINUX LIBERTINE G in matter of ligatures and other opentype specialities. Please, take care to implement
Comment 4 Werner Johann 2012-09-23 10:34:59 UTC
I would like to support the request of that feature.

Werner
Comment 5 Jan_J 2012-11-16 19:54:17 UTC
This feature is crucial for serious typograpy. Its implementation would give an advantage to LibreOffice over many text processors.
Comment 6 Roman Eisele 2012-11-28 16:22:12 UTC
(Definitely an issue which applies to all Platforms.)

I would give my “+1” to this enhancement request -- if there was such a thing as “+1” here ;-) But I fear that this is complex issue ...
Comment 7 Getoar 2013-03-24 21:33:25 UTC
Here's the text I originally posted on the "Get Help" section of the LibreOffice website.  I hope it may explain why we need this advanced algorithm for text justification:

Bidirectional Justification ("WP Justification")

I wish to know how to perform a special type of text justification that appears to be unavailable in LibreOffice Writer. This function would be comparable to the default method for full justification in Corel WordPerfect. Microsoft Word also has a similar option, which it defines as "Do full justification the way WordPerfect 6.x for Windows does." Since no standard term seems to exist for this type of function, I shall refer to it -- if I may -- as bidirectional justification, for practical reasons that I will further explain.

Currently, LibreOffice Writer uses what I would call unidirectional justification, by moving words in a single direction. The program does so by increasing space between words to ensure that a line extends a certain distance (usually the same width as the preceding line). If a word does not fit in the same line, Writer pushes that word onto the next line. This justifies the text on both sides, but may not always create an aesthetically-pleasing output. This issue is especially noticeable when a text lacks hyphenation and is formatted in narrow columns, where a sentence contains on average three to five words.

The shortcomings of unidirectional justification are resolved by bidirectional justification. In other words, a word processing or desktop publishing program could move words forwards as well as backwards to ensure a better appearance. This is done by both increasing and decreasing as necessary space between words to fit them within a certain line width.

Ultimately, I believe that bidirectional justification would be an important feature to include in future updates and releases of LibreOffice Writer that would attract a greater number of users who are currently tied to proprietary software. Thank you in advance for assisting me personally and contributing to the community as a whole!
Comment 8 Emir Sarı 2013-08-02 12:31:59 UTC
I think LibreOffice could benefit from this greatly, +1 from me. Is it possible to make this an EasyHack? So far this enhancement request seems like buried deep within the other ones.
Comment 9 Joe Hudson 2013-08-03 04:04:43 UTC
I don't understand how a feature that's 20+ years old in DTP software and will visibly affect the quality of output of every document using full justification is not given the priority of features like importing graphical bullets from word docs and having gradient frame backgrounds.. maybe there's something in the architecture of LO that makes it very hard to do, or maybe the people doing the dev work don't see it as important? What to do?

Laszlo was working on it amongst other features that have since been integrated in 2011, but not sure where his work on it went.. http://libregraphicsworld.org/blog/entry/libreoffice-is-diving-into-desktop-publishing
Comment 10 Joe Hudson 2013-08-03 04:27:38 UTC
Here's the same issue/feature request over at AOO: https://issues.apache.org/ooo/show_bug.cgi?id=3243 you can see it's been going since 2002..
Comment 11 Michael Meeks 2013-08-03 08:37:18 UTC
Hi Joe,

> I don't understand how a feature ... is not given the priority of features
> like importing graphical bullets from word docs and having gradient frame
> backgrounds...

There is no global feature prioritization in LibreOffice - that is left to the wisdom of the crowds of people contributing to the code-base, either volunteers or companies. In the case of the two features you mentioned, I believe paying customers of contributing companies reported interop. bugs against this - and as such they had their bugs fixed :-)

There are plenty of people eager to be paid to implement features for those willing to incentivise them; failing that - jump in and hack on the code yourself if you want to effect positive change: there is enough mentoring support to help you make that a success - would be great to have you contributing in that way.

Michael.
Comment 12 Joe Hudson 2013-08-05 00:26:07 UTC
(In reply to comment #11)

> There is no global feature prioritization in LibreOffice - that is left to
> the wisdom of the crowds of people contributing to the code-base, either
> volunteers or companies. In the case of the two features you mentioned, I
> believe paying customers of contributing companies reported interop. bugs
> against this - and as such they had their bugs fixed :-)
> 
> There are plenty of people eager to be paid to implement features for those
> willing to incentivise them; failing that - jump in and hack on the code
> yourself if you want to effect positive change: there is enough mentoring
> support to help you make that a success - would be great to have you
> contributing in that way.
> 
> Michael.

Thanks Michael,

I understand the broadly crowd led nature of development, and the influence of funding. A couple of years ago I got in touch of Laszo and offered to help in implementing the pro quality text justification feature that he'd also had in mind to make. If I remember correctly he said it was maybe 6 months away at the time and there wasn't much I could do in way of collaboration.. 

Regarding crowd selection of feature priority, I'm in full agreement with that in principle. I do think though that sometimes the wider value of a feature can be very easy to miss for those not already aware of it. In the case of decent full text justification, if you held two full justified articles side by side, the one with simplistic single line based justification will look ugly and be distracting to read by comparison to the less simplistic paragraph based justified article. It wouldn't take an expert or typography buff to realize it, it's just much nicer looking and easier to read. However, probably not so many people would be able to pinpoint why the latter article looked so much better and say 'well, that's the benefit right there of decent paragraph based full text justification with character scaling'. 

Imagine if the feature was implemented and all the full justified reports, articles and books produced in LibreOffice looked so slick, without tediously going through line by line and stretching words through character properties, or redoing the whole thing in LaTex or Scribus (and losing the convenience and comfort of LO). You'd get more professional users, and thus more people with the funds to contribute to further development. And from that it would add to the prestige and perceived quality of LO.

Currently with the more advanced font features Laszo and also others put into LO from around v3.2, the full justification situation is a bit of an incongruence and bottleneck to more professional update I think. (yes the algorithm isn't going to be perfect, but it would save a huge amount of labour.)

Thus, I think if there was some (non-authoritarian) leadership from prominent LO developers on communicating the value of this feature, then everyone, developers, general users and professionals on a budget might benefit.

What do you think?
Comment 13 Emir Sarı 2013-08-26 14:14:05 UTC
Setting importance to "high", since it is a required feature for professional looking documents. Feel free to set it to "highest" if you feel like.
Comment 14 Getoar 2013-08-26 23:29:37 UTC Comment hidden (me-too)
Comment 15 Michael Meeks 2013-08-27 04:02:15 UTC
> Regarding crowd selection of feature priority, I'm in full agreement with
> that in principle. I do think though that sometimes the wider value of a
> feature can be very easy to miss for those not already aware of it.

Sure - also the wider context of the number and variety of existing problems that need tackling with the codebase is often rather unclear to people certain that their problem is the most important issue we face :-)

> Imagine if the feature was implemented and all the full justified reports
...
> You'd get more professional users, and thus more people with the funds to
> contribute to further development. And from that it would add to the
> prestige and perceived quality of LO.

This argument unfortunately is recursive; there is always another feature that someone could invest to implement that would get more people wanting them to invest in implementing other features that would get more people etc. ;-) Ultimately if you (or someone else) wants to invest in this - go for it, engage a certified consultant to implement it, or jump in yourself - we're happy to help mentor people that want to work on the code.

> Thus, I think if there was some (non-authoritarian) leadership

Which is a request for me to dedicate my time and effort to driving your pet feature to completion ? :-) Let me reflect it back to you: there is nothing stopping you or others working on / investing into this.

There is also nothing stopping you interacting with other users to try to come up with some definitive list of what you think overall bug priorities should be: that might provide another useful source of prioritisation data for some developers. Having lots of individuals begging in their individual pet bugs however just wastes time :-) I suggest a week or so of bug triage to get a view of where the issues with tons of duplicates lie; the QA team would love that:

http://wiki.documentfoundation.org/BugTriage
Comment 16 ⁨خالد حسني⁩ 2013-08-27 11:38:36 UTC
> or redoing the whole thing in … Scribus

Scribus does not support what you are asking for, either (and, unlike LibreOffice, Scribus claims to be a DTP application).
Comment 17 ⁨خالد حسني⁩ 2013-08-27 11:58:04 UTC
Regarding better (i.e. Knuth and Plass) justification in LibreOffice, that is something I’d really be happy to see and I’m even willing to invest time on it (to some degree).

But, last I checked this was far from trivial, the text layout code was not designed with something like this in mind (unlike TeX) and things are scattered around the code base and communication is done in “interesting” ways, so I simply gave up. Of course I don’t understand much of the code, so I’d be wrong, however I’m interested to give a hand if someone can figure out what needs to be done in terms of specific code modifications.

There is also interesting UI interaction due to the nature of Knuth and Plass algorithm (e.g. line breaks changing when editing later lines on the paragraph causing lines to suddenly jump, which can be very annoying to users, so we need to figure out how to handle this). There is also the issue of warping text around images, which would require special handling as well.

A potential way to achieve this that might be relatively easier to implement, is by using extensions, but it depends on the capabilities of our extensions API (I never used), someone wrote a Knuth and Plass implementation in JavaScript, so why not! http://www.bramstein.com/projects/typeset/
Comment 18 ⁨خالد حسني⁩ 2013-09-02 01:20:04 UTC
There is an interesting discussion in Mozilla’s bugzilla on some issues that need to be addressed when implementing Knuth and Plass algorithm on the web (which is not very different from requirements of word processors) https://bugzilla.mozilla.org/show_bug.cgi?id=630181
Comment 19 bdjnks 2013-09-02 18:42:22 UTC
I, like Khaled, delved deep into the code attempting to enhance "justified" paragraph alignment. And like him, I too was surprised by the, what's the word, hideous tangle perhaps? I also, unfortunately, gave up. My chance of improving things without breaking more than I fixed was approximately nil.

This issue needs to be tackled by developers with a deep knowledge of C++, LibreOffice source, and all manner of language typography, if such a rare and beautiful creature even exists, let alone has time to deal with such a "trivial" request.

Don't believe me? Delve into the source yourself. I'd like to work on this, but sadly I'm a developer, not a magician.

If you are a powerful LibreOffice code wizard, please look with beneficence on the supplications of your poor bereft serfs. We rely only on your mercy.
Comment 20 Emir Sarı 2013-09-03 08:24:21 UTC
Created a freedomsponsors thread[1] about this, hopefully it will draw more attention on the subject. 

[1] http://freedomsponsors.org/core/issue/334/better-full-text-justification-with-auto-character-scaling-and-paragraph-level-adjustment
Comment 21 Michael Meeks 2013-09-03 08:44:45 UTC
> I, like Khaled, delved deep into the code attempting to enhance "justified"
> paragraph alignment. And like him, I too was surprised by the, what's the
> word, hideous tangle perhaps?

;-) did you come up with a patch that did anything ? ultimately posting something that doesn't do quite what you want, would help elucidate any questions on the developers list. All experts in that code became so only by struggling with it at length, and we'd love to have the feature if you've the time to invest working on it.

In terms of back-compat - clearly that stuff has a big effect on layout, so we'd need some document setting to turn that new stuff on/off - which we could default to 'on' for new documents, and/or whenever someone changed paper size ;-) or whatever.

Beyond that, in the deep future, we'd love to better document and/or re-work the core layout code, but waiting for that is unlikely to be much fun :-)

Anyhow - thanks so much for trying ! and please do ask questions on the dev. list.
Comment 22 Emir Sarı 2013-11-18 13:31:48 UTC
Just FYI: http://www.bramstein.com/projects/typeset/
Comment 23 Cédric Bosdonnat 2014-01-20 08:57:34 UTC Comment hidden (off-topic)
Comment 24 Werner Johann 2015-01-19 13:43:45 UTC
Would it not be possible to offer at least the same option Microsoft Office is already offering for more than a decade (Do full justification like WordPerfect 6.x for Windows)?

This would already make texts look much better.

(In the long run, of course it would be desirable, if Libreoffice would try to get beyond the limits of traditional office software towards automatic full justification quality of professional desktop publishing software.)
Comment 25 devseppala 2017-01-04 10:31:04 UTC
I thought it would be good to bump this issue a little as I noticed that LO 5.3 has A new cross-platform text layout engine that uses HarfBuzz for consistent text layout on all platforms. Although HarfBuzz does not do (full) justification it self:

http://www.manpagez.com/html/harfbuzz/harfbuzz-1.0.3/hello-harfbuzz.php

"If you want to layout text in paragraphs, you will probably want to send each word of your text to Harfbuzz to determine its shaped width after glyph substitutions, then work out how many words will fit on a line, and then finally output each word of the line separated by a space of the correct size to fully justify the paragraph."

I would like to ask those who have previously looked at implementing this full text justification feature, will the use of HarfBuzz make it easier to implement this feature? I assume it would have at least simplified the code base a little, which seems to have been a major issue.
Comment 26 Hossein 2017-03-03 21:45:31 UTC Comment hidden (me-too)
Comment 27 Art C 2018-10-24 00:17:53 UTC Comment hidden (me-too)
Comment 28 Laurie 2019-08-08 01:04:00 UTC Comment hidden (me-too)
Comment 29 Laurie 2019-08-08 01:04:47 UTC Comment hidden (obsolete)
Comment 30 Xisco Faulí 2019-11-29 13:28:50 UTC
Changing priority back to 'medium' since the number of duplicates is lower than 5
Comment 31 Kevin Cox 2019-11-29 16:19:49 UTC Comment hidden (no-value)
Comment 32 Fahad Al-Saidi 2019-11-29 16:36:52 UTC Comment hidden (no-value)
Comment 33 Justin L 2020-08-04 10:53:04 UTC
related bug 119908 shows how this is affects compatibility with MS Word 2013+, where it seems like Microsoft has implemented their own algorithm in addition to the no-longer-available "Do full justification the way WordPerfect 6.x for Windows does."
Comment 34 Heiko Tietze 2022-08-30 06:31:49 UTC
*** Bug 150670 has been marked as a duplicate of this bug. ***
Comment 35 ⁨خالد حسني⁩ 2023-04-08 11:03:15 UTC
*** Bug 154355 has been marked as a duplicate of this bug. ***
Comment 36 Eyal Rozenberg 2023-11-24 21:28:56 UTC
The bug title mentions both paragraph-level adjustment and "character scaling", meaning, I assume, the possibility of contacting, rather than just expanding, inter-word spacing. While these are of course related - they are not, in fact, dependent. Even with single-line justification, one could squeeze space a bit until a limit, then move words to the next line and expand space.

So, I suggest we have separate bugs for going paragraph-level and one for supporting intra-word space contraction.

Also, we should consider bug 116969: "Justification by alternative methods", which lists:

* Use of alternative character shapes
* Use of ligatures
* Use of expandable "connection" glyphs, e.g. kashida in Arabic

and one could also mention

* Slight expansion and contraction of glyphs, a-la the LaTeX microtype package

The bottom line is that I believe we should have a meta-bug for justification and line breaking logic, supplanting bug 116969, with a separate bug for each capability.

Is that ok with everyone? Would you like to structure the bugs differently?

Finally, I also think maybe paragraph-level line breaking and justification logic is something worth tendering for in 2024, and encourage others to help put that forward.
Comment 37 Frank 2023-11-24 21:51:23 UTC
Re: "Is that ok with everyone?"

Writer's support for basic word processor functionality in most scripts other than Latin (the script behind the English, French, etc. etc. alphabets) certainly ranges from adequate to abysmal, so any attempt to strategize an approach to improving this situation has my vote.

And, as the person posting the question points out, even the justification in English is fairly primitive.

So, yes, it's ok with me.
Comment 38 devseppala 2024-01-27 22:17:17 UTC
FYI, according to LO 24.2.0 RC2 release notes and particularly bug #119908, apparently there is progress in bringing MS Word like 'smart justify' feature to Writer 24.2.0 . However, it seems that initially this will only be enabled only as compatibility feature for imported Word documents and not as something users can set from the alignment options.
Comment 39 devseppala 2024-02-26 15:43:50 UTC
Regarding the recent introduction of the ‘smart justification’ feature of LibreOffice that is apparently planned to be the default justification algorithm of the next major versions of LibreOffice. While I very much welcome any improvements to justification, I also started to think that could there be some easy way to offer users something that is more than just copycatting what M$ Word is doing.

Now that the ‘smart justification’ makes possible to shrink word spacing up to 20%, how difficult would it be to make default space width and maximum shrinkage user selectable?  This would allow for the users to fine tune the justification to their liking. My idea here is that if the you increase the default space width and max shrinkage, you could have some control over space width variability between lines. The benefit of this could vary between languages. The Finnish language has much longer word length on average than English and as such might benefit more from custom settings. Of course the problem with this suggestion is that it not compatible with Word, but users could be warned when enabling this feature and when exporting these documents to Word.

I would love to know does anyone else like this idea or is it just too much trouble for too little gain?
Comment 40 Eyal Rozenberg 2024-02-26 22:23:10 UTC
(In reply to devseppala from comment #39)
> I also started to think that
> could there be some easy way to offer users something that is more than just
> copycatting what M$ Word is doing.

This bug is about implementing that thing (or multiple things), which is well-known and well-established.

> Now that the ‘smart justification’ makes possible to shrink word spacing up
> to 20%, how difficult would it be to make default space width and maximum
> shrinkage user selectable?

Configurability of these settings seems appropriate for a separate, though related, bug, I would say.

>  This would allow for the users to fine tune the
> justification to their liking. 

Users should not need to fine-tune the justification to their liking; but - I see your point. Still, separate bug.

> My idea here is that if the you increase the
> default space width and max shrinkage, you could have some control over
> space width variability between lines.

Paragraph-level justification should take care of this mostly.

> The Finnish language has much longer word length on
> average than English and as such might benefit more from custom settings.

That's something which should be handled automatically, by examining the actual word lengths.
Comment 41 devseppala 2024-02-27 10:19:41 UTC
(In reply to Eyal Rozenberg from comment #40)
> Configurability of these settings seems appropriate for a separate, though
> related, bug, I would say.
Thank you very much for your comments Eyal, based on this I created Bug 159923 - Add max shrink and space width customization options to upcoming smart justification feature. If anyone has comments related to this, please use the new bug report.