Bug 38159 - Better full text justification with auto character scaling and paragraph level adjustment
Summary: Better full text justification with auto character scaling and paragraph leve...
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
Inherited From OOo
Hardware: All All
: high enhancement
Assignee: Not Assigned
URL:
Whiteboard: vote
Keywords:
Depends on:
Blocks: Font-Rendering
  Show dependency treegraph
 
Reported: 2011-06-10 08:59 UTC by Joe Hudson
Modified: 2018-10-24 00:17 UTC (History)
18 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Joe Hudson 2011-06-10 08:59:30 UTC
When formatting a serious piece of text with full justification, the current algorithm seems to work on a line by line basis. The result is often large unsightly gaps between words. This is not acceptable for professional work, e.g. typesetting a book.

Currently if someone wants to produce a book (for example) in LibreOffice with full justification, they need to go through the text, line by line, tweaking the word and character spacing to produce a more pleasing result.

There is a solution to this problem: use the same kind of paragraph level justification algorithm used by Scribus, WordPerfect, Latex, Indesign or Quarkexpress. This approach produces far superior results, with much more even word spacing, and would make jobs such as typesetting books much more efficient and enjoyable while using LibreOffice.

(Why not just use Scribus or Latex? Because LibreOffice has some great usability features that they don't.)

So, the request is for a better full text justification algorithm. It's something that the Openoffice community have also been requesting for almost 10 years now (http://openoffice.org/bugzilla/show_bug.cgi?id=3243) Would be awesome to finally have it.

I have no experience developing for LibreOffice, but if someone was willing to help bring me up to speed, I'd be willing to help make this valuable feature a reality.
Comment 1 Björn Michaelsen 2011-12-23 12:29:14 UTC Comment hidden (obsolete)
Comment 2 Joe Hudson 2012-04-01 19:36:52 UTC
Yes, unfortunately still persists in 3.5
It would make such a great difference for professional document formatting to have this feature. Perhaps someone will make a GSoC project for it? ...
Comment 3 dingodog 2012-05-16 13:38:21 UTC
this a really needed feature, specially with the great improvements to LINUX LIBERTINE G in matter of ligatures and other opentype specialities. Please, take care to implement
Comment 4 Werner Johann 2012-09-23 10:34:59 UTC
I would like to support the request of that feature.

Werner
Comment 5 Jan_J 2012-11-16 19:54:17 UTC
This feature is crucial for serious typograpy. Its implementation would give an advantage to LibreOffice over many text processors.
Comment 6 Roman Eisele 2012-11-28 16:22:12 UTC
(Definitely an issue which applies to all Platforms.)

I would give my “+1” to this enhancement request -- if there was such a thing as “+1” here ;-) But I fear that this is complex issue ...
Comment 7 Getoar 2013-03-24 21:33:25 UTC
Here's the text I originally posted on the "Get Help" section of the LibreOffice website.  I hope it may explain why we need this advanced algorithm for text justification:

Bidirectional Justification ("WP Justification")

I wish to know how to perform a special type of text justification that appears to be unavailable in LibreOffice Writer. This function would be comparable to the default method for full justification in Corel WordPerfect. Microsoft Word also has a similar option, which it defines as "Do full justification the way WordPerfect 6.x for Windows does." Since no standard term seems to exist for this type of function, I shall refer to it -- if I may -- as bidirectional justification, for practical reasons that I will further explain.

Currently, LibreOffice Writer uses what I would call unidirectional justification, by moving words in a single direction. The program does so by increasing space between words to ensure that a line extends a certain distance (usually the same width as the preceding line). If a word does not fit in the same line, Writer pushes that word onto the next line. This justifies the text on both sides, but may not always create an aesthetically-pleasing output. This issue is especially noticeable when a text lacks hyphenation and is formatted in narrow columns, where a sentence contains on average three to five words.

The shortcomings of unidirectional justification are resolved by bidirectional justification. In other words, a word processing or desktop publishing program could move words forwards as well as backwards to ensure a better appearance. This is done by both increasing and decreasing as necessary space between words to fit them within a certain line width.

Ultimately, I believe that bidirectional justification would be an important feature to include in future updates and releases of LibreOffice Writer that would attract a greater number of users who are currently tied to proprietary software. Thank you in advance for assisting me personally and contributing to the community as a whole!
Comment 8 Emir Sarı (away) 2013-08-02 12:31:59 UTC
I think LibreOffice could benefit from this greatly, +1 from me. Is it possible to make this an EasyHack? So far this enhancement request seems like buried deep within the other ones.
Comment 9 Joe Hudson 2013-08-03 04:04:43 UTC
I don't understand how a feature that's 20+ years old in DTP software and will visibly affect the quality of output of every document using full justification is not given the priority of features like importing graphical bullets from word docs and having gradient frame backgrounds.. maybe there's something in the architecture of LO that makes it very hard to do, or maybe the people doing the dev work don't see it as important? What to do?

Laszlo was working on it amongst other features that have since been integrated in 2011, but not sure where his work on it went.. http://libregraphicsworld.org/blog/entry/libreoffice-is-diving-into-desktop-publishing
Comment 10 Joe Hudson 2013-08-03 04:27:38 UTC
Here's the same issue/feature request over at AOO: https://issues.apache.org/ooo/show_bug.cgi?id=3243 you can see it's been going since 2002..
Comment 11 Michael Meeks 2013-08-03 08:37:18 UTC
Hi Joe,

> I don't understand how a feature ... is not given the priority of features
> like importing graphical bullets from word docs and having gradient frame
> backgrounds...

There is no global feature prioritization in LibreOffice - that is left to the wisdom of the crowds of people contributing to the code-base, either volunteers or companies. In the case of the two features you mentioned, I believe paying customers of contributing companies reported interop. bugs against this - and as such they had their bugs fixed :-)

There are plenty of people eager to be paid to implement features for those willing to incentivise them; failing that - jump in and hack on the code yourself if you want to effect positive change: there is enough mentoring support to help you make that a success - would be great to have you contributing in that way.

Michael.
Comment 12 Joe Hudson 2013-08-05 00:26:07 UTC
(In reply to comment #11)

> There is no global feature prioritization in LibreOffice - that is left to
> the wisdom of the crowds of people contributing to the code-base, either
> volunteers or companies. In the case of the two features you mentioned, I
> believe paying customers of contributing companies reported interop. bugs
> against this - and as such they had their bugs fixed :-)
> 
> There are plenty of people eager to be paid to implement features for those
> willing to incentivise them; failing that - jump in and hack on the code
> yourself if you want to effect positive change: there is enough mentoring
> support to help you make that a success - would be great to have you
> contributing in that way.
> 
> Michael.

Thanks Michael,

I understand the broadly crowd led nature of development, and the influence of funding. A couple of years ago I got in touch of Laszo and offered to help in implementing the pro quality text justification feature that he'd also had in mind to make. If I remember correctly he said it was maybe 6 months away at the time and there wasn't much I could do in way of collaboration.. 

Regarding crowd selection of feature priority, I'm in full agreement with that in principle. I do think though that sometimes the wider value of a feature can be very easy to miss for those not already aware of it. In the case of decent full text justification, if you held two full justified articles side by side, the one with simplistic single line based justification will look ugly and be distracting to read by comparison to the less simplistic paragraph based justified article. It wouldn't take an expert or typography buff to realize it, it's just much nicer looking and easier to read. However, probably not so many people would be able to pinpoint why the latter article looked so much better and say 'well, that's the benefit right there of decent paragraph based full text justification with character scaling'. 

Imagine if the feature was implemented and all the full justified reports, articles and books produced in LibreOffice looked so slick, without tediously going through line by line and stretching words through character properties, or redoing the whole thing in LaTex or Scribus (and losing the convenience and comfort of LO). You'd get more professional users, and thus more people with the funds to contribute to further development. And from that it would add to the prestige and perceived quality of LO.

Currently with the more advanced font features Laszo and also others put into LO from around v3.2, the full justification situation is a bit of an incongruence and bottleneck to more professional update I think. (yes the algorithm isn't going to be perfect, but it would save a huge amount of labour.)

Thus, I think if there was some (non-authoritarian) leadership from prominent LO developers on communicating the value of this feature, then everyone, developers, general users and professionals on a budget might benefit.

What do you think?
Comment 13 Emir Sarı (away) 2013-08-26 14:14:05 UTC
Setting importance to "high", since it is a required feature for professional looking documents. Feel free to set it to "highest" if you feel like.
Comment 14 Getoar 2013-08-26 23:29:37 UTC
I am delighted to see the priority for developing "bidirectional
justification" raised to high.  It is a tremendously important feature that
will boost the popularity of LibreOffice.  Personally, I continue to be
bound to proprietary software due to the lack of this feature.

Thank you,
GM


On Mon, Aug 26, 2013 at 9:14 AM, <bugzilla-daemon@freedesktop.org> wrote:

>  Emir Sarı <emir_sari@msn.com> changed bug 38159<https://bugs.freedesktop.org/show_bug.cgi?id=38159>
>  What Removed Added  Priority medium high  QA Contact   emir_sari@msn.com
>
>  *Comment # 13 <https://bugs.freedesktop.org/show_bug.cgi?id=38159#c13>on bug
> 38159 <https://bugs.freedesktop.org/show_bug.cgi?id=38159> from Emir Sarı<emir_sari@msn.com>
> *
>
> Setting importance to "high", since it is a required feature for professional
> looking documents. Feel free to set it to "highest" if you feel like.
>
>  ------------------------------
> You are receiving this mail because:
>
>    - You are on the CC list for the bug.
>
>
Comment 15 Michael Meeks 2013-08-27 04:02:15 UTC
> Regarding crowd selection of feature priority, I'm in full agreement with
> that in principle. I do think though that sometimes the wider value of a
> feature can be very easy to miss for those not already aware of it.

Sure - also the wider context of the number and variety of existing problems that need tackling with the codebase is often rather unclear to people certain that their problem is the most important issue we face :-)

> Imagine if the feature was implemented and all the full justified reports
...
> You'd get more professional users, and thus more people with the funds to
> contribute to further development. And from that it would add to the
> prestige and perceived quality of LO.

This argument unfortunately is recursive; there is always another feature that someone could invest to implement that would get more people wanting them to invest in implementing other features that would get more people etc. ;-) Ultimately if you (or someone else) wants to invest in this - go for it, engage a certified consultant to implement it, or jump in yourself - we're happy to help mentor people that want to work on the code.

> Thus, I think if there was some (non-authoritarian) leadership

Which is a request for me to dedicate my time and effort to driving your pet feature to completion ? :-) Let me reflect it back to you: there is nothing stopping you or others working on / investing into this.

There is also nothing stopping you interacting with other users to try to come up with some definitive list of what you think overall bug priorities should be: that might provide another useful source of prioritisation data for some developers. Having lots of individuals begging in their individual pet bugs however just wastes time :-) I suggest a week or so of bug triage to get a view of where the issues with tons of duplicates lie; the QA team would love that:

http://wiki.documentfoundation.org/BugTriage
Comment 16 Khaled Hosny (inactive) 2013-08-27 11:38:36 UTC
> or redoing the whole thing in … Scribus

Scribus does not support what you are asking for, either (and, unlike LibreOffice, Scribus claims to be a DTP application).
Comment 17 Khaled Hosny (inactive) 2013-08-27 11:58:04 UTC
Regarding better (i.e. Knuth and Plass) justification in LibreOffice, that is something I’d really be happy to see and I’m even willing to invest time on it (to some degree).

But, last I checked this was far from trivial, the text layout code was not designed with something like this in mind (unlike TeX) and things are scattered around the code base and communication is done in “interesting” ways, so I simply gave up. Of course I don’t understand much of the code, so I’d be wrong, however I’m interested to give a hand if someone can figure out what needs to be done in terms of specific code modifications.

There is also interesting UI interaction due to the nature of Knuth and Plass algorithm (e.g. line breaks changing when editing later lines on the paragraph causing lines to suddenly jump, which can be very annoying to users, so we need to figure out how to handle this). There is also the issue of warping text around images, which would require special handling as well.

A potential way to achieve this that might be relatively easier to implement, is by using extensions, but it depends on the capabilities of our extensions API (I never used), someone wrote a Knuth and Plass implementation in JavaScript, so why not! http://www.bramstein.com/projects/typeset/
Comment 18 Khaled Hosny (inactive) 2013-09-02 01:20:04 UTC
There is an interesting discussion in Mozilla’s bugzilla on some issues that need to be addressed when implementing Knuth and Plass algorithm on the web (which is not very different from requirements of word processors) https://bugzilla.mozilla.org/show_bug.cgi?id=630181
Comment 19 bdjnks 2013-09-02 18:42:22 UTC
I, like Khaled, delved deep into the code attempting to enhance "justified" paragraph alignment. And like him, I too was surprised by the, what's the word, hideous tangle perhaps? I also, unfortunately, gave up. My chance of improving things without breaking more than I fixed was approximately nil.

This issue needs to be tackled by developers with a deep knowledge of C++, LibreOffice source, and all manner of language typography, if such a rare and beautiful creature even exists, let alone has time to deal with such a "trivial" request.

Don't believe me? Delve into the source yourself. I'd like to work on this, but sadly I'm a developer, not a magician.

If you are a powerful LibreOffice code wizard, please look with beneficence on the supplications of your poor bereft serfs. We rely only on your mercy.
Comment 20 Emir Sarı (away) 2013-09-03 08:24:21 UTC
Created a freedomsponsors thread[1] about this, hopefully it will draw more attention on the subject. 

[1] http://freedomsponsors.org/core/issue/334/better-full-text-justification-with-auto-character-scaling-and-paragraph-level-adjustment
Comment 21 Michael Meeks 2013-09-03 08:44:45 UTC
> I, like Khaled, delved deep into the code attempting to enhance "justified"
> paragraph alignment. And like him, I too was surprised by the, what's the
> word, hideous tangle perhaps?

;-) did you come up with a patch that did anything ? ultimately posting something that doesn't do quite what you want, would help elucidate any questions on the developers list. All experts in that code became so only by struggling with it at length, and we'd love to have the feature if you've the time to invest working on it.

In terms of back-compat - clearly that stuff has a big effect on layout, so we'd need some document setting to turn that new stuff on/off - which we could default to 'on' for new documents, and/or whenever someone changed paper size ;-) or whatever.

Beyond that, in the deep future, we'd love to better document and/or re-work the core layout code, but waiting for that is unlikely to be much fun :-)

Anyhow - thanks so much for trying ! and please do ask questions on the dev. list.
Comment 22 Emir Sarı (away) 2013-11-18 13:31:48 UTC
Just FYI: http://www.bramstein.com/projects/typeset/
Comment 23 Cédric Bosdonnat 2014-01-20 08:57:34 UTC
Restricted my LibreOffice hacking area
Comment 24 Werner Johann 2015-01-19 13:43:45 UTC
Would it not be possible to offer at least the same option Microsoft Office is already offering for more than a decade (Do full justification like WordPerfect 6.x for Windows)?

This would already make texts look much better.

(In the long run, of course it would be desirable, if Libreoffice would try to get beyond the limits of traditional office software towards automatic full justification quality of professional desktop publishing software.)
Comment 25 devseppala 2017-01-04 10:31:04 UTC
I thought it would be good to bump this issue a little as I noticed that LO 5.3 has A new cross-platform text layout engine that uses HarfBuzz for consistent text layout on all platforms. Although HarfBuzz does not do (full) justification it self:

http://www.manpagez.com/html/harfbuzz/harfbuzz-1.0.3/hello-harfbuzz.php

"If you want to layout text in paragraphs, you will probably want to send each word of your text to Harfbuzz to determine its shaped width after glyph substitutions, then work out how many words will fit on a line, and then finally output each word of the line separated by a space of the correct size to fully justify the paragraph."

I would like to ask those who have previously looked at implementing this full text justification feature, will the use of HarfBuzz make it easier to implement this feature? I assume it would have at least simplified the code base a little, which seems to have been a major issue.
Comment 26 Hossein 2017-03-03 21:45:31 UTC Comment hidden (me-too)
Comment 27 Art C 2018-10-24 00:17:53 UTC
I'm adding this comment mainly to bump this page and remind the community of this issue. I'm working on a book-length document right now, and other than my inability to get the correct chapter titles in the headers and to get page numbers not to restart at a new chapter, the most annoying part of the process is to tweak all the bad justification decisions Writer (v.6.0.6.2) natively makes.

If I could write a line of code, I would happily dig in. I have played around some with LaTeX, which has its own issues, but its justification is about as good as The New Yorker manages, which is high praise indeed.

So: is it possible to port the LaTeX code to LO? It's both a licensing question and a technical one, and I don't know the answer in either case, but surely someone does.  

If the LO code is such a tangle of spaghetti, is it possible that some wholesale rewriting is needed? 

Thank you,
Art