Bug 53009 - Large UserInstallation's user/extensions/bundled/ tree
Summary: Large UserInstallation's user/extensions/bundled/ tree
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: LibreOffice (show other bugs)
Version:
(earliest affected)
3.6.0.4 release
Hardware: All All
: medium normal
Assignee: Stephan Bergmann
URL:
Whiteboard: target:3.7.0 target:3.6.2
Keywords:
Depends on:
Blocks:
 
Reported: 2012-07-31 15:14 UTC by Stephan Bergmann
Modified: 2016-08-22 10:09 UTC (History)
7 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Stephan Bergmann 2012-07-31 15:14:52 UTC
With the band aid fix for bug 51252 gone into LO 3.6.0 RC4, it no longer uses an installation-wide share/prereg/bundled/ tree for some of the registration information about bundled extensions, but instead generates such information exclusively per-user (on first start for a user) in the UserInstallation's user/extensions/bundled/ tree.

Especially on Windows, this tree can become quite large (20MB; see bug 52976).  (And beside the size issue, the per-user generation on first start for a user can also lead to a noticeable start-up delay.)  This is mainly due to three factors:

1  At least on Windows, where there is only a single installation set including all languages (LibO_3.6.0.4_Win_x86_install_multi.msi), even if only a subset of the "Additional user interface languages" is selected during installation, nevertheless all 46 bundled dictionary extensions are always installed.  This needlessly (and substantially) increases the number of bundled extensions that need to be processed per-user, where each bundled extension potentially counts towards the increased space and time requirements.

2  The largest part (18MB, 4096 files and 1335 folders when tested on Windows) is the UserInstallation's user/extensions/bundled/registry/com.sun.star.comp.deployment.help.PackageRegistryBackend/ tree.  It contains computed help index information for each locale for which a bundled extension contains help data.  And some bundled extensions contain help data for more than 100 locales (nlpsolver, presenter-screen, wiki-publisher).

3  The second largest part (1MB, 57 files and 55 folders when tested on Windows) is the UserInstallation's user/extensions/bundled/registry/com.sun.star.comp.deployment.configuration.PackageRegistryBackend/ tree.  It contains copies of bundled extensions' .xcu files that contain %origin% tokens, with those tokens expanded.

I hope to get all three issues addressed eventually.  (1: Bundle the dictionary extensions with the corresponding optional per-language installation parts.  2: Compute the help index information on demand, or pre-compute it into the installation-wide share/extensions/ tree.  3: Remove the use of %origin% tokens from bundled extensions' .xcu files.)
Comment 1 Stephan Bergmann 2012-08-10 14:49:00 UTC
*** Bug 53343 has been marked as a duplicate of this bug. ***
Comment 2 Cor Nouws 2012-08-22 13:34:42 UTC
Just a thought: 
An installation is allways (on a single-user-per-session PC) followed by a first start ...
So can't that work be pushed to the end of the regular installation ?
Comment 3 Stephan Bergmann 2012-08-22 13:43:33 UTC
(In reply to comment #2)
> An installation is allways (on a single-user-per-session PC) followed by a
> first start ...
> So can't that work be pushed to the end of the regular installation ?

That's where we came from, see bug 51252 etc.  The decision to no longer precompute per-user information at installation time was deliberate.  It is a simplification that saves us (a) esp. from a broken implementation of how that was done, but (b) even more importantly, from a lot of general hassle, witness the amount of---often painful---code removed with <http://cgit.freedesktop.org/libreoffice/core/commit/?id=6dcb3d4ef46312729bb6f16c473b433474863f68> "Related fdo#51252: No more prereg, no more unopkg sync".  I hope to get the currently rather noticeable delay on first start reduced significantly in a more pleasing way.
Comment 4 Michael Meeks 2012-08-22 14:30:37 UTC
yep; removing the big bunch of fragile prereg code is really important. It seems like the (somewhat pointless) help compilation for all those languages ie. 2. would be the bit that Cor identifies as slow for the 'WikiPublisher' install I guess.

The irony being that that help is useless without the help-pack also being installed - since you'll get directed to the on-line wiki-help anyway ;-> so we're prolly doing this big chunk of work to no useful purpose anyway.
Comment 5 Not Assigned 2012-08-31 15:15:32 UTC
Stephan Bergmann committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=b23bb8e0de3dbdc2c66c3dedf70dfd318868f76c

fdo#53009: Compile extension help in gbuild
Comment 6 Stephan Bergmann 2012-08-31 15:54:04 UTC
(In reply to comment #5)
> http://cgit.freedesktop.org/libreoffice/core/commit/?id=b23bb8e0de3dbdc2c66c3dedf70dfd318868f76c
> 
> fdo#53009: Compile extension help in gbuild

That removes the user/extensions/bundled/registry/com.sun.star.comp.deployment.help.PackageRegistryBackend/ data for the nlpsolver, presenter-screen, and wiki-publisher extensions (see point 2 of comment 0).

It does not remove the user/extensions/bundled/registry/com.sun.star.comp.deployment.help.PackageRegistryBackend/ data for the LanguageTool extension, as that is built with external languagetool/makefile.mk
rather than gbuild.  One possible solution is to manually compile help in languagetool/makefile.mk.
Comment 7 Stephan Bergmann 2012-09-05 09:20:18 UTC
(In reply to comment #6)
> It does not remove the
> user/extensions/bundled/registry/com.sun.star.comp.deployment.help.PackageRegistryBackend/
> data for the LanguageTool extension

But the "official" (<http://www.libreoffice.org/download/>) installation sets appear to not bundle the LanguageTool extension anyway (i.e., using implicit --disable-ext-languagetool), so there might be no demand to do anything here at all (apart from telling the upstream source of the LanguageTool extension that they could build a new version with precompiled help).

A similar situation is with the Barcode extension, which LO includes prebuilt (extras/source/extensions/Barcode.oxt), which contains non-compiled help, but which LO does not bundle by default (--disable-ext-barcode) either.
Comment 8 Stephan Bergmann 2012-09-05 09:21:17 UTC
(In reply to comment #5)
> Stephan Bergmann committed a patch related to this issue.
> It has been pushed to "master":
> 
> http://cgit.freedesktop.org/libreoffice/core/commit/?id=b23bb8e0de3dbdc2c66c3dedf70dfd318868f76c
> 
> fdo#53009: Compile extension help in gbuild

Requested backport to libreoffice-3-6, <https://gerrit.libreoffice.org/#/c/563/> "Change Ie4bab66d: fdo#53009: Compile extension help in gbuild."
Comment 9 Stephan Bergmann 2012-09-05 15:15:32 UTC
Stephan Bergmann committed a patch related to this issue.
It has been pushed to "libreoffice-3-6":

http://cgit.freedesktop.org/libreoffice/core/commit/?h=libreoffice-3-6&id=4ae49101fd43d052c3556b2db753e4d8ae7d86ce

fdo#53009: Compile extension help in gbuild


It will be available in LibreOffice 3.6.2.
Comment 10 Stephan Bergmann 2012-09-11 15:38:55 UTC
Item 1 of comment 0 has now been addressed on master with <http://cgit.freedesktop.org/libreoffice/core/commit/?id=e2fac98819c00b4fb50f9de9d0f32d20092f3191> "fdo#53009: For msi installer, only default-select a subset of dictionaries" (see also mailing list thread starting at <http://lists.freedesktop.org/archives/libreoffice/2012-August/037671.html> "Selection of dictionaries per installation").

(For backporting purposes, the above commit will need to be augmented with <http://cgit.freedesktop.org/libreoffice/core/commit/?id=8b3ab4b8a580bf4c4013e1baf3474a71fed44663> "Some code simplification") and the fixes for bug 53520.)
Comment 11 Stephan Bergmann 2012-09-21 13:15:39 UTC
(In reply to comment #10)
> Item 1 of comment 0 has now been addressed on master with
> <http://cgit.freedesktop.org/libreoffice/core/commit/?id=e2fac98819c00b4fb50f9de9d0f32d20092f3191>
> "fdo#53009: For msi installer, only default-select a subset of dictionaries"
> (see also mailing list thread starting at
> <http://lists.freedesktop.org/archives/libreoffice/2012-August/037671.html>
> "Selection of dictionaries per installation").
> 
> (For backporting purposes, the above commit will need to be augmented with
> <http://cgit.freedesktop.org/libreoffice/core/commit/?id=8b3ab4b8a580bf4c4013e1baf3474a71fed44663>
> "Some code simplification") and the fixes for bug 53520.)

On backporting that part to LO 3.6 see <http://lists.freedesktop.org/archives/libreoffice/2012-September/038734.html> "[REVIEW 3-6] For msi installer, only default-select a subset of dictionaries."
Comment 12 Not Assigned 2012-09-23 22:14:06 UTC
Stephan Bergmann committed a patch related to this issue.
It has been pushed to "libreoffice-3-6":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=bec93e8d37cbae41a0054a817344c820c0f8b570&g=libreoffice-3-6

fdo#53009: For msi installer, only default-select a subset of dictionaries


It will be available in LibreOffice 3.6.3.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 13 Yousuf Philips (jay) (retired) 2016-08-22 09:58:01 UTC
@Stephan, @Meeks: Should this be closed now or are there still pending commits for this?
Comment 14 Stephan Bergmann 2016-08-22 10:09:17 UTC
Can be closed now.  Item 1 from comment 0 has been addressed as per comment 10.  Items 2 and 3 are not very relevant today, as only dictionaries remain as bundled extensions.