Description: It currently takes two minutes on a 2017 12" MacBook to install LibreOffice from the x86_64 .dmg release -- the Core m3 CPU just can't go over 6 MB/s decompressing bzip2. Maybe we could switch to something that decompresses faster? For example, LZFSE .dmg is supported since macOS 10.11. Using it would not bring any additional compatibility issue because LibreOffice only supports macOS 10.12+, anyways. Now compression ratio would be a concern, but testing with macOS installation media seems to show a similar [actually slightly better] ratio.[1] Interestingly, bzip2 does not improve that much over zlib in that test... [1]: https://mjtsai.com/blog/2015/10/07/lzfse-disk-images-in-el-capitan/ Steps to Reproduce: 1. Get a slow computer. 2. Double-click dmg to mount. 3. Drag app from dmg to Applications as instructed. 4. Stare at the progress bar. Actual Results: Two minutes. Expected Results: How long do YOU expect to take unpacking LibreOffice on ANY computer? I know this is more of Apple's fault, but come on. I guess I should lob a similar complaint at GIMP too now. Reproducible: Always User Profile Reset: Yes Additional Info: Version: 7.4.1.2 / LibreOffice Community Build ID: 3c58a8f3a960df8bc8fd77b461821e42c061c5f0 CPU threads: 4; OS: Mac OS X 13.0; UI render: default; VCL: osx Locale: en-US (en_US.UTF-8); UI: en-US Calc: threaded
Confirming. Anecdotally, I see the problem as having actually worsened over time and through the release versions of LibreOffice. If you use the release from the AppStore, you don't actually see all this, but you still have to sit through the download.
From bug 114679 comment 13 Chat conversation with Cloph: Nothing we can do about it as it is mac that is copying the files from dmg to target disk. Nothing special about that / LibreOffice package has no say in that / is not involved at all. So the complaint is rather: macOS is slow at copying files from a mounted diskimage (dmg). Type of compression could be changed, but I doubt this would make any difference. If someone does research on why it is slow, I'm willing to have another look, but as it stands now: I don't see any way to improve it. Is not the signing/notarization, since the daily builds are not signed or notarized and apparently suffer from the same slowdowns. So leaves only the compression used as a potential candidate, but then again I cannot really imagine that would change things. Potential starting point: play with how the dmg is created, search for the hdiutil invocations in the code etc: https://opengrok.libreoffice.org/xref/core/solenv/bin/modules/installer/simplepackage.pm?r=443f65e9#458 As per above remarks setting to NOTOURBUG
2 things we can try (unless it's already have been tried): - filesystem, use APFS instead of HFS+ since it's available from 10.12, see https://fr.wikipedia.org/wiki/Apple_File_System (and 10.12 is minimum requirement for LO) - UDSB (SPARSEBUNDLE (grows with content; bundle-backed)) instead of UDBZ (UDIF bzip2-compressed image (macOS 10.4+ only)) I'm really not sure if it can help or not, just reading a bit hdutil page https://ss64.com/osx/hdiutil.html
Respectfully, that man page is woefully out of date - it does not include the proposed LZFSE solution. See https://www.unix.com/man-page/mojave/1/hdiutil/ for ULFO, the one with LZFSE. With regard to "won’t probably change things", well please remember it’s bzip2. If it was lzma the assumption would have merit. Sorry for not being able to find that report though.
Anyways, I still have on me LibreOffice_7.4.1_MacOS_x86-64.dmg. Some testing: a2e5@x Downloads % hdiutil convert -format ULFO -o lzf.dmg LibreOffice_7.4.1_MacOS_x86-64.dmg Preparing imaging engine… Reading Protective Master Boot Record (MBR : 0)… (CRC32 $F7A82429: Protective Master Boot Record (MBR : 0)) Reading GPT Header (Primary GPT Header : 1)… (CRC32 $00BB6D2A: GPT Header (Primary GPT Header : 1)) Reading GPT Partition Data (Primary GPT Table : 2)… (CRC32 $F4CF91D5: GPT Partition Data (Primary GPT Table : 2)) Reading (Apple_Free : 3)… (CRC32 $00000000: (Apple_Free : 3)) Reading disk image (Apple_HFS : 4)… ........................................................................ (CRC32 $EDEBB64C: disk image (Apple_HFS : 4)) Reading (Apple_Free : 5)… ............................................................................... (CRC32 $00000000: (Apple_Free : 5)) Reading GPT Partition Data (Backup GPT Table : 6)… ............................................................................... (CRC32 $F4CF91D5: GPT Partition Data (Backup GPT Table : 6)) Reading GPT Header (Backup GPT Header : 7)… ............................................................................... (CRC32 $4E1063A2: GPT Header (Backup GPT Header : 7)) Adding resources… ............................................................................... Elapsed Time: 1m 32.037s File size: 299601906 bytes, Checksum: CRC32 $2A1A05D2 Sectors processed: 3072000, 1654044 compressed Speed: 8.8MB/s Savings: 81.0% created: /Users/a2e5/Downloads/lzf.dmg For comparison, the old file was 294279010 bytes. Small loss of compression there. As the proper timing method (time {} enclosing some commands mounting from command-line and doing copying) is really annoying to write, I opted to manually mount and copy the stuff to ~/Downloads. What do you know, it takes less than 25 seconds now.
(In reply to Telesto from comment #2) > From bug 114679 comment 13 > Chat conversation with Cloph: > > Nothing we can do about it as it is mac that is copying the files from dmg > to target disk. Nothing special about that / LibreOffice package has no say > in that / is not involved at all. So the complaint is rather: macOS is slow > at copying files from a mounted diskimage (dmg). > > Type of compression could be changed, but I doubt this would make any > difference. > > If someone does research on why it is slow, I'm willing to have another > look, but as it stands now: I don't see any way to improve it. Is not the > signing/notarization, since the daily builds are not signed or notarized and > apparently suffer from the same slowdowns. So leaves only the compression > used as a potential candidate, but then again I cannot really imagine that > would change things. > > Potential starting point: play with how the dmg is created, search for the > hdiutil invocations in the code etc: > https://opengrok.libreoffice.org/xref/core/solenv/bin/modules/installer/ > simplepackage.pm?r=443f65e9#458 > > As per above remarks setting to NOTOURBUG Rob Griffiths, Dec 05 2016: The Finder’s GUI tax can be very expensive https://robservatory.com/the-finders-gui-tax-can-be-very-expensive/ (especially Thomas Tempelmann's comment on https://robservatory.com/the-finders-gui-tax-can-be-very-expensive/#comment-13789) Rob Griffiths, Dec 05 2016: Quickly expand compressed files in Finder https://robservatory.com/quickly-expand-compressed-files-in-finder/ Maybe helpful to understand and solve the issue in question?
Sierk: sorry, your message looked like a spam at first glance when reading "tax".
That series of posts is not particularly helpful. Specifically, you just don't "decompress" a dmg; you mount it, you see a big new volume pop up, and you copy the file from it. There is no single command to do that, and the post of course doesn't have it. (Some of the commands it used have more concise alternatives under bsdtar, but I digress.) If y'all read Comment 5, I think the problem should already be considered solved -- just replace `-format UDBZ` with `-format ULFO` in that `simplepackage.pm` thing. But for posterity and for better measurements, let's actually measure the time taken to copy files from a whole disk image. Why not -- it's easier than hand-typing a patch. ```bash unpack() { # Unpacks a DMG file local dir=$(hdiutil attach -mountrandom /tmp "$@" | tail -n 1 | cut -f 3) cp -a "$dir"/* . hdiutil detach "$dir" } ``` Under `bash`, we run: ```bash-session $ time unpack lzf.dmg "disk2" ejected. real 0m30.034s user 0m0.818s sys 0m12.396s $ rm -r LibreOffice.app/ $ time unpack LibreOffice_7.4.1_MacOS_x86-64.dmg "disk2" ejected. real 3m24.980s user 0m1.084s sys 0m14.461s ``` 7x better. How's that for an improvement?
Stephan: thought you might be also interested in this one since you got a mac.
@Patrick Luby From QA perspective horrible. I'm not updating my master build not regularly because of this. From end-user perspective not great either.. Apparently pretty easy to solve..
I have been using a tool to create .dmg files for NeoOffice since at least back to macOS 10.4 Tiger that uses -format UDZO without any problems and man hdiutil on macOS 12.6.2 Monterey) says -format ULFO has been supported since macos 10.11 El Capitan so @Mingye Wang's suggested change appears to be low risk. Can anyone tell me how to actually build the .dmg in the LibreOffice build? From what I can see, the .dmg does not get built in a developer build and I am new to LibreOffice's build process.
@Patrick Luby See comment 2 > Potential starting point: play with how the dmg is created, search for the > hdiutil invocations in the code etc: > https://opengrok.libreoffice.org/xref/core/solenv/bin/modules/installer/ > simplepackage.pm?r=443f65e9#458 Other question can probably be answered by Cloph. LibreOffice’s release manager. Quickest why to contact: LibreOffice IRC# libreoffice-dev https://blog.documentfoundation.org/blog/2017/04/13/cloph-written-interview/
(In reply to Patrick Luby from comment #11) > Can anyone tell me how to actually build the .dmg in the LibreOffice build? that's configure option --with-package-format=dmg
(In reply to Stephan Bergmann from comment #13) > (In reply to Patrick Luby from comment #11) > > Can anyone tell me how to actually build the .dmg in the LibreOffice build? > > that's configure option --with-package-format=dmg @Stephan Bergmann Thank you for the tip. It worked perfectly. For anyone using LODE like me, here are the steps that I used to build a .dmg: ./autogen.sh --with-package-format=dmg make Note: "make check" appears to skip building the .dmg so you
I have changed the .dmg compression from bzip2 to lzfse and I have posted a patch at: https://gerrit.libreoffice.org/c/core/+/144342 The patch needs still needs to be reviewed and tested before it appears in a nightly build but my limited testing showed at least a 5x increase in copying speed on an old MacBook Pro Intel laptop while increasing the .dmg file size by only 4% or 5%. If any LibreOffice release engineers can review the above patch, that would be very much appreciated. Hopefully they can determine if the small increase in .dmg file size won't cause an increase in TDF's bandwidth costs.
Patrick Luby committed a patch related to this issue. It has been pushed to "master": https://git.libreoffice.org/core/commit/5c98d8e79b4b35d30f1198f8c7b4beef44e421f4 tdf#151341 Use lzfse compression instead of bzip2 It will be available in 7.6.0. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Patrick Luby committed a patch related to this issue. It has been pushed to "libreoffice-7-5": https://git.libreoffice.org/core/commit/be0331a93e7691c7d5c4ebbfffe59af83ea3be7a tdf#151341 Use lzfse compression instead of bzip2 It will be available in 7.5.0.0.beta2. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
*** Bug 114679 has been marked as a duplicate of this bug. ***
Were do we nominate developer of the month? Patrick you are a hero! I can't even say for how many years this has been a problem on macOS. 10 or more? Copying LibreOffice has been slow forever. What easily took much longer than 30 seconds is now an under 10 second process. I heard macOS users around the globe were starting to dance and chant when they learned about this change. verified Version: 7.6.0.0.alpha0+ (X86_64) / LibreOffice Community Build ID: 2e8776c181d0e38a8d25481ac0eb33389940bf95 CPU threads: 8; OS: Mac OS X 13.1; UI render: Skia/Raster; VCL: osx Locale: de-DE (en_DE.UTF-8); UI: en-US Calc: threaded
Patrick Luby committed a patch related to this issue. It has been pushed to "libreoffice-7-4": https://git.libreoffice.org/core/commit/5537a4b2c696bfee3c4c85c9ae3e01cd8d3d3257 tdf#151341 Use lzfse compression instead of bzip2 It will be available in 7.4.5. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
(In reply to steve from comment #19) > Were do we nominate developer of the month? Patrick you are a hero! I can't > even say for how many years this has been a problem on macOS. 10 or more? > Copying LibreOffice has been slow forever. What easily took much longer than > 30 seconds is now an under 10 second process. > > I heard macOS users around the globe were starting to dance and chant when > they learned about this change. > I am happy that the fix works so well. But I really can't take credit for the fix. All the credit should to @Mingye Wang who determined what need to be changed. I only tested and submitted their change.
7.4.5 was a hotfix release, updating target in status-whiteboard