Bug 147421 - LibreOffice hangs if there are massively null-byte characters in csv file
Summary: LibreOffice hangs if there are massively null-byte characters in csv file
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
4.2 all versions
Hardware: All All
: medium normal
Assignee: Eike Rathke
URL:
Whiteboard: target:7.4.0 target:7.3.2 target:7.2.6
Keywords: perf
Depends on:
Blocks:
 
Reported: 2022-02-14 15:29 UTC by Hans Müller
Modified: 2022-02-17 14:07 UTC (History)
2 users (show)

See Also:
Crash report or crash signature:


Attachments
sampel file (6.82 MB, text/csv)
2022-02-14 16:32 UTC, Xisco Faulí
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Hans Müller 2022-02-14 15:29:56 UTC
Description:
As described here:

https://ask.libreoffice.org/t/libreoffice-v7-2-5-2-crashes-when-i-open-a-csv-file/73991/4

In 7.2.5.2 version and also in previous ones.

Steps to Reproduce:
1.Install a described version of Calc
2. Open the file in the forum post
3. Wait - infinitely

Actual Results:
Wait - infinitely

Expected Results:
An adequate error message


Reproducible: Always


User Profile Reset: No



Additional Info:
No
Comment 1 Xisco Faulí 2022-02-14 16:32:01 UTC
Created attachment 178272 [details]
sampel file
Comment 2 Xisco Faulí 2022-02-14 16:35:50 UTC
Reproduced in

Version: 7.4.0.0.alpha0+ / LibreOffice Community
Build ID: 2f0c26dc83448a934af5383f9f6b7a607c334744
CPU threads: 8; OS: Linux 5.10; UI render: default; VCL: gtk3
Locale: es-ES (es_ES.UTF-8); UI: en-US
Calc: threaded Jumbo


Version: 5.2.0.4
Build ID: 066b007f5ebcc236395c7d282ba488bca6720265
CPU Threads: 8; OS Version: Linux 5.10; UI Render: default; 
Locale: en-US (es_ES.UTF-8)

Version: 4.2.0.4
Build ID: 05dceb5d363845f2cf968344d7adab8dcfb2ba71
Comment 3 Xisco Faulí 2022-02-14 16:38:16 UTC
@Eike, I thought you might be interested in this issue
Comment 4 Hans Müller 2022-02-15 07:45:06 UTC
Thank you very much. If you need more informations, so pleas don't hesitate to ask me. Kind regards, Hans
Comment 5 Eike Rathke 2022-02-15 13:45:49 UTC
The file is broken, it contains 3577016 trailing null-bytes. Those are read and then in ScImportExport::EmbeddedNullTreatment() to be stripped using OUString::replaceAll() with an empty to-string. It appears that under replaceAll() for each null-byte to be replaced a new rtl_uString instance is allocated and the following remaining string appended just to be stripped again. It appears to hang but eventually it should finish. With this massive amount of stripping that doesn't scale and needs better handling.
Comment 6 Hans Müller 2022-02-15 15:34:20 UTC
Thx!! :-)
Comment 7 Commit Notification 2022-02-15 19:45:21 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/4b0c17609c2cca326bbcc9e8488a327a4a9ea952

Resolves: tdf#147421 Do not use OUString::replaceAll() to strip null-bytes

It will be available in 7.4.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 8 Eike Rathke 2022-02-15 19:45:58 UTC
Pending review https://gerrit.libreoffice.org/c/core/+/129938 for 7-3
Comment 9 Hans Müller 2022-02-16 07:55:58 UTC
OK thx
Comment 10 Xisco Faulí 2022-02-16 08:15:35 UTC
Verified in

Version: 7.4.0.0.alpha0+ / LibreOffice Community
Build ID: 49746f40b06eaf1f61bb54454408a06a49d73c5e
CPU threads: 8; OS: Linux 5.10; UI render: default; VCL: gtk3
Locale: es-ES (es_ES.UTF-8); UI: en-US
Calc: threaded

@Eike, thanks for fixing this issue!!
Comment 11 Commit Notification 2022-02-16 08:17:04 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "libreoffice-7-3":

https://git.libreoffice.org/core/commit/ac5e918bef47a5f7ec4e5c76e94b36753c85e031

Resolves: tdf#147421 Do not use OUString::replaceAll() to strip null-bytes

It will be available in 7.3.2.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 12 Hans Müller 2022-02-16 09:10:18 UTC
That's nice, thank you very much. Then i will try it out if v 7.3.2 is released :-)
Comment 13 Eike Rathke 2022-02-16 11:00:26 UTC
Pending review https://gerrit.libreoffice.org/c/core/+/129945 for 7-2
Comment 14 Commit Notification 2022-02-16 13:44:25 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "libreoffice-7-2":

https://git.libreoffice.org/core/commit/1983ec44061111da206fe73c4da7c325043b00ee

Resolves: tdf#147421 Do not use OUString::replaceAll() to strip null-bytes

It will be available in 7.2.6.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 15 Commit Notification 2022-02-16 17:16:32 UTC
Mike Kaganski committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/4e4a01302a140d75a49055821b3197a2eda81db5

Related: tdf#147421: optimize O(U)String's replaceAll*

It will be available in 7.4.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 16 Commit Notification 2022-02-17 14:07:16 UTC
Mike Kaganski committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/41ca0b7262ca52646e935e41a187c5742c3993bb

Revert "Resolves: tdf#147421 Do not use OUString::replaceAll() to strip null-bytes"

It will be available in 7.4.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.