Bug 141505 - big CSV, Calc hangs when separator used in CSV was not selected when last file was opened
Summary: big CSV, Calc hangs when separator used in CSV was not selected when last fil...
Status: CLOSED DUPLICATE of bug 102846
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
7.1.2.2 release
Hardware: x86-64 (AMD64) All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: CSV-Import
  Show dependency treegraph
 
Reported: 2021-04-06 07:17 UTC by Martin Kozusky
Modified: 2021-08-29 21:12 UTC (History)
2 users (show)

See Also:
Crash report or crash signature:


Attachments
exemple of big CSV file with semicolon separator (1.02 MB, application/zip)
2021-05-02 13:17 UTC, Jean-Baptiste Faure
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Kozusky 2021-04-06 07:17:17 UTC
Description:
Hi,
I have a big (70MB) CSV file, which has "," separator, I open the file (before that I had to choose I will use "," with some smaller CSV) and everything is OK. Then I have another big CSV, which has ";" as separator. But because ";" was not selected with previous file (I didn't want to select multiple separators), it hangs. Normaly, with small files, whole content would be shown on one or two lines in import dialog, when you select wrong separator. But with this big file, it just hangs (may be it would recover after few hours, but I don't want to wait for that).
Would it be possible to fix this somehow? May be to set some limits somewhere, so that it doesn't try to parse the file further when wrong separator is selected?

Steps to Reproduce:
1. open CSV with "," separator (and select no other separator)
2. close the file
3. try to open big CSV file, which has ";" separator

Actual Results:
Calc hangs (probably trying to parse the CSV using lastly selected "," separator to show the preview)

Expected Results:
Calc should not hang, may be there should be some size limit it loads to show the preview


Reproducible: Always


User Profile Reset: No



Additional Info:
Version: 7.1.2.2 (x64) / LibreOffice Community
Build ID: 8a45595d069ef5570103caea1b71cc9d82b2aae4
CPU threads: 4; OS: Windows 6.1 Service Pack 1 Build 7601; UI render: Skia/Raster; VCL: win
Locale: cs-CZ (cs_CZ); UI: cs-CZ
Calc: threaded
Comment 1 Michael Warner 2021-04-06 13:04:55 UTC
Hello mkozusky, thanks for reporting this. I think there's probably some ways we could improve this experience, but it seems more like an enhancement then a bug to me. So, I have updated the importance field accordingly.
Comment 2 m_a_riosv 2021-04-06 17:01:41 UTC
Do you mean that the import window it's not showed at opening every file?
Comment 3 Martin Kozusky 2021-04-06 20:15:44 UTC
The import window is not shown when opening big CSV file with separator that was not selected when opening previous file. May be it would be shown after some long time, I have file with 384 000 lines and 27 cols, so it is quite big, I have waited few hours and nothing ... When I have selected correct separator previously with other file, then preview windows for this big file is shown pretty fast. 

So it only depends if separator in current file was between selected separators when opening last file or not.

When opening small file with wrong separator, it also takes a little time, but then content of the file is shown on one or two lines in import window.
Comment 4 Jean-Baptiste Faure 2021-05-02 13:17:06 UTC
Created attachment 171591 [details]
exemple of big CSV file with semicolon separator

We have a similar problem with the CSV file "donnees-hospitalieres-covid19-2021-05-01-19h09.csv" provided here: https://www.data.gouv.fr/fr/datasets/donnees-hospitalieres-relatives-a-lepidemie-de-covid-19/ and attached here.

Step to reproduce the problem:
1/ open a small CSV file with comma separator with LO Calc then close it;
2/ try to open the attached file (first unzip it); you can use 2 methods:
   2a/ File > Open
   2b/ Sheet > Insert Sheet from File

In both cases Calc hangs and doesn't show the CSV import dialog and consumes 100% of its CPU.

In fact, in this CSV file, the third first columns are text enclosed between "" even if data seems being numerical data.
If, using a text editor, I remove all "" except in the first row, then the file opens without problem in Calc regardless of the separator that the CSV import dialog has stored.

My guess is that the CSV import dialog try to find the first row with data and considers that rows starting with " belong to the header.

Best regards. JBF
Comment 5 Jean-Baptiste Faure 2021-05-10 08:00:39 UTC
Confirmed as bug, it's not en enhancement request. Calc shouldn't freeze and should show the import dialog.

Best regards
Comment 6 Eike Rathke 2021-08-29 21:12:41 UTC
Fixed with bug 102846.

*** This bug has been marked as a duplicate of bug 102846 ***