Bug 141505

Summary:	big CSV, Calc hangs when separator used in CSV was not selected when last file was opened
Product:	LibreOffice	Reporter:	Martin Kozusky <mkozusky>
Component:	Calc	Assignee:	Not Assigned <libreoffice-bugs>
Status:	CLOSED DUPLICATE
Severity:	normal	CC:	jbfaure, miguelangelrv
Priority:	medium
Version:	7.1.2.2 release
Hardware:	x86-64 (AMD64)
OS:	All
Whiteboard:
Crash report or crash signature:		Regression By:
Bug Depends on:
Bug Blocks:	109236
Attachments:	exemple of big CSV file with semicolon separator

Description Martin Kozusky 2021-04-06 07:17:17 UTC

Description:
Hi,
I have a big (70MB) CSV file, which has "," separator, I open the file (before that I had to choose I will use "," with some smaller CSV) and everything is OK. Then I have another big CSV, which has ";" as separator. But because ";" was not selected with previous file (I didn't want to select multiple separators), it hangs. Normaly, with small files, whole content would be shown on one or two lines in import dialog, when you select wrong separator. But with this big file, it just hangs (may be it would recover after few hours, but I don't want to wait for that).
Would it be possible to fix this somehow? May be to set some limits somewhere, so that it doesn't try to parse the file further when wrong separator is selected?

Steps to Reproduce:
1. open CSV with "," separator (and select no other separator)
2. close the file
3. try to open big CSV file, which has ";" separator

Actual Results:
Calc hangs (probably trying to parse the CSV using lastly selected "," separator to show the preview)

Expected Results:
Calc should not hang, may be there should be some size limit it loads to show the preview


Reproducible: Always


User Profile Reset: No



Additional Info:
Version: 7.1.2.2 (x64) / LibreOffice Community
Build ID: 8a45595d069ef5570103caea1b71cc9d82b2aae4
CPU threads: 4; OS: Windows 6.1 Service Pack 1 Build 7601; UI render: Skia/Raster; VCL: win
Locale: cs-CZ (cs_CZ); UI: cs-CZ
Calc: threaded

Comment 1 Michael Warner 2021-04-06 13:04:55 UTC

Hello mkozusky, thanks for reporting this. I think there's probably some ways we could improve this experience, but it seems more like an enhancement then a bug to me. So, I have updated the importance field accordingly.

Comment 2 m_a_riosv 2021-04-06 17:01:41 UTC

Do you mean that the import window it's not showed at opening every file?

Comment 3 Martin Kozusky 2021-04-06 20:15:44 UTC

The import window is not shown when opening big CSV file with separator that was not selected when opening previous file. May be it would be shown after some long time, I have file with 384 000 lines and 27 cols, so it is quite big, I have waited few hours and nothing ... When I have selected correct separator previously with other file, then preview windows for this big file is shown pretty fast. 

So it only depends if separator in current file was between selected separators when opening last file or not.

When opening small file with wrong separator, it also takes a little time, but then content of the file is shown on one or two lines in import window.

Comment 4 Jean-Baptiste Faure 2021-05-02 13:17:06 UTC

Created attachment 171591 [details]
exemple of big CSV file with semicolon separator

We have a similar problem with the CSV file "donnees-hospitalieres-covid19-2021-05-01-19h09.csv" provided here: https://www.data.gouv.fr/fr/datasets/donnees-hospitalieres-relatives-a-lepidemie-de-covid-19/ and attached here.

Step to reproduce the problem:
1/ open a small CSV file with comma separator with LO Calc then close it;
2/ try to open the attached file (first unzip it); you can use 2 methods:
   2a/ File > Open
   2b/ Sheet > Insert Sheet from File

In both cases Calc hangs and doesn't show the CSV import dialog and consumes 100% of its CPU.

In fact, in this CSV file, the third first columns are text enclosed between "" even if data seems being numerical data.
If, using a text editor, I remove all "" except in the first row, then the file opens without problem in Calc regardless of the separator that the CSV import dialog has stored.

My guess is that the CSV import dialog try to find the first row with data and considers that rows starting with " belong to the header.

Best regards. JBF

Comment 5 Jean-Baptiste Faure 2021-05-10 08:00:39 UTC

Confirmed as bug, it's not en enhancement request. Calc shouldn't freeze and should show the import dialog.

Best regards

Comment 6 Eike Rathke 2021-08-29 21:12:41 UTC

Fixed with bug 102846.

*** This bug has been marked as a duplicate of bug 102846 ***