Bug 167664 - slow loading CSV with tokens ending in newline
Summary: slow loading CSV with tokens ending in newline
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
25.8.0.0 alpha0+
Hardware: All All
: medium minor
Assignee: Not Assigned
URL:
Whiteboard: target:26.2.0
Keywords:
Depends on:
Blocks: CSV-Import
  Show dependency treegraph
 
Reported: 2025-07-24 13:47 UTC by Noel Grandin
Modified: 2025-07-25 05:33 UTC (History)
0 users

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Noel Grandin 2025-07-24 13:47:20 UTC
The CSV from here

https://en.www.inegi.org.mx/programas/endireh/2016/#open_data
in 
/conjunto_de_datos_tb_sec_vi_endireh_2016/conjunto_de_datos/

loads very slowly because it has trailing newline chars in its tokens, which fools the CSV loader into thinking we have multiline text, which slows the parsing down a lot.
Comment 1 Noel Grandin 2025-07-24 13:50:13 UTC
fix is here: https://gerrit.libreoffice.org/c/core/+/188286
Comment 2 Commit Notification 2025-07-25 05:30:51 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/749a312ea55eaa0c3dc352a0e8a1f32f8b6fd9cf

tdf#167664 slow loading CSV with tokens ending in newline

It will be available in 26.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.