Bug 57841 - FILEOPEN: Importing CSV file, data columns disappear if binary 0 in data
Summary: FILEOPEN: Importing CSV file, data columns disappear if binary 0 in data
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
3.6.0.0.beta1
Hardware: All All
: medium normal
Assignee: Eike Rathke
URL:
Whiteboard: BSA bibisected40 target:4.1.0 target:...
Keywords: regression
Depends on:
Blocks:
 
Reported: 2012-12-03 14:57 UTC by Eikaskoop
Modified: 2013-04-15 14:50 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:


Attachments
This is an example of file that has been working until 3.5.6 (101.41 KB, application/vnd.ms-excel)
2012-12-03 14:57 UTC, Eikaskoop
Details
Example of failing import (1.71 KB, text/plain)
2013-02-11 09:45 UTC, Danilo
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Eikaskoop 2012-12-03 14:57:49 UTC
Created attachment 70971 [details]
This is an example of file that has been working until 3.5.6

Problem description: 

Opening data file with riched text, organized in columns, and delimited with tabs,  data is lost behind "null" symbol. 

Steps to reproduce:
1. Open file
2. Choose "tab" as delimiter
3. Ok

Current behavior:
Data lost in columns, data not found in other columns.

Expected behavior:
Import all rows from file with all data.

              
Operating System: Windows XP
Last worked in: 3.5.6.2 release
Comment 1 billhook 2012-12-03 22:41:17 UTC
So... is this a text file? How was it originally created? In Excel? Or in LibreOffice?

And what format did you choose when did "Save as"?
Comment 2 billhook 2012-12-04 09:46:19 UTC
I can confirm that Excel 2007 opens this file OK (after warning that the extension is incorrect) and that when opened in LO 3.6.4.1 Windows XP SP3 32bit, columns from E onwards are empty.

It would still be useful to know how this file was created and why it has "nulls" in it, before this can be fixed.
Comment 3 Markus Mohrhard 2012-12-27 02:37:58 UTC
@Eike: It looks like a csv file with.
Comment 4 Aurimas Fišeras 2013-01-27 12:28:31 UTC
Bibisection result:

commit f98bc0bdf78118131e63c79dbc96707c8d9e5020
Author: Bjoern Michaelsen <bjoern.michaelsen@canonical.com>
Date:   Wed Apr 25 07:28:24 2012 +0200

    source-hash-ce97851773a06103504972eb2771eecd7dd81e36
    
    commit ce97851773a06103504972eb2771eecd7dd81e36
    Author:     David Tardon <dtardon@redhat.com>
    AuthorDate: Mon Feb 6 19:12:02 2012 +0100
    Commit:     David Tardon <dtardon@redhat.com>
    CommitDate: Mon Feb 6 19:12:02 2012 +0100
    
        fix typo

# bad: [5b4b36d87517a6ea96ff8c84c46b12f462fc9a1a] source-hash-8450a99c744e9005f19173e4df35d65640bcf5c4
# good: [65fd30f5cb4cdd37995a33420ed8273c0a29bf00] source-hash-d6cde02dbce8c28c6af836e2dc1120f8a6ef9932
git bisect start 'latest' 'oldest'
# bad: [16b0b88cbd4ef0f51816e97277e40c5cf78f7bf9] source-hash-099198a4224778fe6e43f5dc13b5b9b1b4dc828c
git bisect bad 16b0b88cbd4ef0f51816e97277e40c5cf78f7bf9
# bad: [598083cdb5699e7f45183da8b750815f62ff5485] source-hash-ecb1599ad00e71dfe05f3ae9a71bdce5f7540a40
git bisect bad 598083cdb5699e7f45183da8b750815f62ff5485
# good: [7a67618d4eb83613b68e348711ae303d7a37f217] source-hash-71cbcb62028295a98ceee60cb4c4ee425bafcd2e
git bisect good 7a67618d4eb83613b68e348711ae303d7a37f217
# good: [1fd146a6cc80f3a3c2a14d3971501b67fece30a6] source-hash-817bf1d41bb07aeb3ed7649d25c2b44ee4acb1fe
git bisect good 1fd146a6cc80f3a3c2a14d3971501b67fece30a6
# good: [4d2e7c1f9a87c0ee56840ce9460a8a0fc652d911] source-hash-43c7830b03d141ae11d8617c0fdabefa32dd243c
git bisect good 4d2e7c1f9a87c0ee56840ce9460a8a0fc652d911
# bad: [74fa6bec771e6fe10ee8f25d9bfe8f95ab1c4c49] source-hash-4ff7252375b7b85eafbf176ca4e9184cc392d980
git bisect bad 74fa6bec771e6fe10ee8f25d9bfe8f95ab1c4c49
# bad: [3b391fdd0f6bec6561861dfb95efb2bdc64221e7] source-hash-a330f38093e2643a26239557050561afae9ff23d
git bisect bad 3b391fdd0f6bec6561861dfb95efb2bdc64221e7
# bad: [f98bc0bdf78118131e63c79dbc96707c8d9e5020] source-hash-ce97851773a06103504972eb2771eecd7dd81e36
git bisect bad f98bc0bdf78118131e63c79dbc96707c8d9e5020
Comment 5 Danilo 2013-02-11 09:45:57 UTC
Created attachment 74592 [details]
Example of failing import

The problem appears in version 4.0.0.3 also

The attacchment is an example
Comment 6 Eike Rathke 2013-02-12 10:51:33 UTC
What software writes binary 0 into TEXT files? This calls for trouble. If this ever worked it was by accident.
Comment 7 Not Assigned 2013-02-22 13:01:23 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=8970e14d7494859c6079ef2a976416598823ba50

resolved fdo#57841 ignore embedded NULL characters in CSV import



The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 8 Eike Rathke 2013-02-22 13:08:23 UTC
Change pending review for 4-0 as https://gerrit.libreoffice.org/2336
Comment 9 Not Assigned 2013-02-23 03:38:54 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "libreoffice-4-0":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=4a2b187503de7425150528faab568e520bd2474f&h=libreoffice-4-0

resolved fdo#57841 ignore embedded NULL characters in CSV import


It will be available in LibreOffice 4.0.2.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 10 Björn Michaelsen 2013-04-15 14:50:58 UTC
bibisect shows this happened before 3.6 branch off, so no regression in a 3.6.0->3.6.x update