Bug 78513 - valgrind warnings in bundled CPython memory allocator
Summary: valgrind warnings in bundled CPython memory allocator
Status: RESOLVED NOTOURBUG
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: sdk (show other bugs)
Version:
(earliest affected)
4.2.4.1 rc
Hardware: Other Linux (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: BSA
Keywords:
Depends on:
Blocks:
 
Reported: 2014-05-10 02:12 UTC by Jim Avera
Modified: 2015-02-18 20:34 UTC (History)
1 user (show)

See Also:
Crash report or crash signature:


Attachments
Test script which runs valgrind with desired args on libre office (927 bytes, application/x-sh)
2014-05-10 02:12 UTC, Jim Avera
Details
Test calc spreadsheet which causes the buffer overrun when loading (105.02 KB, application/vnd.oasis.opendocument.spreadsheet)
2014-05-10 02:13 UTC, Jim Avera
Details
Complete valgrind log using 4.3-alpha1 (5/4/2014 build) (1.09 MB, text/plain)
2014-05-10 02:26 UTC, Jim Avera
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jim Avera 2014-05-10 02:12:19 UTC
Created attachment 98799 [details]
Test script which runs valgrind with desired args on libre office

Valgrind reports some 8-byte reads which extend 4 bytes beyond the end of malloc'd space.  This would cause unpredictable garbage to be returend in the last 4 bytes, and might possibly explain some "unreproducible" bugs I've been chasing in some Basic macros. The problem exists in LO 4.2.4.0 but I'm attaching traces from 4.3-alpha1 (May 5 build).

Here's typical valgrind output (greatly reduced):

== Invalid read of size 8
==    at: __wcscpy_ssse3 (wcscpy-ssse3.S:146)
==    by: calculate_path (getpath.c:751)
==    by: Py_GetProgramFullPath (getpath.c:879)
==    by: _PySys_Init (sysmodule.c:1615)
==    by: _Py_InitializeEx_Private (pythonrun.c:338)
==    by: pyuno_loader::CreateInstance() in /opt/libreofficedev4.3.SAVED_May4/program/libpythonloaderlo.so
       -snip-
==  Address 0x1fd01478 is 1,544 bytes inside a block of size 1,548 alloc'd

The full trace log will be attached

Steps to reproduce:
1. Download the attached "test.sh" script and "spreadsheet.ods"
2. Edit test.sh to point to your LO install dir, or else pass
    arguments /path/to/libreofficedir spreadsheet.ods
3. When prompted to enable macros, click NO (do not enable)
4. When the document is opened, just close it; the test is done

Current behavior: valgrind reports buffer overruns

Expected behavior: not that
              
Operating System: Ubuntu
Version: 4.2.4.1 rc
Comment 1 Jim Avera 2014-05-10 02:13:56 UTC
Created attachment 98800 [details]
Test calc spreadsheet which causes the buffer overrun when loading
Comment 2 Jim Avera 2014-05-10 02:26:20 UTC
Created attachment 98803 [details]
Complete valgrind log using 4.3-alpha1 (5/4/2014 build)
Comment 3 Michael Meeks 2014-06-05 08:32:14 UTC
In general, I imagine that python has a close relationship to its allocator particularly for this sort of optimised copy, and that this is a false positive =) python generates quite a lot of those.

Having said that - its great to have people pouring over the valgrind output to see if something is really going wrong =) I'd add this particular one to your suppressions if its really verbose / annoying.

So - speculatively closing ... thanks !
Comment 4 Jim Avera 2014-06-05 10:48:24 UTC
Michael, did you specifically look at the code and see that the allocator *expects* storage beyond what is allocated to be used?  This seems like an extremely dangerous situation.  Please don't close (or else please re-open) this bug unless you have actual knowledge that this isn't really a bug.'

I'm saying this because this is not idle lint-checking; there _is_ some bad memory corruption going on somewhere in LibO which makes it unusuable for some people (like me).  My spreadsheets crash every few minutes, whereas they run find with never a crash in OpenOffice 4.1.   What I don't know is whether extremely old versions of LibO also run reliably.


P.S. If valgrind is intercepting malloc() calls then it would be absolutely wrong for any code to allocate beyond the end of a malloc'd storage.  But I don't know exactly what valgrind is doing.
Comment 5 Michael Stahl (allotropia) 2015-02-18 20:34:52 UTC
by the way you can just run "soffice --valgrind" to run LO
in valgrind, no custom scripting required.

most of the errors are in CPython's memory allocation stuff, which we
neither wrote nor maintain; if you think there are problems there
please get in contact with the CPython community.

==21827== Invalid read of size 8
==21827==    at 0x546D511: __wcscpy_ssse3 (wcscpy-ssse3.S:146)

==21827== Invalid read of size 8
==21827==    at 0x53AC1EA: wcschr (wcschr.S:112)

these are extremely likely to be a false positive - the libc
string functions are hyper-optimized and will load
data in 8 byte (or even bigger) chunks into registers
but then only access them until they encounter a 0 byte.

The workaround is to run with --partial-loads-ok=yes

       --partial-loads-ok=<yes|no> [default: no]
           Controls how Memcheck handles 32-, 64-, 128- and 256-bit naturally
           aligned loads from addresses for which some bytes are addressable
           and others are not. When yes, such loads do not produce an address
           error. Instead, loaded bytes originating from illegal addresses are
           marked as uninitialised, and those corresponding to legal addresses
           are handled in the normal way.

           When no, loads from partially invalid addresses are treated the
           same as loads from completely invalid addresses: an illegal-address
           error is issued, and the resulting bytes are marked as initialised.

           Note that code that behaves in this way is in violation of the ISO
           C/C++ standards, and should be considered broken. If at all
           possible, such code should be fixed. This option should be used
           only as a last resort.


perhaps in the meantime your valgrind won't complain any more
because suppressions were added somewhere.