Created attachment 123896 [details]
May be related to: https://bugs.documentfoundation.org/show_bug.cgi?id=88953
I am using a macro to pass values to an existing third party C++ library that does further processing. For bug-reporting purposes I made the sample as small as I could.
Steps to reproduce:
1) Build the attached C++ project (the project file is VS2015), place the resulting DLL in the DLL search path (for example: syswow64 or system32).
2) Create a LibreOffice macro with the following contents:
### START SAMPLE ###
REM ***** BASIC *****
Declare Function fnlotest Lib "lotest.dll" (integer, integer, integer, integer) As Integer
Declare Function fnlotest2 Lib "lotest.dll" (String, String, String, String, String) As Integer
declare sub MyMessageBeep Lib "user32.dll" Alias "MessageBeep" (Long)
MyMessageBeep(5000) ' Test
Dim szBuffer as string
Dim ret as integer
szBuffer = Space(512)
ret = fnlotest(1,2,3,4) ' Expected result: 1+2+3+4 = 10, actual result: 3
fnlotest2(szBuffer, "foo", "bar", "hello", "world")
### END SAMPLE ###
3) Run the sample and verify the return value of fnlotest(1,2,3,4). It should add all numbers.
If you run the sample while having the C++ debugger attached, you will see that the parameters that come through are: 1, 0, 2, 0 instead of 1,2,3,4. Explaining why we get 3 as a result.
Interestingly, the second function with string values is fine.
The third-party DLL uses functions with a signature like:
Because the stack appears to be corrupted after the first integer passed to a function, we get an access violation when accessing the char *.
We tested with older versions, and the last working version is: 22.214.171.124
Currently we are running 126.96.36.199
Verified that it's still an issue in 188.8.131.52, updated the version field to reflect that the last version that is known to work is 184.108.40.206.
The align method seems to create too many or wrong objects that are later converted to function parameters.
Only the first parameter is actually used.
> sblo.dll!`anonymous namespace'::align(std::vector<char,std::allocator<char> > & blob, unsigned int alignment, unsigned int offset, unsigned int add) Line 140 C++
sblo.dll!`anonymous namespace'::add<SbxValues>(std::vector<char,std::allocator<char> > & blob, const SbxValues & data, unsigned int alignment, unsigned int offset) Line 150 C++
sblo.dll!`anonymous namespace'::marshal(bool outer, SbxVariable * variable, bool special, std::vector<char,std::allocator<char> > & blob, unsigned int offset, `anonymous-namespace'::MarshalData & data) Line 333 C++
sblo.dll!`anonymous namespace'::call(const rtl::OUString & dll, const `anonymous-namespace'::ProcData & proc, SbxArray * arguments, SbxVariable & result) Line 497 C++
sblo.dll!SbiDllMgr::Call(const rtl::OUString & function, const rtl::OUString & library, SbxArray * arguments, SbxVariable & result, bool cdeclConvention) Line 719 C++
sblo.dll!SbiRuntime::DllCall(const rtl::OUString & aFuncName, const rtl::OUString & aDLLName, SbxArray * pArgs, SbxDataType eResType, bool bCDecl) Line 1292 C++
sblo.dll!SbiRuntime::StepCALL(unsigned long nOp1, unsigned long nOp2) Line 4164 C++
Verified with the following build:
Build ID: 3b800451b1d0c48045de03b5b3c7bbbac87f20d9
CPU Threads: 4; OS Version: Windows 6.2; UI Render: GL; Layout Engine: new;
Locale: da-DK (da_DK); Calc: group
Right now when running the sample below, instead of getting 1,0,2,0, I get the pointer addresses to the right variables. Tested by debugging and running the following code in the immediate window:
While I now at least would be able to get to the actual values, I think the issue is still not completely resolved.
Additional finding: If I declare the parameters 'byval' in the Macro, as such:
Declare Function fnlotest Lib "lotest.dll" (byval integer, byval integer, byval integer, byval integer) As Integer
It DOES work as expected. To verify, I also executed the same test code in MS Word VBA, and it has exactly the same result when using byval. When not using byval, I still get the addresses.
LibreOffice now behaves exactly as VBA when calling the external DLL. So we can close this issue.