Bug 140525 - Copying cell range with macro is slow
Summary: Copying cell range with macro is slow
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: BASIC (show other bugs)
Version:
(earliest affected)
7.1.0.3 release
Hardware: x86-64 (AMD64) All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords: bibisected, bisected, perf, regression
Depends on:
Blocks:
 
Reported: 2021-02-19 06:30 UTC by Woowbat
Modified: 2022-02-10 14:27 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:


Attachments
Testing files for bug 140525 (18.67 MB, application/vnd.oasis.opendocument.spreadsheet)
2021-02-19 12:23 UTC, Woowbat
Details
File with macro using 1000 iterations (20.28 MB, application/vnd.oasis.opendocument.spreadsheet)
2022-02-07 15:17 UTC, Buovjaga
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Woowbat 2021-02-19 06:30:14 UTC
Hi.
I used to work with big sheets, about 100000 to 200000 rows. Doing some calculation, moving finds.. Trying to upgrade version I've found that each time I do that, it becomes slower and slower. In the end it comes that a new version is about 10 times slower than the old one. For example working on 100000 rows sheet it takes LibreOffice_6.0.7_Win_x64 about 8 minutes. The latest version (and all the previous from the one I've mentioned too)it takes more than an hour. It disqualifies all the good done by the upgrade. Each time I was testing all with the same spread sheet which I kept for spare. What makes it worse, removing a new version and trying replace that with the old one wasn't working. Traces were left and the old version after replacement was working slow as well. I had to remove it and go into windows registry manually removing all the traces left. Then it was working.
Regards
Ziggy
Comment 1 Mike Kaganski 2021-02-19 06:44:29 UTC
Please provide a sample spreadsheet showing this behavior.

The problem looks strange in this part:

> What makes it worse, removing a new version and trying replace that
> with the old one wasn't working. Traces were left and the old version after
> replacement was working slow as well. I had to remove it and go into windows
> registry manually removing all the traces left. Then it was working.

The thing is, LibreOffice does not use Windows Registry at all, and so no traces there should affect the operation of both old and new version. Much greater chance is that removing user profile under %appdata%\LibreOffice could affect that. However, this would be easy to check, if there's a sample file to test - then we could check. Of course, if you also provide information about specific registry entries that you removed, that could help.
Comment 2 Woowbat 2021-02-19 09:10:11 UTC
Hi.
The spreadsheet was doing some calculation on it's own and later working with a macro finding the right data and copying corresponding row to another sheet.
In the registry I was removing any entry with a name LibreOffice in it. I'm not sure which particularly was responsible for making it slow so just in case i did remove all i found. After reinstalling , the old version was working fine.
Regards
Ziggy
Comment 3 Mike Kaganski 2021-02-19 09:36:06 UTC
(In reply to Woowbat from comment #2)

So there is no way for us to reproduce the problem to attempt fixing it? Should it be closed then as impossible to handle?
Comment 4 Woowbat 2021-02-19 12:23:58 UTC
Created attachment 169897 [details]
Testing files for  bug 140525

Sub Main
Dim Doc as Object
Dim SheetCopy as Object
Dim SheetPaste as Object
Dim CellCopy as Object
Dim CellPaste as Object
Doc = ThisComponent
   Sheets = Doc.Sheets()
   SheetCopy = Sheets.getByName("Sheet1")
   SheetPaste = Sheets.getByName("Sheet1")
   CopyRange = SheetCopy.getCellRangeByName("AH1:AH100000")
   PasteRange = SheetPaste.getCellRangeByName("AH1:AH100000")
   PasteRange.DataArray = CopyRange.DataArray
 dim i as long 
 dim j as integer
for i=0 to 100000
	dim document as object
	document = ThisComponent
	sheet = document.Sheets(0)
	REM MsgBox(sheet.getCellByPosition(0, i).String) 
	REM if sheet.getCellByPosition(33, i).String >0 then Print i, (sheet.getCellByPosition(0, i).String)
	if sheet.getCellByPosition(33, i).String >0 then j=j+1
	if sheet.getCellByPosition(2, i).String <0 then i=100000
   Sheets = Doc.Sheets()
	SheetCopy = Sheets.getByName("Sheet1")
	SheetPaste = Sheets.getByName("Sheet2")
	CopyRange = SheetCopy.getCellRangeByPosition(0, i+1, 33, i+1)
	PasteRange = SheetPaste.getCellRangeByPosition(0, j, 33, j)
   PasteRange.DataArray = CopyRange.DataArray
next i
   
End Sub
Comment 5 Woowbat 2021-02-19 12:30:23 UTC
Of course there is. Make sure LibreOffice_6.0.7_Win_x64 is installed on Windows without any version higher than 6.0.7 installed before. On my comp macro needs about 8min 15sec to finish. Compare the difference after upgrade to latest version.
Regards
Ziggy
Comment 6 Buovjaga 2022-02-07 15:16:26 UTC
Bibisected with linux-64-7.1 to
https://git.libreoffice.org/core/commit/861fbd998f2b526c2aea073c9471613bf728fa75
purge shared string pool if ScDocument is closed (tdf#125428)

Adding Cc: to Luboš Luňák

I tested with a macro version with 1k iterations instead of the original 100k (will attach a new file with the modified macro)

The difference on my machine in the 7.1 repo was 2.5 secs vs. 16 secs. However, in the latest of 7.4 the time is only 7 secs, so various improvements were made. The purging commit seems important, so perhaps this can't be seen as a traditional regression. Probably there is a better way to write that Basic macro.
Comment 7 Buovjaga 2022-02-07 15:17:19 UTC
Created attachment 178126 [details]
File with macro using 1000 iterations