Bug 66765 - FILESAVE: saving sheet with some queries in it takes forever (~5 minutes)
Summary: FILESAVE: saving sheet with some queries in it takes forever (~5 minutes)
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
4.0.4.2 release
Hardware: Other All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: BSA target:6.3.0
Keywords: haveBacktrace, perf
Depends on:
Blocks: Save
  Show dependency treegraph
 
Reported: 2013-07-10 10:59 UTC by Tim Jowers
Modified: 2019-06-01 16:01 UTC (History)
6 users (show)

See Also:
Crash report or crash signature:


Attachments
file which is very slow to save. (21.12 KB, application/vnd.oasis.opendocument.spreadsheet)
2013-07-10 10:59 UTC, Tim Jowers
Details
text file which was copied into spreadsheet (3.77 KB, text/plain)
2013-07-11 00:17 UTC, Tim Jowers
Details
Correct source file for the query (1.50 KB, text/plain)
2013-07-11 00:36 UTC, Tim Jowers
Details
Perf flamegraph (766.26 KB, image/svg+xml)
2019-04-18 20:04 UTC, Buovjaga
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Tim Jowers 2013-07-10 10:59:56 UTC
Created attachment 82267 [details]
file which is very slow to save.

Problem description: 
Hit File->Save. It all works correctly but takes about 5 minutes instead of a few seconds.

Steps to reproduce:
1. Open the attached file
2. Save it

Current behavior:
Takes forever to load and to save. Tried renaming file. 
Spreadsheet is quick and normal for other things other than saving.
Saving a new spreadsheet is fast and normal.
Then select all->copy on the slow sheet and then select all->delete, then select cell 1 and paste and then save on the new sheet is again sloooow.


Expected behavior:
Loads/saves quickly like normal files. E.g. a blank doc opens right up and can be used while this slow process is ongoing. The spreadsheet is just slow and blocked while saving.
              
Operating System: Windows 7
Version: 4.0.4.2 release
Comment 1 m.a.riosv 2013-07-10 22:27:00 UTC
Hi Tim, thanks for reporting.
I can reproduce with:
Win7x64Ultimate
Version 4.0.4.2 (Build ID: 9e9821abd0ffdbc09cd8c52eaa574fa09eb08f2)

Clearing the format of cell D8, solves the issue. The cell have a text with 9550 characters, and maybe with the direct format is near/over to the limit 2^16 characters by cell.
Please, how the file was created?.
Comment 2 Tim Jowers 2013-07-11 00:17:46 UTC
Created attachment 82306 [details]
text file which was copied into spreadsheet
Comment 3 Tim Jowers 2013-07-11 00:27:09 UTC
Save with D8 intact is still slow. 
The data was pasted from a text file.
the data in the text file was pasted from MS SQL Server Query Analyzer trace.
I also needed to truncate the other two pasted queries, D8 and D11, to make it save normally quickly.
Comment 4 Tim Jowers 2013-07-11 00:36:23 UTC
Created attachment 82307 [details]
Correct source file for the query

The clipboard source was from the database used from Great Plains and are the stored procedures in there. Notice the sproc/query is a very long line. Attaching the sproc pasted into a txt file. In practice, the last line was copied directly from the SQL Server Modify sproc into Calc.
Comment 5 Tim Jowers 2013-07-11 00:37:30 UTC
(In reply to comment #2)
> Created attachment 82306 [details]
> text file which was copied into spreadsheet

ignore this file. Actual problematic cell was pasted from SQL Server Management studio directly into spreadsheet.
Comment 6 Tim Jowers 2013-07-11 00:38:08 UTC
USE [CPEC]
GO
/****** Object:  StoredProcedure [dbo].[zDP_IV00102SS_1]    Script Date: 7/10/2013 8:34:13 PM ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER OFF
GO
ALTER PROC [dbo].[zDP_IV00102SS_1] (@ITEMNMBR char(31), @RCRDTYPE smallint, @LOCNCODE char(11)) AS /* 11.00.0218.000 */ set nocount on SELECT TOP 1  ITEMNMBR, LOCNCODE, BINNMBR, RCRDTYPE, PRIMVNDR, ITMFRFLG, BGNGQTY, LSORDQTY, LRCPTQTY, LSTORDDT, LSORDVND, LSRCPTDT, QTYRQSTN, QTYONORD, QTYBKORD, QTY_Drop_Shipped, QTYINUSE, QTYINSVC, QTYRTRND, QTYDMGED, QTYONHND, ATYALLOC, QTYCOMTD, QTYSOLD, NXTCNTDT, NXTCNTTM, LSTCNTDT, LSTCNTTM, STCKCNTINTRVL, Landed_Cost_Group_ID, BUYERID, PLANNERID, ORDERPOLICY, FXDORDRQTY, ORDRPNTQTY, NMBROFDYS, MNMMORDRQTY, MXMMORDRQTY, ORDERMULTIPLE, REPLENISHMENTMETHOD, SHRINKAGEFACTOR, PRCHSNGLDTM, MNFCTRNGFXDLDTM, MNFCTRNGVRBLLDTM, STAGINGLDTME, PLNNNGTMFNCDYS, DMNDTMFNCPRDS, INCLDDINPLNNNG, CALCULATEATP, AUTOCHKATP, PLNFNLPAB, FRCSTCNSMPTNPRD, ORDRUPTOLVL, SFTYSTCKQTY, REORDERVARIANCE, PORECEIPTBIN, PORETRNBIN, SOFULFILLMENTBIN, SORETURNBIN, BOMRCPTBIN, MATERIALISSUEBIN, MORECEIPTBIN, REPAIRISSUESBIN, ReplenishmentLevel, POPOrderMethod, MasterLocationCode, POPVendorSelection, POPPricingSelection, PurchasePrice, IncludeAllocations, IncludeBackorders, IncludeRequisitions, PICKTICKETITEMOPT, INCLDMRPMOVEIN, INCLDMRPMOVEOUT, INCLDMRPCANCEL, Move_Out_Fence, DEX_ROW_ID FROM .IV00102 WHERE ITEMNMBR = @ITEMNMBR AND RCRDTYPE = @RCRDTYPE AND LOCNCODE = @LOCNCODE ORDER BY ITEMNMBR ASC, RCRDTYPE ASC, LOCNCODE ASC set nocount off
Comment 7 Alex Thurgood 2015-01-03 17:40:55 UTC Comment hidden (no-value)
Comment 8 QA Administrators 2016-01-17 20:04:16 UTC Comment hidden (obsolete)
Comment 9 Aron Budea 2016-11-28 06:30:02 UTC
Still reproduced with 5.3beta1 / Windows 7.
Comment 10 Telesto 2017-06-17 17:01:27 UTC
Saving attachment 82267 [details] to the same file is slow. However saving it to a new file is pretty fast. Re-saving the new file also.

Version: 6.0.0.0.alpha0+
Build ID: cbf371e07fd5dea1ea08a1f299360d1273961ebd
CPU threads: 4; OS: Windows 6.19; UI render: default; 
TinderBox: Win-x86@42, Branch:master, Time: 2017-06-14_23:13:57
Locale: nl-NL (nl_NL); Calc: CL
Comment 11 QA Administrators 2018-06-18 02:41:47 UTC Comment hidden (obsolete)
Comment 12 Roman Kuznetsov 2019-02-05 13:52:57 UTC
don't repro from scratch in

Version: 6.3.0.0.alpha0+
Build ID: ed707a4806a489467c6d9be7d1b787dab94b5f78
CPU threads: 4; OS: Windows 6.1; UI render: default; VCL: win; 
Locale: ru-RU (ru_RU); UI-Language: en-US
Calc: threaded

but repro with file from attach
Comment 13 Buovjaga 2019-04-18 20:04:57 UTC
Created attachment 150863 [details]
Perf flamegraph

The small spike at the end is probably me accidentally pressing Ctrl-C in Calc instead of terminal.

Arch Linux 64-bit
Version: 6.3.0.0.alpha0+
Build ID: 1fee3f1da6291bfbcd75455512726215d41b3e83
CPU threads: 8; OS: Linux 5.0; UI render: default; VCL: gtk3; 
Locale: fi-FI (fi_FI.UTF-8); UI-Language: en-US
Calc: threaded
Built on 18 April 2019
Comment 14 Noel Grandin 2019-05-20 11:10:32 UTC
with current master, this takes less than 1 second for me
Comment 15 Xisco Faulí 2019-05-20 12:25:30 UTC
(In reply to Noel Grandin from comment #14)
> with current master, this takes less than 1 second for me

Same here in

Version: 6.3.0.0.alpha1+
Build ID: 9c7fac47aacb0877c7d212217089a680400c1377
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); UI-Language: en-US
Calc: threaded

following the steps from comment 10
Comment 16 Buovjaga 2019-05-20 12:48:18 UTC
It's weird, because contrary to comment 10, saving to the same file is fast, but saving as a new file is slow.

Arch Linux 64-bit
Version: 6.3.0.0.alpha1+
Build ID: 7aa30433719faece8c40e41d7aa8c7539287932d
CPU threads: 8; OS: Linux 5.1; UI render: default; VCL: gtk3; 
Locale: fi-FI (fi_FI.UTF-8); UI-Language: en-US
Calc: threaded
Built on 19 May 2019
Comment 17 Xisco Faulí 2019-05-20 13:16:17 UTC
(In reply to Buovjaga from comment #16)
> It's weird, because contrary to comment 10, saving to the same file is fast,
> but saving as a new file is slow.

I can't reproduce it here. it takes 1 second as well
Comment 18 Buovjaga 2019-05-23 16:39:52 UTC
Yesterday Muhammet reproduced the slowness with Save as.
Comment 19 Commit Notification 2019-05-31 07:37:54 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/+/36cee080555c759443896cb3b34c3f710f33e0f0%5E%21

tdf#66765 - FILESAVE: saving sheet with some queries in it takes forever

It will be available in 6.3.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 20 Buovjaga 2019-06-01 16:01:22 UTC
Verified that Save as is now instant! Thanks.

Arch Linux 64-bit
Version: 6.4.0.0.alpha0+
Build ID: 7272b9f752cb74757d6ed504202eefccc589f804
CPU threads: 8; OS: Linux 5.1; UI render: default; VCL: gtk3; 
Locale: fi-FI (fi_FI.UTF-8); UI-Language: en-US
Calc: threaded
Built on 1 June 2019