Bug 134421 - Replace xcu settings storage with sqlite database
Summary: Replace xcu settings storage with sqlite database
Status: UNCONFIRMED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: LibreOffice (show other bugs)
Version:
(earliest affected)
Inherited From OOo
Hardware: All All
: medium enhancement
Assignee: Not Assigned
URL:
Whiteboard:
Keywords: needsDevEval
Depends on:
Blocks: Too-Much-File-Access
  Show dependency treegraph
 
Reported: 2020-06-30 10:06 UTC by Telesto
Modified: 2020-12-10 08:39 UTC (History)
7 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Telesto 2020-06-30 10:06:37 UTC
Description:
Replace xcu settings storage with sqlite database. 
The XCU is read/written rather often (every color change, every toolbar change) and size is rather 'large' (1-3 MB).. So SQLite has some advantages IMHO. Instead of rewriting the XCU thousand of times

Looks like regular standard in lot of software.. and sqlite tables are not to hard to edit either.

Steps to Reproduce:
-

Actual Results:
-

Expected Results:
-


Reproducible: Always


User Profile Reset: No



Additional Info:
Better performance
Reading and writing from an SQLite database is often faster than reading and writing individual files from disk. See 35% Faster Than The Filesystem and Internal Versus External BLOBs.
The application only has to load the data it needs, rather than reading the entire file and holding a complete parse in memory.
Small edits only overwrite the parts of the file that change, reducing write time and wear on SSD drives.
Reduced application cost and complexity
Comment 1 Telesto 2020-06-30 14:47:44 UTC
@Miklos
Any opinion if this should lingering in the bug tracker.. else close it as wontfix or pick some other appropriate status.. As I'm not able to implement this anyhow. Lacking the programming skills required or more precise: can't code.

However using a plaintext xcu isn't my first choice with today standards, I think. Especially when written so often. But of course easier to say than change..
Comment 2 Xisco Faulí 2020-06-30 15:05:15 UTC Comment hidden (obsolete)
Comment 3 Telesto 2020-06-30 15:25:58 UTC Comment hidden (obsolete)
Comment 4 Miklos Vajna 2020-06-30 15:47:47 UTC
> Replace xcu settings storage with sqlite database.

I would say this only makes sense if somebody is interested in doing the work, then profile it and demonstrate that the perf gain is indeed significant. I am not. :-)
Comment 5 Julien Nabet 2020-06-30 17:41:19 UTC
Just passing by, implementing SQLite could help these:
- https://bugs.documentfoundation.org/show_bug.cgi?id=86621
direct SQLite open

- https://bugs.documentfoundation.org/show_bug.cgi?id=128977
Thunderbird Address Book will be changed to sqlite - Development needed

but since both they concern Base and Base is already lacking dev people, perhaps no need to add some extra dev work.
Comment 6 Telesto 2020-06-30 18:34:52 UTC
(In reply to Miklos Vajna from comment #4)
> > Replace xcu settings storage with sqlite database.
> 
> I would say this only makes sense if somebody is interested in doing the
> work, then profile it and demonstrate that the perf gain is indeed
> significant. I am not. :-)

I blindly copy/pasted the promotion text of SQLite... :-). I'm actually more interested: *The application only has to load the data it needs, rather than reading the entire file and holding a complete parse in memory.*

Ever monitored how often the xcu is accessed. It's an immense amount if times (say 1000) times a day. File History/ Color Picker color/ Window/ toolbar position and size. Toolbar items. On every time selecting an item in draw (sidebar). A 'normal' xcu should be around 1 MB.. But with extensions and lots of modifications and stuff .. 2/3 MB maybe

So 1000 MB of read/writes in a single session should be to abnormal.  

--
https://ask.libreoffice.org/en/question/145837/registrymodificationsxcu-writing-to-disk-excessively/

"We encountered same issue. about 200 PC users on other machiens with roaming profile have this file "registrymodifications.xcu" constantly writing and dumping our IO. Is there a way to make disable writing to this file in configuration of Libre ? or editing some option in "registrymodifications.xcu" ?"

--

So not into really interested in the performance gain .. more like a being a bit nicer to SSD's (even if the have technical read/write life time; which maybe not to often hit (of course depending on SSD size etc)

However, proofing if SQLite would useful or not is far beyond my capability's I can only assume, this based on other applications using this for storing profile stuff. Nor can I asses the benefit of this in a local / or roaming setup etc. Nor can I estimate complexity to code this kind of thing. Or if this could qualify as GSOC project for example. 

So if the consensus within the (base) development team is, that this waste of resource/ not worth the effort etc, please close. If the common dev opinion is that it would make some sense to further investigate, leave it open..
Comment 7 Telesto 2020-07-04 07:31:28 UTC
Adding a few people, to make sure it's on the radar/considered/evaluated. If LibreOffice is used on a day to day basis 8 hours a day.. No knowledge about how LibreOffice is deployed in company etc