Bug 106579 - online: serve gzipped file content ...
Summary: online: serve gzipped file content ...
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice Online
Classification: Unclassified
Component: LibreOffice (show other bugs)
Version:
(earliest affected)
unspecified
Hardware: All All
: medium enhancement
Assignee: Aditya Dewan
URL:
Whiteboard: target:5.4.0
Keywords: difficultyBeginner, easyHack, skillCpp, topicCleanup
Depends on:
Blocks:
 
Reported: 2017-03-16 16:49 UTC by Michael Meeks
Modified: 2017-04-12 09:10 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:


Attachments
Strange characters in js file (36.84 KB, image/png)
2017-03-19 15:49 UTC, Aditya Dewan
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Meeks 2017-03-16 16:49:33 UTC
Online serves lots of JS, SVG etc. via HTTP[S]. If the client
flags that they support gzip in their request header; ie. the request
has a header containing 'gzip' eg.

	Accept-Encoding:gzip, deflate, sdch, br

	Then we should send a response with this header:

	Content-Encoding:gzip

	And serve up an internally cached copy of the file which is
gzipped; checkout:

net/Socket.cpp:

namespace HttpHelper
{
    inline void sendFile(const std::shared_ptr<StreamSocket>& socket, const std::string& path,
                         Poco::Net::HTTPResponse& response)

	Ideally we should build and pre-populate a set of all files
and their gzipped' versions in memory on startup - and only serve
these (which would also be useful for security and avoids the need to
do careful path resolution in wsd/FileServer.cpp).

	We can use: Poco's DeflatingOutputStream to read and deflate
those (I hope) - but lets only support 'gzip' encoding.

	Thanks !
Comment 1 Aditya Dewan 2017-03-17 15:42:19 UTC
Hi,

I would like to work on this issue.
Comment 2 Aditya Dewan 2017-03-19 15:48:32 UTC
Hi,

So, i have been working on this issue and trying to solve it. As of now i am able to send gzipped version of requested files with appropriate headers, but there is a small issue that i am facing.

The uncompressed version of the files as shown in the browser, do not exactly match the original file i.e. they contain some stray characters.

I have attached a screenshot of such file.

As per my understanding, this issue might be because of the size of blocks being sent for compression, but i haven't been able to resolve this.

I would be very thankful for any help. : )
Comment 3 Aditya Dewan 2017-03-19 15:49:40 UTC
Created attachment 132014 [details]
Strange characters in js file
Comment 4 Johnny_M 2017-03-19 19:43:47 UTC
Assigning per comment 1. Please reset assignee if you decide to stop working on this.

Solution proposal is on: https://gerrit.libreoffice.org/35429
Comment 5 Commit Notification 2017-03-30 11:09:39 UTC
dewana-dewan committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/online/commit/?id=432204566764e88266370738686ed6a8d1dd31c4

tdf#106579 - serving gzipped file content
Comment 6 Michael Meeks 2017-03-30 11:11:54 UTC
Thanks for the fix; closing the issue.

Another thing to do here is to pre-load all of these files into memory at startup, to pre-compress them (omitting eg. PNGs which are already deflated) - and to serve the files from an internal hash.

Is that something you could work on and/or include in your GSOC proposal (sorry to steal that from you Noel).

If we pre-compress, then getting content-length right should be easy too.

Thanks !
Comment 7 Aditya Dewan 2017-03-30 21:08:23 UTC
Thanks for the suggestion sir, 
I would definitely love to work on this and include this in my gsoc proposal.
Comment 8 Aditya Dewan 2017-04-12 09:10:45 UTC
Submitted a new patch for doing gzip compression instead of deflate, as was initially planned.
https://gerrit.libreoffice.org/#/c/36448/