Bug 100137 - Wrong Hyperlinks in Calc when opening the Calc document over a Symlink
Summary: Wrong Hyperlinks in Calc when opening the Calc document over a Symlink
Status: CLOSED NOTOURBUG
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
4.1.6.2 release
Hardware: All Linux (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-05-30 12:42 UTC by Martin Nathansen
Modified: 2017-06-02 12:33 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Nathansen 2016-05-30 12:42:22 UTC
When opening a calc document over a symlink, a hyperlink path in this calc document is replaced by the symlink path. So the hyperlink is getting wrong and hyperlinked document cannot be opened anymore by Ctrl-Click.

The failure only occurs, when the “save URLs relative to file system” option is enabled and when the calc document is not stored in the home directory, rsp. it should be stored in a subdirectory of the home directory, e.g. /home/user/test/calcdoc.ods.

Verified this bug on Linux with LibreOffice 4.1.6, 5.0.6.3, 5.1.3.2 and 5.2.0 (master) 

Steps to reproduce the failure:

1. Create a calc document in a subdirectory, e.g.:
/home/user/test/calcdoc.ods

2. Make sure the “save URLs relative to file system” option is enabled (it is normally enabled by default):
Tools → Options → Load/Save → General → Save → “save URLs relative to file system”

3. In the calc document insert a hyperlink to any other document in the same folder, e.g.:
Insert → Hyperlink → Document → Document Path: /home/user/test/hyperlinked.pdf

4. Save and Reopen the calc document and check the hyperlink by Ctrl-Click on it, the hyperlinked.pdf should be opening no problem.

5. Create a symbolic link to the calc document for instance from the desktop:
/home/user/Desktop/calcdoc.ods →  /home/user/test/calcdoc.ods

6. Open the calc document by clicking on the symlink /home/user/Desktop/calcdoc.ods and try to open the hyperlinked.pdf by Ctrl-Click on it.
The hyperlinked pdf cannot be opened because the path to the  hyperlinked.pdf has been changed to the symlink path:
/home/user/Desktop/hyperlinked.pdf (wrong)
Comment 1 Dennis Roczek 2016-05-30 13:12:26 UTC
well, Martin, this is how relative links do work. they are saved internally as ./filename.fileextension

How should the symlink know where the "original" file is (and which file is the original)? If you do want to have both working, then use absolute file paths.
Comment 2 Martin Nathansen 2016-05-30 13:18:13 UTC
There are always absolute file paths created even when the "save URLs relative to file system” option is enabled

Those relative path option is meaningless at least for file links.
Comment 3 Martin Nathansen 2016-05-30 13:26:00 UTC
It might be that the paths is saved relative internally but the user just sees absolute paths.

Another problem is that "relative" is the default option.
Comment 4 Markus Mohrhard 2016-05-31 07:38:01 UTC
As Dennis already mentioned there is no bug here. Everything here works as expected.
Comment 5 Martin Nathansen 2016-05-31 09:26:43 UTC
Dennis, Markus, I do not understand why you always close this bug without having a deeper look on it. Here another failure scenario which is faster to read:

1. Preconditions: Fresh (default) LO installation on Linux.

2. The user has a calc or writer document in a sub-folder of his home directory with hyperlinks to other documents in the same folder. These hyperlinks where originally inserted via menu: Insert → Hyperlink → Document → Document Path
All these hyperlinks work perfectly (open with Ctrl-Click).

3. The user opens the same document over a symbolic link from his desktop. But when he try to open the hyperlinked documents via Ctrl-Click only Error-Messages pop up. He has no glue what is going wrong because only absolute hyperlink paths are visible to him....

Dennis, Markus, is this really the desired behavior of LibreOffice? What about usability?

By the way in OpenOffice there seems to be no such bug.
Comment 6 Martin Nathansen 2016-06-02 15:03:40 UTC
Dennis, your suggestion “If you do want to have both working, then use absolute file paths“ is not a good idea, because it leads to the same issue when documents are shared between users: /home/user1/  /home/user2/  In this case the  hyperlinks will break again.

Markus, you said. “Everything here works as expected”. Does that means that the Linux user has to accept that hyperlinks in LibreOffice break when shortcuts or symlinks are used? On MS Windows LibreOffice has no such problems, here the hyperlinks work perfectly in all cases.

In other big issue is that the user has no glue why the hyperlinks break in LO: Visible to the users are only absolute paths even when they are stored internally in the doc as relative paths. The user has no chance to find out the reason of his broken hyperlinks.

So I suggest to treat this issue as it is: a bug at least for the LO users on Linux.
Comment 7 Dennis Roczek 2016-06-08 09:45:07 UTC
get to understand the system of relative and absolute paths including symbolic links. Move the liked object to a shared folder (e.g. a network folder) and use an absolute path to the file on the shared folder.

This has nothing to do with LibreOffice and Linux. That works the same way on Linux, OS X, and Windows and moreover on any other application which can create and kind of links (e.g. HTML pages)
Comment 8 Martin Nathansen 2016-06-08 15:42:51 UTC
> ...Move the liked object to a shared folder (e.g. a network folder) and use an
> absolute path to the file on the shared folder.
This has nothing to do with my bugreport. 

> This has nothing to do with LibreOffice and Linux...
It has. As I already mentioned above the bug does not occurs on MS Windows. I tested it with OpenOffice 4.1 and LibreOffice 5.1 on Windows 7.

> ...That works the same way on Linux, OS X, and Windows
On Linux it is more difficult to resolve the paths, however it is possible:
1) retrieve the absolute path of the opened document (“ls -ali”, "readlink -m <file>", "realpath()", or boost: "path absolute()", "path canonical()"..)
2) from there resolve the relative paths inside the document

> ...and moreover on any other application which can create and kind of
> links (e.g. HTML pages)
I tested the scenario with HTML files. At least Firefox is able to do the job, even on Linux.
Comment 9 Katarina Behrens (Inactive) 2016-06-10 08:36:56 UTC
Let me rephrase this a bit: imho Martin is not disturbed by the fact that relative hyperlinks do not work when opened from a symlink. 

He is disturbed by the fact that in Calc UI, links are displayed and converted to as absolute at all times, even if they have been (or are yet going to be) saved as relative. 

It is clearly documented that this ( = links are always displayed as absolute) is the case and in my opinion, the chapter about differences between relative and absolute links is one of the best written chapters of Calc Users' Guide, but let's be honest, users don't read documentation. 

Moreover, "usability" is frequently considered to be the same as "users actually *don't* have to read documentation and learn anything about the software they are using", but that's another topic, out of scope of this ticket.

And to add insult to injury, the default choice is to save links as relative.

So as a result, the user who isn't aware of all this, who either didn't read the documentation or didn't get the proper training, suddenly doesn't know what happened to her hyperlink and why it no longer works ...
Comment 10 Martin Nathansen 2016-08-02 08:54:48 UTC
Following patch fixes this bug for calc, writer and impress:

https://gerrit.libreoffice.org/#/c/27544/

Hopefully somebody has some time to test it.

The bug is fixed with the help of salhelper::LinkResolver  by
resolving the symlinks or shortcuts before converting the relative hyperlinks
to absolute file paths. So the base URL for the relative/absolute and
absolute/relative link conversion is always the real file path and not the
symlink or desktop shortcut.
Comment 11 Martin Nathansen 2016-08-03 14:23:49 UTC
The new patch solves the issues from Code-Review-1 and is much smaller now. Therefore a new filter had to be included in InetURLObject::smartRel2Abs and InetURLObject::GetNewAbsURL. With this filter the base URI is only resolved and the relative URI only converted if the relative URI starts with dots  “../”. Otherwise the relative file URIs starting with “FILE://” would be converted with a wrong base URI:

https://gerrit.libreoffice.org/#/c/27544/
Comment 12 Eike Rathke 2016-08-09 13:20:49 UTC
Expectations with symlinks may differ. I for example *do* expect that if I open the document via a symlink then all relative external references are relative to the symlink directory and not to the resolved symlink's directory. Doing so enables the user to symlink documents to other places and create scenarios that behave different from the original place. It also allows to place symlinks in the trusted macro directory if documents reside elsewhere.

So, to summarize, for me this is not a bug and I don't want it "fixed".
Comment 13 Mike Kaganski 2016-08-09 13:38:05 UTC
(In reply to Martin Nathansen from comment #8)
> > This has nothing to do with LibreOffice and Linux...
> It has. As I already mentioned above the bug does not occurs on MS Windows.
> I tested it with OpenOffice 4.1 and LibreOffice 5.1 on Windows 7.

Are you sure you've tested it with symlinks under Windows???
I suppose they were shortcuts (.lnk)? If so, then they are entirely different.

Then, how will you "fix" hardlinks?
Comment 14 Eike Rathke 2016-08-09 15:02:17 UTC
I'm quite convinced this can be addressed only properly if the parent document is opened following the symlink on user demand, either through an option or after having determined that it was opened using a symlink and there are relative external links in the document, and then reloading the doc after having asked the user. Everything else IMHO is a hack that will cause other problems.
Comment 15 Martin Nathansen 2016-08-09 15:28:53 UTC
(In reply to Eike Rathke from comment #12)
> Expectations with symlinks may differ. I for example *do* expect that if I
> open the document via a symlink then all relative external references are
> relative to the symlink directory and not to the resolved symlink's
> directory. Doing so enables the user to symlink documents to other places
> and create scenarios that behave different from the original place. It also
> allows to place symlinks in the trusted macro directory if documents reside
> elsewhere.
> 
> So, to summarize, for me this is not a bug and I don't want it "fixed".

The patch should fix three every-day use cases which work properly on MS Windows but fails on Linux:

1) Having LibreOffice documents on a Network-Share with relative hyperlinks.
/home/user1/netshare/....
/home/user2/netshare/...

2) Moving a folder with LO documents with relative hyperlinks  to another folder:
/home/user/folder1/...
/home/user/folder2/subfolder/...

3) Zipping and sharing documents with relative links between users:
/home/user1/...
/home/user2/...

Each of this use cases breaks when a user creates a desktop shortcut and open the documents via this shortcut. Not even the existing hyperlinks break, the user is also not able to create new hyperlinks.
Comment 16 Martin Nathansen 2016-08-09 15:29:25 UTC
(In reply to Mike Kaganski from comment #13)
> (In reply to Martin Nathansen from comment #8)
> > > This has nothing to do with LibreOffice and Linux...
> > It has. As I already mentioned above the bug does not occurs on MS Windows.
> > I tested it with OpenOffice 4.1 and LibreOffice 5.1 on Windows 7.
> 
> Are you sure you've tested it with symlinks under Windows???
> I suppose they were shortcuts (.lnk)? If so, then they are entirely
> different.
> 
> Then, how will you "fix" hardlinks?

No, I tested desktop shortcuts on MS Windows only: On MS Windows there is no such relative hyperlink issue and my intention was to get it working in the same way on Linux.
Comment 17 Dennis Roczek 2016-08-09 15:52:13 UTC
(In reply to Martin Nathansen from comment #16)
> (In reply to Mike Kaganski from comment #13)
> > (In reply to Martin Nathansen from comment #8)
> > > > This has nothing to do with LibreOffice and Linux...
> > > It has. As I already mentioned above the bug does not occurs on MS Windows.
> > > I tested it with OpenOffice 4.1 and LibreOffice 5.1 on Windows 7.
> > 
> > Are you sure you've tested it with symlinks under Windows???
> > I suppose they were shortcuts (.lnk)? If so, then they are entirely
> > different.
> > 
> > Then, how will you "fix" hardlinks?
> 
> No, I tested desktop shortcuts on MS Windows only: On MS Windows there is no
> such relative hyperlink issue and my intention was to get it working in the
> same way on Linux.

well either use / test shortcuts ("soft links") on both systems (linux and windows) or use / test it with symbolic links ("hard links") on both systems. Windows is able to use hard links! (through only by using the command line)
Comment 18 Martin Nathansen 2016-08-09 16:18:34 UTC
(In reply to Eike Rathke from comment #14)
> I'm quite convinced this can be addressed only properly if the parent
> document is opened following the symlink on user demand, either through an
> option or after having determined that it was opened using a symlink and
> there are relative external links in the document, and then reloading the
> doc after having asked the user. Everything else IMHO is a hack that will
> cause other problems.

Most of our users will be overstrained with such a dialog window. 
My intention was to get the three use cases (see  https://bugs.documentfoundation.org/show_bug.cgi?id=100137#c14 ) running in the same way like on MS Windows.
Comment 19 Mike Kaganski 2016-08-09 21:47:56 UTC
(In reply to Dennis Roczek from comment #17)
> well either use / test shortcuts ("soft links") on both systems (linux and
> windows) or use / test it with symbolic links ("hard links") on both
> systems. Windows is able to use hard links! (through only by using the
> command line)

Just a clarification.
Vista+ has symlinks on NTFS: https://en.wikipedia.org/wiki/Symbolic_link
NT4+ has hardlinks on NTFS: https://en.wikipedia.org/wiki/Hard_link
They are different beasts. And removing existing functionality from symlinks (while retaining for hardlinks) would limit the scenarios that Eike mentioned, by only the same filesystem.

In IRC, I suggested that a top yellow banner could help, that would say something like this: "This file was opened with symlink from a different directory, so its relative links currently may work differently. Click here to reopen it from original location. [Don't show this again] [Read more]". But this was supposed to be too intrusive an UI.
Comment 20 Eike Rathke 2016-08-11 08:27:57 UTC
(In reply to Martin Nathansen from comment #15)
> Each of this use cases breaks when a user creates a desktop shortcut and
> open the documents via this shortcut.
The patch on gerrit addresses the desktop shortcut problem but makes all other use cases where symlinks are actually meant for fail. This is unacceptable.

Are there any means to distinguish a desktop shortcut from a normal symlink so the document could be opened differently by following the link in that case? That would solve all problems.
Comment 21 Martin Nathansen 2016-08-12 14:54:18 UTC
(In reply to Eike Rathke from comment #20)
> The patch on gerrit addresses the desktop shortcut problem but makes all
> other use cases where symlinks are actually meant for fail. This is
> unacceptable.
The bugs I have to fix do not only address desktop shortcuts but I think a desktop-shortcut-only-fix helped a lot.

> Are there any means to distinguish a desktop shortcut from a normal symlink
> so the document could be opened differently by following the link in that
> case? That would solve all problems.
This might be an option for us (needs to be checked internally).

However, there are four other bugs which could be fixed with the current patch:

bug 56137 - LO uses absolute pathes to other document, even with "use relative pathes"-option checked

bug 86087 - FILESAVE FILEOPEN VIEWING: Can't open or save relative links

bug 64431 - FILESAVE: Broken hyperlink to another file in PPT 

bug 45435 - XSLX format breaks relative hyperlink path when document is moved
Comment 22 Martin Nathansen 2016-10-11 13:43:13 UTC
Here the slides of my presentation at the LO conference in Brno:

https://conference.libreoffice.org/assets/Conference/Brno/nathansen-fixing-hyperlink-issues-on-linux.pdf
Comment 23 Martin Nathansen 2016-10-12 15:23:36 UTC
I agree with Noels comment on patch set 5:  "I suspect that this works on WIndows because Windows is converting the shortcut to an absolute path before handing it to LibreOffice, whereas Linux is not."

However, since we cannot fix this issue on the Linux file system, it needs to be fixed in LibreOffice.

For us in Munich there are two regressions regarding this issue:
(1) when migrating from MS Windows to Linux and 
(2) when upgrading from our old Limux Basisclient to a newer one. 

IMO this is really a bug which needs to be fixed in the LO master.
Comment 24 Mike Kaganski 2016-10-13 00:20:04 UTC
(In reply to Martin Nathansen from comment #23)
> I agree with Noels comment on patch set 5:  "I suspect that this works on
> WIndows because Windows is converting the shortcut to an absolute path
> before handing it to LibreOffice, whereas Linux is not."

Yes, that's absolutely correct

> However, since we cannot fix this issue on the Linux file system, it needs
> to be fixed in LibreOffice.
> 
> For us in Munich there are two regressions regarding this issue:
> (1) when migrating from MS Windows to Linux and 
> (2) when upgrading from our old Limux Basisclient to a newer one. 
> 
> IMO this is really a bug which needs to be fixed in the LO master.

Whereas this statement is absolutely incorrect.
This is not a bug.
You (or your users) are used to some way of working with documents, that is offered by Windows: using shortcuts. Shortcuts are files that Explorer.exe (shell) opens, not OS or LO. Please take note of this fact! Explorer.exe (or rather Shell32.dll) parses the contents of the file, reads the original path to file, and then starts associated program with original path to file.

Migrating to Linux, you see its Symlink and expect it to behave the same. And when it does not, you assume it's a bug. But it isn't. Symlink is not, and wasn't designed to be, the same as Windows Shortcut. It isn't processed by shell, but its name is passed to program by shell as is. The program (LO) passes it to OS file management functions. And those functions (on OS level) retarget to original file.

Linux Symlinks work exactly like Windows Symlinks. Only Windows doesn't have UI to work with them, while Linux has. Windows Shortcuts are another feature. It has its counterpart on Linux: shell scripts. You may create shell scripts which contain command to open original file, and put them wherever you need. Just Linux doesn't have convenient UI to cteate such "shortcuts".

Of course, program *is able* to do some extra work by itself to detect that the file is actually symlink, and change its processing. But the real problem is, that doing so, it will disable native mode of operation, that users of the OS  are expecting.

In essence, the correct way (if learning correct way of the OS is not an option) would be to patch your distro's shell to provide a means of creation "proper" shell shortcuts.
This could be supplemented by extending LO command line, e.g. to be able to take shell scripts with hashbang like #!path/to/soffice that would contain name(s) of documents to open. Then, the "shell links patch" would create such scripts.
Comment 25 oiaohm 2016-12-09 03:21:29 UTC
https://blogs.windows.com/buildingapps/2016/12/02/symlinks-windows-10/

This fault does exist on windows as describe exists.   Ok latest Windows 10.

This fault is trigger-able on earlier versions of Windows but done slightly differently.
http://www.howtogeek.com/howto/16226/complete-guide-to-symbolic-links-symlinks-on-windows-or-linux/

You mklink the directory holding the ods file to another location and it contains a relative path using ..

documents/test/calcdoc.ods
documents/linked.pdf
documents/test -> desktop/test mklink created link.
When you go to test on XP/7 desktop in this kind of setup and open the ods file no find PDF.   So this is exactly the same error.

Windows 10 latest development you can stuff it up exactly like Linux.

NTFS file system support symbolic and hard links for a long time if you could create them without manually editing the file system has been the issue.   Yes Windows NT 4 NTFS if you manually edit the file system supports symbolic linked files.

There is a difference in handling between a windows .lnk file and a Linux .desktop file.   A .lnk file is not a Windows symbolic link commonly confused as such but should not be.

http://whatis.techtarget.com/fileformat/LNK-Shortcut-file-Microsoft-Windows-9-x
.lnk are officially shortcuts.  .desktop file is the Linux desktop equal to a .lnk file.

0S X is Aliases, Symbolic Links, and Hard Links yes the Aliases that are short cuts behave like .lnk link files under Windows and .desktop files correctly written under Linux.   Yes on OS X if you go down and really use a Symbolic link you will trigger the same issue.

So .lnk(windows shortcut), Aliases(OS X) and .desktop files(Linux) should generate the same kinds of behaviour.  If they don't then that is a bug. 

My issue is how should we handle the case of symbolic link.  The reality is Symbolic link issue exists on Windows, OS X, Linux most all the other OS Libreoffice runs on.  The difference is user accessibility why users on Linux hit it more often.   Not that the issue is not present.

Working with symbolic links was not considered when ODF was written.

.r.ods or equal extensions where the r is resolve/real would be useful.   So a symbolic link with .r.ods triggers libreoffice to resolve out all the symbolic links until it gets not a non symbolic link path before opening document.  Current default behaviour would be able to be left alone.   There are cases where you may want linked documents and images to change based on where a document is linked from.

Of course doing extension change like this would require altering standard so that all implementations of ODF do the same thing.   Also would require training users to use resolve/real extensions when they don't want symbolic link behaviour.

So this is a bug as a usage case has not been considered.   Person reporting bug was not aware that Windows and OS X has the same fault hidden.

 Martin Nathansen you do need to run some training on the difference between lnk/.desktop shortcuts and symbolic links because staff not knowing the difference are going to cause trouble if they start using newer versions of Windows as well.


-- If you do want to have both working, then use absolute file paths.--
Dennis Roczek please no.   Absolute paths in documents are the absolute worse nightmare when you start having items shared on file servers because people many not mount the file share in the same directory path on every system.  So Absolute path option does not work in every usage case.   We need symbolic links handled sanely.   Also Absolute paths makes documents slightly larger to store as well.

One of the other bugs in ODF is not being able to define relative base path in document that relative paths can work from to give Absolute path behaviour without in fact using a Absolute path everywhere and allowing min modification if file path is altered.

So what we have here is people being told to use Absolute path that is only a part work around to issue that will fail badly in many usage cases.   The fix is truly fixing up symbolic link handling and allowing the two use case of symbolic link in a user controlled way.

So this area of ODF is a problem child.  The standard has failed to consider symbolic link case also really does not handle the case of needing to move a Absolute path file successfully.

The default is Relative Path because that works in more cases than Absolute Path.   Any issue forcing user to use Absolute Path over Relative Path really should be considered a bug if there is some way to allow user to remain using Relative path.   This case extending file extension adding a symbolic link resolve option allows user to remain using Relative Path is a solution.
Comment 26 Mike Kaganski 2016-12-09 04:48:10 UTC
The ODF standard doesn't cover symlinks, yes, because it doesn't have to. That's not its area. File storage isn't what it covers. Don't confuse different things.

The LO behaves correctly with symlinks on all platforms as designed. That some OS made the feature more visible, doesn't mean that this feature magically must start working *like another different unrelated feature*. Symlinks are different, and that is very good, because provides other possibilities for those who need them. If you need different operation, use different feature - shortcut.

Creating an extra mode that would also require specific training is even worse idea.

I suggest to close it again as notabug, and stop the reopening wars.
Comment 27 oiaohm 2016-12-10 02:25:51 UTC
**The LO behaves correctly with symlinks on all platforms as designed. **

Mike Kaganski you are miss it.   Designed as per Libreoffice yes as Designed in ODF and releated standard the answer is absolute no.

The ODF standard is complete horible here.  Yes the standard says you can have hyperlinks.  Then goes and references the xlink and then to href standards that does not cover what to-do in case of symlink.

So we can at the moment have 2 programs that are exactly to ODF standard.   One resolves the symlink out and the other does not and they are both to current standard even that to end user they are behaving totally differently.   So Libreoffice is doing here not to  ODF or any linked standard so is undefined action.

Before fixing or closing this bug requires changing what libreoffice is doing from undefined in standard to defined so that all ODF programs will behave the same when they hit symlink. 

Yes ODF has avoided covering file access and has farmed that out to other linked standards.   Problem is those linked xlink and href standards have a flaw.  Fixing the flaw linked standards will break other standards that define symlink handling one way or the other.  So impossible cat fight.   Simplest solution is just define symlink handling in the ODF standard itself somewhere.  Even if it comes the "ODF application behaviour rules" to guide ODF implementers to doing what was possible undefined to the same defined path to follow.  

There is another option how symlink could be handled.  Of course since the current standard has it undefined we are free to go where ever.

Resolve on missing.    This is where you are using a relative path it attempts open hyperlink as per normal if file is missing goes look a document see if it a symlink or not if a symlink resolve and attempt hyperlink with new location with repeat until either a file is found or is to documents real file with no more symlinks to resolve.

Resolve on missing would give a nice feature when working with a master document and you are changing a few sub documents parts to see how it looks.   Symlink master document to a new working directory and place the changed files in the working directory with resolve on missing it would find the unchanged documents where the linked lead back to.    Resolve on missing would have hidden the issue you reported.

So there are at least 3 different ways current ODF Standard supporting program could decide to handle the event of a Symlink file and relative paths.   End users should not have to guess what one and it should not be possible for conforming programs to put users in location where they have to guess.

Martin Nathansen I know this is not exactly the bug you were thinking of.   So that hyperlinks in ODF documents behaving the same no matter the program processing the standard need to be updated to clearly state what should happen with symlinks.

Since this is a case of update start to properly fix we might as well discus the options. 

**File storage isn't what it covers. Don't confuse different things.**
The standard reference for file access by ODF don't cover the Symlink event as anything other than undefined so do what ever you like.

This is the problem accessing file storage was intentionally put out side the ODF standard and no one bothered checking if the standards used for file storage access from ODF in fact covered every possible problem.  The answer is the standards references from ODF for file access all forgot about symlinks.  In fact those standards did not define what should happen in case of symlinks because it would be too big of a cat fight with the different usage cases.
Comment 28 Mike Kaganski 2016-12-10 03:09:18 UTC
1. There is no other ODF-authoring program that behaves differently. Or am I wrong?

2. There is no need to define in ODF standard the things that are OS/filesystem-dependent. Symlink is such a feature, and all that any program has to know when working with such feature is its name. It needs not to bother with its implementation details; the feature is specifically created such that all the program has to do is to pass the filepath to OS file-handling functions, and get back the contents of the file in return. This is the essence of the symlink. It should not be distinguishable from a usual file from application PoV. Only low-level utilities need to do other things, e.g. archiving software that may need to exclude symlinked stuff from archiving because it's actually located in another volume etc.

A feature being udefined in any standard unrelated to the feature means that this standard assumes that this feature should be used as designed, as defined in other relevant standards. It doesn't mean that e.g. ODF standard makes this "undefined" behavior. It *may* be application-defined behavior, which is also fine. So, LibreOffice defines symlinks treatment as they are intended to be used. I find it awesome.
Comment 29 oiaohm 2016-12-10 05:09:20 UTC
(In reply to Mike Kaganski from comment #28)
> 1. There is no other ODF-authoring program that behaves differently. Or am I
> wrong?
At this stage I don't know of one behaving differently.  But history of how Microsoft breaks ODF says if you leave something undefined they will do it differently to everyone else because they can.  Like using MS Office OOXML formula in ODS instead of openformula because open-formula was not formally defined.  Yes everyone was using openformula in ODS at the time bar Microsoft.

Microsoft windows as only recently got symoblic links we can bet on historic averages they will do the resolve just to be different unless is forbid.   If it not Microsoft it will be someone else at some point.

Also current symlink handling some editing ODF applications that are not Libreoffice is broken.

> 2. There is no need to define in ODF standard the things that are
> OS/filesystem-dependent. Symlink is such a feature, and all that any program
> has to know when working with such feature is its name. It needs not to
> bother with its implementation details; the feature is specifically created
> such that all the program has to do is to pass the filepath to OS
> file-handling functions, and get back the contents of the file in return.
> This is the essence of the symlink. It should not be distinguishable from a
> usual file from application PoV. Only low-level utilities need to do other
> things, e.g. archiving software that may need to exclude symlinked stuff
> from archiving because it's actually located in another volume etc.

You want to place a lock file.   So that the file you have open editing is not edited by someone else.   Place lock file with the name and location of the  symlink is  wrong.   The lock file should be placed in the directory with the real file and every application attempting to open file needs to look in the same place.  Otherwise two or more users could open the file modify and attempt to save alterations at the same time resulting in massive file damage.

Symlinks are particularly designed to be distinguishable and resolvable for very practical reasons.

Every time Libreoffice opens a file it detects as a symlink it resolves out the file back to the original file to place the lock file to place a lock file next to the original file.   For windows 10 now that its displaying symbolic link files Libreoffice will need an update resolve those or libreoffice will have it locking broken by symlinks on windows.   Also not all ODF editing programs out there in fact resolve symlinks when locking the file.

So point of view of editing applications they cannot afford to 100 percent ignore symlinks.


The requirements that an application don't have to care about symlinks.
1) Does not run on any OS that support symlinks.
2) Only access files read only.  Never writes so never need to place a lock file.

OS/filesystem-dependent about symlinks is basically an excuse not to look under the hood.   Basic operations of symlink was defined in posix standards and every OS implementing have basically all done the same thing with all the same downsides.   Reality is OS X, Linux and Windows symlinks are all implementations of the same standard with at times to be annoying different ABI names.   I could say that we could ignore the existence of directories because they are OS/Filesystem-dependent.   Yes it basically the same level of stupidity to say ignore existence of directories when designing as saying ignore existance of symlinks.  href and xlink cover directories and files(referenced standards in ODF standard to use in those cases).

So any OS posix related with symlinks you have the same rules to work to or cause hell for yourself or others.   One application not resolving symlinks to place a lock file with ODF could cause a lot of document damage.  It would be good to be able to say that Application is not to ODF standard at least.

Now if ODF applications are locked differently with each ODF editor this is also going to cause nightmares.

The reality is the path to the real file has to be resolved no matter what because it required to lock the file for editing/writing or to know if file is being modified.   Do we use that information else where as well that is a good question.   The fact real file-name has to be resolved no matter want to place required lock file its not very hard to think of an application taking a short cut and only using the resolved name and completely ignoring the symlink so being completely different to the current libreoffice behaviour.

If the behaviour is not defined in standard variation in behaviour between applications using symlinks with odf should be expected to happen.

Individual application-defined behaviour with symlinks is a very fast way to have hell particularly when they are meant to be handling the same files and users scratching head why did it work with this program but not that one at worse why did I magically lose my data.
Comment 30 Mike Kaganski 2016-12-10 05:27:37 UTC
(In reply to oiaohm from comment #29)
> You want to place a lock file.   So that the file you have open editing is
> not edited by someone else.   Place lock file with the name and location of
> the  symlink is  wrong.   The lock file should be placed in the directory
> with the real file and every application attempting to open file needs to
> look in the same place.  Otherwise two or more users could open the file
> modify and attempt to save alterations at the same time resulting in massive
> file damage.

Have you tried that? That's not true.
The lock file is not the only protection. Actually, the OS will lock the original file so that there will be no way to do that massive damage.
But, of course, another LO trying to open it from original location will not know additional info from the lock file, i.e. who exactly opened it. But then: lock file isn't part of the standard; and another ODF-authoring application (say MS Officce) wouldn't take advantage from the lock file even it is there.

As to MS Office that will do a different thing... it will do that *against* symlink specifications. And has nothing with ODF. You seem to have no idea how standards work. If any standard had to re-do what others do, we would be unable to create standards at all.
Comment 31 oiaohm 2016-12-11 02:26:51 UTC
(In reply to Mike Kaganski from comment #30)
> (In reply to oiaohm from comment #29)
> > You want to place a lock file.   So that the file you have open editing is
> > not edited by someone else.   Place lock file with the name and location of
> > the  symlink is  wrong.   The lock file should be placed in the directory
> > with the real file and every application attempting to open file needs to
> > look in the same place.  Otherwise two or more users could open the file
> > modify and attempt to save alterations at the same time resulting in massive
> > file damage.
> 
> Have you tried that? That's not true.
> The lock file is not the only protection. Actually, the OS will lock the
> original file so that there will be no way to do that massive damage.
> But, of course, another LO trying to open it from original location will not
> know additional info from the lock file, i.e. who exactly opened it. But
> then: lock file isn't part of the standard; and another ODF-authoring
> application (say MS Officce) wouldn't take advantage from the lock file even
> it is there.
This is not understanding the problem.

I have tried it.  I did not mention something.  Where you must put the lock file in right place is when you are using NFS or SMB versions/modes that in fact don't mandate exclusivity or provide server side locking. 

OS will always lock the file is a myth.   Not all operating systems will lock the file and not all file systems will let the OS lock the file.   The file systems that will not let OS lock a file will normal be network file system being using by multi-able users.

https://www.samba.org/samba/docs/man/Samba-HOWTO-Collection/locking.html
Yes it a feature of samba that you can in fact disable file locking and there are registry options in windows to disable OS file locking.   There are ways to turn OS file locking off under OS X and Linux as well.   Microsoft windows home editions come with network file locking disabled.

So absolute yes there is ways to cause massive damage due to the fact the standard does not define a lock file.   A lock file is the only thing that almost always works.  Lock files can suffer from the odd race condition.   Every other form of file locking depends on OS having file locking turn on and in the case of file servers them in fact keeping track of locking and not telling every client yes you have that file locked when really its not locking anything.

So the reality is at the moment every ODF-authoring get to create their own locking file so users have to choose to use 1 ODF-authoring program or risk issues if they have a file server that does not respect locking.

> 
> As to MS Office that will do a different thing... it will do that *against*
> symlink specifications. And has nothing with ODF. You seem to have no idea
> how standards work. If any standard had to re-do what others do, we would be
> unable to create standards at all.

Standards are to create uniform behaviour.   The reality is that every ODF-authoring program is forced to create lock files.   Even MS Office first version creates lock files for .doc files.   So this is universal require behaviour.

Define symlink handling would also be about creating a uniform behaviour for ODF programs.   Symlinks by standards around symlinks can be used 3 different ways.   ODF standard does not state what one of those 3 ways applications using ODF files should be using.    For lock files resolving symlinks is mandatory.


So if MS Office decides to resolve out a symlink and use that this is not against symlink standards.

Three different standard usages of symlink
1) treat location of symlink as location of file.
2) Resolve symlink and use location of real file.
3) Do 1 if item cannot be found do 2.

The reality is Libreoffice is using 1 for hyperlinks and 2 for lock files.

This is not a case of redoing this is a case that you need to state what standard mode of symlink people writing application using ODF should use so that every ODF-authoring program are handling symlinks in the same way.   Yes adding lock files to ODF standard would mean using resolve symlink and stating that for lock files that is what is to be used.

Its the Posix standards that define that symlinks can be used the 3 ways.

The problem here is the claim that ODF standard does not need to care about OS/filesystem-dependent.  The reality here when it comes to symlinks the standard that symlink is from defined 3 options.   Libreoffice is using 2 of the 3 options.

Yes symlink is OS/filesystem-dependent but the standards of symlink define 3 usage modes and ODF does not state what option should be used.  Instead you are presuming just because Libreoffice is using option 1 for hyperlinks that this is some how correct.   The fact Libreoffice is using option 2 for lock files just shows that symlink usage mode is not OS/filesystem only defined.  Instead symlink usage mode is Application or Standard defined. 

Reality using a symlink with ODF since mode of symlink to use is not define in standard is using undefined behaviour.

This is not re-doing every thing everyone has done.   When writing complete standards some of it is finding the list of items you have to put answers to in the standard because they are not absolutely defined by everyone else.    The 3 usage modes of symlink is one of those things.

Claiming symlink as OS/filesystem-dependant got out of having to in fact answer the required question of what mode of use for a symlink should be used when processing an ODF document.   In fact standard could define all 3 different uses of symlink in different places.

Posix and other operating standards only get us so far.  After that then it other standards roles to fill in the gaps.  Symlink usage mode selection is just a gap ODF standard has missed.   Lock file is another thing ODF standard cleanly missed.
Comment 32 Mike Kaganski 2016-12-11 02:37:01 UTC
The discussion went off-topick. If you want to change standards, this is not a place for this. You may submit your proposal to OASIS. Please don't forget to mention there a way to create lockfiles in a write-protected directory where only the opened file is write-allowed.
Comment 33 oiaohm 2016-12-11 07:36:00 UTC
(In reply to Mike Kaganski from comment #32)
> The discussion went off-topick. If you want to change standards, this is not
> a place for this. You may submit your proposal to OASIS. Please don't forget
> to mention there a way to create lockfiles in a write-protected directory
> where only the opened file is write-allowed.

If ODF standard had a lock file defined covering the case of a write protected directory with opened file is write allowed is really simple but not 100 percent safe.  Why ODF is a ZIP file so a directory in it own right so fail location to write lock file is inside the zip file directory itself that has to be write allowed.   This is normally not preferred due to risk of collision.

So yes a lock file being part of standard makes 100 percent sense.  Even possible leaving a preallocated file in a ODF to hold lock data if needed would make sense.

Over the symlink behaviour not matching .lnk file with Hyperlink really need to go up to OASIS for ruling if Libreoffice handling of this case is standard or not.  If what Libreoffice is doing is ruled standard then close this bug as not a bug.  Worst case Libreoffice will have to alter its behaviour to match what .lnk does if OASIS rules that way.

This is why I reopened the bug is there is no ruling to close this bug.  Without a ruling what Libreoffice is doing in the case of this bug could be wrong.   If it is wrong the sooner we find out the better due to the number of users that might be effected.

So there are really two standard alterations that need to go to OASIS.

I am not in the Technical Committee so I will have to submit these two by  office-comment@lists.oasis-open.org a person in Technical Committee could taken the problems straight to office@lists.oasis-open.org to get rulings.

This is why I reopened this bug.  If someone in the Technical Committee is watching they could take over this bug until ruling is got resulting in planned alteration to standard that could be just confirming current behaviour.

Now to get ruling you need to provide why this means I do at times need an open bug.   So I really cannot submit up to OASIS for ruling if the bug here is going to closed since I need this for why ruling is required on the hyperlink one.
Comment 34 Buovjaga 2016-12-11 19:56:42 UTC
(In reply to oiaohm from comment #33)
> This is why I reopened this bug.  If someone in the Technical Committee is
> watching they could take over this bug until ruling is got resulting in
> planned alteration to standard that could be just confirming current
> behaviour.
> 
> Now to get ruling you need to provide why this means I do at times need an
> open bug.   So I really cannot submit up to OASIS for ruling if the bug here
> is going to closed since I need this for why ruling is required on the
> hyperlink one.

Then open the issue in the OASIS issue tracker: https://issues.oasis-open.org/

Closing as NOTABUG for the *final time*. I hope I don't have to ban anyone for reopening this again.
Comment 35 Martin Nathansen 2016-12-13 19:33:24 UTC
> 
> Closing as NOTABUG for the *final time*. I hope I don't have to ban anyone
> for reopening this again.

Buovjaga, why did you close my bug report? I do not file bug reports just for fun, and as you can take from the discussion above there is really a bug which needs to be fixed!
What is you intention behind banning people from the LO project who invest a lot of time and energy to  improve LibreOffice?
Comment 36 Buovjaga 2016-12-13 20:06:24 UTC
(In reply to Martin Nathansen from comment #35)
> > 
> > Closing as NOTABUG for the *final time*. I hope I don't have to ban anyone
> > for reopening this again.
> 
> Buovjaga, why did you close my bug report? I do not file bug reports just
> for fun, and as you can take from the discussion above there is really a bug
> which needs to be fixed!
> What is you intention behind banning people from the LO project who invest a
> lot of time and energy to  improve LibreOffice?

Seriously? You closed it yourself on 2016-10-12 15:38:25. I am reclosing it after oiaohm reopened it.
Comment 37 Martin Nathansen 2016-12-13 21:10:43 UTC
(In reply to Buovjaga from comment #36)
> (In reply to Martin Nathansen from comment #35)
> > > 
> > > Closing as NOTABUG for the *final time*. I hope I don't have to ban anyone
> > > for reopening this again.
> > 
> > Buovjaga, why did you close my bug report? I do not file bug reports just
> > for fun, and as you can take from the discussion above there is really a bug
> > which needs to be fixed!
> > What is you intention behind banning people from the LO project who invest a
> > lot of time and energy to  improve LibreOffice?
> 
> Seriously? You closed it yourself on 2016-10-12 15:38:25. I am reclosing it
> after oiaohm reopened it.

After reopening this bug report a second time I received a warning that I could be banned because of that. So I gave up my bug report and also my attempt to include my patch in the master...
Comment 38 Buovjaga 2016-12-14 08:01:01 UTC
(In reply to Martin Nathansen from comment #37)
> After reopening this bug report a second time I received a warning that I
> could be banned because of that. So I gave up my bug report and also my
> attempt to include my patch in the master...

I do not know of any private warnings you have received. At the conference we discussed this issue and how reopening reports is not a fruitful method of communication about such controversial things. I introduced you to Eike and I thought that you guys figured it out.

Now Mike's later explanations here were very enlightening. This is definitely something Munich has to figure out in their own migration process.
Comment 39 oiaohm 2016-12-25 10:27:49 UTC
(In reply to Martin Nathansen from comment #35)
> > 
> > Closing as NOTABUG for the *final time*. I hope I don't have to ban anyone
> > for reopening this again.
> 
> Buovjaga, why did you close my bug report? I do not file bug reports just
> for fun, and as you can take from the discussion above there is really a bug
> which needs to be fixed!
> What is you intention behind banning people from the LO project who invest a
> lot of time and energy to  improve LibreOffice?

(In reply to Buovjaga from comment #34)
> (In reply to oiaohm from comment #33)
> > This is why I reopened this bug.  If someone in the Technical Committee is
> > watching they could take over this bug until ruling is got resulting in
> > planned alteration to standard that could be just confirming current
> > behaviour.
> > 
> > Now to get ruling you need to provide why this means I do at times need an
> > open bug.   So I really cannot submit up to OASIS for ruling if the bug here
> > is going to closed since I need this for why ruling is required on the
> > hyperlink one.
> 
> Then open the issue in the OASIS issue tracker:
> https://issues.oasis-open.org/
> 
> Closing as NOTABUG for the *final time*. I hope I don't have to ban anyone
> for reopening this again.


Only member of the TC can in fact open issues on the OASIS issue tracker https://issues.oasis-open.org/.

This is the problem this bug need to go to someone who has access rights to TC to submit the issue to on the OASIS issue tracker to be resolved at that level.  To make sure that Libreoffice and everything else ODF handles symlink case the same way.

There will be the odd bug like this one where what is required is a TC member to push the issue up the next level.

Now if Buovjaga you have TC access and can submit the issue in the OASIS issue tracker to clearly define how ODF programs should react in case of symlink and had put a link here to that request I would have left this closed in the first place and corrected the status.  Then the bug in the standard has been transfer from Libreoffice bug list to the standard issue list.

My point of view the required step to close the bug properly and make sure of ODF to ODF application compatibility requires this fault opened on the OASIS issue tracker by some means. 

There have been a few bugs closed like this that should have been transferred to standard body.

At this stage using symlinks is undefined behaviour.   So if someone submitted a patch that resulted in Libreoffice not opening symlinks due to limiting to only ODF define behaviour it would be valid.

Buovjaga please read you closed options.   There is a NOTOURBUG option.

Is what going on a bug in Libreoffice the answer is no.   Is this a issue that the standard does not cover a case and Libreoffice is doing something undefined that could be wrong the answer is yes.   So this is someone elses bug being the standards bug. 

So if you were correct closing this bug the closed is.  CLOSED, NOTOURBUG.

You cannot tell someone to report else where and closed bug with NOTABUG.   NOTABUG means is not a bug for libreoffice or for the standard or anything else.

Part of the reason I reopened was NOTABUG close tag is complete wrong for this type of bug.   Since you say you will ban me if I reopen I will just correct the closing status as you should have got right in the first place.

Reality is closed with NOTABUG means I should not be raising the issue else where.    As so you give directions to report else where remember that means the bug is NOTOURBUG.   I would recommend Buovjaga that you check the bugs you have closed with NOTABUG and make sure there are not more that you have failed to use NOTOURBUG tag when you should have.
Comment 40 oiaohm 2017-01-03 01:21:26 UTC
The core of this bug I should have mentioned.   Is that is too hard under Linux currently to create .desktop files.

xdg-open command was created to allow files to be opened with default application.   So its really not that hard to code up a .desktop file to open a document and have it behave like a OS X alias or Windows lnk/shortcut the issue is Linux/BSD/Unixish file managers and graphical environment currently don't provide this.

After that there is then the issue of opening .desktop files with libreoffice and extracting the document hidden.   Of course that bug in libreoffice cannot be opened until Linux file managers and environment work out exactly how they are going to do the .desktop file in this case.

There are a lot of projects closing bugs with NOTABUG that should be closed with NOTOURBUG and then provide the person complaining about issue where they should be taking the issue.   In this case its file managers and environment issues that are not libreoffice domain.

Symlink issue how it should be handled that is a standard body issue that has not been addressed.   So there are more than 2 real bugs here.

Yes every file manager for Linux basically has a bug when it comes to .desktop files so that is quite a number of bugs.
Comment 41 Martin Nathansen 2017-06-02 12:33:41 UTC
(In reply to oiaohm from comment #40)
> The core of this bug I should have mentioned.   Is that is too hard under
> Linux currently to create .desktop files.
Our 18000 Linux clients have KDE desktop installed with “folder ” layout. On such KDE desktops it is very easy to create desktop shortcuts. The user just clicks on the LibreOffice file, moves it to the desktop and clicks “move here” on the pop-up-menu. That's it.
Creating a desktop shortcut on a KDE desktop is even easer than on MS Windows or at least as easy as there. Thence many of our LibreOffice users make use of desktop shortcuts and as a result I received those  bug reports.

> There are a lot of projects closing bugs with NOTABUG that should be closed
> with NOTOURBUG and then provide the person complaining about issue where
> they should be taking the issue.   In this case its file managers and
> environment issues that are not libreoffice domain.
From users point of view LibreOffice should be able to handle hyperlinks independently of the underlying operating system. It has been much easier to fix this issue in LibreOffice -> https://gerrit.libreoffice.org/#/c/27544/  than to ask all the Linux desktop developers to fix it there. But even this is not the whole truth: just make a test with Firefox and a relative HTML link and you will see that only LibreOffice is not able to resolve relative paths (as most users expect).

> Symlink issue how it should be handled that is a standard body issue that
> has not been addressed.   So there are more than 2 real bugs here.
I agree with you: The handling of relative paths should be discussed in the ODF standardization group. However, with closing my bug report as NOTABUG or NOTOURBUG this issue became invisible to the LibreOffice community and will be forgotten soon...