Bug 65209 - FILEOPEN: MS Word document properties creation times etc. time zones not preserved/displayed
Summary: FILEOPEN: MS Word document properties creation times etc. time zones not pres...
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: filters and storage (show other bugs)
Version:
(earliest affected)
4.0.1.2 release
Hardware: All All
: medium minor
Assignee: Not Assigned
URL:
Whiteboard:
Keywords: difficultyMedium, easyHack, filter:doc, skillCpp
Depends on:
Blocks: DOC
  Show dependency treegraph
 
Reported: 2013-05-31 12:44 UTC by declan_moriarty
Modified: 2022-05-02 14:43 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
Contains test time.doc, test timme MS Word properties.png and test time LibreOffice properties.png showing MS Word having correct time & LO having time 1 hour behind. (75.81 KB, application/zip)
2013-05-31 12:44 UTC, declan_moriarty
Details
Opened in Belgium (GMT + 1 + summer hour). Word for Mac left, LibreOffice right (127.12 KB, image/png)
2013-06-01 10:17 UTC, Jorendc
Details
Screenshot of test time file with LibreOffice 4.3.6.2 on Windows 8.1 (49.43 KB, image/png)
2015-03-07 18:38 UTC, Maarten Hoes
Details
LibreOffice Properties of test time.doc on OS X Mavericks 10.9.5 (73.80 KB, image/tiff)
2015-03-08 21:37 UTC, declan_moriarty
Details

Note You need to log in before you can comment on or make changes to this bug.
Description declan_moriarty 2013-05-31 12:44:31 UTC
Created attachment 80089 [details]
Contains test time.doc, test timme MS Word properties.png and test time LibreOffice properties.png showing MS Word having correct time & LO having time 1 hour behind.

Problem description: 

The Creation/Modified date timestamp displayed in Properties dialogue box shows the GMT time rather than the British Summer Time value for the particular time.  This only affects MS Word .doc files created on a Windows XP machine.  I am using a Windows PC to create proper .doc files and opening them in LibreOffice on an iMac.  Both machines are using British Summer Time and are synchronised to the internet using timeservers.

The MS Word documents generated by Word on the Windows box are one hour behind the correct time on the mac.  When opened in MS Word the same documents show the correct time.

This doesn't affect ODF documents created on the mac.

Using LibreOffice 4.0.0.3 on the PC produces ODF documents dated in LO properties as 16th April 2009 11:32 rather than 31st May 2013 12:32!  Note that the windows clock on the PC shows the correct time.


Steps to reproduce:
1. Create MS Word .doc document (using Word XP/2002) on a Windows XP PC set to BST or DST.
2. Transfere to mac running OS X Lion 10.7.3. (but any Mac OS X will probably make no difference) set to British Summer Time or Daylight Saving Time.
3. Note the file creation and modification dates in the Get Info or info pane for the file.
4. Open the file in LibreOffice and open the properties box.
5. The time in the properties box will be one hour less than the Creation/Modification times in Finder.

Current behaviour:  The creation/modification dates in LO Properties box is 1 hour < actual creation dates in Finder for file opened in LO created in MS Word on PC.

Expected behaviour:  Times in LO properties box match times in Finder and Windows explorer for file being opened.

              
Operating System: Mac OS X
Version: 4.0.1.2 release
Last worked in: 4.0.3.3 release
Comment 1 Jorendc 2013-06-01 10:17:58 UTC
Created attachment 80120 [details]
Opened in Belgium (GMT + 1 + summer hour). Word for Mac left, LibreOffice right
Comment 2 Jorendc 2013-06-01 10:20:50 UTC
Thanks for reporting!

I can confirm this behavior. Actually the mentioned time in LibreOffice is 13:15, the mentioned time in Word for Mac 14:15. I'm in time zone UTC+1 (+ 1 summer hour), so Word for Mac is right (12:15 + 2 hour shift = 14:15).

Tested using Mac OSX 10.8.3 with LibreOffice 4.1.0 beta 1.

kind regards,
Joren
Comment 3 Michael Stahl (allotropia) 2013-06-04 19:36:36 UTC
problem is that LO has never supported timezones in Document Properties.

but i believe it should be possible to do this properly with the
API changes to com.sun.star.util.DateTime for LO 4.1,
likely the Document Properties dialog in sfx2/source/dialog/dinfdlg.cxx
needs a little tweak to evaluate the timezone,
and/or the DocumentSummaryInformation
import/export in sfx2/source/doc/docinf.cxx needs to be taught
about timezones.

removing "regression" due to contradictory version information.
Comment 4 Lionel Elie Mamane 2013-06-05 04:29:31 UTC
(In reply to comment #3)
> problem is that LO has never supported timezones in Document Properties.
> but i believe it should be possible to do this properly with the
> API changes to com.sun.star.util.DateTime for LO 4.1,

Nothing happened wrt to timezone for 4.1. I originally planned to
add timezone support in some way, but it did not happen yet.

I expect that such a support is not necessary for this bug, though. That's how I would do it: If the timestamp (datetime) in the file is given "with timezone", it should be converted to UTC upon reading, stored in memory as UTC and converted to localtime for display.

When writing a file, we can just store the time always as UTC.

> likely the Document Properties dialog in sfx2/source/dialog/dinfdlg.cxx
> needs a little tweak to evaluate the timezone,
> and/or the DocumentSummaryInformation
> import/export in sfx2/source/doc/docinf.cxx needs to be taught
> about timezones.

Reading this file, sfx2/source/doc/oleprops.cxx and tools/source/datetime/datetime.cxx, from the comments the intention of the current design seems to be to convert to *local* time upon reading and then keep everything in local time. That can work too.

I don't see this conversion happening, though:

 - SfxOleFileTimeProperty::ImplLoad seems to read two "raw" 32-bit values
   from the file (the SvStream is the file I presume?)
 - and passes them to DateTime::CreateFromWin32FileDateTime
 - which possibly buggily (e.g. it ignores leap seconds) converts that
   to a ::DateTime

But possibly I'm missing it in some other place.

This seems to suggest that the file does not contain any timezone information after all... So the bug is in being *consistent* throughout internal LibreOffice code (either use localtime everywhere, or UTC everywhere), or in the conversion to localtime for display, or something like that.
Comment 5 Björn Michaelsen 2013-10-04 18:46:08 UTC
adding LibreOffice developer list as CC to unresolved EasyHacks for better visibility.

see e.g. http://nabble.documentfoundation.org/minutes-of-ESC-call-td4076214.html for details
Comment 6 Souvik Chakraborty 2015-03-07 16:51:30 UTC
Hi!

I'd like to help with this.

Will tweaking only the two mentioned files resolve this or do things need to be fixed throughout the libreoffice code?
Comment 7 Maarten Hoes 2015-03-07 18:38:19 UTC
Created attachment 113961 [details]
Screenshot of test time file with LibreOffice 4.3.6.2 on Windows 8.1

I cannot reproduce this with LibreOffice 4.3.6.2 on Windows 8.1.
Comment 8 Tomaz Vajngerl 2015-03-08 03:30:49 UTC
(In reply to Souvik Chakraborty from comment #6) 
> Will tweaking only the two mentioned files resolve this or do things need to
> be fixed throughout the libreoffice code?

You have the choice.. resolve the two mentioned files + every other people's file that are wrong (involves visiting every possible LibreOffice user) - HARD,
or fix the cause of this bug in the source code - EASY.
Comment 9 Souvik Chakraborty 2015-03-08 06:36:44 UTC
(In reply to Tomaz Vajngerl from comment #8)
> or fix the cause of this bug in the source code - EASY.

Okay. Let me see what can be done...
Comment 10 declan_moriarty 2015-03-08 21:37:55 UTC
Created attachment 113984 [details]
LibreOffice Properties of test time.doc on OS X Mavericks 10.9.5

I have produced a new attachment that shows I am able to reproduce this bug with the latest version of LO 4.4.0.3 on OS X Mavericks 10.9.5.
Comment 11 Maarten Hoes 2015-03-08 21:44:04 UTC
(In reply to Maarten Hoes from comment #7)
> Created attachment 113961 [details]
> Screenshot of test time file with LibreOffice 4.3.6.2 on Windows 8.1
> 
> I cannot reproduce this with LibreOffice 4.3.6.2 on Windows 8.1.


(In reply to declan_moriarty from comment #10)
> Created attachment 113984 [details]
> LibreOffice Properties of test time.doc on OS X Mavericks 10.9.5
> 
> I have produced a new attachment that shows I am able to reproduce this bug
> with the latest version of LO 4.4.0.3 on OS X Mavericks 10.9.5.

I guess this means that it is reproducible on OS X, but not Windows 8.1. Setting the OS from 'All' to 'MAC OS X'.
Comment 12 Robinson Tryon (qubit) 2015-12-13 10:00:13 UTC Comment hidden (obsolete)
Comment 13 Robinson Tryon (qubit) 2016-02-18 14:52:05 UTC Comment hidden (obsolete)
Comment 14 nassimedu92 2017-01-06 09:49:25 UTC
Hello, we try my friends and me to resolve this bug , I said we try lol. But I would have some explications about the SvStream file please
Comment 15 jani 2017-01-06 09:54:19 UTC
(In reply to nassimedu92 from comment #14)
> Hello, we try my friends and me to resolve this bug , I said we try lol. But
> I would have some explications about the SvStream file please


Why do you need SvStream, look at comment 3, it tell you which files to work with.
Comment 16 Yassine 2017-03-07 17:02:15 UTC
Changing status to "All operating systems", after successfully reproducing the bug on Ubuntu GNU/Linux.
Comment 17 Yassine 2017-03-07 18:05:01 UTC
Steps to reproduce under GNU/Linux:
 - Modify the timezone in your terminal with the following command: 
  sudo dpkg-reconfigure tzdata
 - Create a new file and save it to disk
 - Modify again the timezone and go to your normal timezone again with the same command: sudo dpkg-reconfigure tzdata
 - Open the newly created file, go to File -> Properties and then see that the timezone was not modified according to the change into the timezone.
Comment 18 Yassine 2017-03-20 16:02:18 UTC
While trying to find the place into the code where this issue is situated, we've found multiple files related to datetime

- /core/include/tools/datetime.hxx
- /tools/source/datetime/datetime.cxx
- /core/tools/source/datetime/datetimeutils.cxx
- /core/include/sfx2/dinfdlg.hxx
- /core/sfx2/source/doc/docinfo.cxx
- /core/sfx2/source/dialog/dinfdlg.cxx
- /core/sax/qa/cppunit/test_converter.cxx
- /core/unotools/source/misc/datetime.cxx
- /core/ucb/source/ucp/webdav-neon/DateTimeHelper.cxx
- /core/workdir/UnoApiHeadersTarget/offapi/normal/com/sun/star/util/DateTime.hpp

Could someone point us to the exact file where the date and time are saved into the properties of the document.

Could someone also explain the logic behind the file properties and how they are recorded/loaded while closing/opening a file. 

Thanks for the help
Comment 19 Lionel Elie Mamane 2017-03-20 16:06:20 UTC
(In reply to Yassine from comment #18)
> While trying to find the place into the code where this issue is situated,

> Could someone point us to the exact file where the date and time are saved
> into the properties of the document.

Please read comment 3 and comment 4.
Comment 20 Michael Stahl (allotropia) 2018-10-26 21:39:06 UTC
follow up to comment #3 and comment #4:

some reading reveals that there is exactly 1 timezone in MSO binary files: the type FILETIME is always UTC, as specified in [MS-DTYP] https://msdn.microsoft.com/en-us/library/cc230273.aspx "2.3.3 FILETIME".

hence the import code in oleprops.cxx should set the util::DateTime value to UTC, and the SfxDocumentInfoDialog may need to either display an UTC marker or convert to local time.
Comment 21 Rakshita Varadarajan 2021-02-13 06:51:31 UTC Comment hidden (obsolete)
Comment 22 Rakshita Varadarajan 2021-02-13 06:51:53 UTC Comment hidden (obsolete)
Comment 23 Buovjaga 2021-02-13 07:13:51 UTC
(In reply to Rakshita Varadarajan from comment #22)
> Hello! Can I work on this issue? I just reproduced the error in Ubuntu 20.04.

As you can see from the comments and status, nobody is working on this as of now.
Comment 24 Rakshita Varadarajan 2021-02-13 10:26:08 UTC
(In reply to Buovjaga from comment #23)
> (In reply to Rakshita Varadarajan from comment #22)
> > Hello! Can I work on this issue? I just reproduced the error in Ubuntu 20.04.
> 
> As you can see from the comments and status, nobody is working on this as of
> now.

Okay great, I'll get started
Comment 25 Rakshita Varadarajan 2021-02-14 18:53:15 UTC
(In reply to Michael Stahl (allotropia) from comment #20)
> follow up to comment #3 and comment #4:
> 
> some reading reveals that there is exactly 1 timezone in MSO binary files:
> the type FILETIME is always UTC, as specified in [MS-DTYP]
> https://msdn.microsoft.com/en-us/library/cc230273.aspx "2.3.3 FILETIME".
> 
> hence the import code in oleprops.cxx should set the util::DateTime value to
> UTC, and the SfxDocumentInfoDialog may need to either display an UTC marker
> or convert to local time.

Hello,

I tried doing this, ie, changing util::DateTime value to UTC, but it is not working and I am stuck.
The 3 main changes in oleprops.cxx:

SfxOleFileTimeProperty::SfxOleFileTimeProperty( sal_Int32 nPropId, const util::DateTime& rDateTime ) :
    SfxOlePropertyBase( nPropId, PROPTYPE_FILETIME )
{   
    DateTime trialDateTime(rDateTime);
    trialDateTime.ConvertToUTC();
    maDateTime.Year    = trialDateTime.GetYear();
    maDateTime.Month   = trialDateTime.GetMonth();
    maDateTime.Day     = trialDateTime.GetDay();
    maDateTime.Hours   = trialDateTime.GetHour();
    maDateTime.Minutes = trialDateTime.GetMin();
    maDateTime.Seconds = trialDateTime.GetSec();
    maDateTime.NanoSeconds = trialDateTime.GetNanoSec();
    maDateTime.IsUTC   = true;
}

void SfxOleFileTimeProperty::ImplLoad( SvStream& rStrm )
{
    sal_uInt32 nLower(0), nUpper(0);
    rStrm.ReadUInt32( nLower ).ReadUInt32( nUpper );
    ::DateTime aDateTime = DateTime::CreateFromWin32FileDateTime( nLower, nUpper );
    // note: editing duration is stored as offset to TIMESTAMP_INVALID_DATETIME
    //       of course we should not convert the time zone of a duration!
    // heuristic to detect editing durations (which we assume to be < 1 year):
    // check only the year, not the entire date
    if ( aDateTime.GetYear() != TIMESTAMP_INVALID_DATETIME.GetYear() )
        aDateTime.ConvertToUTC();
    maDateTime.Year    = aDateTime.GetYear();
    maDateTime.Month   = aDateTime.GetMonth();
    maDateTime.Day     = aDateTime.GetDay();
    maDateTime.Hours   = aDateTime.GetHour();
    maDateTime.Minutes = aDateTime.GetMin();
    maDateTime.Seconds = aDateTime.GetSec();
    maDateTime.NanoSeconds = aDateTime.GetNanoSec();
    maDateTime.IsUTC   = true;
}

void SfxOleSection::SetFileTimeValue( sal_Int32 nPropId, const util::DateTime& rValue )
{
    if ( rValue.Year == 0 || rValue.Month == 0 || rValue.Day == 0 )
        SetProperty( std::make_shared<SfxOleFileTimeProperty>( nPropId, TIMESTAMP_INVALID_UTILDATETIME ) );
    else{
        DateTime trialDateTime(rValue);
    trialDateTime.ConvertToUTC();
    util::DateTime rVal;
    rVal.Year    = trialDateTime.GetYear();
    rVal.Month   = trialDateTime.GetMonth();
    rVal.Day     = trialDateTime.GetDay();
    rVal.Hours   = trialDateTime.GetHour();
    rVal.Minutes = trialDateTime.GetMin();
    rVal.Seconds = trialDateTime.GetSec();
    rVal.NanoSeconds = trialDateTime.GetNanoSec();
    rVal.IsUTC   = true;
        SetProperty( std::make_shared<SfxOleFileTimeProperty>( nPropId, rVal ) );
    }
}

However, it is still not working.

On adding ConvertToUTC() in SfxDocumentInfoDialog, it shows the proper UTC time, however fundamentally it is still being saved wrong to the file. Is there any guidance you can provide here? I am trying really hard to see what I may be missing
Comment 26 Michael Stahl (allotropia) 2021-02-15 10:38:41 UTC
hi Rakshita,

i wonder, looking at the SfxOleFileTimeProperty methods, how many times is ConvertToUTC called when storing such a property?
Comment 27 Rakshita Varadarajan 2021-02-15 12:28:59 UTC
(In reply to Michael Stahl (allotropia) from comment #26)
> hi Rakshita,
> 
> i wonder, looking at the SfxOleFileTimeProperty methods, how many times is
> ConvertToUTC called when storing such a property?

AIn the current repo code, it is being called only once in the ImplSave method
 
The constructor SfxOleFileTimeProperty( sal_Int32 nPropId, const util::DateTime& rDateTime ) directly initializes the private variable of util::DateTime to this value got in the constructor, however this time is local time so I changed the code to ConvertToUTC and then initialized the util::DateTime maDateTime variable. 
Another change I did was in the method ImplLoad, it initially converted everything to local time and fed values to maDateTime, so once more I used ConvertToUTC 

I feel I may be missing something, but I am still making changes and trying my best
Comment 28 Michael Stahl (allotropia) 2021-02-16 10:47:30 UTC
(In reply to Rakshita Varadarajan from comment #27)

util::DateTime has an IsUTC flag, right? so i don't understand the point of unconditionally converting a util::DateTime from local time to UTC.

also, can it happen that ImplSave and SetFileTimeValue are both called on storing the file? how many conversions would that be?
Comment 29 Rakshita Varadarajan 2021-02-16 17:25:25 UTC
(In reply to Michael Stahl (allotropia) from comment #28)
> (In reply to Rakshita Varadarajan from comment #27)
> 
> util::DateTime has an IsUTC flag, right? so i don't understand the point of
> unconditionally converting a util::DateTime from local time to UTC.
> 
> also, can it happen that ImplSave and SetFileTimeValue are both called on
> storing the file? how many conversions would that be?

Yes my bad, I realized it last night and added the condition that when IsUTC flag is false, it converts from local to UTC.

Yes, right? They will both be called, and it will be called twice then. 
So I changed the code and removed the converting to UTC in SetFileTimeValue, instead kept it in ImplSave itself. The code still fails. 

I had a doubt: ideally, if there is a static value fed to maDateTime in ImplSave, then the DateTime value that should be shown in the dialogue box should be that static value right? however, it is showing instead the local time of storing the file.
Is it possible there is something else interfering with this process?
Comment 30 Xisco Faulí 2022-05-02 14:43:15 UTC
Dear  Rakshita Varadarajan,
This bug has been in ASSIGNED status for more than 3 months without any
activity. Resetting it to NEW.
Please assign it back to yourself if you're still working on this.