Bug Hunting Session
Bug 67763 - date & time controls: fractional seconds, timezone support, duration, interval
Summary: date & time controls: fractional seconds, timezone support, duration, interval
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: LibreOffice (show other bugs)
Version:
(earliest affected)
unspecified
Hardware: All All
: medium enhancement
Assignee: Not Assigned
URL:
Whiteboard:
Keywords: difficultyInteresting, easyHack, skillCpp, skillSql, skillUI, skillVcl, topicUI
: 122863 (view as bug list)
Depends on:
Blocks: Fields
  Show dependency treegraph
 
Reported: 2013-08-05 03:24 UTC by Lionel Elie Mamane
Modified: 2019-07-31 14:34 UTC (History)
8 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Lionel Elie Mamane 2013-08-05 03:24:11 UTC
In 4.1, we changed our dat&time-related API datatypes to support:

 - negative year

 - nanosecond resolution

 - timezone

The next step is to update our UI elements to support these new features,
as well as introduce UI elements for durations and intervals.


Negative year
=============

Need to decide if negative years are meant astronomically (that is,
before year 1 is year 0) or traditionally (that is, before year 1 is
year -1).

Our code calling OS libraries (in sal/osl/) possibly to be adapted.

Test whether our date formatting is prepared for negative values.


Nanosecond precision
====================

We have two ways to display a timestamp (datetime):

1) "formatted field" with a date format; it works with concepts
   inherited from spreadsheets, namely that everything is a double
   (floating-point number); dates are a number of days since an epoch
   in the late 19th or early 20th century.

   These can already do "arbitrary precision" formatting of timestamps
   with a user-defined format ("just" add ".#########" to the end of
   the format), but the "double" floating-point precision is not
   adequate for full nanosecond resolution, and behaves weirdly anyway
   in that the available precision depends on the distance of the date
   from the epoch.  For the commonest example of dates in the late20th
   / early 21st century, the value of the double is around 4E4
   ( = 4*10^4 = 40000).

   The mantissa has 53 bits of precision; getting to seconds precision
   takes about 32 bits, leaving about 20 bits for the fractional
   seconds.  This around to a precision of the order of magnitude of
   microseconds.

   But dates within a few days of the epoch have a "full" nanosecond
   precision (and beyond) ;)

2) Dedicated "time" and "date" fields, but no single "timestamp"
   control. In base (forms), splitting a datetime into its date and
   time components works (there is special support code for that; each
   control will "touch" only its part of the timestamp).


Wrt 1, we could extend formatted fields to quad floating-point
precision, but IMHO that's just adding to the hackish approach. Unless
Calc decides to do it for its own reasons, I don't think it would be
worth it.


Wrt 2, we need to add cases to
offapi/com/sun/star/text/TimeDisplayFormat.idl for higher
precision. Which ones? Maybe one each for milli, micro and
nanoseconds (times two, for 24-hours and AM/PM variants)?

The implementation is in:

 xmloff/source/forms/handler/vcl_time_handler.*
 (conversion between XML attributes and our internal datatypes)

 include/toolkit/controls/unocontrols.hxx
 include/toolkit/controls/unocontrolbase.hxx
 include/toolkit/controls/unocontrolmodel.hxx
 toolkit/source/controls/unocontrolbase.cxx
 toolkit/source/controls/unocontrolmodel.cxx
 toolkit/source/controls/unocontrols.cxx
 (classes UnoControlTimeFieldModel, UnoTimeFieldControl)

 include/toolkit/awt/vclxwindows.hxx
 toolkit/source/awt/vclxwindows.cxx
 (class class VCLXTimeField)

 include/vcl/field.hxx
 vcl/source/control/field2.cxx
 (classes TimeFormatter, TimeField, TimeBox)
 
 forms/source/component/Time.*
 (classes OTimeControl, OTimeModel)

 svx/source/fmcomp/gridcell.cxx
 (class DbTimeField)

commit 43ea97e1f9cecd6c7cba8db35ce1307c858c6857
Author: Lionel Elie Mamane <lionel@mamane.lu>
Date:   Sun Jul 28 16:08:26 2013 +0200

    fdo#67235 adapt form control code to time nanosecond API change

gives a good overview of the kind of things that need to change.


Timezones
=========

Something like:

 1) Add a property "which timezone should the the display be in".
    (Can be "none" which is different than 0.)

 2) Add a property "assume that untimezoned data from the database is
    in timezone XXX".

 3) Handle conversions in the code.

Do we also want an option "show timezone explicitly"? That would IMO
probably be in a different control, not a setting of the existing
control.


For timestamps, it makes the split in date and time more complicated
(because the timezone can impact both the date and the time), but
possibly still workable. We could also introduce a "datetime" control,
which would IMHO be cleaner / nicer.

What is a timezone in the new property? For isolated times, I think
only "fixed offset to UTC" make sense, and maybe a few special values
like "SYSTEM". But for full datetime, the user would like to enter
something like a city, and have the timezone at that date in the city,
taking into account daylight-saving time and everything. Which
introduces the parsing ambiguity of one hour when going from DST to
winter time...

How to implement that? Ship a copy of the IANA Olson / tz / zoneinfo
database? Rely on the one from the system?

For date-related code, take the list of files and classes above and
replace "Time" by "Date".


Durations and Intervals
=======================

Duration = length of time (e.g. 3 days, 2 hours, 5 minutes and 30s)

Interval = one start datetime and one end datetime
           (e.g. from 4 may 2013 8:00:00 o'clock to 7 september 2013 18:00:00)
           isomorphic to datetime+duration

Take inspiration of our date&time controls to make duration control
and interval control.
Comment 1 Lionel Elie Mamane 2013-08-05 17:33:24 UTC
To save/load the properties of the controls saved to file correctly,
also need to adapt:

1) UnoControlModel::read/write in file toolkit/source/controls/unocontrolmodel.cxx

2) xmlscript/source/xmldlg_imexp/xmldlg_(imp|exp)(models|port).cxx


Wrt to 1, see http://cgit.freedesktop.org/libreoffice/core/commit/?id=700d1cab988524b28c7ce773cf473f42d7741001

 change UNOCONTROL_STREAMVERSION to 3
 change the code introduced in UnoControlModel::write to write out the whole data structures
 change the code introduced in UnoControlModel::read to read:
  - as it does now (backwards compatibility) if nVersion < 3
  - else, the full data structures as written in ::write


Wrt to 2, see http://cgit.freedesktop.org/libreoffice/core/commit/?id=3b388abc1e9d75a045cc0f6ca9306ff3336251bc

 change to code introduced there to write the whole data structures as XMLSchema-2 values
 (like the forms code does)

 need to adapt the XML schemas / DTDs / whatever?
Comment 2 Lionel Elie Mamane 2013-08-05 17:46:49 UTC
> Wrt to 2, see
> http://cgit.freedesktop.org/libreoffice/core/commit/
> ?id=3b388abc1e9d75a045cc0f6ca9306ff3336251bc
> 
>  change to code introduced there to write the whole data structures as
>  XMLSchema-2 values (like the forms code does)

Obviously the reading/parsing code need to sniff whether there is an
integer there or an XMLSchema-2 date/time/duration/... and adapt
accordingly.

We could also change the attribute names to e.g. time-max instead of value-max. Maybe this will help with the DTD, and actually also with the parsing.
Comment 3 Björn Michaelsen 2013-10-04 18:46:25 UTC
adding LibreOffice developer list as CC to unresolved EasyHacks for better visibility.

see e.g. http://nabble.documentfoundation.org/minutes-of-ESC-call-td4076214.html for details
Comment 4 Marcel Adriani 2014-05-02 11:06:14 UTC
Just letting you know that I am having a go at this. Started with the wrt2-branch.
Comment 5 Lionel Elie Mamane 2014-05-02 11:13:03 UTC
(In reply to comment #4)
> Just letting you know that I am having a go at this. Started with the
> wrt2-branch.

Great, I'm assigning it to you then. Let me know if you have any question.
Comment 6 Marcel Adriani 2014-05-02 13:02:08 UTC
(In reply to comment #5)
> Started with the wrt2-branch.
Working my way through the wrt-2 branch, I notice that offapi/com/sun/star/text/TimeDisplayFormat.idl is marked af deprecated. Are you sure that is the right start-point? Thx for yout input@
Comment 7 Marcel Adriani 2014-05-02 13:17:13 UTC
(In reply to comment #6)
> I notice that
> offapi/com/sun/star/text/TimeDisplayFormat.idl is marked af deprecated. Are
> you sure that is the right start-point? 

Is now suspect TimeDisplayFormat.hdl is the way to go. That's where I'm going now.
Comment 8 Lionel Elie Mamane 2014-05-02 14:32:12 UTC
(In reply to comment #6)
> (In reply to comment #5)
> > Started with the wrt2-branch.
> Working my way through the wrt-2 branch, I notice that
> offapi/com/sun/star/text/TimeDisplayFormat.idl is marked af deprecated. Are
> you sure that is the right start-point? Thx for yout input@

At first, I rather suspected that the "deprecated" flag is in error, since it was marked as deprecated when it was introduced (14 Nov 2000, commit ID 20b69e51231b64c9c595ae40dc2c2bb76a8c4e0e).

But then, this constant group is used only in service ::com::sun::star::text::textfield::DateTime, where the property where it is deprecated anyway.

OTOH, concretely the choices in offapi/com/sun/star/text/TimeDisplayFormat.idl are the choices one has in the GUI when choosing the format of a time field or a date field. So - to me - they seem to be in active use within LibreOffice proper, which is rather incompatible with deprecation. Unless I'm wrong and t

Also, the listed alternative (using a NumberFormatter) means shoe-horning a date / time in a double, which leads to the problems listed in comment 0.

Grr... I guess we need to look at this very carefully and maybe un-deprecate it.

Ah no... I think I was just wrong... I thought the list of formats of a date/time control was a value of ::com::sun::star::text::textfield::DateTime, but actually it seems to be listed in forms/source/misc/limitedformats.cxx, function lcl_getFormatTable. So possibly that's the right starting point. But (same file) function OLimitedFormats::ensureTableInitialized, the description is subject to a lookup by "xStandardFormats->queryKey", so (I guess) these "Standard Formats" will have to be extended. Need to find out where they are, though. I guess this corresponds to the "FormatCode" in i18npool/source/localedata/data/*.xml, which would need update for all languages (which in hindsight seems natural). Obviously, those locales that you don't know will be "done" by our i18n teams.

(In reply to comment #7)
> (In reply to comment #6)
> > I notice that
> > offapi/com/sun/star/text/TimeDisplayFormat.idl is marked af deprecated. Are
> > you sure that is the right start-point? 
> 
> Is now suspect TimeDisplayFormat.hdl is the way to go. That's where I'm
> going now.

TimeDisplayFormat.hdl is generated automatically from  TimeDisplayFormat.idl
Comment 9 Marcel Adriani 2014-05-03 13:04:48 UTC
Working on:
2. Dedicated "time" and "date" fields, but no single "timestamp"
   control. In base (forms), splitting a datetime into its date and
   time components works (there is special support code for that; each
   control will "touch" only its part of the timestamp).
I discovered that the time-format mask comes from the i18n language .XML, which gets abstacted in offapi/com/sun/star/i18n/NumberFormatIndex.idl. This is implemented in such a way that changing it (for instance to make room for time-masks with micro- milli- and nanosecond precision) is virtually impossible. I am going to let that rest for now, and work on a proposal to make the way the language files are abstacted more flexible.
Comment 10 Marcel Adriani 2014-07-19 10:58:27 UTC
It's been some time since I looked at this. Turns out I need to work for a living :-(.

I have to admit, though, that this bug is beyond me. Two reasons:

- for a first try at coding for LibreOffice, the piled-up levels of abstractions in the number-formatting sections are staggering. This means that you can never be sure wether one change is going to bring the entire cardhouse down.
- For this bug, changes need to be made in a part of the program that gets used everywhere. I cannot grasp the implications of any change made.

I am going to resign as assignee. I suggest the qualificatie easyhack is dropped: this is not that.
Comment 11 Lionel Elie Mamane 2014-07-19 12:39:19 UTC
(In reply to comment #10)

> I am going to resign as assignee. I suggest the qualificatie easyhack is
> dropped: this is not that.

It is marked as a *difficult* one (DifficultyInteresting)
Comment 12 Alex Thurgood 2015-01-03 17:40:01 UTC Comment hidden (no-value)
Comment 13 Robinson Tryon (qubit) 2015-12-14 04:55:01 UTC Comment hidden (obsolete)
Comment 14 Robinson Tryon (qubit) 2016-02-18 14:51:30 UTC Comment hidden (obsolete)
Comment 15 Fakabbir amin 2017-01-26 15:53:36 UTC
Hi, I would like to work on this.
Comment 16 Lionel Elie Mamane 2017-01-26 16:04:34 UTC
(In reply to Fakabbir amin from comment #15)
> Hi, I would like to work on this.

Great, welcome aboard. Read the explanations in the first comments on this bug. Let me know of any question you have and any progress you make.
Comment 17 Heiko Tietze 2017-01-26 19:52:05 UTC
Added a mockup to the related bug 72662, just in case you want to add an UI. Also relevant are bug 44267 and bug 105176 if you plan to implement various units (e.g. switch timezone per control) and have a defined precision (e.g. seconds in datetime controls). 

If that sounds too confusing just ignore my comment. Easyhacks should remain easy.
Comment 18 jani 2017-01-27 06:59:15 UTC
Please do not expand the scope of an easyhack, not even optionally. Instead make a new one that depends on this one, since it is clearly to different skill sets needed.
Comment 19 Fakabbir amin 2017-01-27 15:50:42 UTC
(In reply to Lionel Elie Mamane from comment #16)
> (In reply to Fakabbir amin from comment #15)
> > Hi, I would like to work on this.
> 
> Great, welcome aboard. Read the explanations in the first comments on this
> bug. Let me know of any question you have and any progress you make.

Hi Mamane,

I had went through the description and progress discussed in comment section. It is quite interesting as changes needed to be made in part of the program that gets used everywhere.
Right now, I am skimming over the code pointers (Thanks for that) and trying to get where Adriani was struck. 

I guess, we need to work separately for this RATHER THAN depending mostly on the implementation already available. 
Since this could be very helpful in implementing timezone challenge.
Comment 20 jani 2017-05-14 07:42:47 UTC Comment hidden (obsolete)
Comment 21 Fakabbir amin 2017-05-14 11:41:35 UTC
(In reply to jani from comment #20)
> A polite ping, still working on this bug


Reverting status to NEW, due to other engagement.
Comment 22 Alex Thurgood 2017-08-25 13:45:33 UTC
Removing easyhack keyword - 2 people have assigned themselves this work in the past and neither has been able to find the time to unwind the code and find a solution, which would suggest that the solution is not "easy".
Comment 23 Alex Thurgood 2019-01-22 09:07:33 UTC
*** Bug 122863 has been marked as a duplicate of this bug. ***
Comment 24 Xisco Faulí 2019-07-17 17:01:07 UTC
(In reply to Alex Thurgood from comment #22)
> Removing easyhack keyword - 2 people have assigned themselves this work in
> the past and neither has been able to find the time to unwind the code and
> find a solution, which would suggest that the solution is not "easy".

I disagree, it's an easyhack + difficultyInteresting. A newcomer shouldn't take this one as the first hack, instead, it should be the natural evolution of someone with some beginning easyhacks upon their shoulders