Bug 143664 - sdf date time recognition fails for march ("Mär") in German
Summary: sdf date time recognition fails for march ("Mär") in German
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
Inherited From OOo
Hardware: All All
: medium minor
Assignee: Eike Rathke
URL:
Whiteboard: target:7.3.0 target:7.2.1 target:7.1.6
Keywords:
Depends on:
Blocks:
 
Reported: 2021-08-01 08:52 UTC by Vollbracht
Modified: 2021-10-18 20:57 UTC (History)
2 users (show)

See Also:
Crash report or crash signature:


Attachments
RKI data sample sdf (3.86 KB, text/plain)
2021-08-01 09:06 UTC, Vollbracht
Details
RKI data sample ods result (16.48 KB, application/vnd.oasis.opendocument.spreadsheet)
2021-08-01 09:17 UTC, Vollbracht
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Vollbracht 2021-08-01 08:52:57 UTC
When using "Textimport" for a table containing a date column, I select this column and chose a column type date "Datum (TMJ)". This works for "31.03.2021" as well as for "01.Apr 2021", "01.Mai 2021", "01.Dez 2020", etc. but nor for "31.Mär 2021" as used in RKI Dash Board, e.g..
Comment 1 Vollbracht 2021-08-01 09:06:49 UTC
Created attachment 174008 [details]
RKI data sample sdf

strange: when opening from within else but Calc it's not recognized as table at all. When opening from within Calc, it is. May be intended.
Comment 2 Vollbracht 2021-08-01 09:17:39 UTC
Created attachment 174009 [details]
RKI data sample ods result
Comment 3 Julien Nabet 2021-08-01 09:45:43 UTC
On pc Debian x86-64 with master sources updated today + German from Germany set for UI + locale, I could reproduce this.

On i18npool/source/localedata/data/de_DE.xml, there's:
        <Month>
          <MonthID>mar</MonthID>
          <DefaultAbbrvName>Mrz</DefaultAbbrvName>
          <DefaultFullName>März</DefaultFullName>
        </Month>
See https://opengrok.libreoffice.org/xref/core/i18npool/source/localedata/data/de_DE.xml?r=8453f6e5#268

Eike: any thoughts here? Is it expected?
Comment 4 Eike Rathke 2021-08-02 11:47:09 UTC
There are several short month names / abbreviations for de-DE März; "Mrz.", "Mär." and even "März".
Similarly Juni and Juli can be used as short month names as well.
This can only be hard-coded in the number/date scanner as locale data doesn't allow for lists of month names.
Comment 5 Eike Rathke 2021-08-02 12:05:43 UTC
The current recommendation of DIN 5008 (standardization of written correspondence) is "Mär."/"März", see
https://www.din-5008-richtlinien.de/startseite/datum/

My take would be to change locale data to "Mär" and in code add "Mrz" to be able to parse that back in.
Then again that DIN 5008 lists abbreviated month names as

Jan., Feb., Mär./März, Apr., Mai, Jun./Juni, Jul./Juli, Aug., Sept., Okt., Nov., Dez.

so we'd need to add abbreviation dots as well, which in turn would need checking date formats for extraneous dots.

Odd is "Sept." because including the dot it's 5 characters instead of 4, where that DIN 5008 talks about always 4 places ("immer 4-stellig").
If we did that then also the shorter "Sep" would need to be recognized in the scanner code (trailing abbreviation dots are always taken into account there).
Comment 6 Julien Nabet 2021-08-02 15:59:15 UTC
Thank you Eike for your feedback!
It seems locale data should be able to deal with lists to have something clean instead of increasing a kludge.

Vollbracht: as a workaround, you may change "Mär" by "Mrz" or "März" so it'll be recognized as a date.
Comment 7 Eike Rathke 2021-08-02 17:39:08 UTC
@Julien:
While enabling lists for short month names may sound appealing, doing so would be a cumbersome endeavour. All day and month names share a common structure (css::i18n::CalendarItem2 inherited from css::i18n::CalendarItem) that is also part of the published API. Changing AbbrevName to a list of strings would require not only to adapt all callers but also to support it as an additional new structure while maintaining the existing API for compatibility.
I think these few exemptions so far are not worth that.
Comment 8 Commit Notification 2021-08-02 23:12:10 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/4961dbaecc9e5cac57d99d2ea9d265a90daa4a8b

Resolves: tdf#143664 {de-*} accept "Mär" and "Mrz" for March

It will be available in 7.3.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 9 Commit Notification 2021-08-02 23:12:22 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/400a44f9ac7792ffe3665f70507324e4a64dca2b

Related: tdf#143664 Change {de-*} March "Mrz" to "Mär"

It will be available in 7.3.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 11 Commit Notification 2021-08-04 10:35:17 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "libreoffice-7-2":

https://git.libreoffice.org/core/commit/5dd4c279a6bb6f297f1c02f2aeb905a81e5fa80d

Resolves: tdf#143664 {de-*} accept "Mär" and "Mrz" for March

It will be available in 7.2.1.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 12 Commit Notification 2021-08-04 10:36:28 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "libreoffice-7-2":

https://git.libreoffice.org/core/commit/e927071730668b37f1721b65721a0e5308e88b00

Related: tdf#143664 Change {de-*} March "Mrz" to "Mär"

It will be available in 7.2.1.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 13 Commit Notification 2021-08-04 10:38:41 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "libreoffice-7-1":

https://git.libreoffice.org/core/commit/c23422c0f320df8ef27dbe3ba0a4ac5ead47d6a8

Resolves: tdf#143664 {de-*} accept "Mär" and "Mrz" for March

It will be available in 7.1.6.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 14 Commit Notification 2021-08-09 14:44:42 UTC
Xisco Fauli committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/ccb1d71fe4c0585a42721f09d984fb699ff8f081

tdf#143664: svl_qa_cppunit: Add unittest

It will be available in 7.3.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 15 Vollbracht 2021-10-18 20:57:25 UTC
Thanks! Good Job!