Bug 148428 - i18n: add dv-MV Dhivehi (Maldives) locale data (was: LibreOffice Calc Vs MS Excel Dhivehi Calendar Issues)
Summary: i18n: add dv-MV Dhivehi (Maldives) locale data (was: LibreOffice Calc Vs MS E...
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Localization (show other bugs)
Version:
(earliest affected)
unspecified
Hardware: All All
: medium enhancement
Assignee: Eike Rathke
URL:
Whiteboard: target:7.4.0
Keywords:
Depends on:
Blocks:
 
Reported: 2022-04-06 16:41 UTC by Adam, M. A.
Modified: 2022-06-05 19:46 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:


Attachments
Eg file for [Bug 148428] i18n: add dv-MV Dhivehi (Maldives) locale data (51.82 KB, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)
2022-04-07 15:56 UTC, Adam, M. A.
Details
Locale dv-MV settings file for [Bug 148428] i18n: add dv-MV Dhivehi (Maldives) locale data (30.76 KB, patch)
2022-04-07 16:00 UTC, Adam, M. A.
Details
changes (5.23 KB, patch)
2022-06-05 18:00 UTC, Eike Rathke
Details
changes to original submission (5.23 KB, patch)
2022-06-05 19:46 UTC, Eike Rathke
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Adam, M. A. 2022-04-06 16:41:24 UTC
Description:
First of all, I congratulate and commend LibreOffice’s performances and contributions to the Opensource community. And I don't honestly believe this to be a bug but amiss of including some locale definitions.

I've made a huge sheet which calculates durations based on dates and it uses DATEDIF(), DAYS() and simple arithmetic operators to get the results. I started it on MS Excel and it works fine. However, after coming across LibreOffice I'm trying to completely switch from MS to LibreOffice. However, one thing is stopping me which is why I'm writing this.

All my MS Excel calculations are correctly displayed in LibreOffice Calc, except the displaying of month names in English and not in Dhivehi although LibreOffice can detect the Maldives language Dhivehi. 

It seems that the Dhivehi names of weekdays and months of Gregorian calendar in Thaana script are not included. Also, need to include the Hijri calendar particulars for same as the Maldives is the only 100% Muslim country after Saudi Arabia in the world.

Steps to Reproduce:
1. Include the Dhivehi names of weekdays and months of Gregorian calendar in Thaana script.
2. Include the Dhivehi names of weekdays and months of Hijri calendar in Thaana script.

Actual Results:
1. In MS Excel blank sheet, enter a date in any language.
2. Copy the same date to another cell.
3. Set the first date's locale to Dhivehi and calendar type to Gregorian by going to the Format Cells dialog box.
4. Set the second date's locale to Dhivehi and calendar type to Hijri by going to the Format Cells dialog box.
5. Save the MS Excel file and exit MS Excel.
6. Open the saved MS Excel file in LibreOffice and see if your dates are correctly displayed in the set locales and with the latest set calendar type.
7. Sure, as it can be, you won't see it. And that why this little work is important to be done.

Expected Results:
The user would expect the same locale and calendar type to be intact no matter which spreadsheet application the user is opening his/her work.


Reproducible: Sometimes


User Profile Reset: Yes


OpenGL enabled: Yes

Additional Info:
If you require any more information or contributions I shall be willing to respond to the best of my ability. Please reach me via: adam7171@gmail.com

I've already prepared a PDF of the confusions this issue may produce in a comparative display of the results in the two applications.
Comment 1 Julien Nabet 2022-04-06 17:16:17 UTC
would it be possible you attach an example file?
Comment 2 Eike Rathke 2022-04-06 17:26:43 UTC
The OP is missing the dv-MV Dhivehi (Maldives) locale data, of which month names are part, which we currently don't have. Someone would have to contribute.

For how to contribute locale data please see
https://wiki.documentfoundation.org/LibreOffice_Localization_Guide/How_To_Submit_New_Locale_Data
Comment 3 Adam, M. A. 2022-04-07 15:56:43 UTC
Created attachment 179382 [details]
Eg file for [Bug 148428] i18n: add dv-MV Dhivehi (Maldives) locale data

Please find the raw XLS file as an example for already reported and logged [Bug 148428] i18n: add dv-MV Dhivehi (Maldives) locale data.
Comment 4 Adam, M. A. 2022-04-07 16:00:52 UTC
Created attachment 179383 [details]
Locale dv-MV settings file for [Bug 148428] i18n: add dv-MV Dhivehi (Maldives) locale data

Contributing suggested and probable fix for already reported and logged [Bug 148428] i18n: add dv-MV Dhivehi (Maldives) locale data.

Kind regards from the Maldives.
ADAM
adam7171@gmail.com [Just in case anyone wants to ask more information.]
Comment 5 Eike Rathke 2022-04-22 12:57:55 UTC
Thanks a lot!

Apparently we don't have your license statement on file, could you
please send us a blanket statement that you contribute all your past and
future patches under the MPLv2 and LGPLv3+ licenses? Best on the dev
mailing list libreoffice@lists.freedesktop.org so we can link to it from
https://wiki.documentfoundation.org/Development/Developers

Something like this does nicely:

All of my past & future contributions to LibreOffice may be
licensed under the MPLv2/LGPLv3+ dual license.

Best use Subject: <your full name> license statement

Sorry for the inconvenience and thank you for cooperating :-)
Comment 6 Eike Rathke 2022-04-27 11:56:27 UTC
Thanks for submitting the license statement.

Test compiling the data I got:

Error: ListSeparator not ';' semicolon. Strongly recommended. Currently required.
Warning: QuotationStart may be wrong: U+2019 ’
Warning: QuotationEnd may be wrong: U+2018 ‘
Warning: DoubleQuotationStart may be wrong: U+201D ”
Warning: DoubleQuotationEnd may be wrong: U+201C “
Error: ThousandSeparator not present in FormatCode formatindex="4".
Warning: formatindex="4","44","45" are the only FormatCode elements checked for separator usage, there may be others that have errors.
Generated date acceptance pattern: 'D-M-Y' from 'DD/MM/YYYY' (formatindex="21" and defined DateSeparator '-')
Generated  2nd acceptance pattern: 'D/M/Y' from 'DD/MM/YYYY' (formatindex="21")
Error: BankSymbol is not ISO 4217
Warning: ReservedWord trueWord seems to be untranslated "true".
Warning: ReservedWord falseWord seems to be untranslated "false".
Warning: ReservedWord quarter1Word seems to be untranslated "1st quarter".
Warning: ReservedWord quarter2Word seems to be untranslated "2nd quarter".
Warning: ReservedWord quarter3Word seems to be untranslated "3rd quarter".
Warning: ReservedWord quarter4Word seems to be untranslated "4th quarter".
Warning: ReservedWord aboveWord seems to be untranslated "above".
Warning: ReservedWord belowWord seems to be untranslated "below".


The list separator currently needs to be ';' semicolon.

The quotation characters seem to be swapped left and right but that may be due to Dhivehi being a RTL script and might be ok. However, other RTL script locales do not have them swapped so the result may be unexpected if ASCII quotes are automatically replaced with typographic quotes.

The ThousandSeparator is defined as <٬> U+066C ARABIC THOUSANDS SEPARATOR but is not used anywhere, the number format codes use <،> U+060C ARABIC COMMA instead. So which is correct? The Arabic locales use U+060C ARABIC COMMA.

The DateSeparator is defined as <-> U+002D HYPHEN-MINUS but the format codes use </> U+002F SOLIDUS. Which is correct?


BankSymbol must be the ISO 4217 code and is swapped with CurrencySymbol so that should be

      <BankSymbol>MVR</BankSymbol>
      <CurrencySymbol>ރ.</CurrencySymbol>

and the replaceTo="[$MVR-101]" be changed accordingly (btw, the LCID is not 101 but 465) to replaceTo="[$ރ.-465]" (note the display is swapped RTL here (at least in the Firefox text edit.. we'll see if that persists), the sequence is "[$" CurrencySymbol "-465]"). To get that in the right order with numbers there's a trick that also Arabic locales use (see for example ar_EG.xml), appending a U+200F RIGHT-TO-LEFT MARK to CurrencySymbol in both the element definition and the replaceTo string so following digits in the currency format display are kept in context. (if you don't know how to do that I can do).


You probably want to translate the elements in ReservedWord as well.


Additionally I found

    <MeasurementSystem>US</MeasurementSystem>

that probably instead should be

    <MeasurementSystem>metric</MeasurementSystem>


Also the LC_INDEX section may need some love. That defines the order in which the Writer table of content (TOC) entries are listed. The current

  <LC_INDEX>
    <IndexKey phonetic="false" default="true" unoid="alphanumeric">A-Z</IndexKey>
    <UnicodeScript>0</UnicodeScript>
    <UnicodeScript>1</UnicodeScript>
    <FollowPageWord> f.</FollowPageWord>
    <FollowPageWord> ff.</FollowPageWord>
  </LC_INDEX>

does not match Dhivehi script. See the explanation in
https://git.libreoffice.org/core/+/HEAD/i18npool/source/localedata/data/locale.dtd#540
However, if you keep that as is it doesn't harm but all index entries will appear only in Unicode order.

You may want to translate the two FollowPageWord abbreviations, first for single "following page", second for multiple "following pages".
Comment 7 Eike Rathke 2022-04-27 12:03:02 UTC
> Writer table of content (TOC) entries
Bah, that's of course wrong and it's the Index section..
Comment 8 Eike Rathke 2022-06-05 18:00:26 UTC
Created attachment 180583 [details]
changes

As there was no follow-up on this I added data with changes I think make sense.
https://gerrit.libreoffice.org/c/core/+/135438
Attached is the diff to the original submission.

The warnings about reserved words and quote characters as mentioned in comment 6 remain. At least the reserved words should be translated.
Comment 9 Commit Notification 2022-06-05 19:42:03 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/2ba57cd896484411fb2c8527802d551f02d419aa

Resolves: tdf#148428 Add Dhivehi (Maldives) {dv-MV} locale data

It will be available in 7.4.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 10 Eike Rathke 2022-06-05 19:46:15 UTC
Created attachment 180584 [details]
changes to original submission

Replacing older changes.