Bug 113974 - Dedicated cell function to fetch / parse / filter JSON data and extract fields
Summary: Dedicated cell function to fetch / parse / filter JSON data and extract fields
Status: REOPENED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
Inherited From OOo
Hardware: All All
: medium enhancement
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
: 109073 (view as bug list)
Depends on:
Blocks: Format-Filters Data-Provider Calc-Function-Requests
  Show dependency treegraph
 
Reported: 2017-11-21 16:04 UTC by Dan Dascalescu
Modified: 2025-12-11 00:04 UTC (History)
16 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Dan Dascalescu 2017-11-21 16:04:24 UTC
Description:
REST is far more popular than SOAP nowadays, and it's time to have a FILTERJSON function equivalent to FILTERXML.

Steps to Reproduce:
The vast majority of REST APIs return JSON data.

Actual Results:  
FILTERJSON (or "PARSEJSON") could use dotted notation to extract object fields.

Expected Results:
A1: =WEBSERVICE("https://api.github.com/repos/DmytroBazunov/LibreOfficeGetRestPlugin/issues/6")
B1: = PARSEJSON(A1, "user.login")  // "dandv"


Reproducible: Always


User Profile Reset: No



Additional Info:
There's a poorly maintained plugin that attempts to do this, https://github.com/DmytroBazunov/LibreOfficeGetRestPlugin/issues/6


User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.89 Safari/537.36
Comment 1 Buovjaga 2017-11-23 10:19:23 UTC
Implementing this will be possible with the nearly-ready dataprovider: https://cgit.freedesktop.org/libreoffice/core/tree/sc/source/ui/dataprovider
Comment 2 Dan Dascalescu 2018-01-11 01:53:55 UTC
Nice, I see very recents commits.

As extra motivation, I was trying to extract Bitcoin prices from this API, 

=SUBSTITUTE(WEBSERVICE("https://min-api.cryptocompare.com/data/pricemulti?fsyms=BTC,ETH,LTC&tsyms=USD"), """:{""USD""", "")

The SUBSTITUTE above is just prep. After that, to get at a value, I came up with a formula looks like this:

=VALUE(MID(A1, SEARCH("(?<=BTC:)\d", A1), FIND("}", A1, SEARCH("(?<=BTC:)\d", A1)) - SEARCH("(?<=BTC:)\d", A1)))

A week later, that thing above looks more cryptic than elliptic curve cryptography :) I have no idea what it does at a glance and why it's so repetitive.
Comment 3 Kevin Suo 2021-10-21 15:46:37 UTC
LibreOffice uses liborcus as an mandatory external lib, and Orcus provides json support. Thus this filer may not be hard to implement.
Comment 4 Kevin Suo 2021-10-21 15:50:35 UTC
*** Bug 109073 has been marked as a duplicate of this bug. ***
Comment 5 Dan Dascalescu 2021-11-29 13:36:45 UTC Comment hidden (obsolete)
Comment 6 Buovjaga 2021-11-29 13:48:25 UTC
(In reply to Dan Dascalescu from comment #5)
> 4 years later, any progress?

Yes, data provider got a UI: https://libreoffice-dataproviders.blogspot.com/2021/08/data-providers.html

But: "The external data currently supported are CSV , HTML , XML and Base". So what Kevin said in comment 3 is still relevant.
Comment 7 Melroy 2022-07-23 23:48:29 UTC
Still really waiting on FILTERJSON() function. Please add it to LibreOffice! PLEASE GOD!
Comment 8 m_a_riosv 2023-01-21 00:20:53 UTC
As in many cases, web-pages to import are in JSON format, it will be really useful.
Comment 9 papou84 2023-08-01 20:10:31 UTC
I started to parse this JSON using this formula :
=PARSEJSON(A1, "outputs.monthly.fixed.get(1).E_m") . I get : 31.29
For month 2 I used : 
=PARSEJSON(A1, "outputs.monthly.fixed.get(2).E_m") . I get : 71.38
untill I get to month 12 :
=PARSEJSON(A1, "outputs.monthly.fixed.get(12).E_m") . and there I get no such node found.

It seems that after 11 it's bugging

JSON file :

{"inputs":{"location":{"latitude":50.0,"longitude":8.0,"elevation":104.0},"meteo_data":{"radiation_db":"PVGIS-SARAH","meteo_db":"ERA-Interim","year_min":2005,"year_max":2016,"use_horizon":true,"horizon_db":"DEM-calculated"},"mounting_system":{"fixed":{"slope":{"value":0,"optimal":false},"azimuth":{"value":0,"optimal":false},"type":"free-standing"}},"pv_module":{"technology":"c-Si","peak_power":1.0,"system_loss":14.0},"economic_data":{"system_cost":null,"interest":null,"lifetime":null}},"outputs":{"monthly":{"fixed":[{"month":1,"E_d":0.59,"E_m":18.36,"H(i)_d":0.8,"H(i)_m":24.67,"SD_m":2.13},{"month":2,"E_d":1.12,"E_m":31.29,"H(i)_d":1.41,"H(i)_m":39.36,"SD_m":6.45},{"month":3,"E_d":2.3,"E_m":71.38,"H(i)_d":2.82,"H(i)_m":87.54,"SD_m":11.19},{"month":4,"E_d":3.68,"E_m":110.26,"H(i)_d":4.57,"H(i)_m":137.06,"SD_m":15.67},{"month":5,"E_d":4.04,"E_m":125.13,"H(i)_d":5.08,"H(i)_m":157.51,"SD_m":17.94},{"month":6,"E_d":4.44,"E_m":133.16,"H(i)_d":5.71,"H(i)_m":171.22,"SD_m":13.81},{"month":7,"E_d":4.27,"E_m":132.5,"H(i)_d":5.57,"H(i)_m":172.78,"SD_m":11.89},{"month":8,"E_d":3.65,"E_m":113.22,"H(i)_d":4.72,"H(i)_m":146.37,"SD_m":7.96},{"month":9,"E_d":2.73,"E_m":81.93,"H(i)_d":3.49,"H(i)_m":104.71,"SD_m":7.8},{"month":10,"E_d":1.58,"E_m":48.97,"H(i)_d":2.02,"H(i)_m":62.65,"SD_m":6.44},{"month":11,"E_d":0.7,"E_m":21.12,"H(i)_d":0.95,"H(i)_m":28.45,"SD_m":2.53},{"month":12,"E_d":0.46,"E_m":14.27,"H(i)_d":0.65,"H(i)_m":20.04,"SD_m":1.24}]},"totals":{"fixed":{"E_d":2.47,"E_m":75.13,"E_y":901.57,"H(i)_d":3.16,"H(i)_m":96.03,"H(i)_y":1152.37,"SD_m":2.73,"SD_y":32.7,"l_aoi":-4.28,"l_spec":"1.45","l_tg":-6.31,"l_total":-21.76}}},"meta":{"inputs":{"location":{"description":"Selected location","variables":{"latitude":{"description":"Latitude","units":"decimal degree"},"longitude":{"description":"Longitude","units":"decimal degree"},"elevation":{"description":"Elevation","units":"m"}}},"meteo_data":{"description":"Sources of meteorological data","variables":{"radiation_db":{"description":"Solar radiation database"},"meteo_db":{"description":"Database used for meteorological variables other than solar radiation"},"year_min":{"description":"First year of the calculations"},"year_max":{"description":"Last year of the calculations"},"use_horizon":{"description":"Include horizon shadows"},"horizon_db":{"description":"Source of horizon data"}}},"mounting_system":{"description":"Mounting system","choices":"fixed, vertical_axis, inclined_axis, two_axis","fields":{"slope":{"description":"Inclination angle from the horizontal plane","units":"degree"},"azimuth":{"description":"Orientation (azimuth) angle of the (fixed) PV system (0 = S, 90 = W, -90 = E)","units":"degree"}}},"pv_module":{"description":"PV module parameters","variables":{"technology":{"description":"PV technology"},"peak_power":{"description":"Nominal (peak) power of the PV module","units":"kW"},"system_loss":{"description":"Sum of system losses","units":"%"}}},"economic_data":{"description":"Economic inputs","variables":{"system_cost":{"description":"Total cost of the PV system","units":"user-defined currency"},"interest":{"description":"Annual interest","units":"%/y"},"lifetime":{"description":"Expected lifetime of the PV system","units":"y"}}}},"outputs":{"monthly":{"type":"time series","timestamp":"monthly averages","variables":{"E_d":{"description":"Average daily energy production from the given system","units":"kWh/d"},"E_m":{"description":"Average monthly energy production from the given system","units":"kWh/mo"},"H(i)_d":{"description":"Average daily sum of global irradiation per square meter received by the modules of the given system","units":"kWh/m2/d"},"H(i)_m":{"description":"Average monthly sum of global irradiation per square meter received by the modules of the given system","units":"kWh/m2/mo"},"SD_m":{"description":"Standard deviation of the monthly energy production due to year-to-year variation","units":"kWh"}}},"totals":{"type":"time series totals","variables":{"E_d":{"description":"Average daily energy production from the given system","units":"kWh/d"},"E_m":{"description":"Average monthly energy production from the given system","units":"kWh/mo"},"E_y":{"description":"Average annual energy production from the given system","units":"kWh/y"},"H(i)_d":{"description":"Average daily sum of global irradiation per square meter received by the modules of the given system","units":"kWh/m2/d"},"H(i)_m":{"description":"Average monthly sum of global irradiation per square meter received by the modules of the given system","units":"kWh/m2/mo"},"H(i)_y":{"description":"Average annual sum of global irradiation per square meter received by the modules of the given system","units":"kWh/m2/y"},"SD_m":{"description":"Standard deviation of the monthly energy production due to year-to-year variation","units":"kWh"},"SD_y":{"description":"Standard deviation of the annual energy production due to year-to-year variation","units":"kWh"},"l_aoi":{"description":"Angle of incidence loss","units":"%"},"l_spec":{"description":"Spectral loss","units":"%"},"l_tg":{"description":"Temperature and irradiance loss","units":"%"},"l_total":{"description":"Total loss","units":"%"}}}}}}
Comment 10 Kohei Yoshida 2025-02-11 02:58:11 UTC
FYI, I just integrated orcus version 0.20.0 into the build, and one thing that's new is the ability to extract JSON subtree:

https://orcus.readthedocs.io/en/latest/overview/json/subtree.html

Perhaps this will be helpful when implementing this feature or something similar.
Comment 11 Kohei Yoshida 2025-10-14 03:00:51 UTC
FYI, Calc will be able to directly open JSON files starting with 26.2.  See

https://wiki.documentfoundation.org/ReleaseNotes/26.2#Generic_XML_and_JSON_mapping_to_Calc

Perhaps we can consider that a solution to this issue?
Comment 12 Buovjaga 2025-10-14 07:05:16 UTC
(In reply to Kohei Yoshida from comment #11)
> FYI, Calc will be able to directly open JSON files starting with 26.2.  See
> 
> https://wiki.documentfoundation.org/ReleaseNotes/26.
> 2#Generic_XML_and_JSON_mapping_to_Calc
> 
> Perhaps we can consider that a solution to this issue?

Dan: are you happy with this?
Comment 13 m_a_riosv 2025-10-14 22:13:47 UTC
Tested with a couple of files from https://sample.json-format.com/

employees_10KB.json (5 level)
employees-10-level_100MB.json

It works:
- opening directly the files.
- Menu>Sheets>Insert sheet from file

But not with
Menu>Sheet>External Links.
Menu>Data>Data Provider.

Opened an enhancement bug for this last.

tdf#168860 - Allow the ability to link to JSON files with Menu>Sheet>External Links and Menu>Data>Data Provider.
Comment 14 Kohei Yoshida 2025-12-10 03:48:16 UTC
Let's call it fixed since the original submitter is no longer responsive & I believe this is now addressed.
Comment 15 chassaing 2025-12-10 03:53:43 UTC
It's definitely not fixed. We need something like the FILTERXML function but FILTERJSON instead. It's crazy that we can directly process XML content (which is a format quickly disappearing from the Internet) but not JSON (which is probably the most popular data interchange format online now).

Being able to open JSON files is definitely not the same as being able to import JSON data within a cell/sheet. Tons of use case, like importing currencies or stock market data, calling online APIs, etc.