Bug 117276 - Autofilter settings being reset in some cases
Summary: Autofilter settings being reset in some cases
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
4.4 all versions
Hardware: All All
: medium normal
Assignee: Balázs Varga (allotropia)
URL:
Whiteboard: target:6.2.0 target:6.3.0 target:6.2....
Keywords:
Depends on:
Blocks: AutoFilter
  Show dependency treegraph
 
Reported: 2018-04-27 08:53 UTC by Samuel Mehrbrodt (allotropia)
Modified: 2022-09-21 13:48 UTC (History)
6 users (show)

See Also:
Crash report or crash signature:
Regression By:


Attachments
Bugdoc (15.57 KB, application/vnd.oasis.opendocument.spreadsheet)
2018-04-27 08:53 UTC, Samuel Mehrbrodt (allotropia)
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Samuel Mehrbrodt (allotropia) 2018-04-27 08:53:37 UTC
Created attachment 141681 [details]
Bugdoc

How to reproduce:

1. open attached file
2. open filter of column B (Fabrikat) and deselect (Citroen, Fiat, Ford, Opel, Peugeot, Renault, Tesla)
3. open filter of column I (Wert) and deselect 8000 (Values 7000 and 9000 are not shown)
4. open filter of column B and select Tesla
5. open filter of column I and select 7000

Result:
Now the filter of column B is reset

Expected:
The filter should still be as configured in step 2.
Comment 1 Marina Latini (SUSE) 2018-04-27 09:05:41 UTC
repro on:

Version: 6.1.0.0.alpha1
Build ID: cb47f0d320994e001bc38dc2ee9b7d957b15e6ab
CPU threads: 4; OS: Linux 4.16; UI render: default; VCL: gtk2; 
Locale: it-IT (it_IT.UTF-8); Calc: group
Comment 2 Xisco Faulí 2018-04-27 09:42:15 UTC
Reproduced in

Version: 5.2.0.0.alpha0+
Build ID: 3ca42d8d51174010d5e8a32b96e9b4c0b3730a53
Threads 4; Ver: 4.13; Render: default;

and

Version: 4.3.0.0.alpha1+
Build ID: c15927f20d4727c3b8de68497b6949e72f9e6e9e
Comment 3 Serge Krot (CIB) 2018-06-05 08:01:15 UTC
Reproduced in Version: 5.3.0.1
Build ID: 3b800451b1d0c48045de03b5b3c7bbbac87f20d9
CPU Threads: 8; OS Version: Windows 6.2; UI Render: default; Layout Engine: new; 
Locale: en-US (en_US); Calc: group
Comment 4 Commit Notification 2018-06-12 14:10:24 UTC
Serge Krot committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=c8a739a2c84f45f878d2ae75eaf16a2f814d1c6e

tdf#117276 filter reset: check complete data range selected

It will be available in 6.2.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 5 Marina Latini (SUSE) 2018-06-16 14:19:12 UTC
(In reply to Samuel Mehrbrodt (CIB) from comment #0)
> Created attachment 141681 [details]
> Bugdoc
> 
> How to reproduce:
> 
> 5. open filter of column I and select 7000
> 
> Result:
> Now the filter of column B is reset
> 
> Expected:
> The filter should still be as configured in step 2.

with 

Version: 6.2.0.0.alpha0+
Build ID: e79dd394deedaeed122717700077b77d94360c12
CPU threads: 4; OS: Linux 4.16; UI render: default; VCL: kde4; 
Locale: it-IT (it_IT.UTF-8); Calc: group threaded

can't execute the step 5.
After Step 4 the value 7000 in column I isn't visible.
Comment 6 Serge Krot (CIB) 2018-06-18 08:24:03 UTC
(In reply to Marina Latini (CIB) from comment #5)
> (In reply to Samuel Mehrbrodt (CIB) from comment #0)
> > Created attachment 141681 [details]
> > Bugdoc
> > 
> > How to reproduce:
> > 
> > 5. open filter of column I and select 7000
> > 
> > Result:
> > Now the filter of column B is reset
> > 
> > Expected:
> > The filter should still be as configured in step 2.
> 
> with 
> 
> Version: 6.2.0.0.alpha0+
> Build ID: e79dd394deedaeed122717700077b77d94360c12
> CPU threads: 4; OS: Linux 4.16; UI render: default; VCL: kde4; 
> Locale: it-IT (it_IT.UTF-8); Calc: group threaded
> 
> can't execute the step 5.
> After Step 4 the value 7000 in column I isn't visible.

This is because you use latest master version where the new strategy of the filter is implemented (which strings to show and which to hide, when multiple filters are in used). If you take for example LO 5.3.0.1, you will see unchecked 7000, 8000, 9000 before 5th step.

The issue "what is to show in the filter when rows are filtered with multiply filters" is just related to this task and should be checked outside of this task. This task is for fix of the filter reset condition.

The reason (why we do not see all possible strings in filter) is coming from this code:

void ScDocument::GetFilterEntries(SCCOL nCol, SCROW nRow, SCTAB nTab, ScFilterEntries& rFilterEntries )
{
...
            if ( bFilter )
            {
                maTabs[nTab]->GetFilteredFilterEntries( nCol, nStartRow, nEndRow, aParam, rFilterEntries );
            }
            else
            {
                maTabs[nTab]->GetFilterEntries( nCol, nStartRow, nEndRow, rFilterEntries );
            }
...
}

There should be provide the better calculation of the bFilter flag that in some cases the strings (hidden bu other filters) will be non hidden in current filter.
Comment 7 Commit Notification 2018-12-21 18:32:40 UTC
Zdeněk Crhonek committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/+/83e957bfbfa22980461b6c0f07231902549504d4%5E%21

uitest for bug tdf#117276

It will be available in 6.3.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 8 Commit Notification 2019-02-02 13:30:48 UTC
Zdeněk Crhonek committed a patch related to this issue.
It has been pushed to "libreoffice-6-2":

https://git.libreoffice.org/core/+/1b36fbaa7f110739873c69f0d5eeb4fc5d1cf335%5E%21

uitest for bug tdf#117276

It will be available in 6.2.1.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 9 Commit Notification 2019-04-12 09:48:26 UTC
Serge Krot committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/+/4e7e3670c31329d60f5cc782abc4568c2aba33a7%5E%21

tdf#117276 sc: autofilter was unexpected reset with OK pressed

It will be available in 6.3.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 10 Commit Notification 2019-07-16 17:33:04 UTC
Xisco Fauli committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/+/9bd7e2cdf7d2b247ae99a2ce71449d9f265032d1%5E%21

tdf#126306: ignore Top10, Empty and NonEmpty in fix for tdf#117276

It will be available in 6.4.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 11 Commit Notification 2019-07-16 22:37:57 UTC
Xisco Fauli committed a patch related to this issue.
It has been pushed to "libreoffice-6-3":

https://git.libreoffice.org/core/+/5f48428533e928868c11ddd00cc449a54d0cdfd9%5E%21

tdf#126306: ignore Top10, Empty and NonEmpty in fix for tdf#117276

It will be available in 6.3.0.2.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 12 raal 2019-12-07 15:16:16 UTC
The test exist, set status to Verified.
Comment 13 Julien Nabet 2020-09-19 10:00:33 UTC
On pc Debian x86-64 with master sources updated today, when following initial comment, I see "Opel" in the first column.
Shouldn't the whole row disappear since step 2 indicates to deselect several trademarks including "Opel"?
Comment 14 Vasily Melenchuk (CIB) 2021-08-19 10:00:57 UTC
(In reply to Julien Nabet from comment #13)
> On pc Debian x86-64 with master sources updated today, when following
> initial comment, I see "Opel" in the first column.
> Shouldn't the whole row disappear since step 2 indicates to deselect several
> trademarks including "Opel"?

I confirm your observations. Seems original problem from bug description is not fixed. At the same time this looks like not a bug from technical point of view: in p.4 we select all known/visible values, so filter is removed and shows all data. Moreover this behavior is identical to MS Excel (tested with Office 365).
Comment 15 Samuel Mehrbrodt (allotropia) 2021-12-09 10:59:19 UTC
(In reply to Vasily Melenchuk (CIB) from comment #14)
> I confirm your observations. Seems original problem from bug description is
> not fixed. At the same time this looks like not a bug from technical point
> of view: in p.4 we select all known/visible values, so filter is removed and
> shows all data. Moreover this behavior is identical to MS Excel (tested with
> Office 365).

Agree with your assessment. If I see this correctly, Autofilter does not remember from which column a row has been hidden - just which rows are hidden.

So technically this is correct: The highlight on column B is reset after step 5, because all items in column B (which are not hidden by other filters) are visible.

Not sure it's worth adding more complexity here.

We might go back to show hidden values in the autofilter dropdown (maybe show them as inactive when it was hidden by another row) - but that makes it harder to use the autofilter dropdown when you have lots of data.

The current state is very simple: You can just continue filtering on the currently visible rows.

So this is something for UX team to evaluate.
Comment 16 Heiko Tietze 2021-12-10 10:08:03 UTC
If you select 7000 the corresponding Opel needs to be shown, that's correct. But the question is whether 7k needs to be listed at all. Adding 8k shows one VW but lists also the Fiat/8k and Citroen/9k. The 9k are not visible unless Citroen is checked. The actual "issue" is that the filter list is rebuild with every iteration

There are two options:

a) Show only those entries that are currently listed plus what has been filtered-out in the actual column; without Opel you neither have 7000 in the sheet not the filter; with 7000 unchecked you have Opel except from 2007. Checking something that matches other filters reverts them.

b) The autofilter always lists all entries; checking Opel does not bring it back to the sheet until 7000 and 9000 are checked as well

We do a) likewise Excel. Changing it might be unexpected.
Comment 17 Mike Kaganski 2021-12-14 10:51:22 UTC
(In reply to Heiko Tietze from comment #16)
> There are two options:
> 
> a) Show only those entries that are currently listed plus what has been
> filtered-out in the actual column; without Opel you neither have 7000 in the
> sheet not the filter; with 7000 unchecked you have Opel except from 2007.
> Checking something that matches other filters reverts them.
> 
> b) The autofilter always lists all entries; checking Opel does not bring it
> back to the sheet until 7000 and 9000 are checked as well
> 
> We do a) likewise Excel. Changing it might be unexpected.

The question is not specifically "what to show", but rather - in case #1 - if the shown list must be considered as "complete" list. The problem is:

(In reply to Samuel Mehrbrodt (allotropia) from comment #15)
> So technically this is correct: The highlight on column B is reset after
> step 5, because all items in column B (which are not hidden by other
> filters) are visible.

Selecting all items *shown* in the list in a given autofilter resets the autofilter - even if some unselected items are hidden from it. Which is wrong.
Comment 18 Heiko Tietze 2021-12-16 14:22:24 UTC
(In reply to Mike Kaganski from comment #17)
> Selecting all items *shown* in the list in a given autofilter resets the
> autofilter - even if some unselected items are hidden from it. Which is
> wrong.

So let's fix the bug but keep the behavior.
Comment 19 Balázs Varga (allotropia) 2022-06-10 15:14:44 UTC
(In reply to Mike Kaganski from comment #17)
> Selecting all items *shown* in the list in a given autofilter resets the
> autofilter - even if some unselected items are hidden from it. Which is
> wrong.

Btw it is also worked the same as excel do.

Google sheets are working better, because its not reset the autofilter. The reason is, because if you filter a column like - "2. open filter of column B (Fabrikat) and deselect (Citroen, Fiat, Ford, Opel, Peugeot, Renault, Tesla)" -, then filtering another column with the remaining checkboxes (google sheets also hide them) will not override the primary filtering column and the first column - what we filtered first - will keep all the values/checkboxes (even the hidden rows one). Ofc other columns's filter hide all the checkboxes which are belongs to a hidden row.
Maybe we should follow that, if we wanna fix this.

What are your opinions about this?
Comment 20 Commit Notification 2022-07-22 06:12:18 UTC
Balazs Varga committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/2d1df9f3dccc10f13b8585ad18afce1542ebc4d1

tdf#117276 sc: Show hidden filter elements as inactive elements

It will be available in 7.5.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 21 Commit Notification 2022-07-27 17:44:49 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/19533948370dc1ccd7334dbe1a8b7fc8330b10c0

Name FilteredRow what it is, not hidden; tdf#117276 follow-up

It will be available in 7.5.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 22 Commit Notification 2022-07-27 18:20:06 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/2085e90fe8ac129bc4dbac4612d1ea7544335dae

FilteredRow is not a property of the column, tdf#117276 follow-up

It will be available in 7.5.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 23 Commit Notification 2022-07-29 16:18:32 UTC
Balazs Varga committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/7321db3cadc8c0e4437ca04e5dcb652734ea9c26

Related tdf#117276 sc: Show hidden filter elements as inactive elements

It will be available in 7.5.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 24 Gabor Kelemen (allotropia) 2022-08-10 13:29:10 UTC
Verified in 

Version: 7.5.0.0.alpha0+ (x64) / LibreOffice Community
Build ID: 4e2ce2a460458f17ee4360c45a2da2fc4b4d753e
CPU threads: 14; OS: Windows 10.0 Build 19044; UI render: Skia/Raster; VCL: win
Locale: en-US (hu_HU); UI: en-US
Calc: threaded

Now the items filtered by column B are visible in the autofilter of the column I as disabled items at the end of the list.
This is something the customer wanted in the internal bug report.

The part of not resetting the columns filter when All non-hidden items are selected back was deemed impractical from user POV, the way the initial patch tried to implement it:
https://gerrit.libreoffice.org/c/core/+/136595/1

To say the least, it's not seen to be such an error, if you try to make filtering work without this feature :).