Bug 169872 - Deduplicate sucks, it's very very slow
Summary: Deduplicate sucks, it's very very slow
Status: RESOLVED DUPLICATE of bug 165566
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
25.8.3.2 release
Hardware: All All
: medium enhancement
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2025-12-07 11:36 UTC by hanzo0101
Modified: 2025-12-19 17:17 UTC (History)
2 users (show)

See Also:
Crash report or crash signature:


Attachments
Here a sample file (257.86 KB, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)
2025-12-17 19:40 UTC, hanzo0101
Details

Note You need to log in before you can comment on or make changes to this bug.
Description hanzo0101 2025-12-07 11:36:15 UTC
Description:
So it's nice to see that you finally added the basic function of deleting duplicates, but you might as well not have bothered because it's very, very slow, compared to Excel or even compared to the old method of filter>not empty>no duplications and then copying to a new sheet so they're actually deleted and not hidden, works instantly, btw. I don't know what's the problem in using this method for the deleting duplicates button, I understand it may not be as simple with multiple columns, but still. 

Steps to Reproduce:
1.Data>Duplicates...
2.Choose your columns like in Excel
3.Wait...

Actual Results:
You for an hour or two, depending on your data quantity, and it doesn't even say how much was deleted or give any confirmation that it's finished

Expected Results:
That it will process it way faster, like in the crutch method we already used before with the filters or like in Excel in a reasonable time.


Reproducible: Always


User Profile Reset: No

Additional Info:
I understand that it's free, and I'm sorry if I sound angry but that's what the feedback page is, right?
Comment 1 m_a_riosv 2025-12-07 22:50:09 UTC
Please attach a sample file, reduce the size as much as possible without private information,
and paste the information in Menu/Help/About LibreOffice, there is a copy icon.
Comment 2 MacAhi 2025-12-09 18:36:25 UTC
I have a side-question.  This feature (Data\Duplicates) isn't available on the MacOS version of 25.8.3.2.  Are the versions always out-of-sync for different platforms, or is there a different issue?
Comment 3 MacAhi 2025-12-17 19:25:50 UTC
I just DLd a build which now contains this feature.  It worked extremely well for me.  

1800 rows of mixed data.  I selected one column (text) and it reduced the set to 15 unique values in about 8 seconds.

After resetting the data, I selected 12 columns of that same data set (dates, text, numbers) and it took about 3 seconds to process.

The one issue I did notice and agree with Hanzo0101 on, is that there is no visual or audible indication that it finished.  In my first test, it was obvious because the change was substantial.  My second test though had a much smaller reduction and it wasn't obvious at all that it had completed.  I ran this test a 2nd time in order to time it correctly.
Comment 4 hanzo0101 2025-12-17 19:36:45 UTC
(In reply to m_a_riosv from comment #1)
> Please attach a sample file, reduce the size as much as possible without
> private information,
> and paste the information in Menu/Help/About LibreOffice, there is a copy
> icon.

Version: 25.8.3.2 (X86_64)
Build ID: 8ca8d55c161d602844f5428fa4b58097424e324e
CPU threads: 8; OS: Windows 11 X86_64 (build 26200); UI render: Skia/Vulkan; VCL: win
Locale: en-US (en_US); UI: en-US
Calc: CL threaded
Comment 5 hanzo0101 2025-12-17 19:40:37 UTC
Created attachment 204696 [details]
Here a sample file

It took excel 2 second
It took libre one minute
Comment 6 Timur 2025-12-17 19:53:50 UTC
I did not test but there is tdf#165566 and tdf#166121.

*** This bug has been marked as a duplicate of bug 165566 ***
Comment 7 MacAhi 2025-12-17 22:00:28 UTC
(In reply to Timur from comment #6)
> I did not test but there is tdf#165566 and tdf#166121.
> 
> *** This bug has been marked as a duplicate of bug 165566 ***

Hi Timur,  Could you please answer my question from Comment 2.
Comment 8 Timur 2025-12-19 10:47:38 UTC
Sorry I cannot explain. Usually menus should be the same. 
Youmay try to also look in Tabbed interface to see if it is there.
Otherwise, you may report specific macOS bug.
Comment 9 MacAhi 2025-12-19 17:17:55 UTC
(In reply to Timur from comment #8)
> Sorry I cannot explain. Usually menus should be the same. 
> Youmay try to also look in Tabbed interface to see if it is there.
> Otherwise, you may report specific macOS bug.

Ok Thanks.  I pulled down LibreOfficeDev_26.8.0.0.alpha0_MacOS_aarch64 a couple of days ago and it's in there, but I also just pulled 25.8.4.2 and it's still not in there.