Bug 126866 - Iteration not only bug-laden, but neglected and misunderstood
Summary: Iteration not only bug-laden, but neglected and misunderstood
Status: RESOLVED INVALID
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
6.2.5.2 release
Hardware: x86-64 (AMD64) Windows (All)
: medium normal
Assignee: Not Assigned
URL: https://wiki.documentfoundation.org/F...
Whiteboard:
Keywords:
Depends on: 46845 54022 114482
Blocks: 33330 117049
  Show dependency treegraph
 
Reported: 2019-08-13 04:20 UTC by hardcoder
Modified: 2019-08-19 09:36 UTC (History)
1 user (show)

See Also:
Crash report or crash signature:


Attachments
Modest action game, comparable to music or dance games. (23.10 KB, application/vnd.oasis.opendocument.spreadsheet)
2019-08-13 04:20 UTC, hardcoder
Details
How I see my spreadsheet in LO 6.2.5.2 x64, under Win 10. (75.75 KB, image/png)
2019-08-13 19:32 UTC, hardcoder
Details

Note You need to log in before you can comment on or make changes to this bug.
Description hardcoder 2019-08-13 04:20:13 UTC
Created attachment 153333 [details]
Modest action game, comparable to music or dance games.

There are several bugs implicated with iteration/iterative calculation in Calc, including several unreported bugs. But reading more about this feature in LibreOffice help and Bugzilla, I soon began to see how that feature wasn't even understood by the LibreOffice team.

Part of the problem exists because of the uneducated assumption that what you call "convergence" is a necessary concern every time iteration is used. If "only" you understood how unnecessary (also narrow-minded) that is (also note that other spreadsheet apps don't work with such assumption). Later, I may post an additional report with objective details on that, if you don't see what I mean.

I know I'll have to eventually write other bug reports to elaborate specifics on this set of problems (which will take time). But if you want one bug per report, the original intention with this one is not just a preface, but to report the bug staff and dev team attitude as a bug in itself -- one that causes and sustains other bugs, so I'd give above-average importance.

While I haven't written more, I attached a little something for you. I made this game inspired by Calc's feature to auto-refresh when there's a (dummy) link to external data, but the game was hindered by the many iteration bugs, to the point I saw I wouldn't be able to make it work on Calc (without macros, which I usually reject). If the bugs are fixed, the game will probably start working with no changes.
Comment 1 hardcoder 2019-08-13 04:24:50 UTC
Steps to reproduce the bug:
- First, understand that I'm not being superfluously sardonic nor joking, so don't quit following. While not all of those steps are necessary for everyone, I reckon they should be _sufficient_ for anyone to see and know what to do on the topic, if they are involved in maintaining LibreOffice.
- Be an advanced spreadsheet user.
- Be willing to know more about iteration. If you seriously try using it (with the help of apps other than LibreOffice, like MS Excel), you may begin to understand it is a unique feature for making electronic spreadsheets more powerful (although currently hard to tame even in Excel -- I'll write further reports on this later). Even if your friends or workmates don't use it, several people can be found on the Internet using the feature, and they have started to do awesome things with iteration. A very modest example: https://www.microsoft.com/en-us/microsoft-365/blog/2007/11/02/iteration-conways-game-of-life/
- Understand that electronic spreadsheets have been around for decades before OpenOffice.org and LibreOffice, with so many features OpenOffice didn't invent. Note that LibreOffice intends to support formats such as .xls and .xlsx.
- Know that iterative calculation has been attested (in various web pages out there) to exist since at least Office 97, well before even OpenOffice.org existed.
- Not be over-questioning about what uses will the users have for spreadsheets; so, even if they are just making a spreadsheet game, they are not living in a sort of "no-bug wonderland", and each unexpected behavior may interfere with serious uses of Calc. One of the lessons of test-driven development is that artificial, out-of-context tests may be legitimate and great. But given the amount of bugs, is your project testable as in TDD?
- Understand that other spreadsheet apps like WPS Office Spreadsheets, SoftMaker PlanMaker, G Sheets (besides MS Excel) are way more mature about iteration.
Comment 2 Mike Kaganski 2019-08-13 07:02:07 UTC
Hi,

Please note that I'm not being superfluously sardonic nor joking, so don't quit following.

This specific report is INVALID, because it doesn't describe some single problem that may be evaluated, tracked, and finally fixed. Instead, it asserts some "bug of attitude" - which also claims something the OP doesn't know about (specifically, what "team" thinks of the feature, especially there's no "team" in strong sense of the word, and so every contributor who decides to dive in and do something related to Calc is part of the "team", thus any wide-scale assertions of this nature are invalid).

Here I do not declare absence of specific bugs in the feature; nor even that there's no flaw in its fundamental design - it's just impossible to understand what the flaw(s) are from this report, and so impossible to act to fix them.

Even the attachment doesn't describe clearly what should be done and what should be observed, and what is the expected result, to see bugs.

So - while I close this RESOLVED INVALID, I ask you (OP) to file specific bug reports on specific problems, describing clearly what is done, what is expected (and why), and what is observed (and with which settings on which LO version); and also ask you be constructive (refrain from assertions that may sound rude, no matter what your emotions might be). Thank you for your effort to make LibreOffice better!
Comment 3 Mike Kaganski 2019-08-13 07:22:04 UTC
Just to be more specific on the "neglected and misunderstood" part:

When in the bug 117049 (that you added to "Blocks:" list), I have made some assertions on the feature that turned out to be wrong, I still asked for help from our experts in the area; and our leading Calc developer has jumped in and corrected me, and fixed the bug. So - it isn't "misunderstood" by "team" (while it could be by individual contributors), and also not "neglected" (and knows problems are fixed as time permits).

If one feels that some problem is not getting enough attention, one needs to familiarize with how this open-source project works, where individuals work on what they *like*, and corporate contributors work on what they are *paid for* by their paying customers. So - one needs to either wait for someone getting interested in a given problem; or to hire someone to fix it; or to jump in and contribute. There's no other way here. And every helping hand is truly welcome: we really try hard to help any newcomer in any questions, e.g. when one needs to setup development environment, to get code pointers, to have changes reviewed, etc.
Comment 4 hardcoder 2019-08-13 15:45:39 UTC
Hi M. Kaganski; thanks for getting some talk going soon.

I figure that for most found bugs I'll eventually have to describe them individually, although that feature is a nest of bugs. I'm particularly slow at the task of writing, and I'm not a native English speaker. Actually it's one of my earliest bug reports, and without internal involvement in LibreOffice. I wasn't quite sure of the tone and how it would be received, and I admit I'm speaking in a vague scale, but I wanted to first interact with your (y'all) stance on iteration. Even now I'm not sure if I should have tried Redmine instead, so I took some chance.

As for things like "Understand that electronic spreadsheets have been around for decades", I know everybody knows that, but I was trying to summon the necessary premises for the deduction (probably out of insecurity). I have a habit of forgetting to mention things when speaking, so I never know why others don't understand me. Still I tried to be objective instead of subjective. I only put "only" in quotes in the first post because it's actually not the only problem to see, although a prominent one to start with. The given URL is a simple example I found of iteration being used outside your expectations, though clearly not the best use.

My logic on the blocked bugs reporting is that they'd require a change of attitude, and I thought bug 117049 would exemplify the incomprehension about the feature, resulting in weird attitudes about iteration, although eventually the user went mad too. The resistance was unnecessary if you better understood the feature, and like I said you would then understand its uniqueness and importance. I'd say it is trivially possible to learn iterative calculation from nothing, so I encourage it to y'all. (The "depends on" bugs seem simpler.)

I also was writing under the assumption LibreOffice had in some way an aim to supersede MS Office, pretty much like we see on Linux distros (against Windows) and several other FLOSS projects. But pondering a bit, I guess I never read that anywhere. There are other features elegantly supported since around Office 2007 like a number of auto-shape effects (glow, shadow, etc.; rich in parameters) and for presentations, animation parameters (a whole other abandoned nest of bugs in Impress), but you (y'all) don't seem to care for them. As a user my heart was initially warmed to see every little of those LO releases had tens of changes and fixes.

I admit my report here doesn't inspire much action, and I know you're waiting for further details, but the report is a suggestion for those involvable in Calc iteration (and I don't know anymore if there's any) to learn and test more about the feature. That is worthwhile, and plausibly is eventually needed to make iteration work. Sincerely, I did want more attention to the feature of iteration, and I'm not sure about the communication there has been, but please from now on, I'd like less misconceptions on the feature, that things get going.

P.S.: I forgot to say, the "other spreadsheet apps" I mentioned are all fortunately available for free (gratis), in case you want to test.
Comment 5 hardcoder 2019-08-13 18:54:52 UTC
I put "convergence" under quotes because it's a plain simple delta; there's absolutely no convergence being checked, and indeed there shouldn't be. Even if there were, it should be presented as a warning (like that little triangle Excel occasionally puts on the top-right of a cell), not an error hiding the value; but that checking would cost CPU time.

If your number varies by 0.1, under default settings, it's said it does not converge. If your rescale things so that it varies by 0.0001, it is literally "convergent". "Furtiveness" would be a better term for this. For those things, I called the "convergence" check uneducated.

Even if the formula is judged to give nonconvergent results (and the results get censored), the underlying iteration is never paused. Later I may paste all this into a new bug report if you need. While I surely wanted to scorn the anti-feature thoroughly, I apologize some of the anger spilled to the originators of the distracted artifact. (I didn't even mention about trying to use iteration with strings.)
Comment 6 hardcoder 2019-08-13 19:05:38 UTC
(In reply to Mike Kaganski from comment #2)
> asserts some "bug of attitude" - which also claims something the OP doesn't
> know about (specifically, what "team" thinks of the feature, especially
> there's no "team" in strong sense of the word, and so every contributor who
(In reply to Mike Kaganski from comment #3)
> in and corrected me, and fixed the bug. So - it isn't "misunderstood" by
> "team" (while it could be by individual contributors), and also not
Of course my statements shouldn't be based on telepathy or delusion, and my point was not about specific minds like yours. I guess I couldn't believe the state of maturity of the feature, given all these years, and given the other apps, so LibreOffice remains unworkable for iteration. After the initial frustration I searched about it in more than one LibreOffice site, like I said, and anyway it's all out there for you all to see -- what's probably missing is someone involved that has (or gets) a different view, and sees that several things are wrong. Every time I refer to LibreOffice team, I refer to what I've seen on the sites. Although I'll never force you to think or speak the same way as I did, I still think what I described as some non-computer bug has the attributes of a bug. Of course, whether it will auto-heal in a decentralized manner, or needs to be tracked, is a different story.

(In reply to Mike Kaganski from comment #2)
> Here I do not declare absence of specific bugs in the feature; nor even that
> there's no flaw in its fundamental design - it's just impossible to
> understand what the flaw(s) are from this report, and so impossible to act
Have you seen Kohei Yoshida's words on bug 33330, comments #5 and #6? What does it mean, "invokes an entirely different calculation engine"??? Well, why do I care.
Comment 7 hardcoder 2019-08-13 19:32:22 UTC
Created attachment 153357 [details]
How I see my spreadsheet in LO 6.2.5.2 x64, under Win 10.
Comment 8 hardcoder 2019-08-13 19:54:07 UTC
> Even the attachment doesn't describe clearly what should be done and what
> should be observed, and what is the expected result, to see bugs.
There are bugs already explicit or hinted there; let's start with them, as I'm already getting specific.
$Sheet1.A3: "you probably will have to press ctrl+shift+f9 (hard recalc) occasionally." means that soft recalc is not enough. Particularly, the ever-unneeded "Err:523" disappears on hard recalc for some reason.
$Sheet1.C5: "(in libreoffice, this has been necessary to force iterative updates without hard recalc.)" refers to including things such as +0*RAND() or +0*NOW() to make iteration keep going on soft recalcs, since those 2 functions force recalc to work (or for strings, &TEXT(0*$B$5,";;;")).
$Sheet1.C6: "(this should match your iteration settings.)" currently there's no function, even in other apps, to inform a spreadsheet on the set number of iterations, which for some purposes has to be compensated, or even better, should be changed on a per-formula basis by a function wrapping the original formula.
Also, "Err:523" pops randomly in the buffer even when nothing has changed, and eventually covers the "your input" field. There's really no apparent or reasonable pattern in that. I had the idea of posting that screenshot, in case the bug doesn't happen with you. I could work my way around the other bugs, but this one (the "your input" and "valid" fields covered up) is the last straw, making the spreadsheet unusable. Unless you throw a hard recalc after every digit you type (in the edit box), and quick enough.
Comment 9 hardcoder 2019-08-13 20:02:15 UTC
There is a more or less sophisticated bunch of mechanisms for: controlling excess iterations; making counters and buffer resettable; only allowing buffer advancing after some time; keeping a buffer (and changing only one row at a time); detecting if the edit box is cleared, and absorbing one extra char at a time; and to slalom the bugs. The reference values are randomly generated at Sheet1.D36. I don't think there's much more relevant to say, but of course I can give any more details if you want.
Comment 10 hardcoder 2019-08-17 20:56:17 UTC
(In reply to hardcoder from comment #1)
> don't quit following. While not all of those steps are necessary for
> everyone, I reckon they should be _sufficient_ for anyone to see and know
> what to do on the topic, if they are involved in maintaining LibreOffice.

While reading again what I wrote I finally realized there was one more rough edge, as I mentioned "everyone", then later only those "involved in maintaining LibreOffice". The quoted part was a late addition, but there wasn't any intention to compare people. I apologize again for the weird wording.

In fact, I wrote with an overall harsh tone because the original focus for me was on software, and I didn't expect to say much about people. The developers and maintainers can easily be wonderful people, and I won't question their intentions. But I still challenge the maturity of the objective product they gave, on some aspects such as Calc's iteration.

I don't understand much about Bugzilla status, but I'm throwing a REOPENED now, as I added some more content since, just in case they haven't seen.
Comment 11 Xisco Faulí 2019-08-19 09:31:16 UTC
Removing keywords set incorreclty.
Comment 12 Xisco Faulí 2019-08-19 09:36:06 UTC
Hello hardcoder,
Thanks for reporting this issue. Unfortunately I agree with Mike Kaganski here, it doesn't describe a single issue that can be fixed, instead, we have already 10 bugs with a lot of literature but nothing specific.
I'm sorry but I'm going to close this issue as RESOLVED INVALID.
Please, create a follow-up bug, with simple steps to reproduce the problem. If the file has different issues, please report each of them in a different bug.
Thank in advance.