NOT Reproducible with "LibreOffice 3.5.3.2 (RC2) German UI/Locale [Build-ID: 235ab8a-3802056-4a8fed3-2d66ea8-e241b80] on German WIN7 Home Premium (64bit).

I created a sample document with a zig zag line similar to the one in reporter's screenshot and was able to create smooth transitions without problems. An additional test with a 18 corners zig zag line also did not show any problem, I modified all corners.

@David F Smith:
Thank you for your report – unfortunately important information is missing.
May be hints on <http://wiki.documentfoundation.org/BugReport> will help you to find out what information will be useful to reproduce your problem? If you believe that that  is really sophisticated please as for Help on a user mailing list
Please:
- Attach a sample document (not only screenshot) or refer to an existing 
  sample document in an other Bug with a link.
- Contribute a sample document related step by step instruction containing 
  every key press and every mouse click how to reproduce your problem 
  (due to example in Bug 43431)
– if possible contribute an instruction how to create a sample document 
  from the scratch
- add information 
  -- concerning your PC (video card, ...)
  -- concerning your OS (Version, Distribution, Language)
  -- concerning your LibO localization (UI language, Locale setting)
  –- Libo settings that might be related to your problems 
    (video hardware acceleration ...)
  -- how you launch LibO and how you opened the sample document
  -- everything else crossing your mind after you read linked texts

Even  if you can not provide all demanded information, every little new information might bring the breakthrough.

Created attachment 60867 [details]
Shows creation of a polygon, then attempts to convert point to Smooth Transition

In the movie, I create the 5-point polygon.  Then I select the second point, a Corner Point, and click Smooth Transition repeatedly.  The Smooth Transition button does not "stick," and in fact the point is still a Corner Point, as I demonstrate by moving one of its control points.
This problem is not completely reproducible for me, either, but as you see, it does happen.

Computer: Dell Inspiron 620, x64-based desktop, Intel Core i5-2320 CPU
OS: Microsoft Windows 7 Professional, v6.1.7601 SP1
Graphics: Intel HD Graphics adapter
LibreOffice 3.5.2.2, Build ID: 281b639-6baa1d3-ef66a77-d866f25-f36d45f
installed from http://mirror.clarkson.edu/tdf/libreoffice/stable/3.5.2/win/x86/LibO_3.5.2_Win_x86_install_multi.msi

I have also previously installed LibreOffice 3.5.1 on an XP Mode Windows Virtual PC (XP v5.1.2600, SP3) on the same computer, and the behavior is exactly the same.

Thanks very much for your efforts to reproduce this.  I hope you can find something.

Created attachment 60873 [details]
Drawing document that shows the behavior

Zigzag.odg is a drawing of a zigzag polyline.  In my copy of LibreOffice Drawing, if I select the middle point and try to change it to a Smooth Transition, it fails.  (If I select the second point from the left, it works correctly.)

Let's distinguish 2 different (but may be related, I do not know) problems:

a) reporter sometimes fails to change a corner t a smooth transition
   This is not reproducible for me 
b) behavior of icon active indicator for several point actions is is at least 
   not easy to understand. I believe it's wrong and will do some further 
   research later

Thanks, Rainer.  You're right that the two issues may be different, but every time the indicator button has gone back to Corner Point and I have checked it (by moving one of the control points), it has in fact been a Corner Point.
Anyway, thanks for your attempts to reproduce the problem.

Created attachment 60918 [details]
Video shows failure to select Smooth Transition

This is a better video to demonstrate my problem, using the attached Zigzag.odg.

I select the middle point of the polyline (a corner point), then click three times on Smooth Transition.  Each time, the Corner Point button remains selected.  I drag one of the control points to show that the point is in fact still a corner point (i.e., it has control points that move independently).

I then select the fourth point of the polyline, another corner point.  I click on Smooth Transition (no success), then on Symmetric Transition (success), and I drag one of the control points to show that the point is Symmetric.  Then I click on Smooth Transition at which time the Corner Point button becomes selected, and I drag one of the control points to show that the point has become a corner again.

I have retested on 3.5.5.3 under Windows 7 Pro, and the problem still remains for me.  The behavior is exactly as shown in my attached video.

(In reply to comment #7)
> I have retested on 3.5.5.3 under Windows 7 Pro, and the problem still remains
> for me.  The behavior is exactly as shown in my attached video.

Testing with:
Version 3.6.0.4 (Build ID: 932b512) Slackware Linux 13.37
Works for me. Please try with the latest version and reply.

I have retested under 3.6.3.2 (Build ID: 58f22d5), still on Win 7 Pro (v6.1.7601), and the problem still remains.  I open zigzag.odg, select the polyline, select the center point, and click on Smooth Transition, and the Corner Point button remains selected.  (And the point is still, in fact, a corner.)

Dear Bug Submitter,

Please read the entire message in its entirety before continuing - also please respond directly to FDO when replying - do not reply via email.

This bug has been in NEEDINFO status with no change for at least 6 months. Please provide the requested information as soon as possible and mark the bug as UNCONFIRMED. Due to regular bug tracker maintenance, if the bug is still in NEEDINFO status with no change in 30 days the QA team will close the bug as INVALID due to lack of needed information.

For more information about our NEEDINFO policy please read the wiki located here: 
https://wiki.documentfoundation.org/QA/FDO/NEEDINFO

If you have already provided the requested information, please mark the bug as UNCONFIRMED so that the QA team knows that the bug is ready to be confirmed.


Thank you for helping us make LibreOffice even better for everyone!


Warm Regards,
QA Team

Created attachment 82615 [details]
Video shows failure to select Smooth Transition for two points of a polyline

It is not clear to me from the message exactly what info is needed.  Therefore I have retested the bug with 4.0.4.2 under Windows 7 Professional (6.1.7601 Service Pack 1) and recorded a new video of the bad behavior.
Here is the script for the video, which shows operations on Zigzag.odg (previously attached).
1. I select the polyline and then the third point.
2. I click the Smooth Transition button, but it does not remain selected.  (Corner Point is selected instead.)  I move one of the control points, but the other point does not move, showing that point 3 is still a corner point.
3. I select the fourth point and then click Smooth Transition.  The Corner Point button remains selected.  I click Smooth Transition twice more.  I move one of the control points, showing that point 4 is still a corner point.

The incorrect behavior does not happen for every point of every polyline, but the above steps for this document have been reproducible in all versions of LibreOffice for me.

David, I can confirm that there is strange behaviour regarding the Smooth Transition button for versions of LO going back to v3.3. The ODG you have provided in comment #3 (created in v3.5.2.2) seems to display the problem for all three corner points, which may be a factor of the draw:transform="rotate (0.988903554179987) translate (3.159cm 13.7cm)" attribute i.e. you have rotated and moved the polygon at some point. When I drew a new M shaped polygon I found the results more variable, which was disheartening. Status set to NEW. Platform set to All/All. Not sure what to do about the Version. It may need to be set back Inherited From OOo, but I would like QA to confirm this.

I first opened your ODG under Ubuntu 10.04 x86_64 running these versions:

- v3.3.0.4 OOO330m19 Build: 6
- v3.4.6.2 OOO340m1 Build: 602
- v3.5.7.2 Build ID: 3215f89-f603614-ab984f2-7348103-1225a5b
- v3.6.7.2 Build ID: e183d5b
- v4.0.6.2 Build ID: 2e2573268451a50806fcd60ae2d9fe01dd0ce24
- v4.1.3.2 Build ID: 70feb7d99726f064edab4605a8ab840c50ec57a

EVERY version displays the indicated behaviour of the Smooth Transition button not staying selected (for each of the three points). The Symmetric Transition button does not display this behaviour. I was so concerned by this result I tried the same tests again under Crunchbang 11 x86_64 Openbox + XFCE system running the last five versions listed above and v3.3.4.1 OOO330m19 Build: 401 (instead of v3.3.0.4) in case I was experiencing a desktop-related error. The results were identical. 

I then tried creating new M diagrams in each of the 12 OS + LO version combinations and while there is definitely a problem with the Smooth Transition button not staying in a selected state I could find no discernible pattern for which points worked as expected and which did not e.g., number of corners, corner angle, ordinal position in the polygon, vertical offset, etc. The only observable pattern seemed to be that once a diagram was drawn the same points would exhibit the same behaviour in all versions of LO.

Oh I forgot, it may be worthwhile changing the summary from "EDITING: Cannot change a polygon point to Smooth Transition" to "EDITING: Smooth Transition button does not remain selected" as this would seem a more accurate reflection of the issue. The actual polygon point does change to a Smooth Transition, it is just that the change / selection is not reflected in the toolbar.

I have retested using LO Version 4.4.1.2
(Build ID: 45e2de17089c24a1fa810c8f975a7171ba4cd432)
under Windows 7 Professional (6.1.7601 Service Pack 1) 
and the bad behavior persists.
I disagree with Owen Genat's characterization of the problem in Comment 14, but it's a subtle point.  When I click on the Smooth Transition button (having selected one of the points of my Zigzag.odg, uploaded with my Comment 3), the shape of the curve changes at that point to the rounded shape that a smooth transition would have.  However, that point does not become a smooth transition, but remains a corner.  On a smooth transition, moving one of the control points causes the other to move in the opposite direction.  If the control points move independently, that's a corner point.

I have retested the bug with LO 5.1.2.2 (Build ID: d3bf12ecb743fc0d20e0be0c58ca359301eb705f) under Windows 7 Pro (v 6.1.7601 SP1).  I used the same procedure described in Comment 12, using the Zigzag.odg file:
1. I select the polyline and then the third (middle) point.
2. I click the Smooth Transition button, but it does not remain selected.  (Corner Point is selected instead.)  The shape of the polyline changes, so that there is a rounded curve at the middle point, but if I move one of the control points, the other point does not move, showing that point 3 is still a corner point.
3. I select the fourth point and then click Smooth Transition.  The shape changes to a rounded curve, as above, but Corner Point button remains selected.  I click Smooth Transition twice more, and Corner Point remains selected.  I move one of the control points, but the other does not move, showing that point 4 is still a corner point.

The behavior is still wrong, and it has been consistently wrong in this test for about four years, back to at least version 3.5.2.

Please note the source of confusion that became apparent in comments 14 and 15.  A sharp angle at one point can become a rounded curve (as if the point became a Smooth Transition), but if the control points can be moved independently, it's still a corner.  The issue is the actual state of the point, not the shape of the curve or the state of the button.

Thanks for the update

For what it's worth, I have retested this with version 5.2.2.2
(Build ID: 8f96e87c890bf8fa77463cd4b640a2312823f3ad)
under Windows 10 (version 10.0.10586)
and it still fails.

The behavior is exactly the same as in my comment 17.

I have tested the behavior in Version: 5.4.0.0.alpha0+
Build ID: eb7b03b052ffe8c2c577b2349987653db6c53f76
CPU threads: 4; OS: Windows 6.1; UI render: default; 
TinderBox: Win-x86@62-merge-TDF, Branch:MASTER, Time: 2017-02-26_22:34:18
Locale: de-DE (de_DE); Calc: group

There are still problems. I have drawn a Bézier curve with four point. Now I cannot leave symmetric transition. Sometimes if I switch the middle segment to straight line and go back to curve, I can then switch the transition kinds. But then again it sticks in one kind. Unfortunately I cannot give you a always working scenario. It is annoying.

The error still exists in Version: 6.2.0.0.alpha0+ (x64)
Build ID: c3e552ac25be001a623469c549ee8d0719b98133
CPU threads: 8; OS: Windows 10.0; UI render: default; 
Locale: de-DE (de_DE); Calc: CL

The toolbar shows "symmetric" and dragging the handle, it acts like symmetric. But when saving the file, it contains a command "c" where it should be a command "s".

I have retested with LO version: 6.0.5.2 (x64)
(Build ID: 54c8cbb85f300ac59db32fe8a675ff7683cd5a16
CPU threads: 4; OS: Windows 10.0; UI render: default; 
Locale: en-US (en_US); Calc: group)
under Windows 10 Pro (10.0.17134 Build 17134).
My test steps, and the results, were exactly the same as in my Comment 17, which means this bug is more than 6 years old.

I have retested with LO Version: 6.2.4.2 (x64)
(Build ID: 2412653d852ce75f65fbfa83fb7e7b669a126d64
CPU threads: 4; OS: Windows 10.0; UI render: default; VCL: win; 
Locale: en-US (en_US); UI-Language: en-US
Calc: threaded)
under Microsoft Windows 10 Pro	
(Version 10.0.17763 Build 17763)
My test steps, and the results, were exactly the same as in my Comment 17

I watched the video but don't know how to make these buttons appear on pc Debian x86-64 with master sources updated today.

(In reply to Julien Nabet from comment #26)
> I watched the video but don't know how to make these buttons appear on pc
> Debian x86-64 with master sources updated today.

Ok it's "Edit points".

I could reproduce this on Draw with the file attached.

Except if I miss something, I don't reproduce this on Writer after having inserted a curve line.

(In reply to Julien Nabet from comment #28)
> I could reproduce this on Draw with the file attached.
> 
> Except if I miss something, I don't reproduce this on Writer after having
> inserted a curve line.

Argh, I reproduced this also on Writer.

After some debugging on gdb, I noticed this part of bt:
#0  basegfx::getContinuity(basegfx::B2DVector const&, basegfx::B2DVector const&) (rBackVector=..., rForwardVector=...) at basegfx/source/vector/b2dvector.cxx:169
#1  0x00007fffec561661 in basegfx::B2DPolygon::getContinuityInPoint(unsigned int) const (this=0x7e99400, nIndex=2) at basegfx/source/polygon/b2dpolygon.cxx:1387
#2  0x00007ffff1be4c35 in ImplPolygon::ImplPolygon(basegfx::B2DPolygon const&) (this=0x7ffffffeeb00, rPolygon=bezier curve basegfx::B2DPolygon = {...}) at tools/source/generic/poly.cxx:414
0x00007ffff1beb7c9 in tools::Polygon::Polygon(basegfx::B2DPolygon const&) (this=0x7ffffffeebb0, rPolygon=bezier curve basegfx::B2DPolygon = {...}) at tools/source/generic/poly.cxx:1862
0x00007ffff35aec43 in XPolygon::XPolygon(basegfx::B2DPolygon const&) (this=0x7ffffffeec50, rPolygon=bezier curve basegfx::B2DPolygon = {...}) at svx/source/xoutdev/_xpoly.cxx:826
0x00007ffff35aef73 in XPolyPolygon::XPolyPolygon(basegfx::B2DPolyPolygon const&) (this=0x7ffffffeed70, rPolyPolygon=open basegfx::B2DPolyPolygon with 1 sub-polygon(s) = {...})
    at svx/source/xoutdev/_xpoly.cxx:855
#6  0x00007ffff3205f10 in SdrPathObj::AddToPlusHdlList(SdrHdlList&, SdrHdl&) const (this=0x35c38c0, rHdlList=..., rHdl=...) at svx/source/svdraw/svdopath.cxx:2006

"B2VectorContinuity::getContinuity" should return "B2VectorContinuity::C1"
but areParallel(rBackVector, rForwardVector) return false where it should return true.
See https://opengrok.libreoffice.org/xref/core/basegfx/source/vector/b2dvector.cxx?r=1d92933d#167

Here are the values retrieved (std::cerr << setprecision(15) << ...):
fValA=33122.8002951872
fValB=33122.8002951847

Here's the code of b2dvector.cxx::areParallel:
118      bool areParallel( const B2DVector& rVecA, const B2DVector& rVecB )
119      {
120          const double fValA(rVecA.getX() * rVecB.getY());
121          const double fValB(rVecA.getY() * rVecB.getX());
122  
123          return fTools::equal(fValA, fValB);
124      }
See https://opengrok.libreoffice.org/xref/core/basegfx/source/vector/b2dvector.cxx?r=1d92933d#118

Just "relaxing" a bit the equal comparison allows to return true in our case.

I submitted this patch for review:
https://gerrit.libreoffice.org/c/core/+/91705

Patch was wrong, I abandoned it.
=> unassign myself

I have retested with 
Version: 7.2.6.2 (x64) / LibreOffice Community
Build ID: b0ec3a565991f7569a5a7f5d24fed7f52653d754
CPU threads: 4; OS: Windows 10.0 Build 19043; UI render: Skia/Raster; VCL: win
Locale: en-US (en_US); UI: en-US
Calc: threaded
My test steps, and the results, were exactly the same as in my Comment 17.