Adding a caption to an image locks the "image space reservation box" where the image currently resides. Now, the image can be moved; but the image box and the caption text do not follow the image. This is made worse by bug 93608...
Another non-intuitive situation... the bounding box, which does not exist on a plain image (caption (other?) must exist for the bounding box to exist) must be the element selected in order to move the image AND its caption.
Hi Pierre, I don't know if you closed this bug yourself, but if you feel something is not intuitive, feel free to file a enhancement request to better the situation!
I closed it rather than waste devs time responding it was a PEBCAK bug. :) The only way to "fix" this would be to always include a bounding box -- it was the difference in UI interactions (image v. image+caption) that got me. Cheers.