The Locale setting in Tools->Options->Language Settings->Languages is not saved. Every time LibreOffice is started, the selected locale setting is reset back to the same language as the setting for "User interface".
E.g. if I set the user interface language to "English (USA)" and the locale to "Swedish (Sweden)", the locale will be set to "English (USA)" the next time I start LibreOffice.
Further tests have showed that this bug is only present i LibreOffice Portable from portableapps.com, so I've changed the status to "not our bug".