Bug 131241 - Hex value gives negative long / single / double number
Summary: Hex value gives negative long / single / double number
Status: RESOLVED NOTABUG
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: BASIC (show other bugs)
Version:
(earliest affected)
6.4.1.2 release
Hardware: All All
: medium normal
Assignee: Andreas Heinisch
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: Macro-StarBasic
  Show dependency treegraph
 
Reported: 2020-03-09 16:26 UTC by Spiros Georgaras
Modified: 2020-03-27 16:21 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Spiros Georgaras 2020-03-09 16:26:37 UTC
Description:
A hex value assigned to a long variable gives a negative number, instead of a correct number.

LO 6.1.5 debian:  aLong = &HFFE0
                     (aLong = 65,504)

LO 6.4.1 manjaro: aLong = &HFFE0
                     (aLong = -32)


Steps to Reproduce:
Just run this and get an overflow error

Sub aLetterExample
	Dim aLetter As Long
	aLetter = &HFFE0
	msgbox chr(aLetter)
End Sub


This would work on both versions

Sub aLetterExampleFix
	Dim aLetter As Long
	aLetter = CLng("&HFFE0")
	msgbox chr(aLetter)
End Sub


Actual Results:
-32

Expected Results:
65504


Reproducible: Always


User Profile Reset: No



Additional Info:
assign 65504 tp variable
Comment 1 Spiros Georgaras 2020-03-11 10:40:35 UTC
This is probably due to
https://wiki.documentfoundation.org/ReleaseNotes/6.4#BASIC

Macros: When converting Hex strings of negative value, treat the high-order bit always as sign bit, e.g., &H8000 = -2^15, &HFFFF = -1, and &H7FFFFFFF = 2^31-1. Behaviour is identical in either modes, i.e., LibO Basic, with Option Compatible, and Option VBA Support 0/1. tdf#62326 (Andreas Heinisch)

So, this is not a bug, but the only way to get around it is to use

aLong = CLng("&HFFE0")
Comment 2 Xisco Faulí 2020-03-18 17:32:12 UTC
@Andreas, could you please comment on this issue ?
Comment 3 Andreas Heinisch 2020-03-18 18:16:58 UTC
So the problem is, that the value of &HFFE0 lies within the Integer range.
Therefore, it will be treated as a negative value, since the high-order bit
is set. Then the negative value will be converted to a long value, when the value to the variable is assigned.

Codepointer:
https://opengrok.libreoffice.org/xref/core/basic/source/comp/scanner.cxx?r=7ddedd25#490

Related:
https://bugs.documentfoundation.org/show_bug.cgi?id=130476
Comment 4 Andreas Heinisch 2020-03-18 18:24:24 UTC
I think the behaviour of LO is correct, but data type characters with literals should affect resulting value type, so the following code should work in the future (note the ampersand at the end of the hex value).

Sub aLetterExample
	Dim aLetter As Long
	aLetter = &HFFE0&
	msgbox chr(aLetter)
End Sub

I will target this as I have finished another bug.
Comment 5 Andreas Heinisch 2020-03-27 13:22:54 UTC
Trailing data type characters should now work due to https://gerrit.libreoffice.org/c/core/+/90978
Comment 6 Xisco Faulí 2020-03-27 16:18:22 UTC
@Andreas, should this issue be closed as RESOLVED ?
Comment 7 Andreas Heinisch 2020-03-27 16:21:25 UTC
The behaviour is as intended like we discussed in https://bugs.documentfoundation.org/show_bug.cgi?id=62326.

I think we can close it