Skip to content

gh-143050: correct PyLong_FromString() to use _PyLong_Negate() #145901

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vstinner merged 13 commits into from
Mar 31, 2026
+34 −18
Merged

gh-143050: correct PyLong_FromString() to use _PyLong_Negate() #145901

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
fbcec5a
gh-143050: correct PyLong_FromString() to use _PyLong_Negate()
skirpichev Mar 13, 2026
b8333ba
cleanup, use _testcapi.test_immortal_small_ints()
skirpichev Mar 13, 2026
20e79c6
+ add also test with nonzero small int
skirpichev Mar 14, 2026
413e56d
_long_is_small_int() -> _PyLong_IsImmortal() helper
skirpichev Mar 14, 2026
b892216
Merge branch 'master' into fix-PyLong_FromString/143050
skirpichev Mar 17, 2026
78c5b4b
address review: rename _PyLong_IsImmortal -> _PyLong_HasImmortalTag
skirpichev Mar 17, 2026
69d35bd
add assert
skirpichev Mar 18, 2026
2f927e0
address review: simplify assert
skirpichev Mar 18, 2026
53523de
Merge branch 'master' into fix-PyLong_FromString/143050
skirpichev Mar 27, 2026
1f363f4
address review: some comments
skirpichev Mar 28, 2026
41961ed
Merge branch 'main' into fix-PyLong_FromString/143050
skirpichev Mar 30, 2026
0b29e28
address review: add assert on _PY_IS_SMALL_INT
skirpichev Mar 30, 2026
45b3b04
Rename to _PyLong_IsSmallInt()
vstinner Mar 31, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion Include/internal/pycore_long.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -231,6 +231,16 @@ _PyLong_IsPositive(const PyLongObject *op)
return (op->long_value.lv_tag & SIGN_MASK) == 0;
}

static inline bool
Comment thread
skirpichev marked this conversation as resolved.
_PyLong_HasImmortalTag(const PyLongObject *op)
Copy link
Copy Markdown
Member

@vstinner vstinner Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO the intent here is to check if an integer is a small integer, rather than checking for a tag. So I would prefer to rename the function to _PyLong_IsSmallInt().

I wanted to suggest referring to _PY_IS_SMALL_INT() but I noticed the macro is wrong. I wrote #146631 to fix the macro.

Copy link
Copy Markdown
Member Author

@skirpichev skirpichev Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are two notions for "small int"'s. One - by value and other is - this. If @colesbury is ok - I'm fine with renaming..

Copy link
Copy Markdown
Member

@vstinner vstinner Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_PyLong_HasImmortalTag() and _PY_IS_SMALL_INT() should be true or false for the same values.

For example, you can add the following check for _PyLong_HasImmortalTag():

+    assert((_PyLong_IsCompact(op) && _PY_IS_SMALL_INT(_PyLong_CompactValue(op)))
+           || (!is_small_int));

(You need to update your branch to retrieve my _PY_IS_SMALL_INT() change.)

For me, testing IMMORTALITY_BIT_MASK is just an implementation detail to check if an integer is "a small int".

Copy link
Copy Markdown
Member Author

@skirpichev skirpichev Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added an assert. BTW, what do you think on #145901 (comment)?

I don't think we should mention _PY_IS_SMALL_INT in the description of the function, assert is readable enough.

For me, testing IMMORTALITY_BIT_MASK is just an implementation detail to check if an integer is "a small int".

For now, we use _PY_IS_SMALL_INT() to check this. But eventually this could be just testing a flag.

Copy link
Copy Markdown
Member

@vstinner vstinner Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer to rename the function to _PyLong_IsSmallInt().

Copy link
Copy Markdown
Member Author

@skirpichev skirpichev Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added suggestions.

Comment thread
vstinner marked this conversation as resolved.
Outdated
{
assert(PyLong_Check(op));
bool is_small_int = (op->long_value.lv_tag & IMMORTALITY_BIT_MASK) != 0;
assert(PyLong_CheckExact(op) || (!is_small_int));
assert((is_small_int && _Py_IsImmortal(op)) || (!is_small_int));
Comment thread
colesbury marked this conversation as resolved.
Outdated
return is_small_int;
}

static inline Py_ssize_t
_PyLong_DigitCount(const PyLongObject *op)
{
Expand Down Expand Up @@ -292,7 +302,9 @@ _PyLong_SetDigitCount(PyLongObject *op, Py_ssize_t size)
#define NON_SIZE_MASK ~(uintptr_t)((1 << NON_SIZE_BITS) - 1)

static inline void
_PyLong_FlipSign(PyLongObject *op) {
_PyLong_FlipSign(PyLongObject *op)
{
assert(!_PyLong_HasImmortalTag(op));
Comment thread
vstinner marked this conversation as resolved.
Outdated
unsigned int flipped_sign = 2 - (op->long_value.lv_tag & SIGN_MASK);
op->long_value.lv_tag &= NON_SIZE_MASK;
op->long_value.lv_tag |= flipped_sign;
Expand Down
7 changes: 7 additions & 0 deletions Lib/test/test_capi/test_long.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -803,6 +803,13 @@ def to_digits(num):
self.assertEqual(pylongwriter_create(negative, digits), num,
(negative, digits))

def test_bug_143050(self):
with support.adjust_int_max_str_digits(0):
int('-' + '0' * 7000, 10)
Comment thread
skirpichev marked this conversation as resolved.
_testcapi.test_immortal_small_ints()
int('-' + '0' * 7000 + '123', 10)
Comment thread
skirpichev marked this conversation as resolved.
_testcapi.test_immortal_small_ints()
Copy link
Copy Markdown
Member Author

@skirpichev skirpichev Mar 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm slightly worry that this pass will non-debug builds.

@corona10 (author of #103962), are you sure there is no other way to implement this helper? Return a different value, raise an error?

Copy link
Copy Markdown
Contributor

@colesbury colesbury Mar 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's okay as long as it catches the regression in a debug build.

Copy link
Copy Markdown
Member

@vstinner vstinner Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_testcapi is always built with assertions, even in release mode.

Copy link
Copy Markdown
Member

@vstinner vstinner Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I built Python in release mode without PyLong_FromString() fix. _PyLong_FlipSign() assertion didn't catch the bug since Python was built without assertions. But at least, test_immortal_small_ints() was able to detect that small integers were corrupted:

test_bug_143050 (test.test_capi.test_long.LongTests.test_bug_143050) ...
python: ./Modules/_testcapi/immortal.c:35: test_immortal_small_ints: Assertion `has_int_immortal_bit' failed.
Fatal Python error: Aborted

Current thread 0x00007f617b91b780 [python] (most recent call first):
  File "/home/vstinner/python/main/Lib/test/test_capi/test_long.py", line 811 in test_bug_143050



if __name__ == "__main__":
unittest.main()
4 changes: 2 additions & 2 deletions Modules/_testcapi/immortal.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,13 @@ test_immortal_small_ints(PyObject *self, PyObject *Py_UNUSED(ignored))
for (int i = -5; i <= 1024; i++) {
PyObject *obj = PyLong_FromLong(i);
assert(verify_immortality(obj));
int has_int_immortal_bit = ((PyLongObject *)obj)->long_value.lv_tag & IMMORTALITY_BIT_MASK;
int has_int_immortal_bit = _PyLong_HasImmortalTag((PyLongObject *)obj);
Comment thread
vstinner marked this conversation as resolved.
Outdated
Comment thread
vstinner marked this conversation as resolved.
Outdated
assert(has_int_immortal_bit);
}
for (int i = 1025; i <= 1030; i++) {
PyObject *obj = PyLong_FromLong(i);
assert(obj);
int has_int_immortal_bit = ((PyLongObject *)obj)->long_value.lv_tag & IMMORTALITY_BIT_MASK;
int has_int_immortal_bit = _PyLong_HasImmortalTag((PyLongObject *)obj);
Comment thread
vstinner marked this conversation as resolved.
Outdated
assert(!has_int_immortal_bit);
Py_DECREF(obj);
}
Expand Down
20 changes: 5 additions & 15 deletions Objects/longobject.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3118,11 +3118,11 @@ PyLong_FromString(const char *str, char **pend, int base)
}

/* Set sign and normalize */
if (sign < 0) {
_PyLong_FlipSign(z);
Comment thread
skirpichev marked this conversation as resolved.
}
long_normalize(z);
z = maybe_small_long(z);
if (sign < 0) {
_PyLong_Negate(&z);
}

if (pend != NULL) {
*pend = (char *)str;
Expand Down Expand Up @@ -3622,21 +3622,11 @@ long_richcompare(PyObject *self, PyObject *other, int op)
Py_RETURN_RICHCOMPARE(result, 0, op);
}

static inline int
/// Return 1 if the object is one of the immortal small ints
_long_is_small_int(PyObject *op)
{
PyLongObject *long_object = (PyLongObject *)op;
int is_small_int = (long_object->long_value.lv_tag & IMMORTALITY_BIT_MASK) != 0;
assert((!is_small_int) || PyLong_CheckExact(op));
return is_small_int;
}

void
_PyLong_ExactDealloc(PyObject *self)
{
assert(PyLong_CheckExact(self));
if (_long_is_small_int(self)) {
if (_PyLong_HasImmortalTag((PyLongObject *)self)) {
Comment thread
vstinner marked this conversation as resolved.
Outdated
// See PEP 683, section Accidental De-Immortalizing for details
_Py_SetImmortal(self);
return;
Expand All @@ -3651,7 +3641,7 @@ _PyLong_ExactDealloc(PyObject *self)
static void
long_dealloc(PyObject *self)
{
if (_long_is_small_int(self)) {
if (_PyLong_HasImmortalTag((PyLongObject *)self)) {
Comment thread
vstinner marked this conversation as resolved.
Outdated
/* This should never get called, but we also don't want to SEGV if
* we accidentally decref small Ints out of existence. Instead,
* since small Ints are immortal, re-set the reference count.
Expand Down
Loading