Skip to content

gh-143050: add helper _PyLong_InitTag() #147956

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vstinner merged 9 commits into from
Apr 1, 2026
Merged

gh-143050: add helper _PyLong_InitTag() #147956

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
d7073e5
gh-143050: add helper _PyLong_Init()
skirpichev Apr 1, 2026
095a186
Merge branch 'master' into add-_PyLong_Init
skirpichev Apr 1, 2026
ddd329f
address review: rename function to _PyLong_InitTag
skirpichev Apr 1, 2026
f6ab450
address review: skip initialization of digit[0]
skirpichev Apr 1, 2026
f5df517
address review: restore comment
skirpichev Apr 1, 2026
21c59bb
XXX redo _PyLong_IsSmallInt
skirpichev Apr 1, 2026
640b326
address review: fix comment
skirpichev Apr 1, 2026
613d2a1
restore initialization of digit[0]
skirpichev Apr 1, 2026
a2bd3c4
Update Include/internal/pycore_long.h
vstinner Apr 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions Include/internal/pycore_long.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -285,6 +285,21 @@ _PyLong_SameSign(const PyLongObject *a, const PyLongObject *b)
return (a->long_value.lv_tag & SIGN_MASK) == (b->long_value.lv_tag & SIGN_MASK);
}

/* Initialize a freshly-allocated int.
*
* Fast operations for single digit integers (including zero)
* assume that there is always at least one digit present.
* The digit has to be initialized explicitly to avoid
* use-of-uninitialized-value.
*/
Comment thread
skirpichev marked this conversation as resolved.
Outdated
static inline void
_PyLong_Init(PyLongObject *op)
{
assert(PyLong_Check(op));
op->long_value.lv_tag = 1; /* non-immortal zero */
Comment thread
vstinner marked this conversation as resolved.
Outdated
op->long_value.ob_digit[0] = 0;
Comment thread
skirpichev marked this conversation as resolved.
Outdated
}

#define TAG_FROM_SIGN_AND_SIZE(sign, size) \
((uintptr_t)(1 - (sign)) | ((uintptr_t)(size) << NON_SIZE_BITS))

Expand All @@ -294,13 +309,15 @@ _PyLong_SetSignAndDigitCount(PyLongObject *op, int sign, Py_ssize_t size)
assert(size >= 0);
assert(-1 <= sign && sign <= 1);
assert(sign != 0 || size == 0);
assert(!_PyLong_IsSmallInt(op));
op->long_value.lv_tag = TAG_FROM_SIGN_AND_SIZE(sign, size);
}

static inline void
_PyLong_SetDigitCount(PyLongObject *op, Py_ssize_t size)
{
assert(size >= 0);
assert(!_PyLong_IsSmallInt(op));
op->long_value.lv_tag = (((size_t)size) << NON_SIZE_BITS) | (op->long_value.lv_tag & SIGN_MASK);
}

Expand Down
29 changes: 17 additions & 12 deletions Objects/longobject.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -185,11 +185,9 @@ long_alloc(Py_ssize_t size)
return NULL;
}
_PyObject_Init((PyObject*)result, &PyLong_Type);
_PyLong_Init(result);
}
_PyLong_SetSignAndDigitCount(result, size != 0, size);
/* The digit has to be initialized explicitly to avoid
* use-of-uninitialized-value. */
result->long_value.ob_digit[0] = 0;
Copy link
Copy Markdown
Member

@vstinner vstinner Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I'm not convinced that this initialization is useful, the comment says that it is. We should keep it, no?

Copy link
Copy Markdown
Member

@vstinner vstinner Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggested to keep this initialization. You marked my comment as resolved, I don't understand. Do you consider that it's not needed?

An alternative would be to fill digits with a pattern to detect usage of non-initialized memory when Python is built in debug mode:

#ifdef Py_DEBUG
    // Fill digits with a known pattern to detect usage of uninitialized memory
    memset(result->long_value.ob_digit, 0xff,
           sizeof(result->long_value.ob_digit[0]) * ndigits);
#endif

We use this strategy for:

  • str: see unicode_fill_invalid() function.
  • PyBytesWriter: memset(byteswriter_data(writer), 0xff, byteswriter_allocated(writer)) in PyBytesWriter_Create()

Copy link
Copy Markdown
Member Author

@skirpichev skirpichev Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you suggest to keep this just here? BTW, tests pass - I rerun one job again.

Copy link
Copy Markdown
Member

@vstinner vstinner Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ob_digit[0] = 0; was added by commit fc130c4 of issue gh-102509 to fix the OSS Fuzz report https://issues.oss-fuzz.com/issues/42516166.

Looks like it's complaining about a use of unit values in _PyLong_FromByteArray
Allocation happens here as part of _PyLong_New

cpython/Objects/longobject.c

Line 911 in 41bc101

v = _PyLong_New(ndigits);

and the claimed use is at

cpython/Objects/longobject.c

Line 957 in 41bc101

return (PyObject *)maybe_small_long(long_normalize(v));

inside of maybe_small_long and IS_SMALL_INT

I understand that the maybe_small_long() at the end of _PyLong_FromByteArray() triggered the issue when Python was built with --with-memory-sanitizer (-fsanitize=memory). From what I understood, the sign was initialized (to zero) but not ob_digit[0].

Oh interesting, k_mul() already does what I proposed above:

#ifdef Py_DEBUG
    /* Fill with trash, to catch reference to uninitialized digits. */
    memset(ret->long_value.ob_digit, 0xDF, _PyLong_DigitCount(ret) * sizeof(digit));
#endif

k_lopsided_mul() sets explicitly all digits to zero:

    memset(ret->long_value.ob_digit, 0, _PyLong_DigitCount(ret) * sizeof(digit));

return result;
}

Expand Down Expand Up @@ -258,6 +256,7 @@ _PyLong_FromMedium(sdigit x)
return NULL;
}
_PyObject_Init((PyObject*)v, &PyLong_Type);
_PyLong_Init(v);
}
digit abs_x = x < 0 ? -x : x;
_PyLong_SetSignAndDigitCount(v, x<0?-1:1, 1);
Expand Down Expand Up @@ -337,6 +336,7 @@ medium_from_stwodigits(stwodigits x)
return PyStackRef_NULL;
}
_PyObject_Init((PyObject*)v, &PyLong_Type);
_PyLong_Init(v);
}
digit abs_x = x < 0 ? (digit)(-x) : (digit)x;
_PyLong_SetSignAndDigitCount(v, x<0?-1:1, 1);
Expand Down Expand Up @@ -6011,29 +6011,34 @@ static PyObject *
long_subtype_new(PyTypeObject *type, PyObject *x, PyObject *obase)
{
PyLongObject *tmp, *newobj;
Py_ssize_t i, n;
Py_ssize_t size, ndigits;
int sign;

assert(PyType_IsSubtype(type, &PyLong_Type));
tmp = (PyLongObject *)long_new_impl(&PyLong_Type, x, obase);
if (tmp == NULL)
return NULL;
assert(PyLong_Check(tmp));
n = _PyLong_DigitCount(tmp);
size = _PyLong_DigitCount(tmp);
/* Fast operations for single digit integers (including zero)
* assume that there is always at least one digit present. */
if (n == 0) {
n = 1;
}
newobj = (PyLongObject *)type->tp_alloc(type, n);
ndigits = size ? size : 1;
newobj = (PyLongObject *)type->tp_alloc(type, ndigits);
if (newobj == NULL) {
Py_DECREF(tmp);
return NULL;
}
assert(PyLong_Check(newobj));
newobj->long_value.lv_tag = tmp->long_value.lv_tag & ~IMMORTALITY_BIT_MASK;
for (i = 0; i < n; i++) {
newobj->long_value.ob_digit[i] = tmp->long_value.ob_digit[i];
if (_PyLong_IsCompact(tmp)) {
sign = _PyLong_CompactSign(tmp);
}
else {
sign = _PyLong_NonCompactSign(tmp);
}
_PyLong_Init(newobj);
_PyLong_SetSignAndDigitCount(newobj, sign, size);
memcpy(newobj->long_value.ob_digit, tmp->long_value.ob_digit,
ndigits * sizeof(digit));
Py_DECREF(tmp);
return (PyObject *)newobj;
}
Expand Down
Loading