gh-143050: add helper _PyLong_InitTag() by skirpichev · Pull Request #147956 · python/cpython

skirpichev · 2026-04-01T03:54:09Z

With this we can assume, that _PyLong_Set*() operate on non-immortal integers.

Issue: Remove redundant decref from _PyLong_Negate #143050

With this we can assume, that _PyLong_Set*() operate on non-immortal integers.

vstinner · 2026-04-01T10:08:57Z

Include/internal/pycore_long.h

+{
+    assert(PyLong_Check(op));
+    op->long_value.lv_tag = 1; /* non-immortal zero */
+    op->long_value.ob_digit[0] = 0;


I would prefer to only initialize the tag and so rename the function to _PyLong_InitTag().

op->long_value.ob_digit[0] = 0; is only used (needed) by long_alloc(). All other functions using _PyLong_Init() will rewrite ob_digit[0] immediately and so it's inefficient.

I think this may trigger a compiler warning. I did renaming, lets do another change in a second commit.

lets do another change in a second commit.

You didn't make the second commit yet.

As I expected, that's relevant:
https://github.com/python/cpython/actions/runs/23846329148/job/69514515704

vstinner · 2026-04-01T11:34:50Z

Objects/longobject.c

    _PyLong_SetSignAndDigitCount(result, size != 0, size);
-    /* The digit has to be initialized explicitly to avoid
-     * use-of-uninitialized-value. */
-    result->long_value.ob_digit[0] = 0;


While I'm not convinced that this initialization is useful, the comment says that it is. We should keep it, no?

I suggested to keep this initialization. You marked my comment as resolved, I don't understand. Do you consider that it's not needed?

An alternative would be to fill digits with a pattern to detect usage of non-initialized memory when Python is built in debug mode:

#ifdef Py_DEBUG // Fill digits with a known pattern to detect usage of uninitialized memory memset(result->long_value.ob_digit, 0xff, sizeof(result->long_value.ob_digit[0]) * ndigits); #endif

We use this strategy for:

str: see unicode_fill_invalid() function.

PyBytesWriter: memset(byteswriter_data(writer), 0xff, byteswriter_allocated(writer)) in PyBytesWriter_Create()

Do you suggest to keep this just here? BTW, tests pass - I rerun one job again.

vstinner · 2026-04-01T12:10:33Z

"Tests / Android (x86_64)" failed with: Error on ZipFile unknown archive. It's unrelated to this change.

  > Task :app:maxVersionSetup
  "Install Android Emulator v.36.4.10" ready.
  Installing Android Emulator in /usr/local/lib/android/sdk/emulator
  "Install Android Emulator v.36.4.10" complete.
  "Install Android Emulator v.36.4.10" finished.
  Checking the license for package AOSP ATD Intel x86_64 Atom System Image in /usr/local/lib/android/sdk/licenses
  License for package AOSP ATD Intel x86_64 Atom System Image accepted.
  Preparing "Install AOSP ATD Intel x86_64 Atom System Image API 32 (revision 1)".
  Warning: An error occurred while preparing SDK package AOSP ATD Intel x86_64 Atom System Image: Error on ZipFile unknown archive.:
  java.io.IOException: Error on ZipFile unknown archive
  	at org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java:383)
  	at org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java:323)
  	at org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java:279)
  	at com.android.repository.util.InstallerUtil.unzip(InstallerUtil.java:101)
  	at com.android.repository.impl.installer.BasicInstaller.doPrepare(BasicInstaller.java:91)

skirpichev · 2026-04-01T12:12:49Z

This yes, but linked above fuzzer failure is relevant.

…

On Wed, Apr 1, 2026, 15:10 Victor Stinner ***@***.***> wrote: *vstinner* left a comment (python/cpython#147956) <#147956?email_source=notifications&email_token=AAQOKGA5T2QHBNZ3SVWPVWT4TUBM7A5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIMJWHE3DGNZXGIZKM4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2LK4DSL5RW63LNMVXHIX3POBSW4X3DNRUWG2Y#issuecomment-4169637722> "Tests / Android (x86_64)" failed with: Error on ZipFile unknown archive. It's unrelated to this change. > Task :app:maxVersionSetup "Install Android Emulator v.36.4.10" ready. Installing Android Emulator in /usr/local/lib/android/sdk/emulator "Install Android Emulator v.36.4.10" complete. "Install Android Emulator v.36.4.10" finished. Checking the license for package AOSP ATD Intel x86_64 Atom System Image in /usr/local/lib/android/sdk/licenses License for package AOSP ATD Intel x86_64 Atom System Image accepted. Preparing "Install AOSP ATD Intel x86_64 Atom System Image API 32 (revision 1)". Warning: An error occurred while preparing SDK package AOSP ATD Intel x86_64 Atom System Image: Error on ZipFile unknown archive.: java.io.IOException: Error on ZipFile unknown archive at org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java:383) at org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java:323) at org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java:279) at com.android.repository.util.InstallerUtil.unzip(InstallerUtil.java:101) at com.android.repository.impl.installer.BasicInstaller.doPrepare(BasicInstaller.java:91) — Reply to this email directly, view it on GitHub <#147956?email_source=notifications&email_token=AAQOKGA5T2QHBNZ3SVWPVWT4TUBM7A5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIMJWHE3DGNZXGIZKM4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2LK4DSL5RW63LNMVXHIX3POBSW4X3DNRUWG2Y#issuecomment-4169637722>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAQOKGB4LCCKKOKT66AWXQD4TUBM7AVCNFSM6AAAAACXIS7L4WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DCNRZGYZTONZSGI> . You are receiving this because you authored the thread.Message ID: ***@***.***>

vstinner · 2026-04-01T12:15:30Z

"Tests / CIFuzz / cpython3 (memory)" CI logs an use-of-uninitialized-value:

==45==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x55af57984f54 in _PyLong_IsSmallInt /src/cpython3/Include/internal/pycore_long.h:243:5
    #1 0x55af57984f54 in _PyLong_SetSignAndDigitCount /src/cpython3/Include/internal/pycore_long.h:305:5
    #2 0x55af579844c6 in long_alloc /src/cpython3/Objects/longobject.c:190:5
    #3 0x55af5798ef9a in PyLong_FromUnsignedLong /src/cpython3/Objects/longobject.c:442:5
    #4 0x55af5798ef9a in PyLong_FromVoidPtr /src/cpython3/Objects/longobject.c:1505:12

pycore_long.h:243 is the following assertion in _PyLong_IsSmallInt():

    assert((_PyLong_IsCompact(op)
            && _PY_IS_SMALL_INT(_PyLong_CompactValue(op)))
           || (!is_small_int));

Oh right, op->long_value.ob_digit[0] = 0; is needed before calling _PyLong_IsSmallInt() to have initialized memory (_PyLong_CompactValue()).

vstinner

What do you think of changing _PyLong_IsSmallInt() implementation like that?

static inline bool
_PyLong_IsSmallInt(const PyLongObject *op)
{
    assert(PyLong_Check(op));
    bool is_small_int = (op->long_value.lv_tag & IMMORTALITY_BIT_MASK) != 0;
    if (is_small_int) {
        assert(PyLong_CheckExact(op));
        assert(_Py_IsImmortal(op));
        assert((_PyLong_IsCompact(op)
                && _PY_IS_SMALL_INT(_PyLong_CompactValue(op))));
    }
    return is_small_int;
}

The current code runs many checks when is_small_int is false which are not needed.

skirpichev · 2026-04-01T12:30:14Z

What do you think of changing _PyLong_IsSmallInt() implementation like that?

Lets try, but I suspect it might fall in other place.

vstinner · 2026-04-01T12:49:00Z

Include/internal/pycore_long.h

+/* Initialize a freshly-allocated int.
+ *
+ * Fast operations for single digit integers (including zero)
+ * assume that there is always at least one digit present.
+ * The digit has to be initialized explicitly to avoid
+ * use-of-uninitialized-value.
+ */


This function doesn't initialize the first digit, so I'm not sure why you reintroduce the "The digit has to be initialized explicitly" comment:

Suggested change

/* Initialize a freshly-allocated int.

*

* Fast operations for single digit integers (including zero)

* assume that there is always at least one digit present.

* The digit has to be initialized explicitly to avoid

* use-of-uninitialized-value.

*/

/* Initialize the tag of a freshly-allocated int. */

vstinner · 2026-04-01T12:57:26Z

Objects/longobject.c

    _PyLong_SetSignAndDigitCount(result, size != 0, size);
-    /* The digit has to be initialized explicitly to avoid
-     * use-of-uninitialized-value. */
-    result->long_value.ob_digit[0] = 0;


I suggested to keep this initialization. You marked my comment as resolved, I don't understand. Do you consider that it's not needed?

An alternative would be to fill digits with a pattern to detect usage of non-initialized memory when Python is built in debug mode:

#ifdef Py_DEBUG // Fill digits with a known pattern to detect usage of uninitialized memory memset(result->long_value.ob_digit, 0xff, sizeof(result->long_value.ob_digit[0]) * ndigits); #endif

We use this strategy for:

str: see unicode_fill_invalid() function.

PyBytesWriter: memset(byteswriter_data(writer), 0xff, byteswriter_allocated(writer)) in PyBytesWriter_Create()

pythongh-143050: add helper _PyLong_Init()

d7073e5

With this we can assume, that _PyLong_Set*() operate on non-immortal integers.

skirpichev added the skip news label Apr 1, 2026

bedevere-app bot mentioned this pull request Apr 1, 2026

Remove redundant decref from _PyLong_Negate #143050

Closed

bedevere-app bot added the awaiting review label Apr 1, 2026

vstinner reviewed Apr 1, 2026

View reviewed changes

skirpichev added 2 commits April 1, 2026 13:16

Merge branch 'master' into add-_PyLong_Init

095a186

address review: rename function to _PyLong_InitTag

ddd329f

skirpichev changed the title ~~gh-143050: add helper _PyLong_Init()~~ gh-143050: add helper _PyLong_InitTag() Apr 1, 2026

address review: skip initialization of digit[0]

f6ab450

vstinner reviewed Apr 1, 2026

View reviewed changes

skirpichev added 2 commits April 1, 2026 15:28

address review: restore comment

f5df517

XXX redo _PyLong_IsSmallInt

21c59bb

vstinner reviewed Apr 1, 2026

View reviewed changes

Uh oh!

Conversation

skirpichev commented Apr 1, 2026 • edited by bedevere-app bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vstinner commented Apr 1, 2026

Uh oh!

skirpichev commented Apr 1, 2026 via email

Uh oh!

vstinner commented Apr 1, 2026

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

skirpichev commented Apr 1, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

skirpichev commented Apr 1, 2026 •

edited by bedevere-app bot

Loading