Destruct internals during interpreter finalization

[b-pass]

Description

pybind11 leaks three PyTypes (metaclass, function_record, static_property) when an interpreter or sub-interpreter is finalized. This is about 1KB each. These should be released during finalization so that simply creating interpreters does not leak memory.

This fixes #5794 (Python itself leaks more memory than this, but we can fix what we're doing at least...)

If a pybind11 module is loaded into a non-pybind11-owned sub-interpreter, then the pybind11 internals are leaked when the sub-interpreter shuts down. This PR also fixes that, by having the internals destruct and free during the interpreter finalization.

Suggested changelog entry:

• Deallocate pybind11 internals during (sub-)interpreter shutdown to avoid memory leaks.

---

📚 Documentation preview 📚: https://pybind11--5958.org.readthedocs.build/

[oremanj]

I think for shared internals you should go ahead and deallocate the internals struct, but not the pointer to it, from the capsule destructor. That is, call get_pp()->reset() followed by unref() on the pp_manager, but don't call destroy(). (For local internals, go ahead and call destroy().) This reduces the leak for the main interpreter to 8 bytes (the std::unique_ptr<internals> itself, but not its contents), and means you can straightforwardly put internals cleanup code in the destructor of struct internals.

The internals state dict is cleared very late in interpreter shutdown - AFAICT it's the last place that calls user-controlled code where the Python C API is available. C-level Py_AtExit handlers are later but those aren't allowed to call into Python. Step 24 in this wonderful list, which I wish had a wider audience than a random GitHub comment: hudson-trading/pymetabind#2 (comment) (that list is for main interpreter shutdown, but I believe subinterpreters do a subset in basically the same order).

If anyone is trying to access internals after the state dict is cleared, it's overwhelmingly likely that whatever they're going to do with the result is invalid anyway.

If somehow someone does manage to ask for internals after they've been destroyed, there are two possibilities for what will occur, and I think both are acceptable.

• If they are accessing the internals through a pp_manager that has had unref() called -- that is, if they are accessing local internals, or shared internals from the extension module that originally created them -- they will create and initialize a new internals struct and add a new entry for it to the interpreter state dict. If this occurred before the state dict was destroyed (the only possibility for that I can think of is if it's done from the destructor of a different entry in the state dict), then the new internals will promptly be destroyed again as PyDict_Clear continues its work. If it occurred after the state dict was destroyed, it will fail because the state dict doesn't exist - that's fine since you're really not supposed to run anything that touches the Python C API after that point.
• If they are accessing through a different pp_manager (different extension module's reference to the same shared internals), they will see that the std::unique_ptr<internals> (which is still allocated) holds null, and fix that by creating and initializing a new internals struct but not registering it in the interpreter state dict. This has the outside potential to create a split-brain scenario where the extension modules in this category share one internals and the extension module in the previous category has a different internals. But everything is shutting down anyway, so it will be difficult for anyone to observe the mildly confusing results.

If we were OK with extension modules in the second category above crashing (since they hold a pointer to a deallocated std::unique_ptr<internals> in their pp_manager) we could go ahead and destroy() the pp_manager in the capsule destructor. But I think eight bytes' leak is a fair price to pay to allow them to not crash.

The perfect solution would be to store inside internals an intrusive linked list of all the pp_managers that refer to it. Then upon internals destruction we could automatically unref() all of them, so that it would be safe to deallocate the internals pointer too. This eliminates the leak and the split-brain possibility, but adding the list head would require an internals version bump. Maybe leave a TODO comment with a link to this thread.

[b-pass]

Ok, I have updated the title and description and code to delete the internals itself in the capsule destructor. I didn't have it call unref currently, so @oremanj's second case applies. If something manages to use pybind11 during finalization after the internals capsule is destroyed, it will re-create (and subsequently might leak) its own new internals. This is an exceedingly rare case, and even in the event it happens the behavior won't result in a crash.

[oremanj]

This check defeats the point. Py_IsInitialized begins returning false quite early in Py_Finalize, well before the state dict is cleared, so adding this check will bring your leak back (at least for the main interpreter).

internals can only be freed by a DECREF on the capsule (in which case its destructor runs at a time when it's safe to DECREF things) or by an explicit call to internals_pp_manager::destroy(). Do any of the latter happen at times when it's not safe to DECREF things?

[b-pass]

If internals is re-created during finalization, then it will be set in the unique_ptr and the current pybind11 finalization code will subsequently call destroy (after PyFinalize has completed) and delete the re-created version.

We could, of course, change the pybind11 finalization process to not behave this way....

[oremanj]

This is not a semantics-preserving change.

There are two possibilities for when the capsule gets destroyed:

• It could be destroyed almost immediately, if another thread inserted the same key before we did. See the comment on line 639. In that case, no one else has seen our newly-allocated payload, so we should delete it.
• It could be destroyed at interpreter finalization. That's when custom destruction logic might come into play.

If two threads concurrently try to create internals, your use of the final dtor from the start means that one of them will leak its unique_ptr. Not a terrible outcome, but easy enough to avoid.

Suggest you change the dtor parameter to dtor_if_inserted. Create the capsule initially with a destructor that deallocates the payload. If insertion succeeds and dtor_if_inserted is not null, then swap the capsule destructor to be dtor_if_inserted instead. Effectively, the old clear_destructor parameter is a special case of the dtor_if_inserted semantics: dtor_if_inserted is a no-op lambda if clear_destructor is true, or is null (i.e. continue with the original dtor that deletes the payload) if clear_destructor is false.

[b-pass]

Ok, I believe the latest change addresses this.

[rwgk]

I'm trying to get my head around the current state of this PR, starting with:

Cursor (Claude 4.5 Opus) generated, with minor manual edits:

---

Problem Being Solved

pybind11 leaks three PyTypeObjects (metaclass, function_record, static_property) when an interpreter or sub-interpreter is finalized (~1KB each). This PR fixes the leak by properly releasing these during interpreter shutdown.

Joshua's (@oremanj) Concerns and Their Resolution

1. Py_IsInitialized() check defeats the point

• Joshua noted that Py_IsInitialized() returns false early in Py_Finalize, before the state dict is cleared.
• Resolved: The code now uses is_interpreter_alive() which checks Py_IsInitialized() || Py_IsFinalizing(). During finalization, Py_IsFinalizing() is true, so the cleanup runs.

2. Use Py_CLEAR instead of Py_XDECREF + set to null

• Resolved: The current code uses Py_CLEAR().

3. Capsule destructor race condition

• Joshua pointed out that if two threads concurrently create internals, one would leak its unique_ptr.
• Resolved: The capsule is created with a simple delete destructor, and only swapped to internals_shutdown after successful insertion.

4. Don't destroy the unique_ptr itself (main architectural suggestion)

• Joshua suggested calling pp->reset() to destroy contents but not deleting the unique_ptr itself (8 bytes), since other modules may still hold references to it.
• Resolved: The internals_shutdown() function does exactly this - calls pp->reset() but leaves the unique_ptr allocated.

Current Implementation

The implementation:

1. Adds destructors to both internals and local_internals that call Py_CLEAR() on their PyType pointers
2. Uses is_interpreter_alive() to safely check if Python APIs can still be called
3. Properly handles the capsule destructor race condition
4. Accepts an 8-byte leak (the unique_ptr shell) to avoid crashes when other modules hold references

Assessment

The PR looks ready to merge. All critical concerns have been addressed. The remaining 8-byte leak per interpreter is an acceptable tradeoff for safety, as Joshua explicitly endorsed. The last commit (by Henry Schreiner) adds a CI cleanup step, suggesting maintainer involvement.

[rwgk]

EDIT: Please see #5961 for the continuation of the leak investigation

---

Cursor (Claude 4.5 Opus) generated:

---

Subinterpreter Memory Leak Investigation

I ran tests to measure memory usage with repeated subinterpreter create/destroy cycles, comparing pure C API against pybind11.

Summary of Results

Test	Per iteration	Notes
Pure C API (Py_NewInterpreter)	~3-6 kB	Simple create/destroy
Pure C API (Py_NewInterpreterFromConfig + PyRun)	~4 kB	With config matching pybind11's defaults
pybind11 upstream/master	~1,698 kB	Before this PR
pybind11 PR #5958	~1,697 kB	After this PR

Key Finding

The massive leak (~1.7 MB per subinterpreter cycle) is NOT primarily from Python itself. Pure C API subinterpreter create/destroy cycles leak only ~4-6 kB, while pybind11 leaks approximately 400x more.

This PR correctly fixes pybind11's leak of the three PyTypeObjects (~3 kB), but there appears to be a much larger leak somewhere else in pybind11's subinterpreter handling that this PR does not address.

Raw Test Output

Pure C API (simple Py_NewInterpreter):

Iterations: 1000
RSS before: 8704 kB
RSS after:  11776 kB
Increase:   3072 kB
Per iter:   3.07 kB

Pure C API (with PyInterpreterConfig + PyRun_SimpleString):

Iterations: 500
RSS before: 8704 kB
RSS after:  10744 kB
Increase:   2040 kB
Per iter:   4.08 kB

pybind11 upstream/master:

Iterations: 1000
RSS before: 11264 kB
RSS after:  1709568 kB
Increase:   1698304 kB
Per iter:   1698.3 kB

pybind11 PR #5958:

Iterations: 1000
RSS before: 11264 kB
RSS after:  1708032 kB
Increase:   1696768 kB
Per iter:   1696.77 kB

Test Environment

• Python: 3.14 (default/GIL build, commit df793163d58)
• pybind11 upstream/master: commit e44aae2 ("chore: bump CMake max policy to 4.2 (chore: bump CMake max policy to 4.2 #5944)")
• pybind11 PR branch: commit 5f9c30b ("Add cleanup step to CI workflow")
• Compiler: g++ -std=c++20 -O0 -g (pybind11) / gcc (pure C)
• Platform: Ubuntu 24.04 (WSL2)
• Memory measurement: /proc/self/status VmRSS

---

Test Code

pybind11 Test (measure_subinterpreter_leak.cpp)

// Memory leak measurement for subinterpreter create/destroy cycles
//
// Compile:
//   g++ -std=c++20 -O0 -g \
//       -I<pybind11>/include \
//       -I<python>/include/python3.14 \
//       -o measure_leak measure_subinterpreter_leak.cpp \
//       -L<python>/lib -Wl,-rpath,<python>/lib \
//       -lpython3.14 -lpthread -ldl -lutil
//
// Run:
//   ./measure_leak 500

#include <pybind11/embed.h>

#ifdef PYBIND11_HAS_SUBINTERPRETER_SUPPORT
#include <pybind11/subinterpreter.h>
#endif

#include <iostream>
#include <fstream>
#include <string>

namespace py = pybind11;

long get_rss_kb() {
    std::ifstream status("/proc/self/status");
    std::string line;
    while (std::getline(status, line)) {
        if (line.rfind("VmRSS:", 0) == 0) {
            // Format: "VmRSS:     12345 kB"
            long rss = 0;
            for (char c : line) {
                if (c >= '0' && c <= '9') {
                    rss = rss * 10 + (c - '0');
                }
            }
            return rss;
        }
    }
    return -1;
}

int main(int argc, char* argv[]) {
    int iterations = 10000;
    if (argc > 1) {
        iterations = std::stoi(argv[1]);
    }

    py::scoped_interpreter guard{};

#ifdef PYBIND11_HAS_SUBINTERPRETER_SUPPORT
    long rss_before = get_rss_kb();
    
    for (int i = 0; i < iterations; ++i) {
        {
            py::scoped_subinterpreter ssi;
        }
    }
    
    long rss_after = get_rss_kb();
    
    std::cout << "Iterations: " << iterations << "\n";
    std::cout << "RSS before: " << rss_before << " kB\n";
    std::cout << "RSS after:  " << rss_after << " kB\n";
    std::cout << "Increase:   " << (rss_after - rss_before) << " kB\n";
    std::cout << "Per iter:   " << (double)(rss_after - rss_before) / iterations << " kB\n";
#else
    std::cout << "Subinterpreter support: NO\n";
#endif

    return 0;
}

Pure C API Test - Simple (measure_leak_pure_c.c)

// Pure C API subinterpreter leak measurement (no pybind11)
// Uses simple Py_NewInterpreter/Py_EndInterpreter
//
// Compile:
//   gcc -o measure_leak_pure_c measure_leak_pure_c.c \
//       -I<python>/include/python3.14 \
//       -L<python>/lib -Wl,-rpath,<python>/lib \
//       -lpython3.14 -lpthread -ldl -lutil
//
// Run:
//   ./measure_leak_pure_c 500

#include <Python.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

long get_rss_kb() {
    FILE *f = fopen("/proc/self/status", "r");
    if (!f) return -1;
    
    char line[256];
    long rss = -1;
    while (fgets(line, sizeof(line), f)) {
        if (strncmp(line, "VmRSS:", 6) == 0) {
            rss = 0;
            for (char *p = line; *p; p++) {
                if (*p >= '0' && *p <= '9') {
                    rss = rss * 10 + (*p - '0');
                }
            }
            break;
        }
    }
    fclose(f);
    return rss;
}

int main(int argc, char *argv[]) {
    int iterations = 500;
    if (argc > 1) {
        iterations = atoi(argv[1]);
    }

    // Initialize main interpreter
    Py_Initialize();
    
    PyThreadState *main_tstate = PyThreadState_Get();
    
    long rss_before = get_rss_kb();
    
    for (int i = 0; i < iterations; i++) {
        // Create subinterpreter
        PyThreadState *sub_tstate = Py_NewInterpreter();
        if (!sub_tstate) {
            fprintf(stderr, "Failed to create subinterpreter at iteration %d\n", i);
            break;
        }
        
        // Destroy subinterpreter
        Py_EndInterpreter(sub_tstate);
        
        // Switch back to main
        PyThreadState_Swap(main_tstate);
    }
    
    long rss_after = get_rss_kb();
    
    printf("Iterations: %d\n", iterations);
    printf("RSS before: %ld kB\n", rss_before);
    printf("RSS after:  %ld kB\n", rss_after);
    printf("Increase:   %ld kB\n", rss_after - rss_before);
    printf("Per iter:   %.2f kB\n", (double)(rss_after - rss_before) / iterations);
    
    Py_Finalize();
    return 0;
}

Pure C API Test - With Config (measure_leak_pure_c_active.c)

This version uses Py_NewInterpreterFromConfig with settings matching pybind11's defaults, and runs a simple Python statement in each subinterpreter.

// Pure C API subinterpreter leak measurement - with PyInterpreterConfig
// More comparable to pybind11's scoped_subinterpreter
//
// Compile:
//   gcc -o measure_leak_pure_c_active measure_leak_pure_c_active.c \
//       -I<python>/include/python3.14 \
//       -L<python>/lib -Wl,-rpath,<python>/lib \
//       -lpython3.14 -lpthread -ldl -lutil
//
// Run:
//   ./measure_leak_pure_c_active 500

#include <Python.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

long get_rss_kb() {
    FILE *f = fopen("/proc/self/status", "r");
    if (!f) return -1;
    
    char line[256];
    long rss = -1;
    while (fgets(line, sizeof(line), f)) {
        if (strncmp(line, "VmRSS:", 6) == 0) {
            rss = 0;
            for (char *p = line; *p; p++) {
                if (*p >= '0' && *p <= '9') {
                    rss = rss * 10 + (*p - '0');
                }
            }
            break;
        }
    }
    fclose(f);
    return rss;
}

int main(int argc, char *argv[]) {
    int iterations = 500;
    if (argc > 1) {
        iterations = atoi(argv[1]);
    }

    // Initialize main interpreter
    Py_Initialize();
    
    PyThreadState *main_tstate = PyThreadState_Get();
    
    long rss_before = get_rss_kb();
    
    for (int i = 0; i < iterations; i++) {
        // Create subinterpreter with config (like pybind11 does)
        PyInterpreterConfig cfg = {
            .use_main_obmalloc = 0,
            .allow_fork = 0,
            .allow_exec = 0,
            .allow_threads = 1,
            .allow_daemon_threads = 0,
            .check_multi_interp_extensions = 1,
            .gil = PyInterpreterConfig_OWN_GIL,
        };
        
        PyThreadState *sub_tstate;
        PyStatus status = Py_NewInterpreterFromConfig(&sub_tstate, &cfg);
        if (PyStatus_Exception(status)) {
            fprintf(stderr, "Failed to create subinterpreter at iteration %d\n", i);
            break;
        }
        
        // We're now on the subinterpreter - run a tiny bit of code
        PyRun_SimpleString("x = 1");
        
        // Destroy subinterpreter
        Py_EndInterpreter(sub_tstate);
        
        // Switch back to main
        PyThreadState_Swap(main_tstate);
    }
    
    long rss_after = get_rss_kb();
    
    printf("Iterations: %d\n", iterations);
    printf("RSS before: %ld kB\n", rss_before);
    printf("RSS after:  %ld kB\n", rss_after);
    printf("Increase:   %ld kB\n", rss_after - rss_before);
    printf("Per iter:   %.2f kB\n", (double)(rss_after - rss_before) / iterations);
    
    Py_Finalize();
    return 0;
}

---

Analysis

The ~400x difference between pure C API (~4 kB/iter) and pybind11 (~1.7 MB/iter) suggests the leak is in pybind11's subinterpreter infrastructure, not in Python itself.

Looking at pybind11/subinterpreter.h, each scoped_subinterpreter cycle does:

1. subinterpreter::create() - Creates the subinterpreter and calls detail::get_internals() to initialize pybind11's per-interpreter state
2. subinterpreter_scoped_activate - Creates a new PyThreadState, switches to it, and stores it in internals.tstate
3. Destructor - Calls Py_EndInterpreter(), then destroy() on both internals_pp_manager and local_internals_pp_manager

The get_internals() call creates the internals struct which includes:

• static_property_type (PyTypeObject)
• default_metaclass (PyTypeObject)
• instance_base (PyObject)
• Various maps and containers

This PR fixes the leak of the PyTypeObjects (~3 kB), but the remaining ~1.7 MB leak suggests something else in the internals infrastructure is not being properly cleaned up.

Possible areas to investigate:

• The various maps in internals (registered_types_cpp, registered_types_py, etc.)
• The instance_base object
• Thread-local storage structures
• Something in local_internals

---

Conclusion

This PR is correct and should be merged - it fixes the leak it claims to fix. However, there's a much larger leak in pybind11's subinterpreter handling that warrants separate investigation.

[rwgk]

@b-pass I'd say go ahead and merge this PR, it looks solid. (It'd be great if you could adopt the suggested expanded comment.)

I'll try to find out more about the "massive leak" as measured with the Cursor-generated code (#5958 (comment)). If you see anything suspicious in the measurements, please let me know.