[kojo1]

Description

Wipe and free previous EVP_PKEY DER cache on reassignment.

Fixes F-8331

Testing

Added test_wolfSSL_EVP_PKEY_reassign_der_cache.

Checklist

• [ x] added tests
• updated/added doxygen
• updated appropriate READMEs
• Updated manual and documentation

[Copilot]

🟡 Changes recommended

The reassignment test checks only replacement-cache sizes, so it can still pass when outgoing DSA/DH buffers are not wiped or freed, leaving the intended regression undetected.

1 open issue

• Test that outgoing cache buffers are wiped and freed exactly once · New

What changed in this PR

This PR addresses F-8331 by clearing previous DER key caches during reassignment in wolfSSL’s OpenSSL compatibility layer.

Changes:

• Wipe and free cached DER data and reset cache metadata before DSA, DH, and RSA reassignment.
• Clear temporary DER buffers before release on affected paths.
• Add and register a regression test covering repeated and cross-key-type reassignment.

File	Description
wolfcrypt/​src/​evp.c	Adds DER cache cleanup and temporary-buffer wiping.
tests/​api/​test_evp_pkey.h	Declares and registers the reassignment test.
tests/​api/​test_evp_pkey.c	Adds reassignment checks for DSA, RSA, and DH caches.

_{Review effort: Balanced}

[github-actions]

MemBrowse Memory Report · Explore what changed →

gcc-arm-cortex-m0plus

• FLASH: .text +392 B (+0.6%, 71,071 B / 262,144 B, total: 27% used)

gcc-arm-cortex-m3

• FLASH: .text +596 B (+0.5%, 132,021 B / 262,144 B, total: 50% used)

gcc-arm-cortex-m4

• FLASH: .text +732 B (+0.4%, 209,292 B / 262,144 B, total: 80% used)

gcc-arm-cortex-m4-baremetal

• FLASH: .text +448 B (+0.6%, 73,783 B / 262,144 B, total: 28% used)

gcc-arm-cortex-m4-crypto-only

• FLASH: .text +384 B (+0.2%, 182,035 B / 262,144 B, total: 69% used)

gcc-arm-cortex-m4-dtls13

• FLASH: .text +896 B (+0.5%, 196,248 B / 1,048,576 B, total: 19% used)

gcc-arm-cortex-m4-min-ecc

• FLASH: .text +384 B (+0.6%, 68,377 B / 262,144 B, total: 26% used)

gcc-arm-cortex-m4-openssl-compat

• FLASH: .rodata +2,472 B, .text +2,304 B (+0.6%, 801,956 B / 1,048,576 B, total: 76% used)
• RAM: .bss -12,048 B (-9.0%, 121,772 B / 262,144 B, total: 46% used)

gcc-arm-cortex-m4-pkcs7

• FLASH: .text +320 B (+0.1%, 223,124 B / 262,144 B, total: 85% used)

gcc-arm-cortex-m4-pq

• FLASH: .text +1,152 B (+0.4%, 313,104 B / 1,048,576 B, total: 30% used)

gcc-arm-cortex-m4-rsa-only

• FLASH: .rodata +8 B, .text +1,216 B (+0.4%, 343,760 B / 1,048,576 B, total: 33% used)
• RAM: .bss +1,320 B (+7.9%, 17,996 B / 262,144 B, total: 7% used)

gcc-arm-cortex-m4-sp-math

• FLASH: .text +384 B (+0.6%, 68,377 B / 262,144 B, total: 26% used)

gcc-arm-cortex-m4-tls12

• FLASH: .text +640 B (+0.5%, 132,817 B / 262,144 B, total: 51% used)

gcc-arm-cortex-m4-tls13

• FLASH: .text +1,024 B (+0.4%, 249,890 B / 262,144 B, total: 95% used)

gcc-arm-cortex-m7

• FLASH: .text +796 B (+0.4%, 209,356 B / 262,144 B, total: 80% used)

gcc-arm-cortex-m7-pq

• FLASH: .text +1,152 B (+0.4%, 314,000 B / 1,048,576 B, total: 30% used)

gcc-arm-cortex-m7-tls13

• FLASH: .text +960 B (+0.4%, 249,890 B / 262,144 B, total: 95% used)

stm32-sim-stm32h753

• FLASH: .text -512 B (-0.3%, 204,076 B / 2,097,152 B, total: 10% used)

zephyr-frdm_rw612

• FLASH: datas +16 B, rodata +28 B, text +8,448 B (+3.0%, 292,508 B / 67,108,864 B, total: 0% used)
• RAM: bss +49 B, datas +16 B (+0.1%, 58,423 B / 983,040 B, total: 6% used)

---

_{Catch memory regressions in CI with MemBrowse. Free for open source.}

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #11965

Scan targets checked: wolfcrypt-src, wolfcrypt-bugs, wolfssl-bugs
Not reviewed (no scan target covers them): tests/api/test_evp_pkey.h
Coverage: 2 of 2 in-scope changed file(s) opened by the reviewer

Findings: 1
1 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Reported findings require changes before merge.

Review tier: Max

[wolfSSL-Fenrir-bot]

Double free of rsaEmpty when the DER watch assertion after assign_RSA fails · Incorrect error handling

DER_WATCH_END() runs between the successful EVP_PKEY_assign_RSA(pkey, rsaEmpty) and the EXPECT_FAIL() check. A failed watch assertion (the regression this test exists to catch) frees rsaEmpty while pkey owns it with ownRsa=1, and EVP_PKEY_free(pkey) at line 1212 frees it again.

Suggested fix: Put the if (EXPECT_FAIL()) RSA_free(rsaEmpty); check right after the EVP_PKEY_assign_RSA call and before DER_WATCH_END(), as the rsa block at line 1172 does.
Basis: OpenSSL EVP_PKEY_set1_RSA(3): EVP_PKEY_assign_RSA() uses the supplied key internally, so the key is freed when the parent pkey is freed.

[kojo1]

Thanks, fixed. The EXPECT_FAIL() ownership check now runs right after
EVP_PKEY_assign_RSA() and before DER_WATCH_END(), matching the rsa block.
Verified with ASan by forcing the watch assertion to fail at that point:
the old order hits heap-use-after-free on EVP_PKEY_free(); the new order
reports a plain test failure.