Fix multiple reported issues.

[kareem-wolfssl]

Description

Fixes zd#22229

Testing

Built in tests, provided reproducers.

Checklist

• added tests
• updated/added doxygen
• updated appropriate READMEs
• Updated manual and documentation

[Copilot]

Pull request overview

This PR addresses multiple reported issues (zd#22229) across certificate verification and TLS handshake processing by tightening cleanup semantics and enforcing negotiated values for RPK certificate-type extensions.

Changes:

• Prevent untrusted temporary CA issuers from persisting in the shared CertManager during/after wolfSSL_X509_verify_cert() and fix object-list cleanup when CRLs are appended.
• Enforce “offered vs received” constraints for client_cert_type / server_cert_type (RFC 7250 / RFC 8446) to reject unsolicited or unoffered negotiated values.
• Reset per-handshake RPK negotiation state on SSL object reuse and harden output-buffer shrinking to avoid freeing non-heap memory and to drop stale accounting.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
src/x509_str.c	Harden temp-CA lifecycle during X509 store verification; fix get0_objects() cleanup when CRLs are present.
src/tls.c	Reject unsolicited/unoffered cert-type negotiation values for RPK-related TLS extensions.
src/ssl.c	Reset RPK negotiation state on WOLFSSL object reuse (preserving isRPKLoaded).
src/internal.c	Make ShrinkOutputBuffer() safe for static buffers and reset output accounting when discarding pending output.

Comments suppressed due to low confidence (1)

src/x509_str.c:1170

• Exit cleanup unconditionally calls wolfSSL_CertManagerUnloadTempIntermediateCerts(), but ignores its return value. If this unload fails (e.g., BAD_MUTEX_E), the function can return success while leaving WOLFSSL_TEMP_CA entries resident in the shared CertManager, undermining the stated goal of always cleaning up temporary trust anchors. Consider propagating cleanup failure (at least when verification otherwise succeeded).

        if (ctx->store != NULL) {
            wolfSSL_CertManagerUnloadTempIntermediateCerts(ctx->store->cm);
        }

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[julek-wolfssl]

This doesn't solve the problem. It just acts as a retry. Erroring out directly from X509StoreRemoveCa should be fine.

[github-actions]

MemBrowse Memory Report

gcc-arm-cortex-m3

• FLASH: .text +8 B (+0.0%, 122,619 B / 262,144 B, total: 47% used)

gcc-arm-cortex-m4-openssl-compat

• FLASH: .text -64 B (-0.0%, 771,348 B / 1,048,576 B, total: 74% used)

gcc-arm-cortex-m4-tls13

• FLASH: .text +64 B (+0.0%, 236,679 B / 262,144 B, total: 90% used)

gcc-arm-cortex-m7-pq

• FLASH: .text +64 B (+0.0%, 280,576 B / 1,048,576 B, total: 27% used)

linuxkm-standard

• Data: __patchable_function_entries +8 B (+0.0%, 49,016 B)
  No memory changes detected for:
• gcc-arm-cortex-m0plus
• gcc-arm-cortex-m4
• gcc-arm-cortex-m4-baremetal
• gcc-arm-cortex-m4-crypto-only
• gcc-arm-cortex-m4-dtls13
• gcc-arm-cortex-m4-min-ecc
• gcc-arm-cortex-m4-pkcs7
• gcc-arm-cortex-m4-pq
• gcc-arm-cortex-m4-rsa-only
• gcc-arm-cortex-m4-sp-math
• gcc-arm-cortex-m4-tls12
• gcc-arm-cortex-m7
• gcc-arm-cortex-m7-tls13
• linuxkm-pie
• stm32-sim-stm32h753

[julek-wolfssl]

comments are way too verbose

[julek-wolfssl]

This doesn't solve the problem. It just acts as a retry. Erroring out directly from X509StoreRemoveCa should be fine.

[kareem-wolfssl]

I removed the X509StoreRemoveCa call. Adding error checking would have caused issues in the case that the cert was already added previously. AddCA would return success, but the cert would not be added as TEMP_CA type, so X509StoreRemoveCa would error out.

The TEMP_CAs will be removed either way by the unconditional wolfSSL_CertManagerUnloadTempIntermediateCerts call.