-Zstaticlib-hide-internal-symbols: Hide non-exported internal symbols from staticlibs

[cezarbbb]

View all comments

According to issue #104707 , when building a staticlib, all Rust internal symbols — mangled symbols, #[rustc_std_internal_symbol] items, allocator shims, etc. — leak out of the static archive. In contrast, cdylib correctly exports only #[no_mangle] symbols via a linker version script.

The approach taken here is to directly post-process ELF object files in the archive: parsing the SHT_SYMTAB sections and setting STV_HIDDEN visibility on any GLOBAL/WEAK defined symbol that is not in the exported symbol set, without changing the binding. This is gated behind -Zstaticlib-hide-internal-symbols and only affects ELF targets (Linux, BSD); a warning is emitted for non-ELF targets.

The rust_eh_personality symbol is always kept visible to ensure .eh_frame unwinding works correctly for C consumers. Using the flag with non-staticlib crate types is a compilation error.

The test code are as follows:

1.a std rust staticlib:

use std::collections::HashMap;
use std::panic::{catch_unwind, AssertUnwindSafe};

#[no_mangle]
pub extern "C" fn my_add(a: i32, b: i32) -> i32 { a + b }

#[no_mangle]
pub extern "C" fn my_hash_lookup(key: u64) -> u64 {
    let mut map = HashMap::new();
    for i in 0..100u64 { map.insert(i, i.wrapping_mul(2654435761)); }
    *map.get(&key).unwrap_or(&0)
}

pub fn internal_reverse(s: &str) -> String { s.chars().rev().collect() }

#[no_mangle]
pub extern "C" fn my_format_number(n: i32) -> i32 {
    let s = format!("number: {}", n); s.len() as i32
}

#[no_mangle]
pub extern "C" fn my_safe_div(a: i32, b: i32) -> i32 {
    match catch_unwind(AssertUnwindSafe(|| {
        if b == 0 { panic!("division by zero!"); }
        a / b
    })) {
        Ok(result) => result,
        Err(_) => -1,
    }
}

#[no_mangle]
pub extern "C" fn my_uncaught_panic() { panic!("uncaught panic across FFI"); }

1.b downstream c program:

extern int my_add(int a, int b);
extern unsigned long my_hash_lookup(unsigned long key);
extern int my_format_number(int n);
extern int my_safe_div(int a, int b);
extern void my_uncaught_panic(void);

int main() {
    int failures = 0;
    if (my_add(10, 20) != 30) failures++;
    if (my_hash_lookup(5) != 5UL * 2654435761UL) failures++;
    if (my_format_number(42) != 10) failures++;
    if (my_safe_div(100, 5) != 20) failures++;
    if (my_safe_div(100, 0) != -1) failures++;
    pid_t pid = fork();
    if (pid == 0) { alarm(5); my_uncaught_panic(); _exit(0); }
    else { waitpid(pid, &status, 0); }
    return failures;
}

The test results with different compiler flags(which might cause binary size reduction) are as follows:

  settings                   OFF        ON  -Zsave     ALL    OFF.dynsym ON.dynsym 
  ------------------------------------------------------------------------
  default                 1.7M      1.5M  204K (12%)    1735       5    1730
  lto_thin                616K      584K  33K (5%)     246       5     241
  lto_fat                 525K      525K    0 (0%)       6       5       1
  opt_s                   1.7M      1.5M  204K (12%)    1735       5    1730
  opt_z                   1.7M      1.5M  204K (12%)    1735       5    1730
  lto_thin_z              602K      570K  32K (5%)     246       5     241
  lto_fat_z               514K      514K    0 (0%)       6       5       1
  full                    514K      514K    0 (0%)       6       5       1

2.a no_std rust staticlib

#![no_std]
#![feature(core_intrinsics)]

use core::panic::PanicInfo;

#[panic_handler]
fn panic(_info: &PanicInfo) -> ! { loop {} }

#[no_mangle]
pub extern "C" fn embedded_add(a: i32, b: i32) -> i32 { a.wrapping_add(b) }

#[no_mangle]
pub extern "C" fn embedded_checksum(data: *const u8, len: usize) -> u8 {
    if data.is_null() { return 0; }
    let slice = unsafe { core::slice::from_raw_parts(data, len) };
    let mut sum: u8 = 0;
    for &byte in slice { sum = sum.wrapping_add(byte); }
    sum
}

fn internal_helper() -> i32 { 42 }
#[no_mangle]
pub extern "C" fn call_internal() -> i32 { internal_helper() }

#[no_mangle]
pub extern "C" fn embedded_trigger_abort() { core::intrinsics::abort(); }

2.b downstream c program

extern int embedded_add(int a, int b);
extern unsigned char embedded_checksum(const unsigned char *data, unsigned long len);
extern int call_internal(void);
extern void embedded_trigger_abort(void);

int main() {
    int failures = 0;
    if (embedded_add(10, 20) != 30) failures++;
    unsigned char data[] = {1, 2, 3};
    if (embedded_checksum(data, 3) != 6) failures++;
    if (call_internal() != 42) failures++;
    pid_t pid = fork();
    if (pid == 0) { embedded_trigger_abort(); _exit(0); }
    else { waitpid(pid, &status, 0); }
    return failures;
}

The test results with different compiler flags(which might cause binary size reduction) are as follows:

  settings                   OFF        ON  -Zsave     ALL    OFF.dynsym ON.dynsym 
  ------------------------------------------------------------------------
  default                 485K      429K  56K (11%)     490       4     486
  lto_thin                180K      180K    0 (0%)       4       4       0
  lto_fat                 179K      179K    0 (0%)       4       4       0
  opt_s                   485K      429K  56K (11%)     490       4     486
  opt_z                   485K      429K  56K (11%)     490       4     486
  lto_thin_z              180K      180K    0 (0%)       4       4       0
  lto_fat_z               179K      179K    0 (0%)       4       4       0
  full                    179K      179K    0 (0%)       4       4       0

Test results show that this compiler option is beneficial for scenarios where LTO cannot be enabled.

r? @bjorn3 @petrochenkov

[rustbot]

r? @petrochenkov

rustbot has assigned @petrochenkov.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

• Owners of files modified in this PR: codegen, compiler
• codegen, compiler expanded to 69 candidates
• Random selection from 16 candidates

[bjorn3]

This would also need to rename symbols to avoid conflicts between two rust staticlibs ending up getting linked together, right?

[bjorn3]

The rust_eh_personality symbol is always kept visible to ensure .eh_frame unwinding works correctly for C consumers.

Why exactly is that the case? rust_eh_personality is actually the symbol that is most likely to cause conflicts as it is the only one whose name doesn't get mangled depending on the rustc version.

[rustbot]

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

[cezarbbb]

This would also need to rename symbols to avoid conflicts between two rust staticlibs ending up getting linked together, right?

My primary goal right now is to reduce binary size, so I don't have immediate plans to implement symbol renaming. This means that linking multiple Rust staticlibs together can still result in multiple definition errors. Would you like me to address that in this PR as well? It seems feasible to implement — for example, by rehashing symbols and updating their references accordingly.

[cezarbbb]

The rust_eh_personality symbol is always kept visible to ensure .eh_frame unwinding works correctly for C consumers.

Why exactly is that the case? rust_eh_personality is actually the symbol that is most likely to cause conflicts as it is the only one whose name doesn't get mangled depending on the rustc version.

I previously assumed this symbol needed to remain externally visible to support scenarios requiring cross-language exception propagation. Do you think we should also set rust_eh_personality as hidden?

[bjorn3]

If it isn't too hard it would be nice to do symbol renaming too. I think doing in-place modification isn't going to work for that though. Adding a unique suffix would require growing the size of the string table.

[bjorn3]

I previously assumed this symbol needed to remain externally visible to support scenarios requiring cross-language exception propagation. Do you think we should also set rust_eh_personality as hidden?

rust_eh_personality is only meant to be referenced by the .eh_frame section of rust object files. The only reason it's name isn't mangled is because LLVM hard codes the name to determine the exception table format to emit.

[cezarbbb]

If it isn't too hard it would be nice to do symbol renaming too. I think doing in-place modification isn't going to work for that though. Adding a unique suffix would require growing the size of the string table.

Got it. I will first fix the rust_eh_personality issue, and then try to implement symbol renaming.

[SparrowLii]

@bors delegate=try

[rust-bors]

✌️ @cezarbbb, you can now perform try builds on this pull request!

You can now post @bors try to start a try build.

[cezarbbb]

@bors try

[rust-bors]

☀️ Try build successful (CI)
Build commit: a9431d3 (a9431d37da1d0346038257cec9d94f2783997621, parent: e8e4541ff19649d95afab52fdde2c2eaa6829965)

[cezarbbb]

@bors try jobs=x86_64-*

[rust-bors]

💔 Test for 12ba282 failed: CI. Failed job:

• Calculate job matrix (web logs, enhanced plaintext logs)

[cezarbbb]

@bors try jobs=aarch64-*

[rust-bors]

☀️ Try build successful (CI)
Build commit: 3e76129 (3e7612913ab931b859e2e0633efea5ee09e9e265, parent: e8e4541ff19649d95afab52fdde2c2eaa6829965)