gccrs#4599: Refactor dynamic object fat pointers and vtable generation

Link: Rust-GCC/gccrs#4599

Note

This is important

Objective

This PR refactors the memory layout of fat pointers (for dynamic trait objects) and vtable generation to match rustc ABI expectations.

Changes Made

Previously, the compiler incorrectly embedded the entire vtable directly inside the fat pointer as an array, and missed essential trait object metadata.

• The fat pointer is now strictly constrained to 2 words (data pointer and vtable pointer).

• The vtable is generated as a separate global static struct.

• The vtable structure now correctly includes drop_in_place, size, and align fields, followed by trait methods.

• Added a caching mechanism in the compilation context to prevent duplicate vtable generation and linker conflicts for identical Type-Trait combinations.

Let’s examine the following example (this example is for 64-bit):

# StructX memory

StructX { val: i32, } -- 4 bytes
├────────────────────┤
│                    │ <- val (4 bytes)
├────────────────────┤

# Previous fat-pointer memory

&StructX -> &dyn TraitA -- 24 bytes
├────────────────────┤
│                    │ <- address of StructX (8 bytes)
├────────────────────┤
│                    │ <- address of method do_a (8 bytes)
├────────────────────┤
│                    │ <- address of method dont_a (8 bytes)
├────────────────────┤

Note: If TraitA had 10 methods, this fat pointer would occupy 88 bytes. It also lacked size, align, and drop_in_place pointers.

# Current fat-pointer memory

&StructX -> &dyn TraitA -- 16 bytes
├────────────────────┤
│                    │ <- address of StructX (8 byte)
├────────────────────┤
│                    │ <- address of vtable (8 bytes)
├────────────────────┤

Generated vtable - 40 bytes
├────────────────────┤
│                    │ <- address of method drop_in_place (8 byte)
├────────────────────┤
│                    │ <- size information (8 byte)
├────────────────────┤
│                    │ <- align information (8 byte)
├────────────────────┤
│                    │ <- address of method dont_a (8 byte)
├────────────────────┤
│                    │ <- address of method dont_a (8 byte)
├────────────────────┤

Note: Regardless of the number of methods in TraitA, the fat pointer size is always 16 bytes. All necessary metadata is successfully preserved in the vtable.

Why is this needed for alloc?

For dynamically sized types (DSTs) and trait objects within alloc to function correctly and efficiently, the fat pointer architecture must exactly match standard Rust behavior.

Test Case

fn print1(label: &str, val: i32) {
    let arrow = "─".repeat(36);
    println!("{:<7}: {:>2} <{}┘   │", label, val, arrow);
}
fn print2(label: &str, val: i32) {
    let arrow = "─".repeat(40);
    println!("{:<7}: {:>2} <{}┘", label, val, arrow);
}

trait TraitA {
    fn do_a(&self) -> i32;
    fn dont_a(&self) -> i32;
}

struct StructX {
    val: i32,
}

impl TraitA for StructX {
    fn do_a(&self) -> i32 { self.val}     // ────────┐
    fn dont_a(&self) -> i32 { -self.val } // ────┐   │
}                                         //     │   │
                                          //     │   │
fn main() {                               //     │   │
    let x = StructX { val: 1 }; // self = 1 ->   │   │
    let dyn_a_x: &dyn TraitA = &x;        //     │   │
                                          //     │   │
    print1("dont_a", dyn_a_x.dont_a());   //     │   │
    print2("do_a", dyn_a_x.do_a());       //     │   │
}                                         //     │   │