Search code examples
delphirustmemory-managementffi

How does one handle memory management across a FFI-boundary between Rust and Delphi?

Passing around allocated memory across a FFI-Boundary between Delphi and Rust

I am currently investigating the feasabilty for our company to use rust as dynamically linked library in our delphi services. Our use case (filling out pdf form automatically) involves passing pointers to heap-memory from delphi to rust (managed by delphi) and from rust to delphi (managed by rust).

I built a toy example, in which the delphi service calls the rust lib with an array of bytes. Rust reads this array and places the modified contents in a Vec<u8>.

Now, the Vec<u8> is turned into a Buffer (see trait impl below), which is returned to delphi. This Buffer record is defined both in delphi and rust.

I have verified that the exchange of data works as expected. The issuse arises when delphi calls the rust library to free the memory which was alloacted by rust (see the drop_buffer function below). The delphi debugger (which apparently can debug assembly) halts on some assembly code and aborts the normal program flow.

Rust Implementation

#[repr(C)]
pub struct Buffer {
    ptr: *const [u8],
    len: usize
}

impl From<Vec<u8>> for Buffer {
    fn from(buffer: Vec<u8>) -> Self {
        let boxed_buffer = buffer.into_boxed_slice();
        let len = boxed_buffer.len();
        let ptr = Box::into_raw(boxed_buffer);
        Self {
            ptr,
            len
        }
    }
}
impl Buffer {
    pub unsafe fn manual_drop(ptr: *const [u8]) {
        let boxed_buffer = Box::from_raw(ptr as *mut [u8]);
        drop(boxed_buffer)
    }
}

To free the memory, delphi calls this function:

#[no_mangle]
pub extern "cdecl" fn drop_buffer(buffer: *const [u8]) {
    unsafe {Buffer::manual_drop(buffer);}
}

I dont know if this is relevant, but this is the code snippet the delphi debugger stops on. If have not yet tried to understand this code.

The assembly code the delphi debuggger halts on

How to proceed

Calling a rust function to free the memory which was allocated in rust seemed like the right idea to me. Is this a sensible approach, and if yes, what could be the cause of my issue?

Solution

  • There is not enough information in your question for us to answer your question definitively, but I can at least point out some issues that jump out to me.

    *const [u8] is a wide pointer (a.k.a. a fat pointer). It actually contains a normal pointer to the data as a *const u8 and the slice's length as a usize. This type is not safe to pass across an FFI boundary, because its representation is not guaranteed to be stable. From the Rustonomicon:

    • DST pointers (wide pointers) and tuples are not a concept in C, and as such are never FFI-safe.

    (DST means dynamically-sized types. [u8] is a DST.)

    You haven't shown the Delphi definition for Buffer, but I suspect you only put a normal pointer and the length (hopefully as NativeUInt, or the equivalent UIntPtr) there. You got "lucky" because the length from the wide pointer happens to be where you thought the len field was.

    You should use *const u8 instead. You'll need to store the length separately. You already have the length in Buffer, but you'll need to add it to drop_buffer as well (or just pass a Buffer value). Use std::slice::from_raw_parts or std::slice::from_raw_parts_mut to reconstruct a slice from a normal pointer and a length. These two functions return a reference, not a pointer, so you'll need to cast the reference to a pointer before calling Box::from_raw.

    drop_buffer should be an unsafe fn; it cannot guarantee memory safety because it receives an arbitrary pointer. Whether a function is marked unsafe or not won't have any effect on the generated code, but it can help avoid mistakes in the future if you end up calling the function (intentionally or not) from Rust code.

    The debugger appears to have encountered an int 3 instruction, which raises an interrupt that's interpreted as a breakpoint. Judging by the address of that instruction, it appears to be in a Windows system DLL, such as ntdll.dll or kernel32.dll. I know from experience that ntdll.dll (and possibly others) have int 3 instructions that are only triggered when a debugger is attached to the process and serve as a hint that something has gone horribly wrong. A call stack with debug symbols would give us more information about how your program got there. (I don't think Delphi's debugger supports symbol servers; you may have more luck using WinDbg or another debugger from Microsoft.)

    My guess for the crash is that the Delphi code only passes a normal pointer to drop_buffer, but the Rust side expected a wide pointer, so the length component is not initialized correctly. Freeing memory in Rust requires information about the layout (memory size and alignment) of the value to be freed in addition to the pointer to the value, unlike in C where free only requires the pointer to the value. A garbage length will likely cause Box::from_raw to construct a layout that doesn't match the layout that was used during allocation, which may cause the allocator to crash (if you're lucky).

    Calling a rust function to free the memory which was allocated in rust seemed like the right idea to me. Is this a sensible approach [...]?

    Yes, absolutely! Even when a library is written in the same programming language as the application consuming it, if the library exposes functions that allocate memory, the best practice is for the library to also provide a function to free that memory. The reason is that there's no guarantee that the library and the application are in fact using the same memory allocator, and in some cases (especially when mixing languages) it may be impossible to use the same memory allocator on both sides.

    The Rustonomicon is a book that covers various aspects of Unsafe Rust. FFI is inherently unsafe, so many sections of that book hold important information that will help you implement an FFI correctly.