Search code examples
haskellmemory-managementmonadshaskell-ffi

UB due to allocaArray automatic cleanup or not?


i have this function in my code that seems to work just fine:

-- type declaration just for reference, i don't have it in my actual code
retrieveVulkanArray :: Storable a => (Ptr Word32 -> Ptr a -> IO b) -> IO (Ptr a, Int)
retrieveVulkanArray' f =
    alloca $ \arrCount -> do
        f arrCount vkNullPtr
        arrCount' <- fromIntegral <$> peek arrCount
        allocaArray arrCount' $ \resArray -> do
            f arrCount resArray
            pure (resArray, arrCount')

(for context this is a helper function to get FFI arrays from Vulkan API, f might be for example vkEnumeratePhysicalDevices)

As i was reviewing my code, i noticed that it returns resArray(which from description of allocaArray seems to only be valid within the inner lambda) to its caller. In C, a code like this would be undefined behaviour. Is my intuition correct here or is there something more going on? I haven't noticed any crashes yet after all :)


Solution

  • The fact that it works certainly doesn't prove that it is correct, in fact this function is indeed very wrong.

    alloca, as well as allocaArray, will allocate a Haskell MutableByteArray# convert it to a pointer. Operate on that pointer and then ensure that the array is still alive with a special touch# function. Problem is that once you loose reference to the actual MutableByteArray#, which is what happens when you exit alloca, GC will clean it up and the Ptr a that was pointing to that array will no longer be valid. So if you continue reading or writing into that pointer Ptr a after you return it from retrieveVulkanArray you are reading/writing into memory that can be used by something else and are now in real danger of a segfault and all sorts of other security vulnerabilities that come with it.

    The proper way would be:

    retrieveVulkanArray
      :: Storable a => (Ptr Word32 -> Ptr a -> IO b) -> IO (ForeignPtr a, Int)
    retrieveVulkanArray' f =
        alloca $ \arrCount -> do
            f arrCount vkNullPtr
            arrCount' <- fromIntegral <$> peek arrCount
            resArray <- mallocForeignPtrArray arrCount'
            _ <- withForeignPtr resArray (f arrCount)
            pure (resArray, arrCount')
    

    ForeignPtr a ensures that you can operate on the raw Ptr a when needed, without worrying that memory it points to is freed until ForeignPtr is no longer used.