Rust's memmap crate has unsafe methods.
I can understand how the returned address space is unsafe to pass to constructors which might validate its contents and then continue to use it.
I'm writing a binary-diff tool which only treats the returned address space as containing bytes (of any value) and does no validation on the contents of the address space.
Can I avoid propagating the unsafe in this case?
One of the main principles of using unsafe correctly is that you take responsibility for ensuring soundness in all affected code - including the safe code that surrounds it.
If you were to expose this function in a library crate, you should mark it as unsafe; users of your library need to understand where UB arises and make their own decisions about safety.
In an application crate, you are in control of all of the code and it's up to you to never dereference the bytes as anything else. If you do that, it is completely acceptable to limit the unsafe code to where you directly interact with the memory map, wrapped in safe functions. In fact, wrapping everything in unsafe is counterproductive because it will be unclear where the danger lies.