Search code examples
opensslfips

Consequences for adding relocation information in fips validated libeay32.dll

We are using the FIPS validated libeay32.dll. This dll uses the /FIXED linker switch so that the libeay32.dll will be loaded at fixed base address. The other modules from our project is consuming the openssl dll in shared mode using LoadLibrary() function. We have observed the intermittent issue while loading the mentioned dll.

As part of resolution, we added the relocation information in the image header of libeay32.dll with the understanding that the dll will be loaded at some base address if not at fixed one to address intermittent loading issue. I checked the open ssl user guide which mentions the following.

The standard OpenSSL build with the fips option will use a base address for libeay32.dll of 0xFB00000 by default. This value was chosen because it is unlikely to conflict with other dynamically loaded libraries. In the event of a clash with another dynamically loaded library which will trigger runtime relocation of libeay32.dll, the integrity check will fail with the error

FIPS_R_FINGERPRINT_DOES_NOT_MATCH_NONPIC_RELOCATED

A base address conflict can be resolved by shuffling the other DLLs or re-compiling OpenSSL with an alternative base address specified with the --with­baseaddr= option.

Here are my questions.

  1. By introducing the relocation information in image header of libeay32.dll, am I making the libeay32.dll vulnerable to security [fips 140-2] ?

  2. What kind of security vulnerabilities am I introducing with their side effect in the modules which are using those open ssl libraries ?

  3. Any cleaner solution to such kind of loading issues ?

Thanks in advance...

Solution

  • By introducing the relocation information in image header of libeay32.dll, am I making the libeay32.dll vulnerable to security

    All Windows PE files have relocation information. Its not like -fPIC on Unix and Linux. The mechanisms are different because the PE/PE+ executable format is different than the ELF/ELF64 specification. Also Microsoft compilers/linkers do not use PC-relative addressing mode, which Unix and Linux often will.

    What you want on Windows is /ASLR, but /ASLR is orthogonal to /FIXED. /FIXED is used by the FIPS module to ensure the integrity of the module. /ASLR is a best practice on Windows and a Microsoft SDLC item.

    Address space layout randomization makes it harder for attackers to guess addresses in some situations. For example, it will help with a "return to libc" attack.

    What kind of security vulnerabilities am I introducing with their side effect in the modules which are using those OpenSSL libraries ?

    Because of /FIXED, the attacker always knows where the binary is loaded, and he may be able to build a ROP chain using it. This assumes the attacker gets a toe hold with a vulnerability.

    One of the reasons I dislike the FIPS validated library is because you lose /ASLR.

    Any cleaner solution to such kind of loading issues ?

    Usually what you can do is, in the executable (EXE) that uses the FIPS DLL, you add the OpenSSL import library as the first item in the list of libraries. It gets written to the PE header in roughly the order specified. The runtime link/loader will then load the library early and not late. Loading it early minimizes the likelihood of [unsatisfied] displacement.

    I believe you can also do the following in your program (EXE). Its taken from Crypto++'s dll.h, which has had validations, too. Put it in your precompiled header, or the "main header" for your program.

    #ifdef NDEBUG
#pragma comment(lib, "msvcrt")
#else
#pragma comment(lib, "msvcrtd")
#endif

#pragma comment(lib, "libeay32")
#pragma comment(lib, "anotherlib")
#pragma comment(lib, "yetanotherlib")

    FIPS is very much about policy and procedure. You must follow it to the letter. If you modify the build procedure by changing /FIXED, then you lose the validation. You may as well just use the regular library.

    As you correctly noted, you are allowed to change the base address used for the module. Authority is provided in the User Guide for the OpenSSL FIPS Object Module v2.0, page 52/208.