Search code examples
shared-librariesdynamic-linkingdynamic-loading

Rebuilding & install Shared library does not impact on process which already loaded that library


I have a question regarding shared library used by multiple processes.

I have a shared library libfoo.so which is used by two different processes, process1 and process2.

The first process (process1) is in the Running state and libfoo.so is loaded in memory. I made some modifications in libfoo.so code, rebuilt and installed, then started process2. The new process2 has loaded the newly-installed library libfoo.so.

But process1 is still running with older libfoo.so. If I restart process1, then it loads the newly-installed libfoo.so as expected.

If the operating system has a single copy of shared library, then why does installing a new shared library not affect currently running processes?


Solution

  • If the operating system has a single copy of shared library, then why does installing a new shared library not affect currently running processes?

    First, your entire notion of a single copy is somewhat flawed.

    Let's talk about an ELF shared library (the concepts apply to other kinds of libraries as well, though the details differ).

    An ELF shared library will usually have at least two loadable segments: a read-only one, and a writable one. The first one contains read-only data, program code (often called .text), relocation sections, etc. The second segment contains initialized, but writable data (often called .data).

    When two processes run and use the same libfoo.so, there will be at least three pages of memory used by libfoo.so: at least one page to "cover" the read-only segment (that page will be shared between the two runing processes), and at least one separate page in each process to "cover" the writable segment.

    As you can see from this, a single copy of the shared library on disk is also replicated into multiple copies in RAM while the library is used by a running program.

    Second, we need to talk about how you update libfoo.so. You could do it in one of two ways:

    1. you could do this: rm -f libfoo.so; gcc -shared -o libfoo.so foo.o, or
    2. you could do this: gcc -shared -o libfoo.so foo.o.

    In the first case, you are not affecting any process that has mmaped libfoo.so at all: the old data for libfoo.so will remain on disk, but not visible to any process which doesn't have it already opened or mmaped. Note also that if libfoo.so is 1GB in size, your disk usage will have gone up by 1GB (both old and new copy are still taking disk space).

    In the second case, you are updating libfoo.so in place (this is not recommended, for reasons that will become obvious shortly). The inode number for libfoo.so will stay the same, and the old data will be gone. Your disk usage will stay constant (assuming new libfoo.so is about the same size as the old one).

    This will affect any running process, but maybe not in a way you expect. The most likely outcome is that your running process will crash.

    Why would it? Think of the library as a book, which has table of contents. During the initial library loading, table of contents will be loaded into RAM, and modified (because shared library can be loaded at arbitrary location in memory). If you now update the book (the library on disk) in such a way that e.g. chapter 3 becomes 3 pages longer, then the table of contents will no longer be valid (at least for chapters 4 through end). Any attempt to follow the pointers in table of contents will land you not at the start of the chapter you are seeking, but in the middle of a chapter. So you would call a function, and land in the middle of a different function. The most likely outcome of this is a crash.

    The picture is even more complicated by demand paging. You may have paged in some of the chapters, but not others. So you may not discover that your process is in fact hosed immediately after update. If your library is small, you may not discover this at all.

    P.S. Some operating systems prohibit the second form of update: opening a library for writing fails with ETXTBSY if the library is currently being used by some process. Linux does that for some file systems, but not all.