Search code examples
c++boost

imbue/facet ignored when used from shared libraries depending on dynamic load order under Android

I'm deploying a C++ app on Android, it uses boost::date_time. It has lots of libraries, some being linked at compile time (shared libraries), others, sort of plugins, being loaded dynamically at runtime through dlopen. In some libraries, setting a boost::posix_time::time_facet to a std::ostream (using imbue) in order to customize a boost::posix_time::ptime display has no effect (is ignored). I could isolate the issue in the following MCVE:

enter image description here

bug_datetime_base is a shared library that uses boost::date_time but only compiles code redirecting a boost::posix_time::ptime to a std::ostream (no boost::posix_time::time_facet used):

MyClass::MyClass( const boost::posix_time::ptime& timeInfo )
{
    std::cout << timeInfo;
}

bug_datetime_lib is a shared library that uses boost::date_time and exports a function that will use a boost::posix_time::time_facet to redirect a boost::posix_time::ptime to a std::ostream with a specific formatting:

#include <sstream>
#include <boost/date_time/posix_time/posix_time.hpp>
#include <QDebug> 

void TestBoost()
{
    boost::posix_time::ptime t1(boost::gregorian::date(2002,boost::gregorian::Jan,10),
                                boost::posix_time::time_duration(1,2,4));

    std::stringstream temp;

    temp << "FROM TestBoost:" << std::endl << "Unformatted:" << t1 << std::endl;

    boost::posix_time::time_facet* facet = new boost::posix_time::time_facet("%Y$%b$%d %H:%M:%S.TestBoost.%f");
    const std::locale loc = std::locale(std::locale::classic(), facet);
    temp.imbue(loc);

    temp << "Formatted:" << t1;

    qDebug() << temp.str().c_str();
}

bug_datetime_wrapper is a shared library that just links to bug_datetime_base and bug_datetime_lib, does nothing more:

MyWrapperClass::MyWrapperClass()
{
}

bug_datetime is the main program that uses boost::date_time, links to bug_datetime_base and dynamically loads bug_datetime_wrapper through dlopen:

#include <sstream>
#include <boost/date_time/posix_time/posix_time.hpp>
#include <QDebug>
#include <QApplication>

#include <dlfcn.h>
typedef void* dllHandle;

int main( int argc, char* argv[] )
{
    QApplication app( argc, argv );

    void* wrapperPtr = NULL;

    // Workaround2:
    // if commenting line below, bug_datetime_wrapper is not loaded, using imbue from any places works perfectly
    wrapperPtr = dlopen( "libbug_datetime_wrapper_armeabi-v7a.so", 0);

    if ( wrapperPtr )
        qDebug() << "Loaded bug_datetime_wrapper, then formatting will fail";
    else
        qDebug() << "Failed to load bug_datetime_wrapper, then formatting will work";

    {
        boost::posix_time::ptime t1(boost::gregorian::date(2002,boost::gregorian::Jan,10),
                                    boost::posix_time::time_duration(1,2,4));

        std::stringstream temp;

        boost::posix_time::time_facet* facet = new boost::posix_time::time_facet("%Y$%b$%d %H:%M:%S.main.%f");
        const std::locale loc = std::locale(std::locale::classic(), facet);
        temp.imbue(loc);

        temp << t1;

        qDebug() << "FROM MAIN: " << temp.str().c_str();
    }

    auto libPtr = dlopen( "libbug_datetime_lib_armeabi-v7a.so", 0);
    if ( libPtr )
    {
        typedef void (*TestBoostFunc)();
        auto func = (TestBoostFunc) dlsym( libPtr, "TestBoost" );
        if ( func )
            (*func)();
        else
            qDebug() << "Failed to load TestBoost function";
    }
    else
    {
        qDebug() << "Failed to load library function";
    }

    return app.exec();
}

Within the main program:

  • Using a boost::posix_time::time_facet to customize redirection of a boost::posix_time::ptime to a std::ostream works fine
  • However, invoking code from bug_datetime_lib doing the same thing does not work (facet is ignored)

So program output is:

D libbug_datetime_armeabi-v7a.so: Loaded bug_datetime_wrapper, then formatting will fail
D libbug_datetime_armeabi-v7a.so: FROM MAIN:  2002$Jan$10 01:02:04.main.000000
D libbug_datetime_armeabi-v7a.so: FROM TestBoost:
D libbug_datetime_armeabi-v7a.so: Unformatted:2002-Jan-10 01:02:04
D libbug_datetime_armeabi-v7a.so: Formatted:2002-Jan-10 01:02:04

While expecting:

D libbug_datetime_armeabi-v7a.so: Loaded bug_datetime_wrapper, then formatting will fail
D libbug_datetime_armeabi-v7a.so: FROM MAIN:  2002$Jan$10 01:02:04.main.000000
D libbug_datetime_armeabi-v7a.so: FROM TestBoost:
D libbug_datetime_armeabi-v7a.so: Unformatted:2002-Jan-10 01:02:04
D libbug_datetime_armeabi-v7a.so: Formatted:2002$Jan$10 01:02:04.TestBoost.000000

The whole code is available here: https://github.com/jporcher/bug_datetime

Note that I use QtCreator to easily compile and deploy the app, but I'm pretty sure the problem can be reproduced with regular ndk-builds.

The libraries architecture makes no sense, it's because I removed lots of code to isolate a MCVE. If I remove bug_datetime_wrapper or bug_datetime_base from the project, the problem is not reproductible anymore.

Note that I found many workarounds that would fix the issue, they are all very surprising:

  • Workaround1: in bug_datetime_base, commenting std::cout << timeInfo; fixes the issue
  • Workaround2: commenting wrapperPtr = dlopen( "libbug_datetime_wrapper_armeabi-v7a.so", 0); (so not loading bug_datetime_wrapper) fixes the issue
  • Workaround3: removing a link below fixes the issue
    • Workaround3.1 not linking bug_datetime program to bug_datetime_base (removing link1)
    • Workaround3.2 not linking bug_datetime_wrapper to bug_datetime_base (removing link2)
    • Workaround3.3 not linking bug_datetime_wrapper to bug_datetime_lib (removing link3)
  • Workaround4: changing link order in bug_datetime_wrapper, linking bug_datetime_lib before bug_datetime_base fixes the issue
  • Workaround5: linking bug_datetime program to bug_datetime_wrapper at compile time fixes the issue

The current code has no undefined behaviour and is perfectly valid, so I'm looking for a rational explanation of what's going wrong and how this should be fixed cleanly (preserving existing links as they are needed in the original project I created this MCVE from).

Edit 07 June: Tried to compile boost as shared libraries rather than static. I still observe the same issue.

Solution

  • This issue is due to ODR violation. Boost date time library is a header only library, this means that the code gets compiled in every translation unit including it.

    Then, see operator<< defined in posix_time_io.hpp:

    template <class CharT, class TraitsT>
  inline
  std::basic_ostream<CharT, TraitsT>&
  operator<<(std::basic_ostream<CharT, TraitsT>& os,
             const ptime& p) {
    boost::io::ios_flags_saver iflags(os);
    typedef boost::date_time::time_facet<ptime, CharT> custom_ptime_facet;
    std::ostreambuf_iterator<CharT> oitr(os);
    if (std::has_facet<custom_ptime_facet>(os.getloc()))
      std::use_facet<custom_ptime_facet>(os.getloc()).put(oitr, os, os.fill(), p);
    else {
      //instantiate a custom facet for dealing with times since the user
      //has not put one in the stream so far.  This is for efficiency 
      //since we would always need to reconstruct for every time period
      //if the locale did not already exist.  Of course this will be overridden
      //if the user imbues as some later point.
      custom_ptime_facet* f = new custom_ptime_facet();
      std::locale l = std::locale(os.getloc(), f);
      os.imbue(l);
      f->put(oitr, os, os.fill(), p);
    }
    return os;
  }

    has_facet checks a static id member of facet object. Then, due to the fact that code is compiled in many different translation unit, you end up with many boost::date_time::time_facet classes defined with different id. If a translation unit creates a boost::posix_time::time_facet facet and a different translation unit uses operator<<, then this operator will not use the facet.

    Solution is to make sure this code is compiled once only.

    So I created a new library boost_datetime with:

    boost_datetime.h:

    #pragma once

#include <ostream>
#include <boost/date_time/posix_time/posix_time.hpp>

namespace boost
{
    namespace posix_time
    {
        class ptime;
    }
}

class BoostDateTime
{
public:
    static void setFacet( std::ostream& os );
};

std::ostream& operator<<( std::ostream& os, const boost::posix_time::ptime& p );

    boost_datetime.cpp:

    #include "boost_datetime.h"

void BoostDateTime::setFacet( std::ostream& os )
{
    boost::posix_time::time_facet* facet = new boost::posix_time::time_facet("%Y$%b$%d %H:%M:%S.%f");
    os.imbue(std::locale(os.getloc(), facet));
}

std::ostream& operator<<( std::ostream& os, const boost::posix_time::ptime& p )
{
    // copied from posix_time_io.hpp

    boost::io::ios_flags_saver iflags(os);
    typedef boost::posix_time::time_facet base_ptime_facet;
    std::ostreambuf_iterator<char> oitr(os);
    if (std::has_facet<base_ptime_facet>(os.getloc()))
      std::use_facet<base_ptime_facet>(os.getloc()).put(oitr, os, os.fill(), p);
    else {
      //instantiate a custom facet for dealing with times since the user
      //has not put one in the stream so far.  This is for efficiency 
      //since we would always need to reconstruct for every time period
      //if the locale did not already exist.  Of course this will be overridden
      //if the user imbues as some later point.
      base_ptime_facet* f = new base_ptime_facet();
      std::locale l = std::locale(os.getloc(), f);
      os.imbue(l);
      f->put(oitr, os, os.fill(), p);
    }
    return os;
}

    Use it in every place instead of directly using boost date_time. This fixes the issue for good.