Python C API: PyDateTime_FromTimestamp causes segmentation fault

Question

I followed this answer to call PyDateTime_FromTimestamp to create a datetime object in C++. But I got a Segmentation fault when PyDateTime_FromTimestamp is called.

Here is my C++ code:

#include <python3.6/Python.h>
#include <stdio.h>

#include <python3.6/datetime.h>
#include <sys/time.h>

static PyObject *iGetDateTime_PyFn(PyObject *Py_UNUSED(self), PyObject *Py_UNUSED(args)) {
    static double doubleValue = 1314761451;
    PyObject *floatObj = NULL;
    PyObject *timeTuple = NULL;
    PyObject *dateTime = NULL;
    floatObj = PyFloat_FromDouble(doubleValue);
    timeTuple = Py_BuildValue("(O)", floatObj);
    printf("timeTuple = %08x\n", (unsigned int)(long long)timeTuple);
    printf("PyTuple_Check(timeTuple) = %d\n", PyTuple_Check(timeTuple));
    dateTime = PyDateTime_FromTimestamp(timeTuple);
    printf("ready to return\n");
    return dateTime;
}

static PyMethodDef all_methods[] = {
    { "get_datetime", iGetDateTime_PyFn, METH_VARARGS, NULL },
    { NULL, NULL, 0, NULL }
};

static struct PyModuleDef main_module = {
    PyModuleDef_HEAD_INIT,
    "cpp",
    NULL,
    -1,
    all_methods
};

PyMODINIT_FUNC PyInit_cpp(void) {
    return PyModule_Create(&main_module);
}

I compiled with this command:

g++ --shared -fPIC -o cpp.so t1.cpp

My g++ version is 7.3.0.

In python, I execute:

import cpp
print(cpp.get_datetime())

And I get the following printed:

timeTuple = a7934358
PyTuple_Check(timeTuple) = 1
Segmentation fault (core dumped)

As we can see the timeTuple is successfully constructed and it is checked as a tuple. But we cannot get to the return sentence.

Answer 1

According to [GitHub]: python/cpython - (3.6) cpython/Include/datetime.h (${PYTHON_SRC_DIR}/Include/datetime.h):

1. PyDateTime_FromTimestamp is a preprocessor macro:

   #define PyDateTime_FromTimestamp(args) \
       PyDateTimeAPI->DateTime_FromTimestamp( \
           (PyObject*) (PyDateTimeAPI->DateTimeType), args, NULL)
   

2. PyDateTimeAPI is initialized to NULL (earlier in file)

   static PyDateTime_CAPI *PyDateTimeAPI = NULL;
   

resulting in segfault (Access Violation) when invoking the macro.

the fix requires initializing PyDateTimeAPI via the PyDateTime_IMPORT macro.

#define PyDateTime_IMPORT \
    PyDateTimeAPI = (PyDateTime_CAPI *)PyCapsule_Import(PyDateTime_CAPSULE_NAME, 0)

Initially, I discovered this while browsing the code (and I did it in the getDateTimePyFn function), then I came across [Python 3.Docs]: DateTime Objects (emphasis is mine)

Before using any of these functions, the header file datetime.h must be included in your source (note that this is not included by Python.h), and the macro PyDateTime_IMPORT must be invoked, usually as part of the module initialization function.

I modified your code, and I will exemplify on Win (as it'e easier for me, and the behavior is reproducible).

cpp.c:

#include <stdio.h>
#include <Python.h>
#include <datetime.h>

#define MOD_NAME "cpp"


static double doubleValue = 1314761451;

static PyObject *getDateTimePyFn(PyObject *Py_UNUSED(self), PyObject *Py_UNUSED(args)) {
    PyObject *floatObj = NULL,
        *timeTuple = NULL,
        *dateTime = NULL;
    floatObj = PyFloat_FromDouble(doubleValue);
    if (!floatObj)
    {
        return NULL;
    }
    timeTuple = Py_BuildValue("(O)", floatObj);
    Py_XDECREF(floatObj);
    if (!timeTuple)
    {
        return NULL;
    }
    dateTime = PyDateTime_FromTimestamp(timeTuple);
    Py_XDECREF(timeTuple);
    return dateTime;
}


static PyMethodDef all_methods[] = {
    { "get_datetime", getDateTimePyFn, METH_VARARGS, NULL },
    { NULL, NULL, 0, NULL }
};


static struct PyModuleDef main_module = {
    PyModuleDef_HEAD_INIT,
    MOD_NAME,
    NULL,
    -1,
    all_methods
};


PyMODINIT_FUNC PyInit_cpp(void) {
    PyDateTime_IMPORT;  // @TODO - cfati: !!! This initializes the struct containing the function pointer !!!
    return PyModule_Create(&main_module);
}

code.py:

#!/usr/bin/env python3

import sys
import cpp


def main():
    print("cpp.get_datetime returned: {:}".format(cpp.get_datetime()))


if __name__ == "__main__":
    print("Python {:s} on {:s}\n".format(sys.version, sys.platform))
    main()
    print("Done.")

Output:

[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q055903897]> sopr.bat
*** Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ***

[prompt]> "c:\Install\x86\Microsoft\Visual Studio Community\2015\vc\vcvarsall.bat" x64

[prompt]> dir /b
code.py
cpp.c

[prompt]> cl /nologo /DDLL /MD /I"c:\Install\x64\Python\Python\03.06.08\include" cpp.c  /link /NOLOGO /DLL /LIBPATH:"c:\Install\x64\Python\Python\03.06.08\libs" /OUT:cpp.pyd
cpp.c
   Creating library cpp.lib and object cpp.exp

[prompt]> dir /b
code.py
cpp.c
cpp.exp
cpp.lib
cpp.obj
cpp.pyd

[prompt]> "e:\Work\Dev\VEnvs\py_064_03.06.08_test0\Scripts\python.exe" code.py
Python 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018, 00:16:47) [MSC v.1916 64 bit (AMD64)] on win32

cpp.get_datetime returned: 2011-08-31 06:30:51
Done.