I use operating system commands in Matlab to launch python scripts. All was working fine till very recently, in fact I suspect that the problem comes with the recent upgrade of Matlab to R2017b and Fedora to FC27.
The fact: If I launch a script python in a shell all is working fine. But if I launch the same script in Matlab, few ImportError exception are raised:
Traceback (most recent call last):
File "/usr/lib64/python3.6/xml/etree/ElementTree.py", line 1448, in __init__
from xml.parsers import expat
File "/usr/lib64/python3.6/xml/parsers/expat.py", line 4, in <module>
from pyexpat import *
ImportError: /usr/lib64/python3.6/lib-dynload/pyexpat.cpython-36m-x86_64-linux-gnu.so: undefined symbol: XML_SetHashSalt
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.6/xml/etree/ElementTree.py", line 1451, in __init__
import pyexpat as expat
ImportError: /usr/lib64/python3.6/lib-dynload/pyexpat.cpython-36m-x86_64-linux-gnu.so: undefined symbol: XML_SetHashSalt
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/data_1/IRM/amigo/src/IRMAGE_python_scripts/IRMAGE_report.py", line 1409, in <module>
outFormat = 'png')
File "/data_1/IRM/amigo/src/IRMAGE_python_modules/IRMAGE_reporting.py", line 1576, in plotPatVsRef
bookRef = xlrd.open_workbook(refFile)
File "/usr/lib/python3.6/site-packages/xlrd/__init__.py", line 422, in open_workbook
ragged_rows=ragged_rows,
File "/usr/lib/python3.6/site-packages/xlrd/xlsx.py", line 784, in open_workbook_2007_xml
ensure_elementtree_imported(verbosity, logfile)
File "/usr/lib/python3.6/site-packages/xlrd/xlsx.py", line 48, in ensure_elementtree_imported
ET.iterparse(_dummy_stream)
File "/usr/lib64/python3.6/xml/etree/ElementTree.py", line 1217, in iterparse
pullparser = XMLPullParser(events=events, _parser=parser)
File "/usr/lib64/python3.6/xml/etree/ElementTree.py", line 1256, in __init__
self._parser = _parser or XMLParser(target=TreeBuilder())
File "/usr/lib64/python3.6/xml/etree/ElementTree.py", line 1454, in __init__
"No module named expat; use SimpleXMLTreeBuilder instead"
ImportError: No module named expat; use SimpleXMLTreeBuilder instead
I am a little bit lost facing this problem since yesterday and I confess that I have no new idea to solve this problem … All idea, information, help about this problem will be very welcome
I just had the idea to check the shared object dependencies concerning the /usr/lib64/python3.6/lib-dynload/pyexpat.cpython-36m-x86_64-linux-gnu.so that raised an ImportError exception !!!
in a shell:
% ldd /usr/lib64/python3.6/lib-dynload/pyexpat.cpython-36m-x86_64-linux-gnu.so
linux-vdso.so.1 (0x00007ffd8f7c5000)
libexpat.so.1 => /lib64/libexpat.so.1 (0x00007ff82fae8000)
libpython3.6m.so.1.0 => /lib64/libpython3.6m.so.1.0 (0x00007ff82f588000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007ff82f369000)
libc.so.6 => /lib64/libc.so.6 (0x00007ff82ef86000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007ff82ed82000)
libutil.so.1 => /lib64/libutil.so.1 (0x00007ff82eb7f000)
libm.so.6 => /lib64/libm.so.6 (0x00007ff82e82a000)
/lib64/ld-linux-x86-64.so.2 (0x00007ff82ff2c000)
In Matlab:
! ldd /usr/lib64/python3.6/lib-dynload/pyexpat.cpython-36m-x86_64-linux-gnu.so
linux-vdso.so.1 (0x00007ffeec596000)
libexpat.so.1 => /usr/local/MATLAB/R2017b/bin/glnxa64/libexpat.so.1 (0x00007fcd2931e000)
libpython3.6m.so.1.0 => /lib64/libpython3.6m.so.1.0 (0x00007fcd28dbe000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fcd28b9f000)
libc.so.6 => /lib64/libc.so.6 (0x00007fcd287bc000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007fcd285b8000)
libutil.so.1 => /lib64/libutil.so.1 (0x00007fcd283b5000)
libm.so.6 => /lib64/libm.so.6 (0x00007fcd28060000)
/lib64/ld-linux-x86-64.so.2 (0x00007fcd29758000)
So it becomes evident that the problem comes from the libexpat.so.1 shared object that is older in the Matlab distrib (libexpat.so.1.5.0), than the one of the system (libexpat.so.1.6.7)!!!:
% ls -l /usr/local/MATLAB/R2017b/bin/glnxa64/libexpat.so*
lrwxrwxrwx. 1 root root 17 Feb 22 12:41 libexpat.so.1 -> libexpat.so.1.5.0
-r-xr-xr-x. 1 root root 182222 Jul 24 2017 libexpat.so.1.5.0
% ls -l /usr/lib64/libexpat.so*
lrwxrwxrwx. 1 root root 17 Nov 6 18:00 /usr/lib64/libexpat.so -> l ibexpat.so.1.6.7
lrwxrwxrwx. 1 root root 17 Nov 6 18:00 /usr/lib64/libexpat.so.1 -> libexpat.so.1.6.7
-rwxr-xr-x. 1 root root 206000 Nov 6 18:00 /usr/lib64/libexpat.so.1.6.7
I fixed the problem by moving to an exclude directory the shared object of the Matlab distrib, to force to use the one of the system !!!
# cd /usr/local/MATLAB/R2017b/bin/glnxa64
# mkdir exclude
# mv libexpat.so* exclude/
All is working fine now !!!
P.S. I experienced few other problems to running correctly Matlab after the upgrade of Matlab and Fedora that I do not detail here. But if somebody have other problems with Matlab and FC26 or FC27 I can give upon request all the changes that I done (all are concerning shared object that are too old in the Matlab distrib !!!)