Search code examples
shared-librariesnvcc

Compiling with nvcc using g++ and additional root libraries results in "undefined references" or "unrecognized command line option"

I just started with cuda and nvcc and cannot progress beyond this particular problem. I would greatly appreciate help.

For the minimal example I have three files: main.cpp

#include "myClass.hh"

int main()
{
    myClass temp;
    return 0;
}

myClass.hh

#include "TTree.h" //can be replaced by any root class

class myClass
{
public:
    myClass();
    ~myClass() = default;
};

and myClass.cu

#include "myClass.hh"

__global__ void myCudaFunction(){};

myClass::myClass() { myCudaFunction<<<1, 1>>>(); }

which I am compiling with

`nvcc -ccbin g++ -o a.out main.cpp myClass.cu -Xcompiler="-L/home/main_user/Tools/ROOT/install/lib" -Xcompiler="-lCore -lImt -lRIO -lNet -lHist -lGraf -lGraf3d -lGpad -lROOTVecOps -lTree -lTreePlayer -lRint -lPostscript -lMatrix -lPhysics -lMathCore -lThread -lMultiProc -lROOTDataFrame -pthread -lm -ldl -rdynamic" --linker-options="-rpath,/home/main_user/Tools/ROOT/install/lib" -Xcompiler="-I$(root-config --incdir)" `

but this returns the error:

`/home/main_user/Tools/ROOT/install/include/TStorage.h:131:37: warning: declaration of ‘static void TStorage::UpdateIsOnHeap(const volatile UInt_t&, volatile UInt_t&)’ with attribute ‘noinline’ follows inline declaration [-Wattributes]
 R__NEVER_INLINE void TStorage::UpdateIsOnHeap(volatile const UInt_t &uniqueID, volatile UInt_t &bits) {
                                 ^~~~~~~~
/home/main_user/Tools/ROOT/install/include/TStorage.h:91:20: note: previous declaration of ‘static void TStorage::UpdateIsOnHeap(const volatile UInt_t&, volatile UInt_t&)’ was here
    static void UpdateIsOnHeap(volatile const UInt_t &uniqueID, volatile UInt_t &bits);
                ^~~~~~~~~~~~~~
/usr/bin/ld: /tmp/tmpxft_0000d4e8_00000000-8_main.o: in function     `__static_initialization_and_destruction_0(int, int)':
main.cpp:(.text+0x99): undefined reference to `TVersionCheck::TVersionCheck(int)'
/usr/bin/ld: /tmp/tmpxft_0000d4e8_00000000-11_myClass.o: in function `__static_initialization_and_destruction_0(int, int)':
tmpxft_0000d4e8_00000000-5_myClass.cudafe1.cpp:(.text+0x38d): undefined reference to  `TVersionCheck::TVersionCheck(int)'
collect2: error: ld returned 1 exit status`

.

Usually the problem with this is, that a library is missing, that is why I copy and pasted the output of

main_user@Notebook:~/cudaTest$ root-config --libs -L/home/main_user/Tools/ROOT/install/lib -lCore -lImt -lRIO -lNet -lHist -lGraf -lGraf3d -lGpad -lROOTVecOps -lTree -lTreePlayer -lRint -lPostscript -lMatrix -lPhysics -lMathCore -lThread -lMultiProc -lROOTDataFrame -Wl,-rpath,/home/main_user/Tools/ROOT/install/lib -pthread -lm -ldl -rdynamic . The output of the other variable is

main_user@Notebook:~/cudaTest$ root-config --incdir /home/main_user/Tools/ROOT/install/include

and I have only one ROOT installation.

My suspicion is, that the error is related to

-Wl,-rpath,/home/main_user/Tools/ROOT/install/lib

, but if I use

-Xcompiler="$(root-config --libs)"

instead, I get

main_user@Notebook:~/cudaTest$ nvcc -ccbin g++ -o a.out main.cpp myClass.cu -Xcompiler="$(root-config --libs)" -Xcompiler="-I$(root-config --incdir)" g++-8: error: unrecognized command line option ‘-Wl’; did you mean ‘-W’? g++-8: error: unrecognized command line option ‘-rpath’ .

I also tried to rearrange the input, but without success. I have also read, that not all library files are suitable for nvcc. It seems only .so files are permissible, so here is the check:

main_user@Notebook:~/cudaTest$ ls /home/main_user/Tools/ROOT/install/lib/ ASImageGui.pcm libcomplexDict.rootmap libmapDict.rootmap libRooFitHS3.so libXMLParser.so libXrdSecProt-5.so Recorder.pcm ASImage.pcm libcomplexDict.so libmapDict.so libRooFitMore.so libXrdAppUtils.so libXrdSecpwd-5.so RGL.pcm _Builtin_intrinsics.pcm libCore.so libMathCore.so libRooFitMultiProcess.so libXrdAppUtils.so.2 libXrdSecsss-5.so RHTTP.pcm _Builtin_stddef_max_align_t.pcm libc.pcm libMathMore.so libRooFitRDataFrameHelpers.so libXrdAppUtils.so.2.0.0 libXrdSecunix-5.so RHTTPSniff.pcm Cling_Runtime_Extra.pcm libcppyy3_8.so libMatrix.so libRooFit.so libXrdBlacklistDecision-5.so libXrdServer.so Rint.pcm Cling_Runtime.pcm libcppyy_backend3_8.so libMinuit2.so libRooFitZMQ.so libXrdBwm-5.so libXrdServer.so.3 RIO.pcm cmdLineUtils.py libdequeDict.rootmap libMinuit.so libRooStats.so libXrdCksCalczcrc32-5.so libXrdServer.so.3.0.0 RMPI.pcm Core.pcm libdequeDict.so libMLP.so libRootAuth.so libXrdClProxyPlugin-5.so libXrdSsi-5.so RMySQL.pcm cppyy libEG.so libmultimap2Dict.rootmap libROOTDataFrame.so libXrdCl.so libXrdSsiLib.so RooFitCore.pcm cppyy_backend libEve.so libmultimap2Dict.so libROOTPythonizations3_8.so libXrdCl.so.3 libXrdSsiLib.so.2 RooFitHS3.pcm DistRDF libFFTW.so libmultimapDict.rootmap libROOTTMVASofie.so libXrdCl.so.3.0.0 libXrdSsiLib.so.2.0.0 RooFitMore.pcm EG.pcm libFitPanel.so libmultimapDict.so libROOTTPython.so libXrdCmsRedirectLocal-5.so libXrdSsiLog-5.so RooFit.pcm Eve.pcm libFITSIO.so libMultiProc.so libROOTVecOps.so libXrdCryptoLite.so libXrdSsiShMap.so RooFitRDataFrameHelpers.pcm FFTW.pcm libFoam.so libmultisetDict.rootmap libSessionViewer.so libXrdCryptoLite.so.2 libXrdSsiShMap.so.2 RooStats.pcm FitPanel.pcm libforward_listDict.rootmap libmultisetDict.so libsetDict.rootmap libXrdCryptoLite.so.2.0.0 libXrdSsiShMap.so.2.0.0 ROOT FITSIO.pcm libforward_listDict.so libNet.so libsetDict.so libXrdCrypto.so libXrdThrottle-5.so RootAuth.pcm Foam.pcm libFumili.so libNetxNG.so libSmatrix32.rootmap libXrdCrypto.so.2 libXrdUtils.so ROOT_Config.pcm Fumili.pcm libGdml.so libNew.so libSmatrix.so libXrdCrypto.so.2.0.0 libXrdUtils.so.3 ROOTDataFrame.pcm Gdml.pcm libGed.so libPhysics.so libSpectrumPainter.so libXrdCryptossl-5.so libXrdUtils.so.3.0.0 ROOT_Foundation_C.pcm Ged.pcm libGenetic.so libPostscript.so libSpectrum.so libXrdFfs.so libXrdXml.so ROOT_Foundation_Stage1_NoRTTI.pcm Genetic.pcm libGenVector32.rootmap libProofBench.so libSPlot.so libXrdFfs.so.3 libXrdXml.so.3 ROOT_Rtypes.pcm GenVector.pcm libGenVector.so libProofDraw.so libSQLIO.so libXrdFfs.so.3.0.0 libXrdXml.so.3.0.0 ROOTTMVASofie.pcm GeomBuilder.pcm libGeomBuilder.so libProofPlayer.so libSrvAuth.so libXrdFileCache-5.so libXrdXrootd-5.so ROOTTPython.pcm GeomPainter.pcm libGeomPainter.so libProof.so libThread.so libXrdHttp-5.so MathCore.pcm ROOTVecOps.pcm Geom.pcm libGeom.so libPyMVA.so libTMVAGui.so libXrdHttpUtils.so MathMore.pcm SessionViewer.pcm Gpad.pcm libGpad.so libQuadp.so libTMVA.so libXrdHttpUtils.so.2 Matrix.pcm Smatrix.pcm Graf3d.pcm libGraf3d.so libRCsg.so libTreePlayer.so libXrdHttpUtils.so.2.0.0 Minuit2.pcm SpectrumPainter.pcm Graf.pcm libGraf.so libRecorder.so libTree.so libXrdN2No2p-5.so Minuit.pcm Spectrum.pcm GuiBld.pcm libGuiBld.so libRGL.so libTreeViewer.so libXrdOfsPrepGPI-5.so MLP.pcm SPlot.pcm GuiHtml.pcm libGuiHtml.so libRHTTPSniff.so libUnfold.so libXrdOssCsi-5.so modules.idx SQLIO.pcm Gui.pcm libGui.so libRHTTP.so libunordered_mapDict.rootmap libXrdOssSIgpfsT-5.so modules.timestamp std.pcm Gviz3d.pcm libGviz3d.so libRint.so libunordered_mapDict.so libXrdPfc-5.so MultiProc.pcm Thread.pcm GX11.pcm libGX11.so libRIO.so libunordered_multimapDict.rootmap libXrdPosixPreload.so Net.pcm TMVAGui.pcm GX11TTF.pcm libGX11TTF.so libRMPI.so libunordered_multimapDict.so libXrdPosixPreload.so.2 NetxNG.pcm TMVA.pcm HistFactory.pcm libHistFactory.so libRMySQL.so libunordered_multisetDict.rootmap libXrdPosixPreload.so.2.0.0 Physics.pcm Tree.pcm HistPainter.pcm libHistPainter.so libRooBatchCompute_AVX2.so libunordered_multisetDict.so libXrdPosix.so Postscript.pcm TreePlayer.pcm Hist.pcm libHist.so libRooBatchCompute_AVX512.so libunordered_setDict.rootmap libXrdPosix.so.3 ProofBench.pcm TreeViewer.pcm Html.pcm libHtml.so libRooBatchCompute_AVX.so libunordered_setDict.so libXrdPosix.so.3.0.0 ProofDraw.pcm Unfold.pcm Imt.pcm libImt.so libRooBatchCompute_CUDA.so libvalarrayDict.rootmap libXrdPss-5.so Proof.pcm X3d.pcm JsMVA libJupyROOT3_8.so libRooBatchCompute_GENERIC.so libvalarrayDict.so libXrdSec-5.so ProofPlayer.pcm XMLIO.pcm JupyROOT liblistDict.rootmap libRooBatchCompute.so libvectorDict.rootmap libXrdSecgsi-5.so __pycache__ XMLParser.pcm libASImageGui.so liblistDict.so libRooBatchCompute_SSE4.1.so libvectorDict.so libXrdSecgsiAUTHZVO-5.so PyMVA.pcm libASImage.so libmap2Dict.rootmap libRooFitCommon.so libX3d.so libXrdSecgsiGMAPDN-5.so Quadp.pcm libCling.so libmap2Dict.so libRooFitCore.so libXMLIO.so libXrdSeckrb5-5.so RCsg.pcm

I was crosschecking the spelling, rearranging the arguments and googling. I tried to include the library path with --linker-options="-rpath,/home/main_user/Tools/ROOT/install/lib" as well as with Xcompiler. Can you help me with this problem, please?

Edit: The solution in https://forums.developer.nvidia.com/t/solved-error-while-linking-with-nvcc/46476/3 did not work for me either ('g++-8: error: unrecognized command line option ‘-Wl’; did you mean ‘-W’?'). Could the problem be, that nvcc uses g++-8, while g++ 9.4.0 is installed?

Solution

  • Replacing -Xcompiler "$(root-config --libs)" with --linker-options "-L/home/main_user/Tools/ROOT/install/lib -lCore -lImt -lRIO -lNet -lHist -lGraf -lGraf3d -lGpad -lROOTVecOps -lTree -lTreePlayer -lRint -lPostscript -lMatrix -lPhysics -lMathCore -lThread -lMultiProc -lROOTDataFrame -pthread -lm -ldl -rdynamic", so without the -Wl part of root-config --libs, seems to work.