I am working on a basic setup for multiple Python projects that are depending on each other (I am using Python 3.6). Later on there will be also Java involved, so I opted for Maven for the build management. So far, I have a parent-pom, a common and a few specific projects.
The general folder structure is:
parent-pom
|- pom.xml
common
|- src/main/python/xxx/yyy/common
|- Utils.py
|- subfolder1
|- subfolder2
|- …
|- pom.xml
project1
|- src/main/python/xxx/yyy/project1
|- subfolderA
|- subfolderB
|- …
|- pom.xml
The actual POM files look like so:
parent-pom project "pom.xml":
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>xxx.yyy</groupId>
<artifactId>parent-pom</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>pom</packaging>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>xxx.yyy</groupId>
<artifactId>common</artifactId>
<version>0.0.1-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>xxx.yyy</groupId>
<artifactId>project1</artifactId>
<version>0.0.1-SNAPSHOT</version>
</dependency>
</dependencies>
</dependencyManagement>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.0</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
<encoding>UTF-8</encoding>
</configuration>
</plugin>
</plugins>
</build>
</project>
common project "pom.xml":
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<artifactId>common</artifactId>
<packaging>pom</packaging>
<parent>
<groupId>aaa</groupId>
<artifactId>parent-pom</artifactId>
<version>0.0.1-SNAPSHOT</version>
</parent>
</project>
project1 project "pom.xml":
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<artifactId>project1</artifactId>
<packaging>pom</packaging>
<parent>
<groupId>xxx.yyy</groupId>
<artifactId>parent-pom</artifactId>
<version>0.0.1-SNAPSHOT</version>
</parent>
<dependencies>
<dependency>
<groupId>xxx.yyy</groupId>
<artifactId>common</artifactId>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.6.0</version>
<executions>
<execution>
<configuration>
<executable>python</executable>
<workingDirectory>src\main\python\xxx\yyy\project1\</workingDirectory>
<arguments>
<argument>Launcher.py</argument>
</arguments>
</configuration>
<id>python-build</id>
<phase>prepare-package</phase>
<goals>
<goal>exec</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
Now I have a class in project1 called Launcher, which should import the class Utils from the common package:
Launcher.py
from xxx.yyy.common.Utils import Utils
class Launcher:
def __init__(self):
Utils.doSomething()
if ('__main__' == __name__):
Launcher()
Utils.py
class Utils:
@staticmethod
def doSomething():
print ("Test")
However, the import
from xxx.yyy.common.Utils import Utils
Cannot be resolved:
[INFO] Scanning for projects...
[INFO]
[INFO] --------------------------< xxx.yyy:project1 >--------------------------
[INFO] Building project1 0.0.1-SNAPSHOT
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ project1 ---
[INFO]
[INFO] --- exec-maven-plugin:1.6.0:exec (python-build) @ project1 ---
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @ project1 ---
[INFO] Installing C:\...\project1\pom.xml to C:\...\.m2\repository\xxx\yyy\project1\0.0.1-SNAPSHOT\project1-0.0.1-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.949 s
[INFO] Finished at: 2019-04-24T12:19:16+02:00
[INFO] ------------------------------------------------------------------------
C:\...\project1>mvn clean install
[INFO] Scanning for projects...
[INFO]
[INFO] --------------------------< xxx.yyy:project1 >--------------------------
[INFO] Building project1 0.0.1-SNAPSHOT
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ project1 ---
[INFO]
[INFO] --- exec-maven-plugin:1.6.0:exec (python-build) @ project1 ---
Traceback (most recent call last):
File "Launcher.py", line 1, in <module>
from xxx.yyy.common.Utils import Utils
ModuleNotFoundError: No module named 'xxx'
[ERROR] Command execution failed.
org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
at org.apache.commons.exec.DefaultExecutor.executeInternal (DefaultExecutor.java:404)
at org.apache.commons.exec.DefaultExecutor.execute (DefaultExecutor.java:166)
at org.codehaus.mojo.exec.ExecMojo.executeCommandLine (ExecMojo.java:804)
at org.codehaus.mojo.exec.ExecMojo.executeCommandLine (ExecMojo.java:751)
at org.codehaus.mojo.exec.ExecMojo.execute (ExecMojo.java:313)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.787 s
[INFO] Finished at: 2019-04-24T12:19:24+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:exec (python-build) on project project1: Command execution failed.: Process exited with an error: 1 (Exit value: 1) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
I guess, declaring POM dependencies is not sufficient in order to access the modules and packages across different Python/Maven projects.
Is there a way to solve this issue within the POM files?
@khmarbaise: The question was how to resolve the ModuleNotFoundError by adding the Python projects to the PYTHONPATH within the POM file. Sorry, I should have asked more clearly.
However, I found out how to solve this. I simply had to add PYTHONPATH paths to the environmentVariables tag in the configurations of the Python execution section ...
Solution:
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.6.0</version>
<executions>
<execution>
<configuration>
<executable>python</executable>
<workingDirectory>src\main\python\xxx\yyy\project1\</workingDirectory>
<arguments>
<argument>Launcher.py</argument>
</arguments>
<environmentVariables>
<PYTHONPATH>
${project.basedir.parent}/common/src/main/python/ <!-- added PYTHONPATH -->
</PYTHONPATH>
</environmentVariables>
</configuration>
<id>python-build</id>
<phase>prepare-package</phase>
<goals>
<goal>exec</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>