Continuous Integration with Talend and Jenkins/Maven

Question

I have a Talend studio Enterprise veresion 6.4.1. I have downloaded the CI builder and installed with maven using the below command.

mvn install:install-file -Dfile=ci.builder-6.4.1.jar -DpomFile=ci.builder-6.4.1.pom

I have exported the job from Talend studio put into the Git. It contains the following files.

1) process a) .items b) .properties c).screenshot 2) talend.project

I have created a pom.xml with the below details

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> 
    <modelVersion>4.0.0</modelVersion> 
    <groupId>org.talend</groupId> 
    <artifactId>buildsources</artifactId> 
    <version>0.1</version> 
    <packaging>pom</packaging> 
    <build> 
      <plugins> 
      <plugin> 
        <groupId>org.talend</groupId> 
        <artifactId>ci.builder</artifactId> 
        <version>6.4.1</version> 
        <executions> 
          <execution> 
            <phase>validate</phase> 
              <goals> 
                <goal>generate</goal> 
              </goals> 
          </execution> 
        </executions> 
      </plugin> 
      </plugins> 
    </build> 
</project>

I have placed pom.xml in the root folder of commandline workspace.

I placed the checked out folder inside command line workspace .

After that I executed the commandline.bat inside Talend studio to start the Talend.

I tried executing the below command to generate the source file.

mvn org.talend:ci.builder:6.4.1:generate -X -Dcommandline.workspace="D:\ci-builder\commandline_workspace\TEST" -Dcommandline.host=localhost -Dcommandline.port=8002 [email protected] -DprojectsTargetDirectory=D:\buildsources\

Now it is able to find the project but at the end it says no job

[INFO] Preparing for the project: TEST
[INFO]  Logon project
[INFO]  Generating sources...
[INFO]  Logoff project
[WARNING] There's no job in current project!

I am using Windows machine to test this - Windows 10 64 bit. Maven version is 3.5.3

I have been referred the below links for doing this.

https://community.talend.com/t5/Architecture-Best-Practices-and/Continuous-Integration-with-Talend-CI-Builder-6-3/ta-p/33033

https://akhileshthipparthi.wordpress.com/2017/01/23/talend-ci-using-jenkins/

But when I check the .Java folder in my Talend installation directory command line workspace, I can see some files generated. pom.xml and pom_routines.xml. But there are no job related files getting generated.

And with those pom.xml , if I do a mvn clean install, it is generating a routines.jar file only in the target directory.

I am not able to proceed further.

Any pointers will be helpful.

Answer 1

One thing I wanted to highlight is that, for talend CI builder to work there is no need to writing test cases.

I was able to build the job using the CI builder using the below steps.

1. Inside the command line work space, you need to keep the complete project structure.
2. Checkout the source files from Git and copy the files to the project folder.
3. All the source files will be present inside the process folder of the project folder.
4. Setup the maven repository for Talend command line. It will be inside the configuration folder inside the Talend folder.
5. Execute the below command to Generate the source code inside the command line workspace.
   Here SayHello is the Job name.

   mvn org.talend:ci.builder:6.4.1:generate -Dcommandline.workspace="D:\ci-builder\commandline_workspace\" -Dcommandline.host=localhost -Dcommandline.port=8002 [email protected] -DprojectsTargetDirectory=D:\ci-builder\target -DitemFilter=(type=process)and(label%SayHello*)

if there is a parent child job, we need to place all the jobs inside a folder and we need to refer that path during generate source.
Here Sample is the folder name

mvn org.talend:ci.builder:6.4.1:generate -Dcommandline.workspace="D:\ci-builder\commandline_workspace\" -Dcommandline.host=localhost -Dcommandline.port=8002 [email protected] -DprojectsTargetDirectory=D:\ci-builder\target -DitemFilter=(type=process)and(Path%Sample*)

6. It will generate the source files and it will be kept in "D:\ci-builder\target"

7. Now navigate to that path and do mvn clean package

8. The job will be built and it will be placed inside the target folder. It will contain all the related Jar's, shell script, batch script and libraries and routines. Also it will have the entire folder structure.

Note : If you get the message like, there is no job in folder, just check if there is any error in the talend command line shell. Mostly u can encounter, the .Java folder is missing in command line workspace or there will be dependency errors (missing Jars in the maven local repo).