Search code examples
antfilesetxmltask

ANT task - multiple files - loop through


I have quite complicated ant task to write. I had to make separate builds files and 1 left. There are 4 files:

 buildpackage.bat, buildpackage.xml, buildpackage.txt, structure.xml

buildpackage.bat is just an ant file that calls buildpackage.xml "ant -f buildpackage.xml". That is fine.

buildpackage.txt is simple and has got comma-separated lines:

server1,client1
server2,client2
genericserver,client[x]
server[x],client[x]

There is always server name and client name:

  • Server name can be letters only, numbers and letters but has variable length.

  • Client name - letters only.

I need to split those files into 2 separate variables: server.name and client.name.

What I have done is:

<loadfile property="buildpackage" srcFile="buildpackage.txt"/>    
<for list="${buildpackage}" param="depl.det" delimiter="${line.separator}">
 <sequential>
  <echo>
   Input:   @{deploy.details}
  </echo>
 </sequential>
</for>

The results I get are:

[echo] Input:   server_01,Client_01
[echo] Input:   server_02,Client_02
[echo] Input:   server_03,Client_03
[echo] Input:   clientserver,client_04

I would like server_01 to be parameter1 and client_02 to be parameter2.

I will perform other tasks in a sequence. I will copy files from client[x] directory to server[x] (that is simple but how do I split that string into 2 separate variables?).

And now the very complicated task.

I also have structure.xml file. The structure is:

 <core>
   <clients>
     <client>
       <name>Client_01</name>
       <branding>brand_01</branding>
       <applications>
         <application>application_01</application>
       </applications>
       <solutions>
         <solution>solution01</solution>
       </solutions>
   </client>
   <client>
     <name>Client_07</name>
     <branding>brand_09</branding>
     <applications>
       <application>application_03</application>
     </applications>
     <solutions>
       <solution>solution01</solution>
       <solution>solution07</solution>
       <solution>solution08</solution>
     </solutions>
   </client>
  <clients>
 <core>

I have plenty of clients, applications, brandings, solutions. Each client can have only 1 application and 1 branding but multiple solutions.

Based on previous sequential task (get server and name) I have to loop through that xml file and get for each matching client (from previous task)

- branding
- application 
- solutions (multiple or one) 

I can read XML using <xmlproperty> but I do not know how to do it sequentially (iterate) and get matching properties.

I know it is complicated. The summary is:

  1. build script starts
  2. build script extracts data from buildpackage.txt as client.name and server.name
  3. build script gets ${client.name} from step 2 as a parameter and loops through structure.xml. It finds matching branding/application/solutions and outputs it for each matching client.

Output:

server.name
client.name
branding.name
application name
solution [X]
solution [y]

Ant has to be processed for each server/client. I can do the rest of the build and compile if I have the properties. I can not touch structure.xml. I can do all I need/want to buildpackage.cml and buildpackge.txt. Buildpackage.txt could be replaced by .xml file or any file (for example .properties). It has to contain those 2 values server and client.

Thank you for your help.

EDIT

Now i have a problem. If I put same application 2x into a buildfile it builds only first one. Here is my build.xml file

<?xml version="1.0" encoding="UTF-8"?>
<project name="DoTheJob" basedir="." xmlns:ac="antlib:net.sf.antcontrib" xmlns:if="ant:if" xmlns:unless="ant:unless">
    <!-- Define classpath and default locations-->
    <property name="root.dir" location="../.."/>
    <property name="dest.dir" location="packages"/>
    <property name="ant.lib.dir" location="../lib"/>
    <property name="repository_url" value="http://repository-url"/> 
    <path id="project.classpath">
        <fileset dir="${ant.lib.dir}">
        <include name="*.jar"/>
        </fileset>
        <fileset dir="${root.dir}/IIT/lib">
            <include name="*.jar"/>
        </fileset>
        <pathelement location="../savedjar/compiled.jar"/>
    </path>
    <path id="svnant.path">
        <fileset dir="../lib/">
            <include name="svnant.jar"/>
            <include name="svnClientAdapter.jar"/>
        </fileset>
    </path>
    <typedef resource="org/tigris/subversion/svnant/svnantlib.xml" classpathref="svnant.path" /> 
    <tstamp>
        <format property="build_time" pattern="YYYY MMMM dd HH:mm:ss"/>
    </tstamp>   
    <!-- Change characters to lowercase -->
    <scriptdef language="javascript" name="lowercase">
        <attribute name="string" />
        <attribute name="to" />
        project.setProperty(attributes.get("to"),attributes.get("string").toLowerCase());
    </scriptdef>
    <!-- Import XMLTask for XML transformations-->
    <taskdef name="xmltask" classname="com.oopsconsultancy.xmltask.ant.XmlTask" classpathref="project.classpath"/>
    <!-- Define buildfiles locations-->
    <property name="structure.file" location="../structure.xml"/>
    <property name="buildpackage.file" location="buildpackage.txt"/>
    <!-- load client/server file-->
    <property file="${buildpackage.file}"/>
    <!-- loop over all clients --> 
    <xmltask source="${structure.file}">
    <call path="//client">
        <param path="name/text()" name="client"/>
        <param path="branding/text()" name="branding"/>
        <param path="applications/application/text()" name="application"/>
            <actions>
                <!-- loop through all XML properties only when @{client} condition is matched --> 
                <ac:if>
                    <isset property="@{client}"/>
                    <then>
                        <echo>=====================================================</echo>
                        <echo>Building package for @{client} on ${@{client}}       </echo>
                        <echo>=====================================================</echo>
                        <!-- creating destination directories -->                           
                        <delete dir="${build.dir}/@{client}"/>
                        <mkdir dir="${dest.dir}/@{client}"/> 
                        <echo>=====================================================</echo>
                        <echo>Copy application files code/temaplates to destination</echo>
                        <echo>=====================================================</echo>                      
                        <!-- creating code build directory -->
                        <mkdir dir="${dest.dir}/@{client}/code"/> 
                        <!-- copy application code --> 
                        <echo>copy application code</echo>
                        <copy todir="${dest.dir}/@{client}/code/application/@{application}" overwrite="yes">
                            <fileset dir="${root.dir}/client-source/application/@{application}" includes="**/*.*"/>
                        </copy>
                        <!-- copy application templates --> 
                        <mkdir dir="${dest.dir}/@{client}/app/@{client}"/> 
                        <echo>copy application templates</echo>
                        <copy todir="${dest.dir}/@{client}/app/@{client}" overwrite="yes">
                            <fileset dir="${root.dir}/applications/apps/@{application}" includes="**/*.*"/>
                        </copy>
                        <echo>===================================================</echo>
                        <echo>copy Solutions files code/temaplates to destination</echo>
                        <echo>===================================================</echo>    
                        <!-- nested xmltask for loop over solutions -->
                        <xmltask source="${structure.file}">
                            <call path="/core/clients/client[name/text()='@{client}']/*/solution">
                                <param path="text()" name="solution"/>
                                    <actions>
                                        <!-- making sure that solution is lowercase -->
                                        <lowercase string="@{solution}" to="solution.lowercase" />
                                        <echo>copy solutions code</echo>
                                        <!-- copy solutions code -->
                                        <copy todir="${dest.dir}/@{client}/code/solutions/${solution.lowercase}" overwrite="yes">
                                            <fileset dir="${root.dir}/client-source/solutions/${solution.lowercase}" includes="**/*.*"/>
                                        </copy>
                                        <!-- copy solutions templates -->
                                        <echo>copy Solutions templates</echo>
                                        <copy todir="${dest.dir}/@{client}/app/@{client}" overwrite="yes">
                                            <fileset dir="${root.dir}/applications/solutions/${solution.lowercase}" includes="**/*.*"/>
                                        </copy>
                                    </actions>
                            </call>
                        </xmltask>
                        <echo>==============================================</echo>
                        <echo>copy Client files code/temaplates to destination</echo>
                        <echo>==============================================</echo> 
                        <!-- copy branding directory --> 
                        <copy todir="${dest.dir}/@{client}/app/@{client}" overwrite="yes">
                            <fileset dir="${root.dir}/applications/brandings/@{branding}" includes="**/*.*"/>
                        </copy>                         
                        <!-- copy client code --> 
                        <copy todir="${dest.dir}/@{client}/code/client/@{client}" overwrite="yes">
                            <fileset dir="${root.dir}/client-source/client/@{client}" includes="**/*.*"/>
                        </copy>                         
                        <!-- set templates directory --> 
                        <copy todir="${dest.dir}/@{client}/app/@{client}" overwrite="yes">
                            <fileset dir="${root.dir}/applications/clients/@{client}" includes="**/*.*"/>
                        </copy>         
                        <echo>==============================================</echo>
                        <echo>compiling code                                </echo>
                        <echo>==============================================</echo> 
                        <!-- making classes directory --> 
                        <mkdir dir="${dest.dir}/@{client}/classes"/>
                        <javac destdir="${dest.dir}/@{client}/classes" classpathref="project.classpath" debug="on" fork="true" includeantruntime="false" memoryinitialsize="256m" memorymaximumsize="1024m">
                            <src path="${dest.dir}/@{client}/code/"/>
                        </javac>
                        <echo>==============================================</echo>
                        <echo>creating Client.jar                           </echo>
                        <echo>==============================================</echo> 
                        <!-- jar classes directory --> 
                        <jar jarfile="${dest.dir}/@{client}/app/Client.jar" basedir="${dest.dir}/@{client}/classes"/>
                        <!-- reaplce text in files --> 
                        <replaceregexp byline="true">
                            <regexp pattern="@ReleaseDate@"/>
                            <substitution expression="${build_time}"/>
                            <fileset dir="${dest.dir}/@{client}/app/@{client}/templates/">
                                <include name="*.htm"/>
                                <include name="*.html"/>            
                                <include name="*.ini"/>
                            </fileset>
                        </replaceregexp>
                        <!-- get svn info --> 
                        <svn username="********" password="************" javahl="false" svnkit="false">
                            <info target="${repository_url}" />
                        </svn>
                        <!-- creating Client.ini file --> 
                        <touch file="${dest.dir}/@{client}/app/Client.ini"/>
                        <concat destfile="${dest.dir}/@{client}/app/Client.ini" append="true">releaseDate=${build_time}${line.separator}</concat>
                        <concat destfile="${dest.dir}/@{client}/app/Client.ini" append="true">svnrevision=${svn.info.lastRev}${line.separator}</concat>
                        <!-- ziping package --> 
                        <zip destfile="${dest.dir}/@{client}/@{client}.zip" basedir="${dest.dir}/@{client}/app/" />
                        <echo>==============================================</echo>
                        <echo>sending @{client}.zip to the server via scp      </echo>
                        <echo>==============================================</echo> 
                        <!-- scp file to remote server -->
                        <property name="scp_username" value="**********"/>
                        <property name="scp_password" value="**********"/>
                        <property name="scp_server" value="${@{client}}"/>
                        <property name="scp_remote_directory" value="*********"/>
                        <scp trust="true" verbose="true" localFile="${dest.dir}/@{client}/@{client}.zip" remoteTodir="${scp_username}:${scp_password}@${scp_server}:${scp_remote_directory}"/>  
                        <echo>${line.separator}${line.separator}${line.separator}</echo>
                    </then>
                </ac:if> 
            </actions>
        </call>
    </xmltask>
</project>

And if I try yo build buildpackage.txt file

client1=server1
client2=server1
client1=server2

I do get client1->server1, client2->server1 but ...... no client 1to server2

EDIT2

All fixed. The solution for scp task is:

       <!-- scp file to remote server -->
        <property name="scp_username" value="XXX"/>
        <property name="scp_password" value="YYY"/>
        <property name="scp_remote_directory" value="path"/>
        <for list="${@{client}}" delimiter="=" param = "val">
            <sequential>
            <property name="token" value="@"/>
            <scp trust="true" verbose="true" localFile="${dest.dir}/@{client}/@{client}.zip" remoteTodir="${scp_username}:${scp_password}${token}@@{val}:${scp_remote_directory}"/> 
        </sequential>
        </for>

All works. Thank you for your help


Solution

  • For any xmldriven build i recommend using the ant addon xmltask, so something like :
    (assuming that clientnames are similar in buildpackage.txt and structure.xml - your snippet has several names Client_01, client1 .. !?)

    <project>
     <!-- Import XMLTask -->
     <taskdef name="xmltask" classname="com.oopsconsultancy.xmltask.ant.XmlTask"/>
    
     <!-- make buildpackage.txt a valid propertyfile to get the job done
          f.e. server1,client1 will get client1=server1 etc. ..
          to make server easily accessible within xml loop
     -->
     <replaceregexp
       file="buildpackage.txt"
       match="(.+),(.+)"
       replace="\2=\1"
       byline="true" />
    
     <!-- now load that file -->
     <property file="buildpackage.txt"/>
    
     <!-- loop over all clients --> 
     <xmltask source="structure.xml">
      <call path="//client">
       <param path="name/text()" name="client"/>
       <param path="branding/text()" name="branding"/>
       <param path="applications/application/text()" name="application"/>
    
       <!-- inside action adress params with @{..} syntax ! -->
       <actions>
        <!-- the copy (or whatever) action
             here demonstrated with echoxml -->
       <echoxml>
        <!-- access the corresponding server (set via buildpackage.txt)
             with nested ${@{..}} syntax ! -->
        <copy todir="//${@{client}}/share">
         <fileset dir="@{client}/somedir"/>
        </copy>
       </echoxml>
    
       <echo>${@{client}}${line.separator}@{client}${line.separator}@{application}${line.separator}@{branding}</echo>
    
       <!-- another nested xmltask for loop over solutions -->
       <xmltask source="structure.xml">
        <call path="//client[name/text()='@{client}']/*/solution">
         <param path="text()" name="solution"/>
          <actions>
           <echo>@{solution}${line.separator}</echo>
          </actions>
         </call>
       </xmltask>
    
       </actions>
      </call>
     </xmltask>
    </project>
    

    If possible you should create buildpackage.txt as valid propertyfile
    with the format => Client[x]=Server[x] right away.
    Then all you need is the xmltask. Put all your steps to be done into the nested action section as demonstrated with echoxml/copy.

    Output :

    <?xml version="1.0" encoding="UTF-8"?>
    <copy todir="//server1/share">
      <fileset dir="client1/somedir" />
    </copy>
         [echo] server1
         [echo] client1
         [echo] application_03
         [echo] brand_01
         [echo] solution01
    <?xml version="1.0" encoding="UTF-8"?>
    <copy todir="//server4/share">
      <fileset dir="client4/somedir" />
    </copy>
         [echo] server4
         [echo] client4
         [echo] application_03
         [echo] brand_09
         [echo] solution01
         [echo] solution07
         [echo] solution08
    BUILD SUCCESSFUL
    

    -- EDIT after question about conditional action, only if ${@{client}} is set --

    if propertyfile buildpackage.txt contains wrong properties, means server=client instead of client=server, f.e. :

    Sol-e-45-D=PRODUCTION1
    CONNECTION=PRODUCTION3
    PRODUCTION4=DFHH
    PRODUCTION5=MEGA-5
    

    you need some if isset logic. The best is to use the new if/unless feature (introduced with ant191) if you're on ant >= 1.9.3, f.e. :

    <!-- you need to activate that feature via namespaces -->
    <project
      xmlns:if="ant:if"
      xmlns:unless="ant:unless"
    >
    
     <!-- Import XMLTask -->
     <taskdef name="xmltask" classname="com.oopsconsultancy.xmltask.ant.XmlTask"/>
    
     <!-- load propertyfile with client=server mappings -->
     <property file="buildpackage.txt"/>
    
     <!-- loop over all clients -->
     <xmltask source="structure.xml">
      <call path="//client">
       <param path="name/text()" name="client"/>
       <param path="branding/text()" name="branding"/>
       <param path="applications/application/text()" name="application"/>
    
       <!-- inside action adress params with @{..} syntax ! -->
       <actions>
        <!-- the copy (or whatever) action
             here demonstrated with echoxml -->
       <echoxml if:set="@{client}">
        <!-- access the corresponding server (set via buildpackage.txt)
             with nested ${@{..}} syntax ! -->
        <copy todir="//${@{client}}/share">
         <fileset dir="@{client}/somedir"/>
        </copy>
       </echoxml>
    
       <echo if:set="@{client}">${@{client}}${line.separator}@{client}${line.separator}@{application}${line.separator}@{branding}</echo>
    
       <!-- another nested xmltask for loop over solutions -->
       <xmltask source="structure.xml">
        <call path="//client[name/text()='@{client}']/*/solution">
         <param path="text()" name="solution"/>
          <actions>
           <echo if:set="@{client}">@{solution}${line.separator}</echo>
          </actions>
         </call>
       </xmltask>
    
       </actions>
      </call>
     </xmltask>
    </project>
    

    Note => the xmltask call has also an if attribute, but the clientname is not known when starting the loop, can't use <call path="//client" if="@{client}">
    output :

    <?xml version="1.0" encoding="UTF-8"?>
    <copy todir="//PRODUCTION1/share">
      <fileset dir="Sol-e-45-D/somedir" />
    </copy>
         [echo] PRODUCTION1
         [echo] Sol-e-45-D
         [echo] Sol-e-45-D
         [echo] Branding-BLUE
         [echo] Sol-e-45-D
    <?xml version="1.0" encoding="UTF-8"?>
    <copy todir="//PRODUCTION3/share">
      <fileset dir="CONNECTION/somedir" />
    </copy>
         [echo] PRODUCTION3
         [echo] CONNECTION
         [echo] Sol-e-45-D
         [echo] Branding-BLUE
         [echo] Sol-e-45-D
    

    Otherwise when bound to ant <= 1.9.3 you may use antcontrib if task, as you're already have that taskdef in your snippet :

    <if>
     <isset property="@{client}"/>
      <then>
       <!-- ... -->
      </then>
    </if> 
    

    Beware, your antcontrib taskdef definition is wrong !!, the antcontrib.properties contains only taskdefs for ant < 1.6, declare it like that (assuming antcontrib.jar already on path) :

      <project xmlns:ac="antlib:net.sf.antcontrib">
    
        <ac:if>
        ...
    

    or without namespaces :

      <project>
    
      <!-- Import AntContrib -->
      <taskdef resource="net/sf/antcontrib/antlib.xml"/>
      ...