I am trying to transfer files using IBM Websphere MQ FTE with parameter -bs (matches per task). I thought that using this parameter will split total files trasnfered into several tasks, according to its value. E.g. I used "-bs 10" and tried to transfer 16 files. The result should be two separate transfers, one with 10 files, second with 6 files. Am I correct? But for me the result is all 16 files in one transfer/task :/.
As stated in IBM documentation: By setting the batch size of a resource monitor (say, to 100) the resource monitor would create a new transfer for every 100 files it matched. So with 1000 .txt files you'd only get 10 new transfers being started.
I used these commands to create transfer.
fteCreateTransfer -sa AGENT1 -sm TQM.FTE -da AGENT2 -dm TQM.FTE -dd /var/MQFTE/test -de overwrite -sd delete -gt /var/IBM/WMQFTE/config/TQM.FTE/FTE_TEST.xml c:\\workspace\\FTE_tests\\IN\\*.xml
fteCreateMonitor -ma AGENT1 -mn FTE_TEST_MONITOR -md c:\\workspace\\FTE_tests -mt /var/IBM/WMQFTE/config/TQM.FTE/FTE_TEST.xml -tr match,test.run -pi 30 -pu seconds -f -c -bs 10.
Any suggestions? Thanks!
Your understanding is correct but the problem you have is because in your transfer template definition XML you are transferring source files "*.xml" which is a wildcard pattern and you are matching only on a single trigger file, "test.run".
It sounds like you want to transfer the files you match on as you don't know these in advance.
Your WebSphere MQ FTE directory resource monitor will be triggering x number of times on the poll where x is the number of files you have in the directory match your trigger pattern. These will then be batched into a number of transfer x/batchSize.
As you are only matching on one file, "test.run" the resource monitor triggers once. When the WebSphere MQ FTE agent process the single managed transfer request, you have told it to look for any files ending in .XML so it transfers all of them.
Take a look in the Information Centre for "Customizing Tasks with Variable Substitution" and use that functionality in your transfer template XML file.
Basically, you could, in your first command (fteCreateTransfer) change "c:\workspace\FTE_tests\IN\*.xml" to "c:\workspace\FTE_tests\IN\${fileName}" and then change the second command (fteCreateMonitor) to monitor in the same directory "c:\workspace\FTE_tests\IN" and match on "*.xml".
When the resource monitor triggers, it will replace ${fileName} with the name of one file for which it has triggered one and will repeat this x number of times, as it will trigger on every file ending in .xml. If there are 16 files, it will do this 16 times.
Based on the information you've given I think it will give you the behaviour you want.
Any further questions or clarifications you need, feel free to ask.
Hi Miroslav, I don't seem able to comment so I'll add a bit here to address your comment.
I'm glad to hear the changes suggested have given you the behaviour you were expecting. Indeed not knowing the exact name of the file to transfer is common but by using wildcards and certain naming conventions you can hopefully configure WebSphere MQ FTE resource monitor(s) to give the behaviour you want.
When using a trigger file of a certain name e.g., "test.run" the resource monitor is only looking for that file so will know nothing about any other files ending in .xml, for example. Hence why the batching only works on files that the resource monitor is triggering/matching on. Remember that the -bs option on the fteCreateMonitor command is for "matches per task" as defined in the Infocenter page for the command. So in your second case, the behaviour you see is expected.
I hope this helps!