Mixed record formats in a flat file with mule

Question

I have managed to confuse myself once more in mule.

I must accept a flat file from a vendor and cannot get the file reformatted. The file has a large number of records with fixed-width fields, perfect for the mule data-mapper. This works fine, mapping to a java POJO to create a collection of records to process.

The issue: The last record in the file is a summary record of a completely different format. The problem is, the format not only changes the data layout, the record length also changes. This exceptions the data-mapper as the record is too short for the format definition.

Are there any ideas on how to allow for this type of mixed formatting within a file, or instead of standard mule features do I need to make custom transformers to split the records and react to record lengths to separate formats?

Yes, I am aware this type of use of flat files is outdated, but dealing with legacy systems requires accepting the data as produced until the system can be retired.

Answer 1

I remain open to other solutions, but I doubt that the out of box data-mapper can achieve this. A bit too esoteric for a general solution I would think.

Pending hearing a brilliant work around, my current approach is to use a java component to split the file by record and extract the control value and pass through a choice control to separate the records by type. Java components are then called to map the fields and perform the business logic in each branch for the special record and the body records separately such as:

<file:connector name="File" autoDelete="true" streaming="true" 
         validateConnections="true" doc:name="File"/>
<spring:beans>  
    <spring:bean name="sumRec" class="com.mypackage.SummaryRecord" />                
    <spring:bean name="detailRec" class="com.mypackage.DetailRecord" />
</spring:beans>

<flow name="FlatToFile">
    <file:inbound-endpoint path="/SourceDir/Work"
        moveToDirectory="/SourceDir/Archive" 
        doc:name="FlatFileIn" connector-ref="File" >
        <file:filename-wildcard-filter pattern="*.txt" caseSensitive="false" />
    </file:inbound-endpoint>

    <component class="com.mypackage.FlatFileReader" doc:name="Split file and dispatch to VM" />

</flow>

<flow name="processRecords">
    <vm:inbound-endpoint exchange-pattern="one-way" path="in" doc:name="VM" />

    <object-to-string-transformer doc:name="Object to String"/>
    <set-variable variableName="recType" value="#message.payload.substring(0,7)]" doc:name="Variable"/>

    <choice doc:name="Choice">
        <when expression="flowVars.recType == '9999999'" >
            <expression-component doc:name="Expression">
                message.outboundProperties.recCount =     app.registry.sumRec.process(message.payload);
            </expression-component>
        </when>
        <otherwise>
            <expression-component doc:name="Expression">
                app.registry.detailRec.process(message.payload);
            </expression-component>
        </otherwise>
    </choice>
</flow>
</mule>

The VM endpoint is called from Java by the FlatFileReader.