ftpeclipse-rcpcobolapache-commons-netjrecord

JRecord - Formatting file transferred from Mainframe


I am trying to display a mainframe file in a eclipse RCP application using JRecord library.I already have the COBOL copybook as a text file. to accomplish that,

  1. I am transferring the file from mainframe to my desktop through apache commons net FTPClient API
  2. Now I have a text file
  3. I am removing the newline and carriage return characters
  4. then I read it via ., a CobolIoProvider and convert it into a ArrayList of type AbstractLine

But I have offset issues because of some special charcters . here are the issues

  1. when I dont perform step #3 , there are offset issues right from record 1. hence I included step #3
  2. even when I perform step #3 , the first few thounsands of records seem to be formatted(or read ) by the AbstractLineReader correctly unless it encounters a special character (not sure but thats my assumption).

Code snippet:

ArrayList<AbstractLine> lines = new ArrayList<AbstractLine>();
        InputStream copyStream;
        InputStream fis;
        try {
            copyStream = new FileInputStream(new File(copybookfile));

            String filec = FileUtils.readFileToString(new File(datafile));
            System.out.println("initial len: "+filec.length());
            filec=filec.replaceAll("\r", "");
            filec=filec.replaceAll("\n", "");
            System.out.println("initial len: "+filec.length());

            fis= new ByteArrayInputStream(filec.getBytes());

            CobolIoProvider ioProvider = CobolIoProvider.getInstance();
            AbstractLineReader reader = ioProvider.newIOBuilder(copyStream, "REQUEST",
                    Convert.FMT_MAINFRAME).newReader(fis);
            AbstractLine line;
            while ((line = reader.read()) != null) {
                lines.add(line);
            }
        } catch (FileNotFoundException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }

What am I missing here ? is there an additional preprocessing that I need to do for the file transferred from mainframe ?


Solution

  • If it is a Text File (no binary data) with \r\n line delimiters try:

        ArrayList<AbstractLine> lines = new ArrayList<AbstractLine>();
        InputStream copyStream;
        InputStream fis;
        try {
            copyStream = new FileInputStream(new File(copybookfile));
    
            AbstractLineReader reader = CobolIoProvider.getInstance() 
                .newIOBuilder(copyStream, "REQUEST", ICopybookDialects.FMT_MAINFRAME)
                    .setFileOrganization(Constants.IO_STANDARD_TEXT_FILE)
                .newReader(datafile);
            AbstractLine line;
            while ((line = reader.read()) != null) {
                lines.add(line);
            }
        } catch (FileNotFoundException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }
    

    Note: The setFileOrganization tells JRecord what type of file it is. So .setFileOrganization(Constants.IO_STANDARD_TEXT_FILE) tells JRecord it is a Text file with \n or \r\n end-of-line markers. Here is a Description of FileOrganisation in JRecord.

    The special charcters worry me though, if there is a \n in the 'Data' it will be treated as an end-of-line. You may need to do binary transfer and keep the RDW (Record-Descriptor-Word) if it is a VB file.

    If The file contains Binary data, you will need:

    • do a binary transfer (with RDW if it is a VB file)
    • use the appropriate File-Organisation
    • Specify Ebcdic (.setFont("cp037") tells JRecord is US-Ebcdic)

    I will add a second answer for Generating Code using the RecordEditor


    If you are absolutely sure all the records are the same length you can use the low-level routines to do the reading see the ReadAqtrans.java program in https://sourceforge.net/p/jrecord/discussion/678634/thread/4b00fed4/

    basically you would do:

        ICobolIOBuilder iobuilder = CobolIoProvider.getInstance()
                .newIOBuilder("copybookFileName", ICopybookDialects.FMT_MAINFRAME)
                        .setFont("CP037")
                        .setFileOrganization(Constants.IO_FIXED_LENGTH);
        LayoutDetail layout = iobuilder.getLayout();
        FixedLengthByteReader br 
             = new FixedLengthByteReader(layout.getMaximumRecordLength() + 2);
        br.open("...");
    
        byte[] bytes;
        while ((bytes = br.read()) != null) {           
            lines.add(iobuilder.newLine(bytes));
        }