Search code examples
perlbashtemplatestext-processingtemplate-toolkit

BASH: importing data from flat file into template


I have a flat file of records, each 33 lines long. I need to format this file to specs in a template. The template is in DOS format while the source file is in NIX format. The template has specific indenting and spacing which must be adhered to. I've thought of a few options:

  • BASH with classic nix tools: sed, awk, grep etc...
  • BASH with template toolkit
  • Perl eith template toolkit
  • Perl

These are in order of my familiarity. Here's a sample source record ( NIX format ): I've reduced the number of newlines to save space ( normally 33 lines ):

JACKSON HOLE SANITARIUM AND REPTILE ZOO
45 GREASY HOLLER LN
JACKSON HOLE, AK   99999


Change Service Requested


BUBBA HOTEP
3 DELIVERANCE RD
MINNEAPOLIS, MN   99998


BUBBA HOTEP 09090909090909

You have a hold available for pickup as of 2012-01-04:

Title: Banjo for Fun and Profit
Author: Williams, Billy Dee
Price: $10 

Here's the template ( DOS format -- lines reduced - 66 lines normally):

     <%BRANCH-NAME%>
     <%BRANCH-ADDR%>
     <%BRANCH-CTY%>


<%CUST-NAME%> <%BARCODE%>
You have a hold available for pickup as of <%DATE%>:

Title: <%TITLE%>
Author: <%AUTHOR%>
Price: <%PRICE%>


             <%CUST-NAME%>
             <%CUST-ADDR%>
             <%CUST-CTY%>

end of file

It actually does say "end of file" at the end of each record.

Thoughts? I tend to over-complicate things.

UPDATE2

Figured it out.

My answer is below. Feel free to suggest improvements.


Solution

  • This is what I am using for this project. Feel free to suggest improvements, or, submit better solutions.

    cp $FILE $WORKING # we won't mess with original
    
    NUM_RECORDS=$( grep "^Price:" "$FILE" | wc -l ) # need to know how many records we have 
                                                  # counting occurences of end of record r
    
    TMP=record.txt # holds single record, used as temp storage in loop below
    
    # Sanity
    # Make sure temp storage exists. If not create -- if so, clear it.
    [ ! -f $TMP ] && touch $TMP || cat /dev/null >$TMP
    
    # functions
    function make_template () {
        local _file="$1"
        mapfile -t filecontent < "$_file"
        _loc_name="${filecontent[0]}"
        _loc_strt="${filecontent[1]}"
        _loc_city="${filecontent[2]}"
        _pat_name="${filecontent[14]}"
        _pat_addr="${filecontent[15]}"
        _pat_city="${filecontent[16]}"
        _barcode=${filecontent[27]:(-14)} # pull barcode from end of string
        _date=${filecontent[29]:(-11)}    # pull date from end of string
        # Test title length - truncate if necessary - 70 chars.
        _title=$(grep -E "^Title:" $_file)
        MAXLEN=70
        [ "${#_title}" -gt "$MAXLEN" ] && _title="${filecontent[31]:0:70}" || :
        _auth=$(grep -E "^Author:" $_file)
        _price=$(grep -E "^Price:" $_file)
        sed "
            s@<%BRANCH-NAME%>@${_loc_name}@g
            s@<%BRANCH-ADDR%>@${_loc_strt}@g
            s@<%BRANCH-CTY%>@${_loc_city}@g
            s@<%CUST-NAME%>@${_pat_name}@g
            s@<%CUST-ADDR%>@${_pat_addr}@
            s@<%CUST-CTY%>@${_pat_city}@
            s@<%BARCODE%>@${_barcode}@g
            s@<%DATE%>@${_date}@
            s@<%TITLE%>@${_title}@
            s@<%AUTHOR%>@${_auth}@
            s@<%PRICE%>@${_price}@" "$TEMPLATE"
    }
    
    ####################################
    #  MAIN
    ####################################
    
    for((i=1;i<="$NUM_RECORDS";i++))
    do
        sed -n '1,/^Price:/{p;}' "$WORKING" >"$TMP"  # copy first record with end of record
                                                    # and copy to temp storage.
    
        sed -i '1,/^Price:/d' "$WORKING"             # delete first record using EOR regex.
        make_template "$TMP"                        # send temp file/record to template fu
    done
    
    # cleanup
    exit 0