Search code examples
pythonflat-file

fixed length flat file without separators


I have a comprehension question not related to any particular language, but since I am writing in python, I tagged python. I am asked to provide some data in "fixed length, flatfile without separators". It confuses me, since I understand it like:

Input: Column A: date (len6)

Input: Column B: name (len20)

Output: "20170409MYVERYSHORTNAME[space][space][space][space][space]"

"MYVERYSHORTNAME" is only 15 char long, but since it's fixed 20-length output, I am supposed to fill 5 times it with something ? It's not specified.

Why do someone even needs a file without separators? He/she will need to break it down to separated fields anyway, what's the point?


Solution

  • This kind of flat (binary) file is meant to be faster/easier to read by machines, and more memory efficient than the equivalent in a more human friendly representation (eg, JSON, CSV, etc.). For example, the machine can preallocate the appropriate amount of memory before reading the contents. Nowadays, with the virtually unlimited quantity of RAM and dynamic nature of the languages, nobody uses flat files anymore (unless it is specifically needed).

    In Python, in order to deal properly with this kind of binary files, you can for example use the struct module from the standard library: https://docs.python.org/3.6/library/struct.html#module-struct

    Example:

    import struct
    from datetime import datetime
    mydate = datetime.now()
    myshortname = "HelloWorld!"
    
    struct.pack("8s20s", mydate.strftime('%Y%m%d').encode(), myshortname.encode())
    
    >>> b'201709HelloWorld!\x00\x00\x00\x00\x00\x00\x00\x00\x00'