Search code examples
python-2.7avroavsc

Python - How to convert .txt/.csv file holding table schema to .avsc file


Need to extract table schema (using describe\list columns) into .txt or .csv file and later want to convert those files into .avsc(avro schema) file using python.

sample.txt:

COLUMN_NAME |TYPE_NAME|DEC&|NUM&|COLUM&|COLUMN_DEF|CHAR_OCTE&|IS_NULL&

AIRLINE |CHAR |NULL|NULL|2 |NULL |4 |NO AIRLINE_FULL |VARCHAR |NULL|NULL|24 |NULL |48 |YES

Need to convert sample.txt into sample.avsc


Solution

    1. Read CSV into python dict
    2. Get header part to other dict(header_dict)
    3. create empty dict(final_dict) and append type,namespace reletad thing and pass rows from header_dict to this final_dict.
    4. Dump final_dict to file, which will be your avsc