Search code examples
pythonargparse

How to process a file like each line is a command with argparse or similar


I am creating a program that reads a list of files from some text file and then does various things with them. I want to be able to read each line and then process it like it was typed in in a CLI. For example:

~/test1.txt -r
~/test2.txt -w "hello"

Where the flag -r saves the text in the file to some object and -w appends whatever string argument follows that flag to the file. In this case would append "hello" to test2.txt.

I know of argparse, and it does exactly what I want, but it only reads and processes commands from the command line. Is there a way to parse strings that I already have in my program like I described above?

If it cant be done with argparse, are there any other decent solutions other then coding it myself?


Solution

  • My guess is your want to read from a text file and treat each line as command line arguments. Here is what get you started.

    import argparse
    import pathlib
    import shlex
    
    # Create a parser
    parser = argparse.ArgumentParser()
    parser.add_argument("-r", action="store_true")
    parser.add_argument("-w")
    parser.add_argument("filename")
    
    # Read the lines from the text file
    data_path = pathlib.Path(__file__).with_name("input.txt")
    with open(data_path, "r", encoding="utf-8") as stream:
        for line in stream:
            print(f"\n# {line.strip()}")
            tokens = shlex.split(line)
            options = parser.parse_args(tokens)
            print(options)
    
            # Do something with options
    

    Output:

    
    # ~/test1.txt -r
    Namespace(r=True, w=None, filename='~/test1.txt')
    
    # ~/test2.txt -w "hello"
    Namespace(r=False, w='hello', filename='~/test2.txt')
    

    Notes

    Normally, we call parse.parse_args(), which parses the command-line arguments.

    In order to parse a text line, we need to split that line and pass the list of tokens into parse_args.

    Do not use line.split(): it does not understand quotes and escaped characters. Instead, we use shlex.split() which take care of those details.

    Update

    Thank you to Charles Duffy for pointing out the ~ expansion. That can be easily fixed with the following changes:

    import os
    ...
    
    parser.add_argument("filename", type=os.path.expanduser)