Generating list of modules from all .py files and installing without specifying module version

Question

I have hundreds of python files and I need to install all the existing modules, I thought I'd create a file with all the module names that are imported in the files:

import os

def remove_duplicates():
    lines = open('all_modules.txt', 'r').readlines()
    lines_set = set(lines)
    out  = open('all_modules.txt', 'w')
    for line in lines_set:
        out.write(line)

def main():
    fileDir = r"C:\Users\Computador\Desktop\tests vscode"
    fileExt = r".py"
    py_file_list = [_ for _ in os.listdir(fileDir) if _.endswith(fileExt)]

    with open('all_modules.txt', 'w+', newline='', encoding='UTF-8') as f:

        for py_file in py_file_list:
            with open(py_file) as stacktest:
                stacktest_list = [line.strip() for line in stacktest.readlines()]

            for line in stacktest_list:
                if 'import ' in line and "'" not in line:
                    after_import = line.split('import ',1)[1]
                    before_space = after_import.split(' ')[0]
                    before_comma = before_space.split(',')[0]
                    f.write(before_comma + '\n')

    remove_duplicates()

if __name__ == '__main__':
    main()

Example result in all_modules.txt:

pandas
time
dataframe_image
webbrowser
num2words
glob
Pool
schedule
move
csv
os
feedparser

But to install with pip install using requirements.txt it is necessary to have the version of the module I want to install, in this case, the computer is completely clean and I want the latest version of all modules.

Is there a way to install these modules at once without having to write one by one manually?

Answer 1

But to install with pip install using requirements.txt it is necessary to have the version of the module I want to install

That's not true. You can take the file you have generated and run pip install -r all_modules.txt

Pip will go through each name in the file and attempt to install the latest version available (that might not be the same version you used when writing the code originally - things might not work, but it will at least install the package).

The other problems you will likely encounter are:

1. the names in your file are extracted from import statements, but the import name of a package does not necessarily match the name of the package in PyPI that you need to pip install
2. you will find some imported packages that do not need to be pip installed, such as packages from Python stdlib, or from other modules in your own project... these will give an error when you try to pip install them - or worse you may install a different package of the same name that exists in PyPI by coincidence