Search code examples
pythonpipelinewindows-shell

How to pipe multiple sql- and py-scripts


I have the following code copied from github gtfs_SQL_importer:

cat gtfs_tables.sql \
  <(python import_gtfs_to_sql.py path/to/gtfs/data/directory) \
  gtfs_tables_makeindexes.sql \
  vacuumer.sql \
  | psql mydbname

I tried to run this on windows and replaced the call to the UNIX-command cat by the windows equivalent type which should work similar as of is-there-replacement-for-cat-on-windows.

However when I execute that code I get some error:

The syntax for the filename, directory or filesystem is wrong.

So I tried to limit the number of piped files to only combine the call to python and the call to psql:

type <(C:/python27/python path/to/py-script.py path/to/file-argument) | psql -U myUser -d myDatabase

which results in the same error.

However when I execute the python-script alone it works as expected:

C:/python27/python path/to/py-script.py path/to/file-argument

So I assume the error results from using type in order to pipe the result of the script directly to psql.

Does anyone know the correct syntax?

EDIT: To ensure the problem is not related to a file not being found I used absolute paths for all arguments within my command except the type and the psql-command (which are both handled via the %PATH%-variable).


Solution

  • Comment: So I can´t combine the sql-files and the output from my pythonscript together and pipe them to psql?

    Another approach, create your own cat with Python, or add the first three line of code to import_gtfs_to_sql.py, for instance:

    # Usage
    python import_gtfs_to_sql.py... | python myCat.py gtfs_tables.sql | psql mydbname
    
    #myCat.py
    import sys
    with open(sys.argv[1]) as fh:
        sys.stdout.write(fh.read())
    
    sys.stdout.write(sys.stdin.read())
    

    Comment: I already know the error comes from type<(python...)

    TYPE command does not accept stdin.

    Therefore your only solution is Option 2.

    Another approach is to use your Python Script to do print gtfs_tables.sql.

    Question: the syntax for the filename, directory or filesystem is wrong.

    1. Find out from which part the above Error comes from.

       a) type gtfs_tables.sql
       b) type <(python ...   
       c) type gtfs_tables.sql <(python ...  
       d) type gtfs_tables.sql | psql mydbname
       e) type <(python ... | psql mydbname  
      
    2. Save the Output of <(python ... to a File and try

       python ... > tmp_python_output
       type gtfs_tables.sql tmp_python_output | psql mydbname