Search code examples
node.jsstreampg-promise

Stream "RETURNING" from insert for pg-promise


The goal here is to stream the RETURNING data from INSERT INTO .... to a write stream.

ie.

                const write_stream = fs.createWriteStream('file.csv')
                client.query(`
                    INSERT INTO table1  (columns) 
                        SELECT ...
                        FROM table2
                        RETURNING *
                    `)
                    .then(returned => {
                        returned.rows.pipe(write_stream)                      
                    })

Is this possible with pg-promise?

At the moment the response from returned.rows contains all the data in an array[{}] thus above example does not work, so the only solution might need to be an insert and a select separately, but would be great if I could simply get the returned data.


Solution

  • Publishing the right answer, as the question author chose not to follow the example provided.

    Here's how to stream query results into a csv file properly (from the author of pg-promise):

    import QueryStream from 'pg-query-stream';
    import CsvWriter from 'csv-write-stream';
    import {createWriteStream} from 'fs';
    
    const csv = new CsvWriter();
    const file = createWriteStream('out.csv');
    
    const qs = new QueryStream('select * from my_table');
    
    await db.stream(qs, s => {
        s.pipe(csv).pipe(file);
    });
    //=> resolves with: {processed, duration}
    

    See stream method.