I am trying to do some big bulk inserts to Postgres via node-postgres When the bindings array exceeds 65536 values then passes to postgres the rest of values and when the query it runs I take the error
[error: bind message supplies 4 parameters, but prepared statement "" requires 65540]
Any thoughts? Thanks in advance.
Prepared Statements within node-postgres are not suitable for bulk inserts, because they do not support multi-query statements. And you shouldn't stretch the array of variables across all inserts at the same time, this won't scale well, it has its own limits, like the one you hit there.
Instead, you should use multi-value inserts, in the format of:
INSERT INTO table(col1, col2, col3) VALUES
(val-1-1, val-1-2, val-1-3),
(val-2-1, val-2-2, val-2-3),
...etc
split your bulk inserts in queries like this, with up to 1,000 - 10,000 records, depending on the size of each record, and execute as a simple query.
See also Performance Boost article, to understand INSERT scalability better.