Search code examples
c#postgresqlnpgsql

Why is it so much faster passing an array as a literal rather than a parameter?


I have a query like the following:

SELECT
   table1.field1,
   table1.field2,
   table3.field1,
   table3.field2,
   table3.field3,
   table3.field4

FROM table1

INNER JOIN table2
ON table1.field1 = table2.field1

INNER JOIN table3
ON table2.field2 = table3.field5

WHERE table1.field1 = ANY(@pArray);

All of these comparisons are with string/text fields. There are indexes on all of the foreign key fields. This is a poorly designed database that I have mostly just read-only access to.

What I'm wondering is why the above parameter causes this query to take 10+ seconds, whereas if I pass the array manually as an array literal (using string concatenation), then it takes about 40-50 milliseconds.

My array contains 195 8-character numeric string elements.

My only guess is that it has something to do with sanitizing the parameter values, but making the query take hundreds of times longer seems a bit excessive. What I've done is add my own check to make sure that all of the strings are 100% numeric, which I think should avoid SQL injection attacks.

My NpgSql code looks like this:

void MyQuery(
   string[] pArray
) {
   var sql = @"
      // as above
   ";

   using var connection = new NpgsqlConnection(mConnectionString);
   connection.Open();

   using var command = new NpgsqlCommand(sql, connection);
   command.Parameters.AddWithValue("pArray", pArray);

   // takes ~12 seconds with parameter, or ~50ms with array literal
   using var reader = command.ExecuteReader();

When removing the parameter, the WHERE clause ends up looking more like this, after concating:

WHERE table1.field1 = ANY('{12345678,12345679,12345680,12345681}');

Is the slowness due only to sanitization, or a bug in NpgSql, or something else?


Solution

  • The problem as indicated in the comments was adding a parameter without specifying the type. Changing the parameter line to the following solved the problem:

    command.Parameters.AddWithValue("pArray", NpgsqlDbType.Array | NpgsqlDbType.Char, pArray);
    

    Notice that to specify an array type, you OR the Array value with another value, in this case Char. The actual PostgreSQL type in the database is something like character varying(15).