I have a query like the following:
SELECT
table1.field1,
table1.field2,
table3.field1,
table3.field2,
table3.field3,
table3.field4
FROM table1
INNER JOIN table2
ON table1.field1 = table2.field1
INNER JOIN table3
ON table2.field2 = table3.field5
WHERE table1.field1 = ANY(@pArray);
All of these comparisons are with string/text fields. There are indexes on all of the foreign key fields. This is a poorly designed database that I have mostly just read-only access to.
What I'm wondering is why the above parameter causes this query to take 10+ seconds, whereas if I pass the array manually as an array literal (using string concatenation), then it takes about 40-50 milliseconds.
My array contains 195 8-character numeric string elements.
My only guess is that it has something to do with sanitizing the parameter values, but making the query take hundreds of times longer seems a bit excessive. What I've done is add my own check to make sure that all of the strings are 100% numeric, which I think should avoid SQL injection attacks.
My NpgSql code looks like this:
void MyQuery(
string[] pArray
) {
var sql = @"
// as above
";
using var connection = new NpgsqlConnection(mConnectionString);
connection.Open();
using var command = new NpgsqlCommand(sql, connection);
command.Parameters.AddWithValue("pArray", pArray);
// takes ~12 seconds with parameter, or ~50ms with array literal
using var reader = command.ExecuteReader();
When removing the parameter, the WHERE
clause ends up looking more like this, after concating:
WHERE table1.field1 = ANY('{12345678,12345679,12345680,12345681}');
Is the slowness due only to sanitization, or a bug in NpgSql, or something else?
The problem as indicated in the comments was adding a parameter without specifying the type. Changing the parameter line to the following solved the problem:
command.Parameters.AddWithValue("pArray", NpgsqlDbType.Array | NpgsqlDbType.Char, pArray);
Notice that to specify an array type, you OR the Array
value with another value, in this case Char
. The actual PostgreSQL type in the database is something like character varying(15)
.