I am trying to make a subquery in a DELETE
statement in order to improve performance. The plain DELETE
statement works but the subquery ones are either deleting all rows indiscriminately, or only one row per invocation. I am confused as to why the statements are not equivalent.
Jump down to "What doesn't work" to see the problem statements.
I am using sqlite3
from python2
to manage a database of pictures with associated tags.
The schema for the table is:
CREATE VIRTUAL TABLE "files" USING fts3(fname TEXT, orig_name TEXT, tags TEXT, md5sum TEXT);
Tags are organized as a comma separated list so direct string comparison in sqlite
doesn't (easily) work, so I've add a helper function TAGMATCH
def tag_match(tags, m):
i = int(m in [i.strip() for i in tags.split(',')])
return i
db.create_function('TAGMATCH', 2, tag_match)
This does what I want/expect. It deletes all rows where the column tags
contains the tag 'DELETE'
. Downside is, as far as I understand, this requires a linear scan of the table. Because of the "dangers" of deleting something from the table I did want to use MATCH
in case, in some hypothetical situation, a match occurs with another unintended tag ie. 'DO NOT DELETE THIS'
.
DELETE FROM files WHERE TAGMATCH(tags, 'DELETE')
To speed things up I tried a trick I read in another stackoverflow post where MATCH
is used to narrow down the search, then a direct string comparison is done on those results ie
SELECT * FROM (SELECT * FROM table WHERE words MATCH keyword) WHERE words = keyword
I tried using this trick here, but instead it deletes every row in the table.
DELETE FROM files WHERE TAGMATCH((
SELECT tags FROM files WHERE tags MATCH 'DELETE'), 'DELETE')
This is what I first came up with. I realize now it's not a particularly good solution, but since its effect puzzles me I'm including it. This statement only deletes one row containing the tag 'DELETE'
. If invoked again, it deletes another row, and so on until all the rows with 'DELETE'
are removed:
DELETE FROM files WHERE rowid = (
SELECT rowid FROM (
SELECT rowid, tags FROM files WHERE tags MATCH 'DELETE')
WHERE TAGMATCH(tags, 'DELETE'))
Following query deletes everything because the WHERE
clause evaluates to a number which if by itself evaluates to TRUE
:
DELETE FROM files WHERE TAGMATCH((
SELECT tags FROM files WHERE tags MATCH 'DELETE'), 'DELETE')
Equivalent to
DELETE FROM files WHERE 1 -- or whatever ##
Instead, consider using EXISTS
with subquery that correlates to main query:
DELETE FROM files WHERE EXISTS
(SELECT 1
FROM (SELECT rowid, tags FROM files WHERE tags MATCH 'DELETE') sub
WHERE TAGMATCH(sub.tags, 'DELETE') AND sub.rowid = files.rowid)
Alternatively, using your attempt, turn =
into an IN
as former only uses the first record found.
DELETE FROM files WHERE rowid IN
(SELECT rowid
FROM (SELECT rowid, tags FROM files WHERE tags MATCH 'DELETE') sub
WHERE TAGMATCH(sub.tags, 'DELETE'))