I have this select query, ItemType is varchar type and ItemComments is int type:
select * from ItemInfo where ItemType="item_type" order by ItemComments desc limit 1
You can see this query has 3 conditions:
The interesting thing is, when I select rows with all three conditions, it's getting very slow. But if I drop any one of the three (except condition 2), the query runs quite fast. See:
select * from ItemInfo where ItemType="item_type" order by ItemComments desc limit 1;
/* Affected rows: 0 Found rows: 1 Warnings: 0 Duration for 1 query: 16.318 sec. */
select * from ItemInfo where ItemType="item_type" order by ItemComments limit 1;
/* Affected rows: 0 Found rows: 1 Warnings: 0 Duration for 1 query: 0.140 sec. */
select * from ItemInfo order by ItemComments desc limit 1;
/* Affected rows: 0 Found rows: 1 Warnings: 0 Duration for 1 query: 0.015 sec. */
Plus,
I have searched many possible explanation like MySQL support for descending index, composite index and so on. But these still can't explain why query #1 runs slowly while query #2 and #3 runs well.
It would be very appreciated if anyone could help me out.
Updates:create table and explain info
Create code:
CREATE TABLE `ItemInfo` (
`ItemID` VARCHAR(255) NOT NULL,
`ItemType` VARCHAR(255) NOT NULL,
`ItemPics` VARCHAR(255) NULL DEFAULT '0',
`ItemName` VARCHAR(255) NULL DEFAULT '0',
`ItemComments` INT(50) NULL DEFAULT '0',
`ItemScore` DECIMAL(10,1) NULL DEFAULT '0.0',
`ItemPrice` DECIMAL(20,2) NULL DEFAULT '0.00',
`ItemDate` DATETIME NULL DEFAULT '1971-01-01 00:00:00',
PRIMARY KEY (`ItemID`, `ItemType`),
INDEX `ItemDate` (`ItemDate`),
INDEX `ItemComments` (`ItemComments`),
INDEX `ItemType` (`ItemType`)
)
COLLATE='utf8_general_ci'
ENGINE=InnoDB;
Explain result:
mysql> explain select * from ItemInfo where ItemType="item_type" order by ItemComments desc limit 1;
+----+-------------+-------+------------+-------+---------------+--------------+---------+------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+-------+---------------+--------------+---------+------+------+----------+-------------+
| 1 | SIMPLE | i | NULL | index | ItemType | ItemComments | 5 | NULL | 83 | 1.20 | Using where |
+----+-------------+-------+------------+-------+---------------+--------------+---------+------+------+----------+-------------+
mysql> explain select * from ItemInfo where ItemType="item_type" order by ItemComments limit 1;
+----+-------------+-------+------------+-------+---------------+--------------+---------+------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+-------+---------------+--------------+---------+------+------+----------+-------------+
| 1 | SIMPLE | i | NULL | index | ItemType | ItemComments | 5 | NULL | 83 | 1.20 | Using where |
+----+-------------+-------+------------+-------+---------------+--------------+---------+------+------+----------+-------------+
mysql> explain select * from ItemInfo order by ItemComments desc limit 1;
+----+-------------+-------+------------+-------+---------------+--------------+---------+------+------+----------+-------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+-------+---------------+--------------+---------+------+------+----------+-------+
| 1 | SIMPLE | i | NULL | index | NULL | ItemComments | 5 | NULL | 1 | 100.00 | NULL |
+----+-------------+-------+------------+-------+---------------+--------------+---------+------+------+----------+-------+
Query from O. Jones:
mysql> explain
-> SELECT a.*
-> FROM ItemInfo a
-> JOIN (
-> SELECT MAX(ItemComments) ItemComments, ItemType
-> FROM ItemInfo
-> GROUP BY ItemType
-> ) maxcomm ON a.ItemType = maxcomm.ItemType
-> AND a.ItemComments = maxcomm.ItemComments
-> WHERE a.ItemType = 'item_type';
+----+-------------+------------+------------+-------+----------------------------------------+-------------+---------+---------------------------+---------+----------+--------------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+------------+------------+-------+----------------------------------------+-------------+---------+---------------------------+---------+----------+--------------------------+
| 1 | PRIMARY | a | NULL | ref | ItemComments,ItemType | ItemType | 767 | const | 27378 | 100.00 | Using where |
| 1 | PRIMARY | <derived2> | NULL | ref | <auto_key0> | <auto_key0> | 772 | mydb.a.ItemComments,const | 10 | 100.00 | Using where; Using index |
| 2 | DERIVED | ItemInfo | NULL | index | PRIMARY,ItemDate,ItemComments,ItemType | ItemType | 767 | NULL | 2289466 | 100.00 | NULL |
+----+-------------+------------+------------+-------+----------------------------------------+-------------+---------+---------------------------+---------+----------+--------------------------+
I'm not sure if I execute this query right but I couldn't get the records within quite a long time.
Query from Vijay. But I add ItemType join condition cause with only max_comnt return items from other ItemType:
SELECT ifo.* FROM ItemInfo ifo
JOIN (SELECT ItemType, MAX(ItemComments) AS max_comnt FROM ItemInfo WHERE ItemType="item_type") inn_ifo
ON ifo.ItemComments = inn_ifo.max_comnt and ifo.ItemType = inn_ifo.ItemType
/* Affected rows: 0 Found rows: 1 Warnings: 0 Duration for 1 query: 7.441 sec. */
explain result:
+----+-------------+------------+------------+-------------+-----------------------+-----------------------+---------+-------+-------+----------+-----------------------------------------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+------------+------------+-------------+-----------------------+-----------------------+---------+-------+-------+----------+-----------------------------------------------------+
| 1 | PRIMARY | <derived2> | NULL | system | NULL | NULL | NULL | NULL | 1 | 100.00 | NULL |
| 1 | PRIMARY | ifo | NULL | index_merge | ItemComments,ItemType | ItemComments,ItemType | 5,767 | NULL | 88 | 100.00 | Using intersect(ItemComments,ItemType); Using where |
| 2 | DERIVED | ItemInfo | NULL | ref | ItemType | ItemType | 767 | const | 27378 | 100.00 | NULL |
+----+-------------+------------+------------+-------------+-----------------------+-----------------------+---------+-------+-------+----------+-----------------------------------------------------+
And I want to explain why I use order with limit at the first place: I was planning to fetch record from the table randomly with a specific probability. The random index generated from python and send to MySQL as a variable. But then I found it cost so much time so I decided to just use the first record I got.
After inspiring by O. Jones and Vijay, I tried using max function, but it doesn't perform well:
select max(ItemComments) from ItemInfo where ItemType='item_type'
/* Affected rows: 0 Found rows: 1 Warnings: 0 Duration for 1 query: 6.225 sec. */
explain result:
+----+-------------+------------+------------+------+---------------+----------+---------+-------+-------+----------+-------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+------------+------------+------+---------------+----------+---------+-------+-------+----------+-------+
| 1 | SIMPLE | ItemInfo | NULL | ref | ItemType | ItemType | 767 | const | 27378 | 100.00 | NULL |
+----+-------------+------------+------------+------+---------------+----------+---------+-------+-------+----------+-------+
Thanks for all contribute to this question. Hope you could bring more solutions based on information above.
Please provide CURRENT SHOW CREATE TABLE ItemInfo.
For most of those queries, you need the composite index
INDEX(ItemType, ItemComments)
For the last one, you need
INDEX(ItemComments)
For that especially slow query, please provide EXPLAIN SELECT ....
Discussion - Why does INDEX(ItemType, ItemComments) help with where ItemType="item_type" order by ItemComments desc limit 1?
An index is structured in a BTree (see Wikipedia), thereby making searching for an individual item very fast, and making scanning in a particular order very fast.
where ItemType="item_type" says to filter on ItemType, but there are a lot of such in the index. In this index, they are ordered by ItemComments (for a given ItemType). The direction desc suggests to start with the highest value of ItemContents; that is the 'end' of the index items. Finally limit 1 says to stop after one item is found. (Somewhat like finding the last "S" in your Rolodex.)
So the query is to 'drill down' the BTree to the end of the entries for ItemType in the composite INDEX(ItemType, ItemContents) and grab one entry -- a very efficient task.
Actually SELECT * implies that there is one more step, namely to get all the columns for that one row. That info is not in the index, but over in the BTree for ItemInfo -- which contains all the columns for all the rows, ordered by the PRIMARY KEY.
The "secondary index" (INDEX(ItemType, ItemComments)) implicitly contains a copy of the relevant PRIMARY KEY columns, so we now have the values of ItemID and ItemType. With those, we can drill down this other BTree to find the desired row and fetch all (*) the columns.