Search code examples
mysqljoinleft-joinquery-optimizationinner-join

MySQL: Optimizing JOINs to find non-matching records


We have a host management system (let's call it CMDB), and a DNS system, each using different tables. The former syncs to the latter, but manual changes cause them to get out of sync. I would like to craft a query to find aliases in CMDB that do NOT have a matching entry in DNS (either no entry, or the name/IP is different)

Because of the large size of the tables, and the need for this query to run frequently, optimizing the query is very important.

Here's what the tables look like:

cmdb_record: id, ipaddr
cmdb_alias: record_id, host_alias
dns_entry: name, ipaddr

cmdb_alias.record_id is a foreign key from cmdb_record.id, so that one IP address can have multiple aliases.

So far, here's what I've come up with:

SELECT cmdb_alias.host_alias, cmdb_record.ipaddr
FROM cmdb_record
INNER JOIN cmdb_alias ON cmdb_alias.record_id = cmdb_record.id
LEFT JOIN dns_entry
    ON dns_entry.ipaddr = cmdb_record.ipaddr
    AND dns_entry.name = cmdb_alias.host_alias
WHERE dns_entry.ipaddr IS NULL OR dns_entry.name IS NULL

This seems to work, but takes a very long time to run. Is there a better way to do this? Thanks!

EDIT: As requested, here are the SHOW CREATE TABLEs. There are lots of extra fields that aren't particularly relevant, but included for completeness.

Create Table: CREATE TABLE `cmdb_record` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `ip_version` int(11) DEFAULT NULL,
  `ipaddr` varchar(40) DEFAULT NULL,
  `ipaddr_numeric` decimal(40,0) DEFAULT NULL,
  `block_id` int(11) NOT NULL,
  `record_commented` tinyint(1) NOT NULL,
  `mod_time` datetime NOT NULL,
  `deleted` tinyint(1) NOT NULL,
  `deleted_date` datetime DEFAULT NULL,
  `record_owner` varchar(50) DEFAULT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `ipaddr` (`ipaddr`),
  KEY `cmdb_record_fe30f0f7` (`ipaddr`),
  KEY `cmdb_record_2b8b575` (`ipaddr_numeric`),
  KEY `cmdb_record_45897ef2` (`block_id`),
  CONSTRAINT `block_id_refs_id_ed6ed320` FOREIGN KEY (`block_id`) REFERENCES `cmdb_block` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=104427 DEFAULT CHARSET=latin1

Create Table: CREATE TABLE `cmdb_alias` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `host_alias` varchar(255) COLLATE latin1_general_cs NOT NULL,
  `record_id` int(11) NOT NULL,
  `record_order` int(11) NOT NULL,
  PRIMARY KEY (`id`),
  KEY `cmdb_alias_fcffc3bb` (`record_id`),
  KEY `alias_lookup` (`host_alias`),
  CONSTRAINT `record_id_refs_id_8169fc71` FOREIGN KEY (`record_id`) REFERENCES `cmdb_record` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=155433 DEFAULT CHARSET=latin1 COLLATE=latin1_general_cs

Create Table: CREATE TABLE `dns_entry` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `rec_grp_id` varchar(40) NOT NULL,
  `parent_id` int(11) NOT NULL,
  `domain_id` int(11) DEFAULT NULL,
  `name` varchar(255) DEFAULT NULL,
  `type` varchar(6) DEFAULT NULL,
  `ipaddr` varchar(255) DEFAULT NULL,
  `ttl` int(11) DEFAULT NULL,
  `prio` int(11) DEFAULT NULL,
  `status` varchar(20) NOT NULL,
  `op` varchar(20) NOT NULL,
  `mod_time` datetime NOT NULL,
  `whodunit` varchar(50) NOT NULL,
  `comments` longtext NOT NULL,
  PRIMARY KEY (`id`),
  KEY `dns_entry_a2431ea` (`domain_id`),
  KEY `dns_entry_52094d6e` (`name`)
) ENGINE=InnoDB AUTO_INCREMENT=49437 DEFAULT CHARSET=utf8

Solution

  • If you don't have one already, create an index on dns_entry(ipaddr, name). This might be all you need to speed the query.