Zeek Scripting "Error field missing value"

Question

I'm trying to write a Zeek script to divide the dns traffic into two log files (query and reply) The error is "Field missing value" for the code $TTL=c$dns$TTLs in dns_query_reply event. I don't understand the reason for this error since the dns.log file correctly contains the value.

The code is as follows:

 module DnsFeatureExtractor;

export{

    redef enum Log::ID += {QueryDNS};
    redef enum Log::ID += {ReplyDNS};


    type InfoQuery: record{
        uid:                string              &log;
        id:                 conn_id             &log;
        domain:             string              &log    &optional;
        query_type:         string              &log    &optional;
        timestamp:          time                &log;
    };


    type InfoReply: record{
        uid:                string              &log;
        id:                 conn_id             &log;
        response_code:      count               &log    &optional;
        TTL:                vector of interval  &log    &optional;
        resolved_IP:        vector of string    &log    &optional;
        timestamp:          time                &log;
    };  
}

event zeek_init(){

    Log::create_stream(QueryDNS, [$columns=InfoQuery, $path="QueryDNS"]);
    Log::create_stream(ReplyDNS, [$columns=InfoReply, $path="ReplyDNS"]);
}

event dns_request(c: connection, msg: dns_msg, query: string, qtype: count, qclass: count){

    local name_qtype = DNS::query_types[qtype];

    local rec: DnsFeatureExtractor::InfoQuery = [$uid=c$uid, $id=c$id, $domain=query, $query_type=name_qtype, $timestamp=c$start_time];
    Log::write(DnsFeatureExtractor::QueryDNS, rec);
}

event dns_query_reply(c: connection, msg: dns_msg, query: string, qtype: count, qclass: count){

    local rec_r: DnsFeatureExtractor::InfoReply = [$uid=c$uid, $id=c$id, $response_code=msg$rcode, $TTL=c$dns$TTLs, $resolved_IP=c$dns$answers, $timestamp=c$start_time];
    Log::write(DnsFeatureExtractor::ReplyDNS, rec_r);
}

Answer 1

This one's subtle: you're seeing an ordering problem. The dns_query_reply event handler is invoked before that of the event that actually populates the TTLs (such as dns_A_reply).

The c$dns fields in question get populated in the DNS::do_reply hook, so your best bet is to ensure you add to the hook. If you use the default priority, it'll run after the one that populates the fields. Try this instead of the dns_query_reply handler:

hook DNS::do_reply(c: connection, msg: dns_msg, ans: dns_answer, reply: string) {
    local rec_r: DnsFeatureExtractor::InfoReply = [$uid=c$uid, $id=c$id, $response_code=msg$rcode, $TTL=c$dns$TTLs, $resolved_IP=c$dns$answers, $timestamp=c$start_time];
    Log::write(DnsFeatureExtractor::ReplyDNS, rec_r);
}

Note that you're still dealing with optional record values, so it's possible that despite the above you still don't see all field values. To guard against that, you could check whether the optional values are actually defined, and copy them over only in that case:

hook DNS::do_reply(c: connection, msg: dns_msg, ans: dns_answer, reply: string)
{
    local rec_r: DnsFeatureExtractor::InfoReply = [$uid=c$uid, $id=c$id, $response_code=msg$rcode, $timestamp=c$start_time];

    if ( c$dns?$TTLs )
        rec_r$TTL = c$dns$TTLs;
    if ( c$dns?$answers )
        rec_r$resolved_IP = c$dns$answers;

    Log::write(DnsFeatureExtractor::ReplyDNS, rec_r);
}