marking packet for sending over raw socket

Question

I have the following function which send packets over raw socket.

#include <unistd.h>
#include <stdio.h>
#include <sys/socket.h>
#include <netinet/ip.h>
#include <netinet/udp.h>

#include "pkt-types.h"
#include "pkt-log.h"
#include "pkt-utils.h"

int
send_packet_raw (void *data, int size)
{
  log_message (LOG_DEBUG, " inside send_packet_raw");
  int sd;
  struct iphdr *iph = (struct iphdr *) data;
  struct udphdr *udph = (struct udphdr *) (data + sizeof (struct ip));
  struct sockaddr_in sin;
  // needed for notify kernel to not to build header for this
  int one = 1;
  const int *val = &one;
  // creating a socket
  if ((sd = socket (PF_INET, SOCK_RAW, IPPROTO_UDP)) < 0)
    {
      log_message (LOG_ERROR, " problem creating a socket");
      return EXITCODE_SOCK_CREATION_FAILED;
    }
  // setting address family
  sin.sin_family = AF_INET;
  // setting port
  sin.sin_port = udph->dest;
  // setting ip
  sin.sin_addr.s_addr = iph->daddr;
  // notifying kernel do not fill up the packet structure.
  if (setsockopt (sd, IPPROTO_IP, IP_HDRINCL, val, sizeof (one)) < 0)
    {
      log_message (LOG_ERROR, "error notifying kernel about raw socket");
      return EXITCODE_SOCK_KERN_NOTIF_FAILED;
    }
  /* setting socket option to use MARK value */
  if (setsockopt (sd, SOL_SOCKET, SO_MARK, val, sizeof (one)) < 0)
  {
    log_message (LOG_ERROR, "error notifying kernel about MARK");
    return EXITCODE_SOCK_MARK_FAILED;
  }
  #ifdef CHECKSUM
  /* compute checksum */
  udph->check = udp_checksum (data + IP_OFFSET, size - IP_OFFSET, iph->saddr, iph->daddr);
  /* testing purposed */
  #else
  udph->check = 0x00;
  #endif
  /* dscp 101000 means express forwarding */
  if (sendto (sd,               /* our socket */
              data,             /* data to send */
              size,     /* total length of our ip packet */
              0,                /* routing flag, normally always zero */
              (struct sockaddr *) &sin, /* socket addr */
              sizeof (sin)) < 0)
    {
      log_message (LOG_ERROR, "sending over raw socket failed");
      return EXITCODE_SOCK_SEND_FAILED;
    }
  else
  {
    /* shutdown the socket */
    if(shutdown (sd, 2)) /* shutdown ok */
      return EXITCODE_OK;
  }
}

Now i'm setting mark from nfq_set_verdict2() from libnetfilter_queue :http://www.netfilter.org/projects/libnetfilter_queue/doxygen/group__Queue.html

int nfq_set_verdict2    (   struct nfq_q_handle *   qh,
u_int32_t   id,
u_int32_t   verdict,
u_int32_t   mark,
u_int32_t   data_len,
const unsigned char *   buf  
)           
nfq_set_verdict2 - like nfq_set_verdict, but you can set the mark.

Parameters:
qh  Netfilter queue handle obtained by call to nfq_create_queue().
id  ID assigned to packet by netfilter.
verdict     verdict to return to netfilter (NF_ACCEPT, NF_DROP)
mark    mark to put on packet
data_len    number of bytes of data pointed to by buf
buf     the buffer that contains the packet data

when i receive the packet from netfilter_queue i do something following :

nfq_set_verdict(..,NF_DROP,MARK,...);
process_packet();

This process_packet() calls send_packet_raw().

Associated iptable rules :

$iptables -t mangle -A PREROUTING -m mark --mark 0xa -j ACCEPT
$iptables -t mangle -A PREROUTING -p udp --dport $PORT -j NFQUEUE
$iptables -t mangle -A OUTPUT -m mark --mark 0xa -j ACCEPT
$iptables -t mangle -A OUTPUT -p udp --sport $PORT -j NFQUEUE

I have also put up some -j LOG rules to see if packets are in-fact matching. but as it seems neither packet goes out or goes in as no log entries are shown. Could not understand how to find the problem here.

Answer 1

Not exactly sure what the question was, but

nfq_set_verdict(..,NF_DROP,MARK,...);
process_packet();

looks bad. I wouldn't call NF_DROP before processing the packet. I have written a couple of tunneling programs, and I first process the packet, put it in my buffer, the issue a NF_DROP. After this I can reissue the packets from buffer using the raw socket. So:

process_packet();    
nfq_set_verdict(..,NF_DROP,MARK,...);

would be better. At least copy the packet data before issuing the verdict.