I am trying to write a utility to extract the payload from RTP packet. From my understanding about the network packets and protocols, RTP is wrapped in UDP. How can we decide whether the UDP packet that we have captured has RTP packet wrapped in it? I mean it can also be NBNS or MDNS right? I am using Winpcap library and am new to it. I have been given a task to extract the RTP packets from any given pcap dump file.
Have a look at the RTP packet format at Wikipedia. By studying it you will find a few magic bytes which help you to detect potential RTP packets. Also, RTP packets don't travel alone and you can find related packets by source/dst IP+port and by sequence numbers: