Can P frame of HEVC video be dropped on streaming?

Question

I'm creating a live streaming app using HEVC encoding/decoding on unreliable network. There are IDR/I/P/B frame types available in HEVC, from which I'm using IDR and P frame since the source video is captured in realtime and the latency from capture device to display is critical.

Since the network is unreliable, it needs to be expected that any frame in streaming may be dropped. To avoid making decoded images jaggy, I implemented the receiver as follows:

1. 1 IDR frame and 29 P frames in seconds (= 30fps in total) in normal operation
2. If IDR frame is dropped, upcoming P frames are not passed to decoder and waits for a new IDR frame
3. If P frame is dropped, just skip the frame and passes the next received frame to decoder

This procedure comes from my understanding that P frame refers the latest IDR frame, so any P frame can be dropped while any IDR frame cannot be dropped.

However, the decoded images are still jaggy (it looks the decoder fails compensating motion vectors)

Here the following questions came to my mind:

• Is it true that P frame refers the last IDR frames? Or any frame (incl. IDR/I/P/B) cannot be dropped on HEVC in specification?
• Is there any way to avoid the jaggy in decoded video on streaming over unreliable network?

Thank you very much in advance!

Answer 1

P frames can refer to other P frames. You can not drop them. You can use reference frame invalidation to force them to only refer to the IDR, but that will increase bitrate/decrease image quality.