Considering that most of the MediaCodec YUV color format constants have been deprecated in API 23, then, when selecting COLOR_FormatYUV420Flexible as the color format for a MediaCodec “encoder”, how can be found which specific YUV (planar, semiplanar, etc) format is actually expecting the encoder to be fed with?
When using COLOR_FormatYUV420Flexible, you won't know upfront when configuring the encoder exactly which layout it will be using.
When feeding the data into the encoder, instead of using getInputBuffers() or getInputBuffer(int index), use getInputImage(int index). This returns an Image, which describes (implicitly) the layout of the buffer and the data you should feed into it. Image.getPlanes() returns three Image.Plane objects, and each of them gives you a ByteBuffer to the start of that buffer, and allows you to query the row and pixel stride of that buffer.
The Image.Plane class allows describing all potential variants of planar and semiplanar frame layouts (and it does allow you to describe other layouts which don't fit into the normal planar/semiplanar distinctions as well).
For the normal planar case, all three planes have a pixel stride of 1, i.e. all pixels are written consecutively, and you need to use the row stride to advance from the start of one row to the next (as there can be padding at the end of each row).
For semiplanar, the two chroma planes have a pixel stride of 2, and the ByteBuffers for the two planes overlap, with the second chroma plane's buffer starting one byte ahead of the first chroma plane's buffer.
In practice, you'll most probably want to fill in the buffers with native code. It can make sense to try to map the buffer pointers and pixel stride values to known common cases (planar if all planes have pixel stride 1, semiplanar if chroma planes have pixel stride 2 and the buffers start at the same point with offset 1) with efficient implementations, and then have a catch-all fallback implementation which works for any unexpected combination of pixel strides and buffer pointers.