According to Romain Guy's answer here:
glsurfaceview inside a scrollview, moving but not clipping
one should not use SurfaceView inside ScrollView, or ListView, etc. Since VideoView is subclass of SurfaceView, I was wondering how this applies to CardScrollView?
As stated in the comments, the proper way to do this is with a TextureView.