video: add mp_image_params.hw_flags and add an example

It seems this will be useful for Rokchip DRM hwcontext integration.

DRM hwcontexts have additional internal structure which can be different
depending on the decoder, and which is not part of the generic hwcontext
API. Rockchip has 1 layer, which EGL interop happens to translate to a
RGB texture, while VAAPI (mapped as DRM hwcontext) will use multiple
layers. Both will use sw_format=nv12, and thus are indistinguishable on
the mp_image_params level. But this is needed to initialize the EGL
mapping and the vo_gpu video renderer correctly.

We hope that the layer count is enough to tell whether EGL will
translate the data to a RGB texture (vs. 2 texture resembling raw nv12
data). For that we introduce MP_IMAGE_HW_FLAG_OPAQUE.

This commit adds the flag, infrastructure to set it, and an "example"
for D3D11.

The D3D11 addition is quite useless at this point. But later we want to
get rid of d3d11_update_image_attribs() anyway, while we still need a
way to force d3d11vpp filter insertion, so maybe it has some
justification (who knows). In any case it makes testing this easier.
Obviously it also adds some basic support for triggering the opaque
format for decoding, which will use a driver-specific format, but which
is not supported in shaders. The opaque flag is not used to determine
whether d3d11vpp needs to be inserted, though.

1 files changed, 19 insertions, 1 deletions

diff --git a/video/decode/d3d.c b/video/decode/d3d.c
index a5408f2133..0a44c0f507 100644
--- a/video/decode/d3d.c
+++ b/video/decode/d3d.c
@@ -111,7 +111,12 @@ void d3d_hwframes_refine(struct lavc_ctx *ctx, AVBufferRef *hw_frames_ctx)
     if (fctx->format == AV_PIX_FMT_D3D11) {
         AVD3D11VAFramesContext *hwctx = fctx->hwctx;
 
-        hwctx->BindFlags |= D3D11_BIND_DECODER | D3D11_BIND_SHADER_RESOURCE;
+        hwctx->BindFlags |= D3D11_BIND_DECODER;
+
+        // According to hwcontex_d3d11va.h, yuv420p means DXGI_FORMAT_420_OPAQUE,
+        // which has no shader support.
+        if (fctx->sw_format != AV_PIX_FMT_YUV420P)
+            hwctx->BindFlags |= D3D11_BIND_SHADER_RESOURCE;
     }
 }
 
@@ -132,3 +137,16 @@ AVBufferRef *d3d11_wrap_device_ref(ID3D11Device *device)
 
     return device_ref;
 }
+
+static void d3d11_complete_image_params(struct AVHWFramesContext *hw_frames,
+                                        struct mp_image_params *p)
+{
+    // According to hwcontex_d3d11va.h, this means DXGI_FORMAT_420_OPAQUE.
+    p->hw_flags = hw_frames->sw_format == AV_PIX_FMT_YUV420P
+                ? MP_IMAGE_HW_FLAG_OPAQUE : 0;
+}
+
+const struct hwcontext_fns hwcontext_fns_d3d11 = {
+    .av_hwdevice_type = AV_HWDEVICE_TYPE_D3D11VA,
+    .complete_image_params = d3d11_complete_image_params,
+};