Le mardi 02 septembre 2025 à 15:30 +0200, Michael Nazzareno Trimarchi a écrit : > Hi Nicolas > > On Tue, Sep 2, 2025 at 3:13 PM Nicolas Dufresne > <nicolas.dufresne@xxxxxxxxxxxxx> wrote: > > > > Hi, > > > > Le mardi 02 septembre 2025 à 15:01 +0200, Michael Nazzareno Trimarchi a écrit : > > > > If you only have one format support, you don't need to force anything in > > > > GStreamer. Some extra information about Hantro post-processor. When used, you > > > > need to provide 2 sets of buffers. The decoder will still produce NV16, and a > > > > secondary set of buffers is (in parallel, pipeline mode) written back into the > > > > format you have configured on the PP register set. > > > > > > Those sets of buffers are provided by gstreamer or how does this > > > handle in practice for > > > other decoders? > > > > > > > > > > > The post-processor can also be used in standalone mode, but this feature is > > > > often fused out. In that mode, in can input interleaved YUV, as often produced > > > > as raw format by USB cameras (and most cameras using serial links). > > > > > > Much better. Now I need to understand better about the two set of buffers > > > > the V4L2 API does not have the notion of primary vs secondary buffers (unlike as > > an example Vulkan Video, which is 20 years younger :-D). So we endup hiding the > > primary buffers inside the kernel driver [0]. Just notice the this allocation > > allocates reference buffers for the decoder, and the decoder actually allocate > > the final buffers. This is quite convoluted imho. Note that some important fixes > > have happen on this part of the postproc code (which arguably still need a lot > > of cleanup, a proper split between G1 and G2 (and merging back VC8000 into G2 > > since this is largely compatible). > > > > https://gitlab.freedesktop.org/linux-media/media-committers/-/blob/next/drivers/media/platform/verisilicon/hantro_postproc.c?ref_type=heads#L224 > > https://gitlab.freedesktop.org/linux-media/media-committers/-/commit/db300ab0e9d38b1e1b0b561333b66a5beacb9994 > > > > Ok, I read the code, but I have some doubt: > > - the NV16 and NV12 are both available as the primary buffer for the > decoder but you don't know until you plug the camera and start > streaming, so I suppose that in this > case the size of the primary buffer should be computed based on the > biggest format. Since JPEG is a stateful decoder, it needs to implement the respective portion of the stateful decoder specification. In short, - Userspace should prepare the OUTPUT queue and provide a first buffer - The driver should parse the frame header** - The driver should notify that the resolution/format has been found - Userspace will enum or get the information and configura capture queue ** JPEG is the only codec we allow in-kernel parsing to take place, other codec must defer this to a firmware or implement the stateless spec With that mechanism in-place, there is no need to allocate bigger then needed. > - I think that hantro_set_reference_frames_format assumption is to > keep the one supported from decoder and match the first one match the > bit_depth, > so it can set as reference the NV12 and not the NV16 > > Is this correct? Its the bitstream that dictate which of NV12 and NV16. The main issue you are facing is that only bit depth (8 and 10bit) has been implemented so far. There might be more work to support 422. Note that G1 is unique, since the decoder part already support 2 formats. Though, in mainline we did not expose the tiled format (8x4 tiling). So if you have a 422 JPEG stream, your choices are NV16 or 8x4 NV16. > > The postproc_enable happens if we have prepare_run it means that the > buffers are allocated already but the buffer will be known only when > someone > enqueues them to the decoder. In such a scenario is there any driver > already solving it? As this code have only been used through the stateless specification, some work will be needed to support stateful for that context. CODA960 support might serve as inspiration. Nicolas > > Michael > > > > > regards, > > Nicolas
Attachment:
signature.asc
Description: This is a digitally signed message part