Raspberry Pi Forums

Posted: Tue Oct 27, 2020 5:39 pm

by RedMarsBlueMoon

Hi, I'm trying to understand how a hardware decoder using mmal/omx would need to be set up and am looking for an example that does that.
I have a bunch of other examples which are really useful but it would be great to have an example pipeline for recieving and decoding a network h264 stream. Does anyone know of any?

Cheers, Fred

Posted: Tue Oct 27, 2020 5:44 pm

by 6by9

It depends what the stream is, but simplest to use Gstreamer.

Generally use v4l2h264dec instead of the omx libraries. IL will not be supported on 64bit OSes, whereas V4L2 is and will be.

Posted: Tue Oct 27, 2020 5:59 pm

by RedMarsBlueMoon

I did start looking at Gstreamer yesterday as I hadn't come across it before.
Would Gstreamer be good for 'zero latency' streaming Ie not zero as such but for game streaming?

When you say 'v4l2h264dec' would that be equivalent to this one listed out of ffmpeg/ffplay:
h264_v4l2m2m ?
I tried encoding screen capture with that and it seemed to do it but the resulting file was unplayable with ffplay, vlc and omxplayer. (edit: i fixed the h264_omx typo - that was another one I tried)

The same command but using the libx264 codec produces a playable file.

Similarly I haven't be able to get any ffplay command with any codec option Iv'e tried to play with hw decoding.

Posted: Tue Oct 27, 2020 6:08 pm

by 6by9

So are you playing a network stream, or streaming to the network? Your original post refers to decoders, but this latest post is using x11grab.

Yes, Gstreamer's v4l2h264dec is equivalent to FFmpeg's decoder h264_v4l2m2m.

Posted: Tue Oct 27, 2020 6:15 pm

by RedMarsBlueMoon

My goal is to decode a network h264 stream and I would love to be able to use avcodec for that.
Testing on the files is me trying to verify that the hardware decoding in ffmpeg (which I believe is using avcodec?) is actually working. Otherwise I won't spend time going down that route.

I had read that ffmpeg hw support was down to the _omx h264 variants but that not being available for decode I tried _omx and h264_v4l2m2m for encode to see if that worked. (which it didn't)
The thinking being if I could verify something that did hardware decode OR encode, than that would be a good clue for what might be missing in the decode side.

Posted: Tue Oct 27, 2020 6:47 pm

by 6by9

FFmpeg has MMAL for decode (h264_mmal, mpeg4_mmal, mpeg2_mmal, vc1_mmal) and OMX for encode (h264_omx). Don't ask why, that's just the way they developed it.
V4L2 is the main API if you want it to work on 64bit systems, but it will also work on 32bit systems.

GStreamer is slightly better behaved with the V4L2 devices than FFmpeg.

Posted: Tue Oct 27, 2020 7:23 pm

by RedMarsBlueMoon

I don't see any of that working like I expect though. When I play the same video files (a Mulan mp4 trailer and a mulan demuxed h264 video file) with omxplayer I get full playback speed at ~15% CPU useage. VLC is the same as omxplayer.

With ffplay I get a stuttery ~15fps and ~45% CPU useage.

ffplay version says this: and has amongst other things:
--enable-omx-rpi --enable-mmal --enable-neon --enable-rpi --enable-omx

I also built ffmpeg myself with instructions that were made to enable the hw decode on the pi but that version behaves exactly like the above apt installed version.

If someone really has this working and is able to decode h264 with the above good specs, 1080p at full speed and 15% CPU use, please I'd love to hear exactly what ffmpeg/ffplay version you are using and what data you are playing!?

Posted: Tue Oct 27, 2020 7:46 pm

by 6by9

That is ffplay's fault for not rendering through any efficient path.
Omxplayer only parses the container on the arm, everything else is on the gpu. There is no way directly in ffmpeg to do that as it is a generic playback library for multiple platforms, and no others offer the near full offload the pi does.

You wanted to use ffmpeg, so your choice to investigate rendering paths.

Posted: Tue Oct 27, 2020 7:53 pm

by RedMarsBlueMoon

So the implication then is that avcodec is not able to do the decoding as that's what ffplay is using. Which would align with the observations I had that prompted me to investigate this.

You mention Gstreamer but also said that "Gstreamer's v4l2h264dec is equivalent to FFmpeg's decoder h264_v4l2m2m."
When you say equivalent is that they are both based on avcodec? And therefore Gstreamer would have the same problems doing this?

Or is it just the spec of the codec that is the same and Gstreamers implementation and use of mmal/omx is different and should work?

EDIT: I will look more at Gstreamer, as you suggest.

Posted: Mon Nov 02, 2020 12:21 am

by RedMarsBlueMoon

For anyone reading I ended up going with some code that was derived from the Raspberry OpenMax example 'hello_video'. (Was that yours @6by9 ? :) )
I tried with Gstreamer for a bit and I got my raspberry camera working well just doing a command line pipeline, but after a day and a half of trying to play a h264 video file and failing I gave up and went back to the original plan. Maybe Gstreamer would have worked but I wasn't having luck with it.

Here's what I ended up with.
https://youtu.be/xsVvuWcEPC4

Cheers
Fred

Posted: Sat Feb 13, 2021 12:30 pm

by SynAckFin

Try this https://github.com/SynAckFin/CCTVplexer

The render.c file contains all the code necessary for rendering a H264 stream.