We use some essential cookies to make our website work.

We use optional cookies, as detailed in our cookie policy, to remember your settings and understand how you use our website.

3 posts • Page 1 of 1
jaween
Posts: 2
Joined: Fri Dec 27, 2024 8:20 am

[SOLVED] Decoding H265 on Raspberry Pi 5 via V4L2

Fri Dec 27, 2024 8:52 am

Hi,

I'm trying to use the V4L2 API on a Raspberry Pi 5 to decode H265 video. (I'm not able to use ffmpeg due to an unrelated issue with how my project will be deployed.)

I'm not familiar with V4L2, but through reading the Raspberry Pi camera docs and the docs on V4L2 at kernel.org I've managed to query /dev/video19 through C++, and set up the capture buffer, but I'm not able to queue an output buffer (the encoded H265 video input). It consistently fails with "Invalid argument". Here's the shorted version of my code. I compile it with g++ v4l2_test.cpp:

Code: Select all

#include <fstream>
#include <iostream>
#include <functional>
#include <linux/videodev2.h>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <unistd.h>
#include <string.h>
#include <vector>
#include <stdexcept>
#include <cerrno>
std::vector<char> readBytesFromFile(std::string filePath, size_t maxCount) {
 std::ifstream file(filePath, std::ios::binary);
 if (!file) {
 throw std::runtime_error("Failed to read file");
 }
 std::vector<char> buffer(maxCount);
 file.read(buffer.data(), buffer.size());
 std::streamsize bytesRead = file.gcount();
 buffer.resize(bytesRead);
 return buffer;
}
int main() { 
 std::vector<char> hevc = readBytesFromFile("/home/openup/source.h265", 4096);
 int hevc_byte_count = hevc.size();
 std::cout << "Read " << hevc_byte_count << " bytes" << std::endl;
 int fd = open("/dev/video19", O_RDWR);
 if (fd < 0) {
 throw std::runtime_error("Failed to open device");
 }
 struct v4l2_capability capability;
 if (ioctl(fd, VIDIOC_QUERYCAP, &capability) < 0) {
 std::perror("Failed to query capabilities");
 return 1;
 }
 std::cout << "Card: " << capability.card << std::endl;
 struct v4l2_format encodedFormat = {};
 encodedFormat.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
 encodedFormat.fmt.pix_mp.pixelformat = V4L2_PIX_FMT_HEVC_SLICE;
 encodedFormat.fmt.pix_mp.field = V4L2_FIELD_NONE;
 encodedFormat.fmt.pix_mp.plane_fmt[0].sizeimage = hevc_byte_count;
 encodedFormat.fmt.pix_mp.num_planes = 1;
 encodedFormat.fmt.pix.bytesperline = hevc_byte_count;
 if (ioctl(fd, VIDIOC_S_FMT, &encodedFormat) < 0) {
 std::perror("Failed to set encoded format");
 return 1;
 }
 struct v4l2_buffer v4l2_buf = {};
 struct v4l2_plane planes[1] = {};
 struct timeval timestamp = {};
 v4l2_buf.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
 v4l2_buf.memory = V4L2_MEMORY_USERPTR;
 v4l2_buf.timestamp = timestamp;
 v4l2_buf.length = 1;
 v4l2_buf.m.planes = planes;
 planes[0].m.userptr = reinterpret_cast<unsigned long>(hevc.data());
 planes[0].length = hevc_byte_count;
 planes[0].bytesused = hevc_byte_count;
 if (ioctl(fd, VIDIOC_QBUF, &v4l2_buf)) {
 std::perror("Failed to queue buffer");
 return 1;
 }
 std::cout << "Successfully queued buffer" << std::endl;
 return 0;
}

And here's the output of v4l2-ctl:

Code: Select all

$ v4l2-ctl --dev /dev/video19 --all
Driver Info:
 Driver name : rpivid
 Card type : rpivid
 Bus info : platform:rpivid
 Driver version : 6.6.62
 Capabilities : 0x84204000
 Video Memory-to-Memory Multiplanar
 Streaming
 Extended Pix Format
 Device Capabilities
 Device Caps : 0x04204000
 Video Memory-to-Memory Multiplanar
 Streaming
 Extended Pix Format
Media Driver Info:
 Driver name : rpivid
 Model : rpivid
 Serial : 
 Bus info : platform:rpivid
 Media version : 6.6.62
 Hardware revision: 0x00000000 (0)
 Driver version : 6.6.62
Interface Info:
 ID : 0x0300000c
 Type : V4L Video
Entity Info:
 ID : 0x00000001 (1)
 Name : rpivid-source
 Function : V4L2 I/O
 Pad 0x01000002 : 0: Source
 Link 0x02000008: to remote pad 0x1000004 of entity 'rpivid-proc' (Video Decoder): Data, Enabled, Immutable
Priority: 2
Format Video Capture Multiplanar:
 Width/Height : 1920/1088
 Pixel Format : 'NC12' (Y/CbCr 4:2:0 (128b cols))
 Field : None
 Number of planes : 1
 Flags : 
 Colorspace : Default
 Transfer Function : Default
 YCbCr/HSV Encoding: Default
 Quantization : Default
 Plane 0 :
 Bytes per Line : 1632
 Size Image : 3133440
Format Video Output Multiplanar:
 Width/Height : 1920/1088
 Pixel Format : 'S265' (HEVC Parsed Slice Data)
 Field : None
 Number of planes : 1
 Flags : 
 Colorspace : Default
 Transfer Function : Default
 YCbCr/HSV Encoding: Default
 Quantization : Default
 Plane 0 :
 Bytes per Line : 0
 Size Image : 4096
Stateless Codec Controls
 hevc_sequence_parameter_set 0x00a40a90 (unknown): type=270 value=unsupported payload type flags=has-payload
 hevc_picture_parameter_set 0x00a40a91 (unknown): type=271 value=unsupported payload type flags=has-payload
 slice_param_array 0x00a40a92 (unknown): type=272 dims=[4096] flags=has-payload, 0x00000800
 hevc_scaling_matrix 0x00a40a93 (unknown): type=273 value=unsupported payload type flags=has-payload
 hevc_decode_parameters 0x00a40a94 (unknown): type=274 value=unsupported payload type flags=has-payload
 hevc_decode_mode 0x00a40a95 (menu) : min=0 max=0 default=0 value=0 (Slice-Based)
 0: Slice-Based
 hevc_start_code 0x00a40a96 (menu) : min=0 max=1 default=0 value=0 (No Start Code)
 0: No Start Code
 1: Annex B Start Code
Based on querying the capabilities, it seems like it only supports multi-planar buffer types, so that's why I'm sending the H265 data in a mulitplanar buffer with one plane. It also mentions it's slice based, so I assume I'll have to parse the bitstream and queue up H265 slices, but I'm not sure if that's the problem I'm facing yet.

I'm wondering if any someone can point out what I'm doing wrong here?
Thank you

Edit: Cleanup up the code a little
Last edited by jaween on Thu Jan 09, 2025 2:34 am, edited 1 time in total.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 18476
Joined: Wed Dec 04, 2013 11:27 am

Re: Decoding H265 on Raspberry Pi 5 via V4L2

Fri Dec 27, 2024 8:59 pm

The HEVC decoder follows the stateless decoder API - https://www.kernel.org/doc/html/latest/ ... coder.html

Yes you need to have decoded the bitstream headers and provide the relevant controls via the media request API.
Software Engineer at Raspberry Pi Ltd. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

jaween
Posts: 2
Joined: Fri Dec 27, 2024 8:20 am

Re: Decoding H265 on Raspberry Pi 5 via V4L2

Thu Jan 09, 2025 2:34 am

6by9 wrote:
Fri Dec 27, 2024 8:59 pm
The HEVC decoder follows the stateless decoder API - https://www.kernel.org/doc/html/latest/ ... coder.html

Yes you need to have decoded the bitstream headers and provide the relevant controls via the media request API.
Thank you, the stateless decoder API was what I needed.

I was then stuck for a few days with a dequeued Capture Buffer with the flag V4L2_BUF_FLAG_ERROR=1, but no other feedback. Using trial and error to interpret a bit of the documentation of the V4L2 hevc control parameters, I got frames decoding. I was so happy, I did a little dance!
Slightly more struggle after that to properly interpret the SAND 128 Column pixel format from the docs, but not too much trouble.

And now I've integrated Raspberry Pi 5's 4K 60 H265 decoding into my project, very pleased! Will mark as solved.

3 posts • Page 1 of 1

Return to "Graphics programming"

AltStyle によって変換されたページ (->オリジナル) /