[SOLVED] Decoding H265 on Raspberry Pi 5 via V4L2
Hi,
I'm trying to use the V4L2 API on a Raspberry Pi 5 to decode H265 video. (I'm not able to use ffmpeg due to an unrelated issue with how my project will be deployed.)
I'm not familiar with V4L2, but through reading the Raspberry Pi camera docs and the docs on V4L2 at kernel.org I've managed to query /dev/video19 through C++, and set up the capture buffer, but I'm not able to queue an output buffer (the encoded H265 video input). It consistently fails with "Invalid argument". Here's the shorted version of my code. I compile it with g++ v4l2_test.cpp:
And here's the output of v4l2-ctl:
Based on querying the capabilities, it seems like it only supports multi-planar buffer types, so that's why I'm sending the H265 data in a mulitplanar buffer with one plane. It also mentions it's slice based, so I assume I'll have to parse the bitstream and queue up H265 slices, but I'm not sure if that's the problem I'm facing yet.
I'm wondering if any someone can point out what I'm doing wrong here?
Thank you
Edit: Cleanup up the code a little
I'm trying to use the V4L2 API on a Raspberry Pi 5 to decode H265 video. (I'm not able to use ffmpeg due to an unrelated issue with how my project will be deployed.)
I'm not familiar with V4L2, but through reading the Raspberry Pi camera docs and the docs on V4L2 at kernel.org I've managed to query /dev/video19 through C++, and set up the capture buffer, but I'm not able to queue an output buffer (the encoded H265 video input). It consistently fails with "Invalid argument". Here's the shorted version of my code. I compile it with g++ v4l2_test.cpp:
Code: Select all
#include <fstream>
#include <iostream>
#include <functional>
#include <linux/videodev2.h>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <unistd.h>
#include <string.h>
#include <vector>
#include <stdexcept>
#include <cerrno>
std::vector<char> readBytesFromFile(std::string filePath, size_t maxCount) {
std::ifstream file(filePath, std::ios::binary);
if (!file) {
throw std::runtime_error("Failed to read file");
}
std::vector<char> buffer(maxCount);
file.read(buffer.data(), buffer.size());
std::streamsize bytesRead = file.gcount();
buffer.resize(bytesRead);
return buffer;
}
int main() {
std::vector<char> hevc = readBytesFromFile("/home/openup/source.h265", 4096);
int hevc_byte_count = hevc.size();
std::cout << "Read " << hevc_byte_count << " bytes" << std::endl;
int fd = open("/dev/video19", O_RDWR);
if (fd < 0) {
throw std::runtime_error("Failed to open device");
}
struct v4l2_capability capability;
if (ioctl(fd, VIDIOC_QUERYCAP, &capability) < 0) {
std::perror("Failed to query capabilities");
return 1;
}
std::cout << "Card: " << capability.card << std::endl;
struct v4l2_format encodedFormat = {};
encodedFormat.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
encodedFormat.fmt.pix_mp.pixelformat = V4L2_PIX_FMT_HEVC_SLICE;
encodedFormat.fmt.pix_mp.field = V4L2_FIELD_NONE;
encodedFormat.fmt.pix_mp.plane_fmt[0].sizeimage = hevc_byte_count;
encodedFormat.fmt.pix_mp.num_planes = 1;
encodedFormat.fmt.pix.bytesperline = hevc_byte_count;
if (ioctl(fd, VIDIOC_S_FMT, &encodedFormat) < 0) {
std::perror("Failed to set encoded format");
return 1;
}
struct v4l2_buffer v4l2_buf = {};
struct v4l2_plane planes[1] = {};
struct timeval timestamp = {};
v4l2_buf.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
v4l2_buf.memory = V4L2_MEMORY_USERPTR;
v4l2_buf.timestamp = timestamp;
v4l2_buf.length = 1;
v4l2_buf.m.planes = planes;
planes[0].m.userptr = reinterpret_cast<unsigned long>(hevc.data());
planes[0].length = hevc_byte_count;
planes[0].bytesused = hevc_byte_count;
if (ioctl(fd, VIDIOC_QBUF, &v4l2_buf)) {
std::perror("Failed to queue buffer");
return 1;
}
std::cout << "Successfully queued buffer" << std::endl;
return 0;
}And here's the output of v4l2-ctl:
Code: Select all
$ v4l2-ctl --dev /dev/video19 --all
Driver Info:
Driver name : rpivid
Card type : rpivid
Bus info : platform:rpivid
Driver version : 6.6.62
Capabilities : 0x84204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Media Driver Info:
Driver name : rpivid
Model : rpivid
Serial :
Bus info : platform:rpivid
Media version : 6.6.62
Hardware revision: 0x00000000 (0)
Driver version : 6.6.62
Interface Info:
ID : 0x0300000c
Type : V4L Video
Entity Info:
ID : 0x00000001 (1)
Name : rpivid-source
Function : V4L2 I/O
Pad 0x01000002 : 0: Source
Link 0x02000008: to remote pad 0x1000004 of entity 'rpivid-proc' (Video Decoder): Data, Enabled, Immutable
Priority: 2
Format Video Capture Multiplanar:
Width/Height : 1920/1088
Pixel Format : 'NC12' (Y/CbCr 4:2:0 (128b cols))
Field : None
Number of planes : 1
Flags :
Colorspace : Default
Transfer Function : Default
YCbCr/HSV Encoding: Default
Quantization : Default
Plane 0 :
Bytes per Line : 1632
Size Image : 3133440
Format Video Output Multiplanar:
Width/Height : 1920/1088
Pixel Format : 'S265' (HEVC Parsed Slice Data)
Field : None
Number of planes : 1
Flags :
Colorspace : Default
Transfer Function : Default
YCbCr/HSV Encoding: Default
Quantization : Default
Plane 0 :
Bytes per Line : 0
Size Image : 4096
Stateless Codec Controls
hevc_sequence_parameter_set 0x00a40a90 (unknown): type=270 value=unsupported payload type flags=has-payload
hevc_picture_parameter_set 0x00a40a91 (unknown): type=271 value=unsupported payload type flags=has-payload
slice_param_array 0x00a40a92 (unknown): type=272 dims=[4096] flags=has-payload, 0x00000800
hevc_scaling_matrix 0x00a40a93 (unknown): type=273 value=unsupported payload type flags=has-payload
hevc_decode_parameters 0x00a40a94 (unknown): type=274 value=unsupported payload type flags=has-payload
hevc_decode_mode 0x00a40a95 (menu) : min=0 max=0 default=0 value=0 (Slice-Based)
0: Slice-Based
hevc_start_code 0x00a40a96 (menu) : min=0 max=1 default=0 value=0 (No Start Code)
0: No Start Code
1: Annex B Start CodeI'm wondering if any someone can point out what I'm doing wrong here?
Thank you
Edit: Cleanup up the code a little
Last edited by jaween on Thu Jan 09, 2025 2:34 am, edited 1 time in total.
- 6by9
- Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator - Posts: 18476
- Joined: Wed Dec 04, 2013 11:27 am
Re: Decoding H265 on Raspberry Pi 5 via V4L2
The HEVC decoder follows the stateless decoder API - https://www.kernel.org/doc/html/latest/ ... coder.html
Yes you need to have decoded the bitstream headers and provide the relevant controls via the media request API.
Yes you need to have decoded the bitstream headers and provide the relevant controls via the media request API.
Software Engineer at Raspberry Pi Ltd. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.
I'm not interested in doing contracts for bespoke functionality - please don't ask.
Re: Decoding H265 on Raspberry Pi 5 via V4L2
Thank you, the stateless decoder API was what I needed.6by9 wrote: ↑Fri Dec 27, 2024 8:59 pmThe HEVC decoder follows the stateless decoder API - https://www.kernel.org/doc/html/latest/ ... coder.html
Yes you need to have decoded the bitstream headers and provide the relevant controls via the media request API.
I was then stuck for a few days with a dequeued Capture Buffer with the flag V4L2_BUF_FLAG_ERROR=1, but no other feedback. Using trial and error to interpret a bit of the documentation of the V4L2 hevc control parameters, I got frames decoding. I was so happy, I did a little dance!
Slightly more struggle after that to properly interpret the SAND 128 Column pixel format from the docs, but not too much trouble.
And now I've integrated Raspberry Pi 5's 4K 60 H265 decoding into my project, very pleased! Will mark as solved.
Return to "Graphics programming"
Jump to
- Community
- General discussion
- Announcements
- Other languages
- Deutsch
- Español
- Français
- Italiano
- Nederlands
- 日本語
- Polski
- Português
- Русский
- Türkçe
- User groups and events
- Raspberry Pi Official Magazine
- Using the Raspberry Pi
- Beginners
- Troubleshooting
- Advanced users
- Assistive technology and accessibility
- Education
- Picademy
- Teaching and learning resources
- Staffroom, classroom and projects
- Astro Pi
- Mathematica
- High Altitude Balloon
- Weather station
- Programming
- C/C++
- Java
- Python
- Scratch
- Other programming languages
- Windows 10 for IoT
- Wolfram Language
- Bare metal, Assembly language
- Graphics programming
- OpenGLES
- OpenVG
- OpenMAX
- General programming discussion
- Projects
- Networking and servers
- Automation, sensing and robotics
- Graphics, sound and multimedia
- Other projects
- Media centres
- Gaming
- AIY Projects
- Hardware and peripherals
- Camera board
- Compute Module
- Official Display
- HATs and other add-ons
- Device Tree
- Interfacing (DSI, CSI, I2C, etc.)
- Keyboard computers (400, 500, 500+)
- Raspberry Pi Pico
- General
- SDK
- MicroPython
- Other RP2040 boards
- Zephyr
- Rust
- AI Accelerator
- AI Camera - IMX500
- Hailo
- Software
- Raspberry Pi OS
- Raspberry Pi Connect
- Raspberry Pi Desktop for PC and Mac
- Beta testing
- Other
- Android
- Debian
- FreeBSD
- Gentoo
- Linux Kernel
- NetBSD
- openSUSE
- Plan 9
- Puppy
- Arch
- Pidora / Fedora
- RISCOS
- Ubuntu
- Ye Olde Pi Shoppe
- For sale
- Wanted
- Off topic
- Off topic discussion