Raspberry Pi 3 and V4L2 C++ Capture Code for JPEGs

Question

I have a Raspberry Pi 3 with the latest Raspbian Stretch, and the Sony 8MB official camera.

I can save a good looking JPEG with the following commands:

v4l2-ctl --set-fmt-video=width=2592,height=1944,pixelformat=3
v4l2-ctl --stream-mmap=3 --stream-count=1 --stream-to=somefile.jpg

I can list all formats with this command:

root@raspberrypi:/home/pi# v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
 Index : 0
 Type : Video Capture
 Pixel Format: 'YU12'
 Name : Planar YUV 4:2:0
 Size: Stepwise 32x32 - 3280x2464 with step 2/2
 ...
 Index : 3
 Type : Video Capture
 Pixel Format: 'JPEG' (compressed)
 Name : JFIF JPEG
 Size: Stepwise 32x32 - 3280x2464 with step 2/2

Now I want to translate the v4l2-ctl capture of a JPEG to C++ / Qt5 with libv4l2. My code is shown below and whenever I run it on the Pi, I get the following error:

root@raspberrypi:/home/pi# ./v4l2-test 
Begin Capture
VIDIOC_STREAMON: Operation not permitted

C++ Code:

#include <QCoreApplication>
// C++ / V4L2 Includes
#include <linux/videodev2.h>
#include <fcntl.h> /* low-level i/o */
#include <unistd.h>
#include <errno.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/time.h>
#include <sys/mman.h>
#include <sys/ioctl.h>
// Debugging
#include <QDebug>
int main(int argc, char *argv[])
{
 QCoreApplication a(argc, argv);
 qDebug() << "Begin Capture";
 int fd;
 if((fd = open("/dev/video0", O_RDWR)) < 0){
 perror("open");
 exit(1);
 }
 struct v4l2_capability cap;
 if(ioctl(fd, VIDIOC_QUERYCAP, &cap) < 0){
 perror("VIDIOC_QUERYCAP");
 exit(1);
 }
 if(!(cap.capabilities & V4L2_CAP_VIDEO_CAPTURE)){
 fprintf(stderr, "The device does not handle single-planar video capture.\n");
 exit(1);
 }
 struct v4l2_format format;
 format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
 format.fmt.pix.pixelformat = V4L2_PIX_FMT_JPEG;
 format.fmt.pix.width = 2592;
 format.fmt.pix.height = 1944;
 if(ioctl(fd, VIDIOC_S_FMT, &format) < 0){
 perror("VIDIOC_S_FMT");
 exit(1);
 }
 struct v4l2_requestbuffers bufrequest;
 bufrequest.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
 bufrequest.memory = V4L2_MEMORY_MMAP;
 bufrequest.count = 1;
 if(ioctl(fd, VIDIOC_REQBUFS, &bufrequest) < 0){
 perror("VIDIOC_REQBUFS");
 exit(1);
 }
 struct v4l2_buffer bufferinfo;
 memset(&bufferinfo, 0, sizeof(bufferinfo));
 bufferinfo.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
 bufferinfo.memory = V4L2_MEMORY_MMAP;
 bufferinfo.index = 0;
 if(ioctl(fd, VIDIOC_QUERYBUF, &bufferinfo) < 0){
 perror("VIDIOC_QUERYBUF");
 exit(1);
 }
 void* buffer_start = mmap(
 NULL,
 bufferinfo.length,
 PROT_READ | PROT_WRITE,
 MAP_SHARED,
 fd,
 bufferinfo.m.offset
 );
 if(buffer_start == MAP_FAILED){
 perror("mmap");
 exit(1);
 }
 memset(buffer_start, 0, bufferinfo.length);
 // Activate streaming
 int type = bufferinfo.type;
 if(ioctl(fd, VIDIOC_STREAMON, &type) < 0){
 perror("VIDIOC_STREAMON");
 exit(1);
 }
 /* Here is where you typically start two loops:
 * - One which runs for as long as you want to
 * capture frames (shoot the video).
 * - One which iterates over your buffers everytime. */
 bool capture_is_running = true;
 while(capture_is_running){
 for(int i = 0; i < bufrequest.count; i++){
 // Put the buffer in the incoming queue.
 if(ioctl(fd, VIDIOC_QBUF, &bufferinfo) < 0){
 perror("VIDIOC_QBUF");
 exit(1);
 }
 // The buffer's waiting in the outgoing queue.
 if(ioctl(fd, VIDIOC_DQBUF, &bufferinfo) < 0){
 perror("VIDIOC_QBUF");
 exit(1);
 }
 }
 }
 // Deactivate streaming
 if(ioctl(fd, VIDIOC_STREAMOFF, &type) < 0){
 perror("VIDIOC_STREAMOFF");
 exit(1);
 }
 close(fd);
 return EXIT_SUCCESS;
 return a.exec();
}

What am I doing wrong? The camera works and I run as root, but for some reason I get "Permission Denied". I appreciate any help - thanks.

UPDATE - The exact error I see is shown below as well as the terminal output showing that "root" is in the "video" group

root@raspberrypi:/home/pi# groups root
root : root video
root@raspberrypi:/home/pi# ./v4l2-test 
Begin Capture
VIDIOC_STREAMON: Operation not permitted

UPDATE: Running 'file' on the v4l2-ctl saved image shows the 2592x1944 sized frame:

root@raspberrypi:/tmp# file somefile.jpg 
somefile.jpg: JPEG image data, Exif standard: [TIFF image data, big-endian, direntries=9, height=0, model=imx219, xresolution=130, yresolution=138, resolutionunit=2, datetime=.&checktime(1970,01,01,':') 00:00:00, width=0], baseline, precision 8, 2592x1944, frames 3

UPDATE: Here is a paste of the complete STRACE output of all commands:

https://zerobin.net/?84fee190a4a904a0#ZZOwDe7ftBKjpqpP/lPfnTMPrUlILyJaCCCyunO6aug=

UPDATE: Linking step of application compilation

/home/me/Desktop/buildroot/output/host/bin/arm-buildroot-linux-gnueabihf-g++ --sysroot=/home/me/Desktop/buildroot/output/host/arm-buildroot-linux-gnueabihf/sysroot -o v4l2-test main.o -lrt -lpthread -ldl -latomic 

UPDATE: I removed my S_FMT ioctl, but the issue still continues:

 struct v4l2_format format;
 format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
 format.fmt.pix.pixelformat = V4L2_PIX_FMT_JPEG;
 // 3280x2464
 format.fmt.pix.width = 3280;
 format.fmt.pix.height = 2464;
// if(ioctl(fd, VIDIOC_S_FMT, &format) < 0){
// perror("VIDIOC_S_FMT");
// exit(1);
// }
 struct v4l2_requestbuffers bufrequest;
 bufrequest.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
 bufrequest.memory = V4L2_MEMORY_MMAP;
 bufrequest.count = 1;

UPDATE: I just checked dmesg again I'm seeing the following error - however Google has not turned up anything so far...

Jan 06 01:40:19 raspberrypi kernel: video0: VIDIOC_QUERYCAP
Jan 06 01:40:19 raspberrypi kernel: video0: VIDIOC_S_FMT
Jan 06 01:40:19 raspberrypi kernel: video0: VIDIOC_REQBUFS
Jan 06 01:40:19 raspberrypi kernel: video0: VIDIOC_QUERYBUF
Jan 06 01:40:20 raspberrypi kernel: bcm2835-v4l2: Failed to enable capture port - error -28. Disabling camera port again
Jan 06 01:40:20 raspberrypi kernel: video0: VIDIOC_STREAMON: error -1

UPDATE: Per suggestion - I tried increasing the buffer count to 3 - but it did not help:

struct v4l2_requestbuffers bufrequest;
bufrequest.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
bufrequest.memory = V4L2_MEMORY_MMAP;
bufrequest.count = 3;
if(ioctl(fd, VIDIOC_REQBUFS, &bufrequest) < 0){
 perror("VIDIOC_REQBUFS");
 exit(1);
}
fprintf(stderr,"Buffer Request is for 3 buffers...\n");

• jdonald: Your program runs fine for me (my different camera so different resolution, added loop exit condition, added printout to confirm buffers). Questions (please edit your post to clarify): 1. Can you check that your user pi is in video according to /etc/group? Chances are it is as that's the Raspbian default and if it weren't you'd get open: Permission denied, but good to confirm. 2. In your post you wrote you're getting Operation not permitted (EPERM) but at the end you said "Permission Denied" (EACCES) in quotes. Could you clarify? 3. guvcview and cheese also work fine as non-root?
• PhilBot: @jdonald - both root and pi are in the video group. Also, the error is "Operation not permitted", when the VIDIOC_STREAMON ioctl is executed. The weird thing I don't understand is that the v4l2-ctl application successfully saves a good looking JPEG from both users.
• jdonald: Thanks. I looked through bcm2835-camera.c and v4l2-ioctl.c but it's unlikely you're hitting either of those EPERM cases. Can you compare the strace output of v4l2-ctl --stream-mmap... vs your program (starting from open("/dev/video0", O_RDWR) down to VIDIOC_STREAMON)? My USB camera works in both cases, but I notice that v4l2-ctl swaps the order of the first VIDIOC_QBUF with VIDIOC_STREAMON.
• PhilBot: @jdonald - thank you for the help. Your suggestion to check strace allowed me to see the libc errors I've updated my post with. I cross compiled the app with the official Raspberry Pi toolchain and a Buildroot build. It seems those files do not exist in my buildroot build however. Do you have any suggestions? Thanks for the help!
• jdonald: Those errors are just localization for perror() and those happen after the EPERM anyway. I also realized that ordering of VIODIC_QBUF/VIDIOC_STREAMON may be an artifact of strace order inaccuracy. Next: somefile.jpg is viewable, but if you run file somefile.jpg is it the requested resolution 2592x1944? And 2592x1944 is for Camera Module V1 but your v4l2-ctl --list-formats-ext indicates this is Camera Module V2 (3280x2464), plus you said it's the Sony module. Strange if wrong resolution has different behavior on root vs video, but might as well fix your code and rule it out?

Answer 1

We have a case of a program that runs without error on some cameras (such as mine), but gives an error specifically with Raspberry Pi Camera Module V2. We can track this down starting with the dmesg error:

Jan 06 01:40:20 raspberrypi kernel: bcm2835-v4l2: Failed to enable capture port - error -28. Disabling camera port again

Error 28 (ENOSPC) is "No space left on device". Grepping the kernel, the code flagging this error is in bcm2835/mmal-vchiq.cpp:

 /* ensure there are enough buffers queued to cover the buffer headers */
 if (port->buffer_cb != NULL) {
 hdr_count = 0;
 list_for_each(buf_head, &port->buffers) {
 hdr_count++;
 }
 if (hdr_count < port->current_buffer.num)
 return -ENOSPC;
 }

So this driver expects all buffers to have been allocated by the time VIDIOC_STREAMON is called. That wasn't the case in the original code, while you'll see that VIDIOC_QBUF comes before VIDIOC_STREAMON in the strace log for v4l2-ctl --stream-mmap... which does work.

However, there's an additional mismatch in v4l2-test.cpp's strace log:

ioctl(3, VIDIOC_REQBUFS, {count=1, type=V4L2_BUF_TYPE_VIDEO_CAPTURE, memory=V4L2_MEMORY_MMAP}) = 0 ({count=3})

That means although the program requests a single buffer, V4L2 is insisting on three buffers! Perhaps the code was originally designed for a different camera, one like my own that permits bufrequest.count = 1 as is.

This revised program creates a loop to prepare all three buffers. To allocate and queue each one, we do VIDIOC_QUERYBUF, mmap(), then VIDIOC_QBUF. Once all buffers are set up, we can start the loop that pulls camera images from the circular buffer.

Another way of looking at the revised code: it runs a sequence very similar to what's found in the strace log for v4l2-ctl --stream-mmap.... Comparing the strace logs did initially bring red herrings such as extra linked libraries or O_LARGEFILE. Fortunately once those were ruled out, the ordering of ioctl() and other system calls provided a useful reference implementation.

Answer 2

I was getting VIDIOC_STREAMON: Operation not permitted while using cv2.VideoCapture(0). Setting the GPU memory to a higher value, solved the problem.

enter image description here

Answer 3

Adding to Pedro Lobito's response:

While I was testing at the command line using v4l2-ctl to perform test captures a lot of the supported pixelformats failed with "Operation not permitted" when trying for the full resolution supported by the chip 3280x2464 or just hang indefinitely with no error message and restarting the terminal and running any v4l2-ctl commands would from then on hang requiring a reboot of the Pi to fix.

But increasing the GPU memory in the "Raspberry Pi Configuration" to 256 fixed the "Operation not permitted" and the hanging for me.

I was using the Raspberry Pi Camera Module V2.