[Feature request] Record video with GStreamer using a callback. #1

sblantipodi commented 2023年05月31日 12:55:23 +02:00 (Migrated from github.com)

Hello,
is there any plan of adding some extensive examples with GStreamer?
A good examples could be recording the screen with GStreamer using a callback.

The libs seems so cool but it's a bit hard to use without proper examples :)

Thanks
Davide

Hello, is there any plan of adding some extensive examples with GStreamer? A good examples could be recording the screen with GStreamer using a callback. The libs seems so cool but it's a bit hard to use without proper examples :) Thanks Davide

jwharm commented 2023年05月31日 15:00:01 +02:00 (Migrated from github.com)

Hi Davide,

That would certainly be a cool example (though supporting multiple operating systems and windowing systems (Xorg/Wayland) would probably go a bit too far).

I'm currently working on a few other things though, but I'll keep it in mind.

Regards,
Jan-Willem

Hi Davide, That would certainly be a cool example (though supporting multiple operating systems and windowing systems (Xorg/Wayland) would probably go a bit too far). I'm currently working on a few other things though, but I'll keep it in mind. Regards, Jan-Willem

sblantipodi commented 2023年05月31日 15:24:28 +02:00 (Migrated from github.com)

@jwharm thanks for the answer and for considering my "feature request".
Please take in mind that GStreamer uses the same interfaces for screen capture on Windows, Linux and Mac.

This are some examples on Windows and Linux.

Windows
./gst-launch-1.0 d3d11screencapturesrc ! d3d11convert ! d3d11download ! autovideosink

Linux
./gst-launch-1.0 ximagesrc ! videoscale ! videoconvert ! autovideosink

as you can see the pipelines are a bit different but the interface from the bindings should be identical.

@jwharm thanks for the answer and for considering my "feature request". Please take in mind that GStreamer uses the same interfaces for screen capture on Windows, Linux and Mac. This are some examples on Windows and Linux. Windows `./gst-launch-1.0 d3d11screencapturesrc ! d3d11convert ! d3d11download ! autovideosink` Linux `./gst-launch-1.0 ximagesrc ! videoscale ! videoconvert ! autovideosink` as you can see the pipelines are a bit different but the interface from the bindings should be identical.

jwharm commented 2023年06月24日 22:39:20 +02:00 (Migrated from github.com)

@sblantipodi I added a "screen recorder" example with a pipeline that contains a Tee that splits the stream into a Vorbis encoder and saves to file, and a second stream that goes to an AppSink that allows you to do custom operations on the stream data.

It is currently in the "screen-recorder" branch:
https://github.com/jwharm/java-gi-examples/tree/screen-recorder

If you clone the java-gi-exapmles repo, checkout the "screen-recorder" branch, navigate to the ScreenRecorder folder and run gradle run the app should record 5 seconds of screentime, and display an asterisk every time a buffer sample is received in the appsink callback function.

Hope this helps

@sblantipodi I added a "screen recorder" example with a pipeline that contains a Tee that splits the stream into a Vorbis encoder and saves to file, and a second stream that goes to an AppSink that allows you to do custom operations on the stream data. It is currently in the "screen-recorder" branch: https://github.com/jwharm/java-gi-examples/tree/screen-recorder If you clone the java-gi-exapmles repo, checkout the "screen-recorder" branch, navigate to the `ScreenRecorder` folder and run `gradle run` the app should record 5 seconds of screentime, and display an asterisk every time a buffer sample is received in the appsink callback function. Hope this helps

sblantipodi commented 2023年06月25日 14:43:47 +02:00 (Migrated from github.com)

hi @jwharm, this is a great news, thanks for creating this example.
I can't wait to explore this bindings... it looks awesome.

image

I am trying to gradle run the software but it seems that it can't find the gstreamer lib even if it is installed.

image

do you have any suggestion? thanks!

hi @jwharm, this is a great news, thanks for creating this example. I can't wait to explore this bindings... it looks awesome.  I am trying to gradle run the software but it seems that it can't find the gstreamer lib even if it is installed.  do you have any suggestion? thanks!

jwharm commented 2023年06月25日 22:26:51 +02:00 (Migrated from github.com)

The library libgstreamer-1.0.so cannot be found in the library path.

If you're on a Debian/Ubuntu based distribution, can you please run the following command and let me know the output:

ls /lib/x86_64-linux-gnu/libgstreamer-1.0.so* -l

Or if you're on Fedora, please run:

ls /lib64/libgstreamer-1.0.so* -l

Or if you're using Arch:

ls /usr/lib/libgstreamer-1.0.so* -l

Also, the following info might be useful:

ldd /usr/bin/gst-launch-1.0

The library `libgstreamer-1.0.so` cannot be found in the library path. If you're on a Debian/Ubuntu based distribution, can you please run the following command and let me know the output: `ls /lib/x86_64-linux-gnu/libgstreamer-1.0.so* -l` Or if you're on Fedora, please run: `ls /lib64/libgstreamer-1.0.so* -l` Or if you're using Arch: `ls /usr/lib/libgstreamer-1.0.so* -l` Also, the following info might be useful: `ldd /usr/bin/gst-launch-1.0`

sblantipodi commented 2023年06月26日 12:02:15 +02:00 (Migrated from github.com)

@jwharm

I'm using Ubuntu 23.04

this is the output of the commands:

sblantipodi@DeskStar:~$ ls /lib/x86_64-linux-gnu/libgstreamer-1.0.so* -l
lrwxrwxrwx 1 root root 28 apr 12 17:47 /lib/x86_64-linux-gnu/libgstreamer-1.0.so.0 -> libgstreamer-1.0.so.0.2202.0
-rw-r--r-- 1 root root 1383640 apr 12 17:47 /lib/x86_64-linux-gnu/libgstreamer-1.0.so.0.2202.0
sblantipodi@DeskStar:~$ ldd /usr/bin/gst-launch-1.0
	linux-vdso.so.1 (0x00007ffd8e131000)
	libgstreamer-1.0.so.0 => /lib/x86_64-linux-gnu/libgstreamer-1.0.so.0 (0x00007fd563e6c000)
	libglib-2.0.so.0 => /lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007fd563d25000)
	libgobject-2.0.so.0 => /lib/x86_64-linux-gnu/libgobject-2.0.so.0 (0x00007fd563cc4000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd563a00000)
	libgmodule-2.0.so.0 => /lib/x86_64-linux-gnu/libgmodule-2.0.so.0 (0x00007fd563cbd000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fd563917000)
	libunwind.so.8 => /lib/x86_64-linux-gnu/libunwind.so.8 (0x00007fd563c9f000)
	libdw.so.1 => /lib/x86_64-linux-gnu/libdw.so.1 (0x00007fd56386c000)
	libpcre2-8.so.0 => /lib/x86_64-linux-gnu/libpcre2-8.so.0 (0x00007fd563c05000)
	libffi.so.8 => /lib/x86_64-linux-gnu/libffi.so.8 (0x00007fd563861000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fd563fde000)
	liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007fd56382d000)
	libelf.so.1 => /lib/x86_64-linux-gnu/libelf.so.1 (0x00007fd563810000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fd5637f2000)
	libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007fd5637df000)

@jwharm I'm using Ubuntu 23.04 this is the output of the commands: ``` sblantipodi@DeskStar:~$ ls /lib/x86_64-linux-gnu/libgstreamer-1.0.so* -l lrwxrwxrwx 1 root root 28 apr 12 17:47 /lib/x86_64-linux-gnu/libgstreamer-1.0.so.0 -> libgstreamer-1.0.so.0.2202.0 -rw-r--r-- 1 root root 1383640 apr 12 17:47 /lib/x86_64-linux-gnu/libgstreamer-1.0.so.0.2202.0 sblantipodi@DeskStar:~$ ldd /usr/bin/gst-launch-1.0 linux-vdso.so.1 (0x00007ffd8e131000) libgstreamer-1.0.so.0 => /lib/x86_64-linux-gnu/libgstreamer-1.0.so.0 (0x00007fd563e6c000) libglib-2.0.so.0 => /lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007fd563d25000) libgobject-2.0.so.0 => /lib/x86_64-linux-gnu/libgobject-2.0.so.0 (0x00007fd563cc4000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd563a00000) libgmodule-2.0.so.0 => /lib/x86_64-linux-gnu/libgmodule-2.0.so.0 (0x00007fd563cbd000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fd563917000) libunwind.so.8 => /lib/x86_64-linux-gnu/libunwind.so.8 (0x00007fd563c9f000) libdw.so.1 => /lib/x86_64-linux-gnu/libdw.so.1 (0x00007fd56386c000) libpcre2-8.so.0 => /lib/x86_64-linux-gnu/libpcre2-8.so.0 (0x00007fd563c05000) libffi.so.8 => /lib/x86_64-linux-gnu/libffi.so.8 (0x00007fd563861000) /lib64/ld-linux-x86-64.so.2 (0x00007fd563fde000) liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007fd56382d000) libelf.so.1 => /lib/x86_64-linux-gnu/libelf.so.1 (0x00007fd563810000) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fd5637f2000) libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007fd5637df000) ```

jwharm commented 2023年06月26日 14:51:52 +02:00 (Migrated from github.com)

@sblantipodi Java-GI expects the soname libgstreamer-1.0.so, without the .0 extension. I will log an issue to change the library loader to consider the .0 soname; in the meantime, you can create a symlink as a workaround:

cd /lib/x86_64-linux-gnu
sudo ln -s libgstreamer-1.0.so.0 libgstreamer-1.0.so

Let me know if you encounter other issues!

@sblantipodi Java-GI expects the soname `libgstreamer-1.0.so`, without the `.0` extension. I will log an issue to change the library loader to consider the `.0` soname; in the meantime, you can create a symlink as a workaround: ``` cd /lib/x86_64-linux-gnu sudo ln -s libgstreamer-1.0.so.0 libgstreamer-1.0.so ``` Let me know if you encounter other issues!

sblantipodi commented 2023年06月27日 21:31:32 +02:00 (Migrated from github.com)

@jwharm

done it but same error

sblantipodi@DeskStar:/lib/x86_64-linux-gnu$ ls -la libgstre*
lrwxrwxrwx 1 root root 21 giu 27 21:27 libgstreamer-1.0.so -> libgstreamer-1.0.so.0
lrwxrwxrwx 1 root root 28 apr 12 17:47 libgstreamer-1.0.so.0 -> libgstreamer-1.0.so.0.2202.0
-rw-r--r-- 1 root root 1383640 apr 12 17:47 libgstreamer-1.0.so.0.2202.0

@jwharm done it but same error ``` sblantipodi@DeskStar:/lib/x86_64-linux-gnu$ ls -la libgstre* lrwxrwxrwx 1 root root 21 giu 27 21:27 libgstreamer-1.0.so -> libgstreamer-1.0.so.0 lrwxrwxrwx 1 root root 28 apr 12 17:47 libgstreamer-1.0.so.0 -> libgstreamer-1.0.so.0.2202.0 -rw-r--r-- 1 root root 1383640 apr 12 17:47 libgstreamer-1.0.so.0.2202.0 ```

jwharm commented 2023年06月28日 20:03:46 +02:00 (Migrated from github.com)

I reproduced the issue on an Ubuntu install.

It seems that the folder /lib/x86_64-linux-gnu is not on the Java library path by default.

I added it to the Java library path in the build.gradle with the -Djava.library.path= option.
Change this block:

tasks.withType(JavaExec) {
 jvmArgs += "--enable-preview"
 jvmArgs += "--enable-native-access=ALL-UNNAMED"

To this:

tasks.withType(JavaExec) {
 jvmArgs += "--enable-preview"
 jvmArgs += "--enable-native-access=ALL-UNNAMED"
 jvmArgs += "-Djava.library.path=/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib:/lib/x86_64-linux-gnu"

Be aware that every example has its own build.gradle file, so if you want to run the other examples, you will have to change their build files too.

Afterwards, there were a bunch of missing dependencies. I solved that by installing all packages that are listed on this page under the "Install GStreamer on Ubuntu or Debian" header.

After that, it worked for me.

Let me know if it works!

I reproduced the issue on an Ubuntu install. It seems that the folder `/lib/x86_64-linux-gnu` is not on the Java library path by default. I added it to the Java library path in the `build.gradle` with the `-Djava.library.path=` option. Change this block: ``` tasks.withType(JavaExec) { jvmArgs += "--enable-preview" jvmArgs += "--enable-native-access=ALL-UNNAMED" ``` To this: ``` tasks.withType(JavaExec) { jvmArgs += "--enable-preview" jvmArgs += "--enable-native-access=ALL-UNNAMED" jvmArgs += "-Djava.library.path=/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib:/lib/x86_64-linux-gnu" ``` Be aware that every example has its own `build.gradle` file, so if you want to run the other examples, you will have to change their build files too. Afterwards, there were a bunch of missing dependencies. I solved that by installing all packages that are listed on [this page](https://gstreamer.freedesktop.org/documentation/installing/on-linux.html?gi-language=c) under the "Install GStreamer on Ubuntu or Debian" header. After that, it worked for me. Let me know if it works!

sblantipodi commented 2023年06月30日 10:55:13 +02:00 (Migrated from github.com)

hi @jwharm thanks for the answer and for the awesome work here...
it's amazing how new this binding is and what great results you achieved here...

in this way it works, I'm going to explore this binding in depth, so please expect some questions :)

are you considering to support Windows too?

the corresponding ximagesrc for windows is d3d11screencapturesrc

hi @jwharm thanks for the answer and for the awesome work here... it's amazing how new this binding is and what great results you achieved here... in this way it works, I'm going to explore this binding in depth, so please expect some questions :) are you considering to support Windows too? the corresponding ximagesrc for windows is d3d11screencapturesrc

jwharm commented 2023年06月30日 23:32:52 +02:00 (Migrated from github.com)

Just create a d3d11screencapturesrc Element and use it instead of the ximagesrc.

Just create a d3d11screencapturesrc Element and use it instead of the ximagesrc.

sblantipodi commented 2023年07月05日 11:39:29 +02:00 (Migrated from github.com)

this is a very greats news, I need some days to try it out because I'm very busy this period but I can't wait to find out more about this bindings... I'll report back if I find any problems... thank you for your great work.

this is a very greats news, I need some days to try it out because I'm very busy this period but I can't wait to find out more about this bindings... I'll report back if I find any problems... thank you for your great work.

jwharm commented 2023年07月05日 14:04:30 +02:00 (Migrated from github.com)

OK, I will keep the issue ticket open for now.

OK, I will keep the issue ticket open for now.

sblantipodi commented 2023年07月17日 10:27:22 +02:00 (Migrated from github.com)

Hi @jwharm,
I am trying to run the screenrecorder demo on Windows 11.

When I run the gradlew.bat I have this output:

C:\dev\dpsoftware\java-gi-examples\ScreenRecorder>gradlew.bat
Starting a Gradle Daemon, 3 incompatible and 47 stopped Daemons could not be reused, use --status for details
> Task :help
Welcome to Gradle 8.0.
To run a build, run gradlew <task> ...
To see a list of available tasks, run gradlew tasks
To see more detail about a task, run gradlew help --task <task>
To see a list of command-line options, run gradlew --help
For more detail on using Gradle, see https://docs.gradle.org/8.0/userguide/command_line_interface.html
For troubleshooting, visit https://help.gradle.org
BUILD SUCCESSFUL in 2s
1 actionable task: 1 executed
C:\dev\dpsoftware\java-gi-examples\ScreenRecorder>

what I am doing wrong?

Hi @jwharm, I am trying to run the screenrecorder demo on Windows 11. When I run the gradlew.bat I have this output: ``` C:\dev\dpsoftware\java-gi-examples\ScreenRecorder>gradlew.bat Starting a Gradle Daemon, 3 incompatible and 47 stopped Daemons could not be reused, use --status for details > Task :help Welcome to Gradle 8.0. To run a build, run gradlew <task> ... To see a list of available tasks, run gradlew tasks To see more detail about a task, run gradlew help --task <task> To see a list of command-line options, run gradlew --help For more detail on using Gradle, see https://docs.gradle.org/8.0/userguide/command_line_interface.html For troubleshooting, visit https://help.gradle.org BUILD SUCCESSFUL in 2s 1 actionable task: 1 executed C:\dev\dpsoftware\java-gi-examples\ScreenRecorder> ``` what I am doing wrong?

jwharm commented 2023年07月18日 19:32:19 +02:00 (Migrated from github.com)

Try gradlew.bat run

Try `gradlew.bat run`

sblantipodi commented 2023年07月20日 13:03:58 +02:00 (Migrated from github.com)

@jwharm

image

PS C:\dev\dpsoftware\java-gi-examples\ScreenRecorder> gradle --version
------------------------------------------------------------
Gradle 8.2.1
------------------------------------------------------------
Build time: 2023年07月10日 12:12:35 UTC
Revision: a38ec64d3c4612da9083cc506a1ccb212afeecaa
Kotlin: 1.8.20
Groovy: 3.0.17
Ant: Apache Ant(TM) version 1.10.13 compiled on January 4 2023
JVM: 20.0.1 (Eclipse Adoptium 20.0.1+9)
OS: Windows 11 10.0 amd64
PS C:\dev\dpsoftware\java-gi-examples\ScreenRecorder> 

should I use a different gradle version?

@jwharm  ``` PS C:\dev\dpsoftware\java-gi-examples\ScreenRecorder> gradle --version ------------------------------------------------------------ Gradle 8.2.1 ------------------------------------------------------------ Build time: 2023年07月10日 12:12:35 UTC Revision: a38ec64d3c4612da9083cc506a1ccb212afeecaa Kotlin: 1.8.20 Groovy: 3.0.17 Ant: Apache Ant(TM) version 1.10.13 compiled on January 4 2023 JVM: 20.0.1 (Eclipse Adoptium 20.0.1+9) OS: Windows 11 10.0 amd64 PS C:\dev\dpsoftware\java-gi-examples\ScreenRecorder> ``` should I use a different gradle version?

jwharm commented 2023年07月20日 13:08:49 +02:00 (Migrated from github.com)

No, the line "One element could not be created. Exiting" is printed from the example app, so gradle is working fine.

You might have to change that piece of code to find out which element exactly is null, and continue from there.

No, the line "One element could not be created. Exiting" [is printed from the example app](https://github.com/jwharm/java-gi-examples/blob/screen-recorder/ScreenRecorder/src/main/java/io/github/jwharm/javagi/examples/screenrec/ScreenRecorder.java#L94), so gradle is working fine. You might have to change that piece of code to find out which element exactly is null, and continue from there.

jwharm commented 2023年07月23日 15:37:50 +02:00 (Migrated from github.com)

@sblantipodi I've setup a matrix room: #java-gi:matrix.org. If you want, you can contact me there.

@sblantipodi I've setup a matrix room: [#java-gi:matrix.org](https://matrix.to/#/#java-gi:matrix.org). If you want, you can contact me there.

jwharm commented 2023年11月08日 20:55:12 +01:00 (Migrated from github.com)

I'm closing this issue; if there are any other questions or issues, please log a new issue.

I'm closing this issue; if there are any other questions or issues, please log a new issue.