Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Go #312

ggerganov started this conversation in Bindings
Go #312
Dec 22, 2022 · 31 comments · 64 replies
Discussion options

General discussion about the go bindings for whisper.cpp

Current state: bindings/go

You must be logged in to vote

Replies: 31 comments 64 replies

Comment options

mutablelogic/go-whisper#5

Can't seem to get the app to transcribe just yet - something I'm doing wrong here? Thanks

You must be logged in to vote
9 replies
Comment options

ok, figured it out I think, I had to create a build folder inside the go package and drop in libwhisper.a there

Comment options

Hi Chris, happy new year. So there is a "Makefile" in the bindings/go directory which will do all of this for you. In that directory, you can run:

  • make all to build the library and examples and put the result in a build temporary folder;
  • make test to run the unit tests
  • make modtidy and make clean to clean things up.

If you are placing your code in the "examples" folder you can build it with:

  • make examples/stream which builds the example and puts the result in the build directory.

I hope that helps!

Comment options

I wanted to use this binding in my project. Also had problem to build it with missing header and cannot find -lwhisper.

I realized the CGO directives in the source files require the header to be in $SRCDIR/../.. and the libwhisper.a in build. So I cloned the repo into my Go package root, compiled the bindings using make whisper in the Go bindings package and then in my root project go.mod added:

replace github.com/ggerganov/whisper.cpp/bindings/go => ./whisper.cpp/bindings/go

Now I was able to compile it program with just go build.

Comment options

Yeah, I've put in a PR #379 now which removes the hard-coded paths from the .go file in preference for setting the C_INCLUDE_PATH and LIBRARY_PATH variables which invoking the go toolchain, and included some documentation to describe this. I hope that helps!

Comment options

thanks, that worked great! Is there any way I can help with the streaming part of this?

@chrisbward I added a very basic streaming example here:

https://github.com/djthorpe/go-whisper/

It creates all sorts of crashes (I need to look into thread safety, I assume it's bacause I am launching goroutines where I shouldn't). But may help you a bit more.

Comment options

Hello Everybody, First of all thank you very much for all the work that was poured into this.
I am trying to initialise a whisper model using Golang.

I created my build folder with all executables.
I downloaded the models.
I created a dependencies Folder where I added lib whisper.an and whisper.h accordingly.
I created the environment variable that point to the dependencies folder.
With all these done, I still get the following error.

github.com/ggerganov/whisper.cpp/bindings/go
../../go/pkg/mod/github.com/ggerganov/whisper.cpp/bindings/go@v0.0.0-20230116195735-49b529ba74f3/params.go:11:10: fatal error: 'whisper.h' file not found
#include <whisper.h>

Screenshot 2023年01月17日 at 13 10 09

Any advice/piece of help is very much appreciated.

You must be logged in to vote
7 replies
Comment options

As you can notice, in the dependencies folder on the left I stored bot .h and .a files.
Also, given the tree on the left, the path should be correct..

Comment options

I must mention that, even if I try to run the package tests from the original repository, with the original path, it fails from the same reason..

Comment options

Thanks @mbaxamb3333 can you run the following and see what happens, in a fresh empty directory:

git clone https://github.com/ggerganov/whisper.cpp.git
cd whisper.cpp/bindings/go
make test

I'd be interested to see if that compiles. If it doesn't work, can you send me the output?

Comment options

On OSX, following those steps, I get

 $ make all
Clean
Mkdir build
Mkdir models
Build whisper
make[1]: *** No rule to make target `libwhisper.a'. Stop.
make: *** [whisper] Error 2
Comment options

On OSX, following those steps, I get

 $ make all
Clean
Mkdir build
Mkdir models
Build whisper
make[1]: *** No rule to make target `libwhisper.a'. Stop.
make: *** [whisper] Error 2

I encountered the same error, and it seems that the Go bindings are no longer being maintained.

Comment options

Screenshot 2023年01月17日 at 14 36 04

When running the command from the makefile everything works fine.
Although If I try to run, within whisper_test.go, run package tests, I get the same error.

github.com/ggerganov/whisper.cpp/bindings/go

/var/folders/y0/dbsx2zd55lz31wt__nmdgm840000gn/T/go-build1048629891/b084/params.cover.go:11:10: fatal error: 'whisper.h' file not found
#include <whisper.h>
^~~~~~~~~~~
1 error generated.
FAIL github.com/ggerganov/whisper.cpp/bindings/go [build failed]
FAIL

You must be logged in to vote
1 reply
Comment options

Great. In that case, I would take a look at the Makefile in the bindings/go directory and see what it is doing differently to make it work. The most important part of the makefile is as follows:

C_INCLUDE_PATH=${INCLUDE_PATH} LIBRARY_PATH=${LIBRARY_PATH} go build <folder>

It's really just important that INCLUDE_PATH contains the file whisper.h and the LIBRARY_PATH contains the file libwhisper.a - these should be on the same line as the "go build" command.

Comment options

Golang bindings are 45 times slower than the C++ binary when transcoding samples/jfk.wav using the ggml-tiny.en.bin model.

C++ binary Golang bindings
1.428s 63.919s

C++ and Golang examples are compiled following the readme. I haven't profiled the Golang bindings yet.

Raw results

C++ binary

time ./main -m ./models/ggml-tiny.en.bin -f ./samples/jfk.wav
whisper_init_from_file: loading model from './models/ggml-tiny.en.bin'
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 384
whisper_model_load: n_audio_head = 6
whisper_model_load: n_audio_layer = 4
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 384
whisper_model_load: n_text_head = 6
whisper_model_load: n_text_layer = 4
whisper_model_load: n_mels = 80
whisper_model_load: f16 = 1
whisper_model_load: type = 1
whisper_model_load: mem required = 387.00 MB (+ 3.00 MB per decoder)
whisper_model_load: kv self size = 2.62 MB
whisper_model_load: kv cross size = 8.79 MB
whisper_model_load: adding 1607 extra tokens
whisper_model_load: model ctx = 73.58 MB
whisper_model_load: model size = 73.54 MB
system_info: n_threads = 4 / 16 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 | 
main: processing './samples/jfk.wav' (176000 samples, 11.0 sec), 4 threads, 1 processors, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:07.740] And so my fellow Americans ask not what your country can do for you
[00:00:07.740 --> 00:00:10.740] ask what you can do for your country
whisper_print_timings: load time = 186.77 ms
whisper_print_timings: mel time = 83.82 ms
whisper_print_timings: sample time = 16.98 ms
whisper_print_timings: encode time = 791.23 ms / 197.81 ms per layer
whisper_print_timings: decode time = 289.46 ms / 72.36 ms per layer
whisper_print_timings: total time = 1374.19 ms
real 0m1.428s
user 0m4.208s
sys 0m0.280s

Golang bindings

time ./build/go-whisper -model ./models/ggml-tiny.en.bin samples/jfk.wav
whisper_init_from_file: loading model from './models/ggml-tiny.en.bin'
whisper_model_load: loading model
whisper_model_load: n_vocab = 51864
whisper_model_load: n_audio_ctx = 1500
whisper_model_load: n_audio_state = 384
whisper_model_load: n_audio_head = 6
whisper_model_load: n_audio_layer = 4
whisper_model_load: n_text_ctx = 448
whisper_model_load: n_text_state = 384
whisper_model_load: n_text_head = 6
whisper_model_load: n_text_layer = 4
whisper_model_load: n_mels = 80
whisper_model_load: f16 = 1
whisper_model_load: type = 1
whisper_model_load: mem required = 387.00 MB (+ 3.00 MB per decoder)
whisper_model_load: kv self size = 2.62 MB
whisper_model_load: kv cross size = 8.79 MB
whisper_model_load: adding 1607 extra tokens
whisper_model_load: model ctx = 73.58 MB
whisper_model_load: model size = 73.54 MB
Loading "./samples/jfk.wav"
 ...processing "./samples/jfk.wav"
[ 0s-> 7.74s] And so my fellow Americans ask not what your country can do for you
[ 7.74s->10.74s] ask what you can do for your country
real 1m3.919s
user 4m7.851s
sys 0m6.771s

Related issue: #421

You must be logged in to vote
4 replies
Comment options

Could this hard coded processors = 0 contributes to the slowness? Any reason we cannot use Whisper_full_parallel at this moment?

image

Comment options

I made three samples and tested their performance on MLK's full "I have a dream" speech:

  • binding libwhisper.so directly and using cgo: 24s
  • using the "low-level" bindings: 31s
  • using the "high-level" bindings: 55s

So there's a significant cost to using either of the official bindings. I'm not sure where the cost comes in on the low-level bindings, because my cgo looks extremely similar to what it's doing, but it's repeatably much faster.

All programs are available here (I've linked directly to the version I used for these tests)

The default example program gives 27s (!) when run with time ./whisper-cpp -m ~/.local/share/blisper/ggml-small.bin samples/mlk_ihaveadream_long.wav, which should be the roughly equivalent call. Not sure why my cgo program is running faster than it.

Comment options

@jaybinks on a 53 minute gil strang lecture, with the large model this time, on my Mac OS laptop, plugged in:

whisper-cpp main (4-thread default): 14:15
whisper-cpp main (8 threads): 18:45
binding libwhisper.so directly, 2 CPUs (4-thread default): 12:38
binding libwhisper.so directly, 1 CPU (4-thread default): 14:25
low-level bindings: 62:28
high-level bindings: I cut it off because it didn't see to be going. very long.

system_info from whisper-cpp:

system_info: n_threads = 4 / 10 | AVX = 0 | AVX2 = 0 | AVX512 = 0 | FMA = 0 | NEON = 1 | ARM_FMA = 1 | F16C = 0 | FP16_VA = 1 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 0 | VSX = 0 | COREML = 0 | 
main: processing 'samples/strang.wav' (51080185 samples, 3192.5 sec), 4 threads, 1 processors, lang = en, task = transcribe, timestamps = 1
whisper_print_timings: load time = 1080.02 ms
whisper_print_timings: fallbacks = 9 p / 5 h
whisper_print_timings: mel time = 3814.76 ms
whisper_print_timings: sample time = 4009.11 ms / 9769 runs ( 0.41 ms per run)
whisper_print_timings: encode time = 446511.75 ms / 126 runs ( 3543.74 ms per run)
whisper_print_timings: decode time = 390127.31 ms / 9760 runs ( 39.97 ms per run)
whisper_print_timings: total time = 845733.56 ms
./whisper-cpp -m ~/.local/share/blisper/ggml-large.bin samples/strang.wav 3471.29s user 36.83s system 414% cpu 14:05.80 total

I noticed that whisper-cpp spread the work across all my processors but kept them only about half-utilized. Possibly because it had only 4 threads on my 8-processor machine?

With 8 threads, it clearly does a better job of keeping the processors busy, but actually took longer (?):

whisper_print_timings: load time = 1570.18 ms
whisper_print_timings: fallbacks = 9 p / 5 h
whisper_print_timings: mel time = 2104.77 ms
whisper_print_timings: sample time = 4008.01 ms / 9769 runs ( 0.41 ms per run)
whisper_print_timings: encode time = 672592.88 ms / 126 runs ( 5338.04 ms per run)
whisper_print_timings: decode time = 445225.22 ms / 9760 runs ( 45.62 ms per run)
whisper_print_timings: total time = 1125743.62 ms
./whisper-cpp --threads 8 -m ~/.local/share/blisper/ggml-large.bin 8434.05s user 43.55s system 753% cpu 18:45.81 total

system_info from sotest (custom bindings):

2 CPUs

AVX = 0 | AVX2 = 0 | AVX512 = 0 | FMA = 0 | NEON = 1 | ARM_FMA = 1 | F16C = 0 | FP16_VA = 1 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 0 | VSX = 0 | COREML = 0 | 
./sotest -model large ../samples/strang.wav /dev/stdout 5336.24s user 79.33s system 714% cpu 12:37.85 total

1 CPU

./sotest -model large ../samples/strang.wav /dev/stdout 3446.45s user 49.32s system 403% cpu 14:25.43 total

low-level bindings

With the "low-level" bindings 62m 28s:
./lltest -model large ../samples/strang.wav /dev/stdout 7388.36s user 90.48s system 199% cpu 1:02:27.94 total

Hardware and OS info:

$ sw_vers && system_profiler SPHardwareDataType  9:45AM
ProductName:		macOS
ProductVersion:		13.4
BuildVersion:		22F66
Hardware:
 Hardware Overview:
 Model Name: MacBook Pro
 Model Identifier: MacBookPro18,4
 Model Number: Z15H00107LL/A
 Chip: Apple M1 Max
 Total Number of Cores: 10 (8 performance and 2 efficiency)
 Memory: 32 GB
Comment options

For kicks, I edited the bindings to remove the progress callbacks on the theory that that caused frequent calls across the go -> C boundary, and that dropped the time for the low-level bindings to 14:19 from 62:28 - pretty conclusive evidence that they are a major issue.

Here's the diff:

--- whisper.go	2023年06月26日 08:49:04
+++ whisper.2.go	2023年06月29日 15:43:00
@@ -326,10 +326,10 @@
 // It seems this approach can offer some speedup in some cases.
 // However, the transcription accuracy can be worse at the beginning and end of each chunk.
 func (ctx *Context) Whisper_full_parallel(params Params, samples []float32, processors int, encoderBeginCallback func() bool, newSegmentCallback func(int)) error {
-	registerEncoderBeginCallback(ctx, encoderBeginCallback)
-	registerNewSegmentCallback(ctx, newSegmentCallback)
-	defer registerEncoderBeginCallback(ctx, nil)
-	defer registerNewSegmentCallback(ctx, nil)
+	// registerEncoderBeginCallback(ctx, encoderBeginCallback)
+	// registerNewSegmentCallback(ctx, newSegmentCallback)
+	// defer registerEncoderBeginCallback(ctx, nil)
+	// defer registerNewSegmentCallback(ctx, nil)
 
 	if C.whisper_full_parallel((*C.struct_whisper_context)(ctx), (C.struct_whisper_full_params)(params), (*C.float)(&samples[0]), C.int(len(samples)), C.int(processors)) == 0 {
 		return nil
Comment options

Can we reuse the same model instance to process multiple wav files which are in different languages?

I have 20 wav files which are in different languages and totally unrelated. I create one model using a multi-lang model, and then create a new Context using model.NewContext() to process each audio file, with language auto-detection turned on for each context. The transcription for most of the files are sane, but 2 or 3 of them are just completely rambles, even though the language were correctly detected.

So my question is - are we supposedly to reuse one model and just create a new context for each new wav file? My understanding is that the model inference is somewhat stateful - would the first wav file / context influence the second wav file /context in some way?

Btw thanks for building such a great project!

You must be logged in to vote
2 replies
Comment options

The C api does not have the concept of model. It only has whisper_context created by WHISPER_API struct whisper_context * whisper_init_from_file(const char * path_model);. However in Go, we create a Model instance by loading the model file, we can then use Model.NewContext() to create one or more contexts - this encourages reusing of the same model instance, which seem consistent with the C api?

Comment options

ggerganov Feb 4, 2023
Maintainer Author

I haven't looked into the Go API in details, but regarding the C API, you should be able to achieve what you need by simply setting params.no_context = true when calling whisper_full():

https://github.com/ggerganov/whisper.cpp/blob/86ef64a8555e2fb11d47bf17bf7a208501006a4c/whisper.h#L248

This will tell whisper to ignore any past transcriptions (i.e. clear the state).

Comment options

I'm right that we can't do something like this

for i := 0; i < 10; i++{
 go func(){ // new goroutine
 ctx := whisper.NewContext()
 ctx.Process(...)
 }()
}

I'm having a data race error right now.

You must be logged in to vote
2 replies
Comment options

2yrs in the future and i'm stuck on the same issue, according to the documentation (cpp) multi threading on one model should be possible if you use different contexts, and is unstable when doing it in one context.
But the above piece of code doesn't work, go crashes when two contexts are busy at the same time..

Comment options

Hi, the above code might work if there was a "WaitGroup" in there? What is the crash you're getting?

var wg sync.WaitGroup
for i := 0; i < 10; i++{
 wg.Add(1)
 go func(){ // new goroutine
 defer wg.Done()
 ctx := whisper.NewContext()
 ctx.Process(...)
 }()
}
// Wait for all goroutines to finish
wg.Wait()
Comment options

Any qualms against including pre-compiled static libraries for the go bindings (if not more?). The https://github.com/rogchap/v8go is a nice demonstration of how doing prevents having to worry about the c toolchain and the library can effectively be treated as a native go library.

See https://github.com/rogchap/v8go/blob/master/cgo.go and https://github.com/rogchap/v8go/blob/master/.github/workflows/v8build.yml for how they build those static libraries.

You must be logged in to vote
2 replies
Comment options

lol such good sense, I agree

Comment options

does make it a little more platform restricted...I can see why it'd be good to at least make sure the full compile works.

Comment options

I've done similar to what your saying here. https://github.com/jaybinks/goConstmeWhisper And yea, I kinda agree. This allows you to simply download a binary .so of whisper without being required to build it yourself. That being said, it's also not super feasible since your going to get a very unoptimized whisper library
...
On Sun, 25 June 2023, 9:00 am Travis Cline, ***@***.***> wrote: Any qualms against including pre-compiled libraries for the go bindings (if not more?). The https://github.com/rogchap/v8go is a nice demonstration of how doing prevents having to worry about the c toolchain and the library can effectively be treated as a native go library. See https://github.com/rogchap/v8go/blob/master/cgo.go and https://github.com/rogchap/v8go/blob/master/.github/workflows/v8build.yml for how they build those binaries. — Reply to this email directly, view it on GitHub <#312 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AALQR67V4XJG7HTBBHL6GUDXM5WRBANCNFSM6AAAAAATG4MHKQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.*** com>
You must be logged in to vote
0 replies
Comment options

I've done similar to what your saying here. https://github.com/jaybinks/goConstmeWhisper And yea, I kinda agree. This allows you to simply download a binary .so of whisper without being required to build it yourself. That being said, it's also not super feasible since your going to get a very unoptimized whisper library

That doesn't seem to match my definition of 'infeasible' -- can you elaborate?

You must be logged in to vote
0 replies
Comment options

It would be interesting to see how your performance observations scale with much longer audio durations. or, simply looping over the same audio multiple times after loading the model etc. My gut says the difference must be memory copies vs pointers or something like this.
...
On 2023年6月29日 at 03:11, Bill Mill ***@***.***> wrote: I made three samples and tested their performance on MLK's full "I have a dream" speech: - binding libwhisper.so directly and using cgo: 24s - using the "low-level" bindings: 31s - using the "high-level" bindings: 55s So there's a significant cost to using either of the official bindings. I'm not sure where the cost comes in on the low-level bindings, because my cgo looks extremely similar to what it's doing, but it's repeatably much faster. All programs are available here <https://github.com/llimllib/blisper/tree/abad02796b6edd51fb01a9e529a46fd4420b6d85> (I've linked directly to the version I used for these tests) — Reply to this email directly, view it on GitHub <#312 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AALQR6YIEVHHHBRGH3QUCFDXNRQVTANCNFSM6AAAAAATG4MHKQ> . You are receiving this because you commented.Message ID: ***@***.***>
-- Sincerely Jay
You must be logged in to vote
0 replies
Comment options

I had read that the go > boundary was a choke point , but never in my wildest dreams did I ever consider it was THIS bad. WOW . Just for kicks, how does your sotest / lltest compare against the native C CLI app on the same file / system / settings ??
...
On 2023年6月30日 at 05:44, Bill Mill ***@***.***> wrote: For kicks, I edited the bindings to remove the progress callbacks on the theory that that caused frequent calls across the go -> C boundary, and that dropped the time for the low-level bindings to 14:19 from 62:28 - pretty conclusive evidence that they are a major issue. Here's the diff: --- whisper.go 2023年06月26日 08:49:04+++ whisper.2.go 2023年06月29日 15:43:00@@ -326,10 +326,10 @@ // It seems this approach can offer some speedup in some cases. // However, the transcription accuracy can be worse at the beginning and end of each chunk. func (ctx *Context) Whisper_full_parallel(params Params, samples []float32, processors int, encoderBeginCallback func() bool, newSegmentCallback func(int)) error {- registerEncoderBeginCallback(ctx, encoderBeginCallback)- registerNewSegmentCallback(ctx, newSegmentCallback)- defer registerEncoderBeginCallback(ctx, nil)- defer registerNewSegmentCallback(ctx, nil)+ // registerEncoderBeginCallback(ctx, encoderBeginCallback)+ // registerNewSegmentCallback(ctx, newSegmentCallback)+ // defer registerEncoderBeginCallback(ctx, nil)+ // defer registerNewSegmentCallback(ctx, nil) if C.whisper_full_parallel((*C.struct_whisper_context)(ctx), (C.struct_whisper_full_params)(params), (*C.float)(&samples[0]), C.int(len(samples)), C.int(processors)) == 0 { return nil — Reply to this email directly, view it on GitHub <#312 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AALQR62BBVO76TAOQ4RFJ4DXNXLJZANCNFSM6AAAAAATG4MHKQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>
-- Sincerely Jay
You must be logged in to vote
1 reply
Comment options

Sorry I missed this message, github discussions doesn't thread correctly via email it seems, so I got no notification about it.

Using jfk.wav from the whisper.cpp repository, whisper-cpp built with make on commit 3998465, and blisper v0.0.1 from here, on a small file blisper is about 15% slower

$ hyperfine 'blisper -model large jfk.wav /tmp/a.srt'
Benchmark 1: blisper -model large jfk.wav /tmp/a.srt
 Time (mean ± σ): 10.264 s ± 0.244 s [User: 36.698 s, System: 1.280 s]
 Range (min ... max): 9.911 s ... 10.717 s 10 runs
 
$ hyperfine 'whisper-cpp --model ~/.local/share/blisper/ggml-large.bin -of /tmp/a.srt jfk.wav'
Benchmark 1: whisper-cpp --model ~/.local/share/blisper/ggml-large.bin -of /tmp/a.srt jfk.wav
 Time (mean ± σ): 9.025 s ± 0.383 s [User: 26.350 s, System: 6.531 s]
 Range (min ... max): 8.329 s ... 9.428 s 10 runs
Comment options

@ggerganov @djthorpe

When I use make whisper, the console always reports the following error.

Mkdir build
Mkdir models
Build whisper
sysctl: unknown oid 'hw.optional.arm64'
I whisper.cpp build info:
I UNAME_S: Darwin
I UNAME_P: i386
I UNAME_M: x86_64
I CFLAGS: -I. -O3 -DNDEBUG -std=c11 -fPIC -D_DARWIN_C_SOURCE -pthread -mfma -mavx -mavx2 -DGGML_USE_ACCELERATE
I CXXFLAGS: -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -D_DARWIN_C_SOURCE -pthread
I LDFLAGS: -framework Accelerate
I CC: Apple clang version 14.0.0 (clang-1400029.202)
I CXX: Apple clang version 14.0.0 (clang-1400029.202)
make[1]: `libwhisper.a' is up to date.

I have seen the same query raised by someone. issue#597.

I also tried removing CFLAGS += -mf16c. This did not work for me.

Can you please tell me how I should resolve this issue as I want to bind in my own Golang package. It asks me to compile libwhisper.a, that is, to run make whisper in the go/buildings folder.

Could this error be due to the low version of my Mac system? I'm using an Intel Core i7 processor and version 12.6.3.

If so, I'll update my system (it's slow to update and I want to make sure it's not the system version first)

You must be logged in to vote
4 replies
Comment options

When I run make test in the go/buildings folder, it succeeds!

image

Comment options

I'm not an expert, but it seems like that's just a warning and the build is working

Comment options

Thanks for the reply, but make whisper doesn't generate a binary file, so I think there's something wrong with it

Comment options

The execution of make whisper reads :

whisper: mkdir
	@echo Build whisper
	@${MAKE} -C ... /... libwhisper.a

Then in the parent directory, it should generate a target file called libwhisper.a, but it doesn't do what I expect it to do.

Comment options

Could you please tell me how to use flags?

./build/go-whisper -model models/ggml-base.bin samples/speech.wav --language ru

`Loading "--language"
open --language: no such file or directory

system_info: n_threads = 8 / 8 | AVX = 0 | AVX2 = 0 | AVX512 = 0 | FMA = 0 | NEON = 1 | ARM_FMA = 1 | METAL = 0 | F16C = 0 | FP16_VA = 1 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 0 | SSSE3 = 0 | VSX = 0 | COREML = 0 | OPENVINO = 0 |

Loading "ru"
open ru: no such file or directory
`

You must be logged in to vote
1 reply
Comment options

Try ./build/go-whisper -model models/ggml-base.bin -language ru samples/speech.wav

  • flags go before arguments
  • flags only use one dash
Comment options

Hi, I am trying to execute make test but got this error:

# github.com/ggerganov/whisper.cpp/bindings/go.test
/usr/local/go/pkg/tool/darwin_arm64/link: running clang failed: exit status 1
ld: Undefined symbols:
 _MTLCopyAllDevices, referenced from:
 _ggml_metal_init in libwhisper.a[7](ggml-metal.o)
 _MTLCreateSystemDefaultDevice, referenced from:
 _ggml_metal_init in libwhisper.a[7](ggml-metal.o)
 OBJC_CLASS$_MTLComputePassDescriptor, referenced from:
 in libwhisper.a[7](ggml-metal.o)
 OBJC_CLASS$_NSBundle, referenced from:
 in libwhisper.a[7](ggml-metal.o)
 OBJC_CLASS$_NSProcessInfo, referenced from:
 in libwhisper.a[7](ggml-metal.o)
 OBJC_CLASS$_NSString, referenced from:
 in libwhisper.a[7](ggml-metal.o)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
FAIL	github.com/ggerganov/whisper.cpp/bindings/go [build failed]
FAIL
make: *** [test] Error 1
OS: macOS Sonoma v.14
Chip: Apple Mac M1 MAX
Memory: 32 GB
You must be logged in to vote
2 replies
Comment options

I'm getting the same error in different scenario, I used the bindings and I'm running this command:

LIBRARY_PATH=./whisper.cpp C_INCLUDE_PATH=./whisper.cpp go build .

And I get this output:

/opt/homebrew/Cellar/go/1.21.0/libexec/pkg/tool/darwin_arm64/link: running cc failed: exit status 1
ld: Undefined symbols:
 _MTLCopyAllDevices, referenced from:
 _ggml_metal_init in libwhisper.a[7](ggml-metal.o)
 _MTLCreateSystemDefaultDevice, referenced from:
 _ggml_backend_metal_buffer_type_alloc_buffer in libwhisper.a[7](ggml-metal.o)
 _ggml_backend_metal_buffer_type_get_max_size in libwhisper.a[7](ggml-metal.o)
 _ggml_backend_metal_buffer_from_ptr in libwhisper.a[7](ggml-metal.o)
 _ggml_metal_init in libwhisper.a[7](ggml-metal.o)
 _OBJC_CLASS_$_MTLCaptureDescriptor, referenced from:
 in libwhisper.a[7](ggml-metal.o)
 _OBJC_CLASS_$_MTLCaptureManager, referenced from:
 in libwhisper.a[7](ggml-metal.o)
 _OBJC_CLASS_$_MTLCompileOptions, referenced from:
 in libwhisper.a[7](ggml-metal.o)
 _OBJC_CLASS_$_MTLComputePassDescriptor, referenced from:
 in libwhisper.a[7](ggml-metal.o)
 _OBJC_CLASS_$_NSBundle, referenced from:
 in libwhisper.a[7](ggml-metal.o)
 _OBJC_CLASS_$_NSProcessInfo, referenced from:
 in libwhisper.a[7](ggml-metal.o)
 _OBJC_CLASS_$_NSString, referenced from:
 in libwhisper.a[7](ggml-metal.o)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [build] Error 1
Comment options

#1899 Related issue.

Comment options

I've created a PR to setup Makefile env vars properly and got my binding running. #1530

You must be logged in to vote
0 replies
Comment options

Hi , does someone has similar issue? I got it run as a container in Kubernetes cluster (intel CPU) but it's very slow. I suspect no hardware acceleration enabled but not sure how to do that. below is my dockerfile. Thanks

############################
# STEP 1 build libwhisper.a
############################
FROM golang:alpine AS libBuilder
RUN apk add git build-base
WORKDIR /
RUN git clone https://github.com/ggerganov/whisper.cpp.git
WORKDIR /whisper.cpp/bindings/go
RUN make whisper
############################
# STEP 2 build executable binary
############################
FROM golang:alpine AS builder
RUN apk add git build-base
RUN adduser -S myuser -G root -s /bin/bash
COPY . /app/
COPY --from=libBuilder /whisper.cpp/libwhisper.a /app/whisper.cpp2/
COPY --from=libBuilder /whisper.cpp/*.h /app/whisper.cpp2/
WORKDIR /app
RUN C_INCLUDE_PATH="/app/whisper.cpp2/" LIBRARY_PATH="/app/whisper.cpp2/" go test -ldflags "-linkmode 'external' -extldflags '-static'" -v ./...
RUN C_INCLUDE_PATH="/app/whisper.cpp2/" LIBRARY_PATH="/app/whisper.cpp2/" go build -ldflags "-linkmode 'external' -extldflags '-static'" -v -o myapp
############################
# STEP 3 build image based on scratch
############################
FROM scratch
# Copy user
COPY --from=builder /etc/passwd /etc/passwd
# Copy our static executable
COPY --from=builder /app/myapp /bin/myapp
COPY ./models /models
USER myuser
ENTRYPOINT ["/bin/myapp"]
You must be logged in to vote
3 replies
Comment options

Hey, that's a different thing - and very related to your OS and docker setup, not with metal and mac os + arm. In any case look at docker capabilities to enable gpu access - that's very dependent on drivers and so on: https://docs.docker.com/compose/gpu-support/

Comment options

no GPU, just Intel CPU for the inference. I suspect the AVX or F16C capacity is not leveraged by the golang binary.

system_info: n_threads = 24 / 24 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | METAL = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | CUDA = 0 | COREML = 0 | OPENVINO = 0
Comment options

from the log, it looks like the AVX is already enabled when building libwhisper.a
Is any flag need to be set when building the final binary? Thanks.

RUN C_INCLUDE_PATH="/app/whisper.cpp2/" LIBRARY_PATH="/app/whisper.cpp2/" go build -ldflags "-linkmode 'external' -extldflags '-static'" -v -o myapp
#11 [libbuilder 5/6] RUN make whisper
#11 0.172 Mkdir build
#11 0.173 Mkdir models
#11 0.175 Build whisper
#11 0.176 make[1]: Entering directory '/whisper.cpp'
#11 0.191 I whisper.cpp build info: 
#11 0.191 I UNAME_S: Linux
#11 0.191 I UNAME_P: unknown
#11 0.191 I UNAME_M: x86_64
#11 0.191 I CFLAGS: -I. -O3 -DNDEBUG -std=c11 -fPIC -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -pthread -mavx -mavx2 -mfma -mf16c -msse3 -mssse3
#11 0.191 I CXXFLAGS: -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -pthread -mavx -mavx2 -mfma -mf16c -msse3 -mssse3
#11 0.191 I LDFLAGS: 
#11 0.191 I CC: cc (Alpine 13.2.1_git20231014) 13.2.1 20231014
#11 0.191 I CXX: g++ (Alpine 13.2.1_git20231014) 13.2.1 20231014
#11 0.191 
#11 0.192 cc -I. -O3 -DNDEBUG -std=c11 -fPIC -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -pthread -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -c ggml.c -o ggml.o
#11 11.77 cc -I. -O3 -DNDEBUG -std=c11 -fPIC -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -pthread -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -c ggml-alloc.c -o ggml-alloc.o
#11 12.55 cc -I. -O3 -DNDEBUG -std=c11 -fPIC -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -pthread -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -c ggml-backend.c -o ggml-backend.o
#11 13.73 cc -I. -O3 -DNDEBUG -std=c11 -fPIC -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -pthread -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -c ggml-quants.c -o ggml-quants.o
#11 21.87 g++ -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -pthread -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -c whisper.cpp -o whisper.o
#11 37.16 ar rcs libwhisper.a ggml.o ggml-alloc.o ggml-backend.o ggml-quants.o whisper.o
#11 37.17 make[1]: Leaving directory '/whisper.cpp'
#11 DONE 37.3s
Comment options

I have some questions about this golang binding implementation.
First thank you, finaly something that is free and able to be used by selfhosting without any phone-home code or License requirement! :)

I made it so far, that I can use the if on the demo files that come with it. Also one other wav-file gets analysed properly, another wav-file is getting analysed as "Musik" :D even tho it i a normal speech.

So in general I have some questions:

  1. does this support "live-transcription"? So I do not have to pass a wav-file, but a wav-stream to it and it writes the text back to a writer? Currently the text is displayed AFTER all the analyses is getting done, not line by line as it analyses it.
  2. performance is not very good, even with the ggml-small.en.bin (which is the smallest model which is presice enough for me. Is it currently beeing worked on this?
  3. my currently sysinfo (on my dev VPS) is this:
system_info: n_threads = 10 / 10 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | METAL = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | CUDA = 0 | COREML = 0 | OPENVINO = 0

Does it benefit the tools performance, if this is getting run on an Intel CPU with an iGPU?

Thanks in advance!

You must be logged in to vote
3 replies
Comment options

It would also be awesome if it could take all common audio codecs/formats (AC3, DTS, DTS-HD MA, TrueHD, TrueHD Atmos, OGG/OGA, MP4/M4A, ASF/WMA, MP3, FLAC, ALAC, DST, OPUS) of another process, that would be able to take in anything and convert it to WAV, so it can process it :)

At least it would be cool on the long run. So this can be used for analysing every Video/Playbook and eighter create subtitles, or just to index the words it contains.

Comment options

I'm not the author but based on my usage, live streaming would be possible if you buffered chunks (as in each 10s streaming you run the model on a goroutine). Converting from stream to chunks would be on your own. I think that even Context.Process taking a full buffer and its segment/progress callbacks can be wrapped with an io.Reader.
Running the model over the whole wav file takes a long time because it is expensive on CPU and GPU, with impressive improvements depending on your hardware (I got a 10x speed bump testing the same code on a mac M1 vs mac M3), but AFAIK it doesn't benefit of any live tuning/feedback so by my tests you get a pretty decent result chopping a wav file (which you can test with /bin/dd locally).
I figure that while you are streaming if you can detect silence or volume drops it would prevent mid-tokens.
About converting formats I think that the choice is to have the library understand the most basic/available format so we can overlay specialized libraries with their optimization to understand other formats and containers. I've been piping audio from ffmpeg. Hope that helps.

Comment options

You make your Go code read float32 from standard input which is what the whisper model reads

Then feed it from ffmpeg -hide_banner -i i.mp4 -ac 1 -ar 16k -f f32le - | myGoCode
or any other float stream you can come up with
(little endian hardware)

use large-v3-turbo model of 2024-12: it’s ×ばつ better

With Core ML on M, you be ×ばつ at 20% CPU load
— the host can still be used normally or compile things

On Linux Intel cpu whisper will eat all cores 0%id, 2 GiB RAM and still only do ×ばつ
— Quantize no faster

Core ML is ×ばつ and 80% of the cpu better

Comment options

Question: C++/C noobie here. Can we also expose the basic stream implementation in the bindings? Or is that not possible for some reason?

I saw djthorpe has his own Go implementation that's a bit buggy and hard to get running relative to this. Wondering if we can do without another implementation and only maintain this one stream here!

You must be logged in to vote
0 replies
Comment options

Hello trying to follow the Readme but getting this error when I run make test on my m3 macbook air

 "_ggml_view_tensor", referenced from:
 whisper_build_graph_encoder(whisper_context&, whisper_state&) in libwhisper.a[2](whisper.o)
 whisper_build_graph_cross(whisper_context&, whisper_state&) in libwhisper.a[2](whisper.o)
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [examples/go-whisper] Error 1

Any ideas?

You must be logged in to vote
12 replies
Comment options

It looks like it's not linking with the CUDA-version of GGML - so you should build GGML with:

GGML_CUDA=1 make libggml.a

And that might make the CUDA-enabled version of ggml

This didn't help, unfortunately.

Comment options

Sorry to hear that. I do have an example of buiding with CUDA enabled libggml / libwhisper / go bindings in my project here, if that's at all useful?

https://github.com/mutablelogic/go-whisper

I use pkgconfig to bind to all the libraries needed and I start by generating all the pkgconfig files first.

Comment options

Thanks David, appreciate your taking the time to reply. Will check out your project to see if I can get some pointers.

Comment options

Hey! If anyone is still interested:
I submitted a patch which finally allows us to compile the go examples with cuda support in this pr: #2416

Comment options

Definitely still interested! Thank you very much! Will take a look.

Comment options

It seems that go bindings cannot be build using apple clang, but if I use brew version of llvm, I get errors as it is not able to use apple frameworks named in bindings. its M1 mac on macOS 14.4

You must be logged in to vote
1 reply
Comment options

Here are commands for macOS 15.2: Core ML and OpenMP
#2689 (comment)

Comment options

Hello, I am trying to use the Go bindings for Whisper.cpp, after following the documentation I am having an error with the make target 'libwhisper.a'

Anyone having this issue on Linux and have a fix?

I also tried with Docker like this, and I still have the error:

FROM golang:alpine AS libBuilder
RUN apk add git build-base
WORKDIR /
RUN git clone https://github.com/ggerganov/whisper.cpp.git
WORKDIR /whisper.cpp/bindings/go
RUN make whisper
You must be logged in to vote
3 replies
Comment options

Looked into it a bit and you need to replace the following in bindings/go/Makefile:33:
@${MAKE} -C ../.. libwhisper.a -> @${MAKE} -C ../.. build

But now if I run make test I get the following error:
/tmp/go-build2035026264/b001/go.test: error while loading shared libraries: libwhisper.so.1: cannot open shared object file: No such file or directory

Comment options

I also took a look at this. Version 1.7.3 of whisper.cpp has deprecated building via Makefile, so the new way to do it is to update the Makefile targets is:

whisper: mkdir
	@echo Build whisper
	@cmake -S ../.. -B ${BUILD_DIR} -DBUILD_SHARED_LIBS=0
	@cmake --build ${BUILD_DIR} -j --config Release	

On my system, this places some libraries in the build folder, which are required to link (relative to ${BUILD_DIR}):

  • GGML ggml/src/libggml.a, ggml/src/libggml-cpu.a, ggml/src/libggml-base.a
  • GGML Extensions ggml/src/ggml-blas/libggml-blas.a, ggml/src/ggml-metal/libggml-metal.a,
  • Whisper src/libwhisper.a

I know from also compiling with CUDA that there are other libraries created when GPU-specific features are also included. The whisper process creates a file ${BUILD_DIR}/whisper.pc which can be used to bind the go bindings to whisper, but only if you install whisper first, or you could I guess fix-up this file in a new Makefile target.

The bigger issue is that the go bindings are out-of-date as the whisper API has changed. I have written some new bindings here but didn't test them enough to merge them back into the whisper.ccp repo.

Comment options

With a bit more testing, a modified version of the whisper.pc file would look something like this (testing on my MacOS x86):

prefix=${PATH_TO}/whisper.cpp
exec_prefix=${prefix}/bindings/go
libdir=${exec_prefix}/build
Name: whisper
Description: Port of OpenAI's Whisper model in C/C++
Version: 1.7.3
Libs: -L${libdir}/src -lwhisper -L${libdir}/ggml/src -lggml -lggml-base -lggml-cpu -L${libdir}/ggml/src/ggml-metal -lggml-metal -L${libdir}/ggml/src/ggml-blas -lggml-blas 
Cflags: -I${prefix}/include -I${prefix}/ggml/include

Not sure yet how you'd like to create this pkgconfig file. I have some examples in my own repo on using go generate to do this, but not sure it's the right long-term approach, given all the variations of GGML extensions.

Comment options

Help with Setting Up whisper.cpp in a Wails2 Project

Hello, I’m a beginner with both C++ and Go, and I am trying to integrate whisper.cpp into my application built using the Wails2 framework to recognize text from audio data. I followed the instructions in the Go bindings but encountered an issue while building the project.

Initially, I tried to compile the project using make whisper as instructed, expecting to generate the libwhisper.a file. However, I received the following error:

make[1]: Entering directory 'F:/proj/whisper/whisper.cpp'
make[1]: *** No rule to make target 'libwhisper.a'. Stop.
make[1]: Leaving directory 'F:/proj/whisper/whisper.cpp'
make: *** [Makefile:33: whisper] Error 2

Then, based on the main README, I proceeded with the cmake build process:

# build the project
cmake -B build
cmake --build build --config Release

This successfully generated the libwhisper.a file in the build directory. I then copied both libwhisper.a and whisper.h into my project, but when I tried to run wails dev to debug, I encountered the following error:

# github.com/ggerganov/whisper.cpp/bindings/go
 In file included from D:\go\pkg\mod\github.com\ggerganov\whisper.cpp\bindings\go@v0.0.0-20241224073007-7d55637f0bc8\params.go:11:
 F:\proj\echo\lib/whisper.h:4:10: fatal error: ggml.h: No such file or directory
 4 | #include "ggml.h"

From the error message, I understand that it’s looking for the ggml.h file. However, I could not find this file anywhere in the entire whisper.cpp repository, nor is there any mention of how to build it.

Could someone kindly help me? My goal is to use whisper.cpp in my Wails2 project, and any advice or guidance would be greatly appreciated!

My environment:

Windows 11
CPU: 7800X3D
Go version: 1.23.2
Wails CLI v2.9.2

Wishing everyone a happy and prosperous New Year! May 2025 bring you success and happiness!

You must be logged in to vote
5 replies
Comment options

The include directory flags are as follows, replacing ${prefix} with the repository path (whisper.cpp)

-I${prefix}/include -I${prefix}/ggml/include
Comment options

Thank you for your reply. I executed the following command:

$env:CGO_CFLAGS = "-I F:/proj/whisper/whisper.cpp/include -I F:/proj/whisper/whisper.cpp/ggml/include -I F:/proj/echo/lib"

With this setup, the error about missing ggml.h was resolved. However, now I am encountering a new issue: undefined references to various ggml methods. The error messages are as follows (there are many similar errors, so I am only providing a snippet):

。。。many similar error logs
F:/proj/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: F:\\proj\\echo\\lib/libwhisper.a(whisper.cpp.obj):whisper.cpp:(.text+0x21b23): undefined reference to `ggml_backend_dev_name'
F:/proj/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: F:\\proj\\echo\\lib/libwhisper.a(whisper.cpp.obj):whisper.cpp:(.text+0x21c0e): undefined reference to `ggml_backend_tensor_set'
F:/proj/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: F:\\proj\\echo\\lib/libwhisper.a(whisper.cpp.obj):whisper.cpp:(.text+0x21ccd): undefined reference to `ggml_backend_alloc_ctx_tensors'
F:/proj/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: F:\\proj\\echo\\lib/libwhisper.a(whisper.cpp.obj):whisper.cpp:(.text+0x2b12e): undefined reference to `ggml_backend_buffer_free'
F:/proj/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: F:\\proj\\echo\\lib/libwhisper.a(whisper.cpp.obj):whisper.cpp:(.text+0x2b258): undefined reference to `ggml_backend_buffer_clear'
F:/proj/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: F:\\proj\\echo\\lib/libwhisper.a(whisper.cpp.obj):whisper.cpp:(.text+0x2b455): undefined reference to `ggml_time_us'
F:/proj/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: F:\\proj\\echo\\lib/libwhisper.a(whisper.cpp.obj):whisper.cpp:(.text+0x2b646): undefined reference to `ggml_time_us'
F:/proj/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: F:\\proj\\echo\\lib/libwhisper.a(whisper.cpp.obj):whisper.cpp:(.text+0x2b6ca): undefined reference to `ggml_time_us'
F:/proj/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: F:\\proj\\echo\\lib/libwhisper.a(whisper.cpp.obj):whisper.cpp:(.text+0x2c33b): undefined reference to `ggml_time_us'
F:/proj/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: F:\\proj\\echo\\lib/libwhisper.a(whisper.cpp.obj):whisper.cpp:(.text+0x2c4ae): undefined reference to `ggml_time_us'
F:/proj/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: F:\\proj\\echo\\lib/libwhisper.a(whisper.cpp.obj):whisper.cpp:(.text+0x2c7b1): more undefined references to `ggml_time_us' follow
F:/proj/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: F:\\proj\\echo\\lib/libwhisper.a(whisper.cpp.obj):whisper.cpp:(.text+0x1108d): undefined reference to `ggml_log_set'
collect2.exe: error: ld returned 1 exit status

I have also tried copying all ggml-related files into my project's lib directory and linking them, but I encountered the same errors.

Do you have any suggestions on how I can resolve this issue? Thank you in advance for your help!

Comment options

Yes, you'll also need to link the GGML libraries. The library flags (CGO_ LDFLAGS) will be something like:

-L${prefix}/ggml -lggml

but it depends on the version of whisper.cpp (you can see the discussion above about the recent changes to building)

Comment options

Thanks for your willingness to help, even though it didn’t quite solve my issue. For me, looking into the Makefile is still a bit challenging, but I’ll try with ChatGPT’s assistance.

By the way, why is this Go building tool in such a nearly unmaintained state? I thought Go was supposed to be a pretty popular language!

Comment options

It's popular, but known to have a "difficult" time binding with third party libraries :-( Good luck and let me know if you need some more help.

Comment options

large-v3-turbo whisper.cpp 1.7.4 Go Bindings macOS 15.2 Core ML

sustained transcription throughput:

M1 Max: ×ばつ or ×ばつ
2020 Mac mini: ×ばつ or ×ばつ

after 10+ hours, instances converge towards ×ばつ
if running beyond main memory or GPU capabilities, one or all instances will exhibit periods ×ばつ or hang
longer streams = higher numbers, M1 Max: max 3 instances, mini: max 2. Main cpu load only 20%

GPU load 40% per instance

macOS compile commands: #2696 (comment)

You must be logged in to vote
0 replies
Comment options

go build macOS:
CC="$(brew --prefix)/opt/llvm/bin/clang" go build...
// clang: error: unsupported option '-fopenmp'
// add: CC="$(brew --prefix)/opt/llvm/bin/clang"
// have brew llvm installed #2696 (comment)

go run macOS:
W='.../whisper.cpp' LIBRARY_PATH="$W/build/src" CC="$(brew --prefix)/opt/llvm/bin/clang" DYLD_LIBRARY_PATH="$W/build/src" go run ./cmd/myGoMainDirectory
// whisper.h error: add: W='.../whisper.cpp' C_INCLUDE_PATH="$W/include:$W/ggml/include"
// link/clang error: add:W='.../whisper.cpp' LIBRARY_PATH="$W/build/src"

Go executable macOS:
DYLD_LIBRARY_PATH='.../whisper.cpp/build/src' myGoExecutable
// dyld[13346]: Library not loaded: @rpath/libwhisper.1.dylib
// add DYLD_LIBRARY_PATH

You must be logged in to vote
2 replies
Comment options

Hello @resolutecacke, do you have something for

ework Metal -framework Foundation -framework CoreGraphics -O2 -g -framework CoreFoundation
clang: error: unsupported option '-fopenmp'
make: *** [run] Error 1

Maybe? I running on M1 Pro and I run using: C_INCLUDE_PATH="/.../projects/whisper.cpp/include" LIBRARY_PATH="/.../projects/whisper.cpp/bindings/go/build/src" CGO_CFLAGS="-I /.../projects/whisper.cpp/include -I /.../projects/whisper.cpp/ggml/include" go run ./cmd/serve

Comment options

I'm stupid, I'm using default clang, you need to use llvm ^

Comment options

after 5+ days of transcribing, macOS may slow down to around ×ばつ. Possibly macOS restart may be required to regain capabilities. If there is a period of high memory pressure or GPU load, macOS may not recover. It is unclear whether transcription quality is affected

It is necessary to constantly measure and monitor throughput per context-process method invocation, per audio file and per executable invocation. We know macOS is garbage and this is how that shows. It’s the processor that’s great

The speed multiple is fairly constant, but higher for non-speech noise or silence, or longer files
eg. if speech transcribes at ×ばつ, noise may transcribe at ×ばつ and average may be ×ばつ

If it’s not ×ばつ+, restart the executable or reboot macOS. Should be done to secure transcription quality

You must be logged in to vote
0 replies
Comment options

Hello all.
I'll does it for Darwin arm64.

You can use it if you need, or make PRs to support other platforms.
https://github.com/AlexSnet/whisper.go

The main idea is to store *.a with go code. Ok, it's not very good for main repo, but it is not very bad way for Golang binding, IMHO.

A tiny explanations how to get c-archive files:

  1. clone main whisper.cpp repo
  2. cd into it and configure cmake -B build -DBUILD_SHARED_LIBS=OFF
  3. build it cmake --build build
  4. get libwhisper.a from build/src/libwhisper.a

Also you need to do something like above with ggml repo and get all the c-archive files from build directory.

After all you can put them into one directory and point them as libraries using CGO like this: #cgo LDFLAGS: -ldl -lm -lpthread -lcommon -lcommon-ggml -lggml -lggml-base -lggml-blas -lggml-cpu -lggml-metal -lwhisper -lstdc++

Maybe my method is not good and there is something to fix or remake.

Thank you.

PS> All what I did was rewriting a few lines of original whisper.cpp Golang bindings and gluing it together with builded c-archive files and headers.

You must be logged in to vote
0 replies
Comment options

I have troubles in building for CUDA with Golang bindings in Docker:

  1. I can NOT use environment variable, GGML_CUDA, to control use CUDA or not. I have to specific it with -DGGML_CUDA 1 in cmake.
  2. Missing ggml-cuda lib when I build Golang with GGML_CUDA=1
  3. And somehow, I have trouble to loss CUDA_PATH in Makefile

I forked this project, made some changes, then it worked. But I don't know why...😂
Help of explanation will be very appreciated 😁.

And here is my Dockerfile when I encounter troubles above.

ARG UBUNTU_VERSION=22.04
ARG CUDA_VERSION=12.6.3
ARG BASE_CUDA_DEV_CONTAINER=nvidia/cuda:${CUDA_VERSION}-devel-ubuntu${UBUNTU_VERSION}
FROM ${BASE_CUDA_DEV_CONTAINER}
WORKDIR /app
ARG CUDA_DOCKER_ARCH=all
ENV CUDA_DOCKER_ARCH=${CUDA_DOCKER_ARCH}
ENV GGML_CUDA=1
RUN test -n "$http_proxy" && echo "Acquire::http::Proxy \"$http_proxy\";" || exit 0 >> /etc/apt/apt.conf.d/proxy.conf
RUN test -n "$https_proxy" && echo "Acquire::https::Proxy \"$https_proxy\";" || exit 0 >> /etc/apt/apt.conf.d/proxy.conf
RUN apt-get update \
 && apt-get install -y build-essential libsdl2-dev ffmpeg wget cmake git curl
RUN rm -f /etc/apt/apt.conf.d/proxy.conf
ENV CUDA_MAIN_VERSION=12.6
ENV LD_LIBRARY_PATH="/usr/local/cuda-${CUDA_MAIN_VERSION}/compat:$LD_LIBRARY_PATH"
RUN git clone --depth=1 "https://github.com/allape/whisper.cpp.git" .
RUN make build
WORKDIR /tmp
ENV GO_VERSION="1.24.1"
ENV GO_ARCH="linux-amd64"
ENV GO_ZIP_URL="https://go.dev/dl/go$GO_VERSION.$GO_ARCH.tar.gz"
RUN curl -L -o go.tar.gz "$GO_ZIP_URL" \
 && rm -rf /usr/local/go \
 && tar -C /usr/local -xzf go.tar.gz
ENV PATH="/usr/local/go/bin:$PATH"
WORKDIR /app/bindings/go
RUN make whisper
WORKDIR /app
COPY go.mod .
COPY go.sum .
RUN go mod download
COPY main.go .
RUN CGO_CFLAGS="-I/app/include -I/app/ggml/include" \
 CGO_LDFLAGS="-L/app/build_go/src -L/app/build_go/ggml/src -L/app/build_go/ggml/src/ggml-cuda -L/usr/local/cuda/targets/x86_64-linux/lib -lggml-cuda -lcudart -lcuda -lcublas" \
 /usr/local/go/bin/go build -o whisper main.go
EXPOSE 9090
CMD ["./whisper"]
You must be logged in to vote
0 replies
Comment options

If anyone needs the vulkan link, you can refer to the following patch:

diff --git a/bindings/go/Makefile b/bindings/go/Makefile
index edcc016..0e062cc 100644
--- a/bindings/go/Makefile
+++ b/bindings/go/Makefile
@@ -16,12 +16,18 @@ MODELS_DIR := models
 EXAMPLES_DIR := $(wildcard examples/*)
 INCLUDE_PATH := $(abspath ../../include):$(abspath ../../ggml/include)
 LIBRARY_PATH := $(abspath ../../${BUILD_DIR}/src:$(abspath ../../${BUILD_DIR}/ggml/src))
+CMAKE_EXT :=
 
 ifeq ($(GGML_CUDA),1)
 	LIBRARY_PATH := $(LIBRARY_PATH):$(CUDA_PATH)/targets/$(UNAME_M)-linux/lib/
 	BUILD_FLAGS := -ldflags "-extldflags '-lcudart -lcuda -lcublas'"
 endif
 
+ifeq ($(GGML_VULKAN),1)
+	BUILD_FLAGS := -ldflags "-extldflags '-L $(abspath ../../${BUILD_DIR}/ggml/src/ggml-vulkan) -lvulkan -lggml-vulkan'"
+	CMAKE_EXT:= -DGGML_VULKAN=ON
+endif
+
 ifeq ($(UNAME_S),Darwin)
 	EXT_LDFLAGS := -framework Foundation -framework Metal -framework MetalKit
 endif
@@ -31,7 +37,7 @@ all: clean whisper examples
 whisper: mkdir
 	cmake -S ../.. -B ../../${BUILD_DIR} \
 		-DCMAKE_BUILD_TYPE=Release \
-		-DBUILD_SHARED_LIBS=OFF
+		-DBUILD_SHARED_LIBS=OFF $(CMAKE_EXT)
 	cmake --build ../../${BUILD_DIR} --target whisper
 
 test: model-small whisper modtidy

then:

GGML_VULKAN=1 make examples

thanks: https://github.com/AlexSnet/whisper.go

You must be logged in to vote
0 replies
Comment options

I have this working inside of a Docker container:

https://github.com/joaquindelcanto/go-whisper.cpp/tree/main

I was pretty new to many of the technologies involved, so It took me some days of learning and troubleshooting to get to this point. This has been a fun project to help me learn! I hope that this template repo will help others get started more easily. 🙏😎

You must be logged in to vote
0 replies
Comment options

How annoying.

You must be logged in to vote
0 replies
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

AltStyle によって変換されたページ (->オリジナル) /