Dec 22, 2022 · 31 comments · 64 replies
-
General discussion about the go bindings for whisper.cpp
Current state: bindings/go
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
-
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
9 replies
-
ok, figured it out I think, I had to create a build folder inside the go package and drop in libwhisper.a there
Beta Was this translation helpful? Give feedback.
-
Hi Chris, happy new year. So there is a "Makefile" in the bindings/go directory which will do all of this for you. In that directory, you can run:
make all to build the library and examples and put the result in a build temporary folder;make test to run the unit testsmake modtidy and make clean to clean things up.If you are placing your code in the "examples" folder you can build it with:
make examples/stream which builds the example and puts the result in the build directory.I hope that helps!
Beta Was this translation helpful? Give feedback.
{{actor}} deleted this content .
-
I wanted to use this binding in my project. Also had problem to build it with missing header and cannot find -lwhisper.
I realized the CGO directives in the source files require the header to be in $SRCDIR/../.. and the libwhisper.a in build. So I cloned the repo into my Go package root, compiled the bindings using make whisper in the Go bindings package and then in my root project go.mod added:
replace github.com/ggerganov/whisper.cpp/bindings/go => ./whisper.cpp/bindings/go
Now I was able to compile it program with just go build.
Beta Was this translation helpful? Give feedback.
-
Yeah, I've put in a PR #379 now which removes the hard-coded paths from the .go file in preference for setting the C_INCLUDE_PATH and LIBRARY_PATH variables which invoking the go toolchain, and included some documentation to describe this. I hope that helps!
Beta Was this translation helpful? Give feedback.
-
thanks, that worked great! Is there any way I can help with the streaming part of this?
@chrisbward I added a very basic streaming example here:
https://github.com/djthorpe/go-whisper/
It creates all sorts of crashes (I need to look into thread safety, I assume it's bacause I am launching goroutines where I shouldn't). But may help you a bit more.
Beta Was this translation helpful? Give feedback.
-
Hello Everybody, First of all thank you very much for all the work that was poured into this.
I am trying to initialise a whisper model using Golang.
I created my build folder with all executables.
I downloaded the models.
I created a dependencies Folder where I added lib whisper.an and whisper.h accordingly.
I created the environment variable that point to the dependencies folder.
With all these done, I still get the following error.
github.com/ggerganov/whisper.cpp/bindings/go
../../go/pkg/mod/github.com/ggerganov/whisper.cpp/bindings/go@v0.0.0-20230116195735-49b529ba74f3/params.go:11:10: fatal error: 'whisper.h' file not found
#include <whisper.h>
Any advice/piece of help is very much appreciated.
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
7 replies
-
As you can notice, in the dependencies folder on the left I stored bot .h and .a files.
Also, given the tree on the left, the path should be correct..
Beta Was this translation helpful? Give feedback.
-
I must mention that, even if I try to run the package tests from the original repository, with the original path, it fails from the same reason..
Beta Was this translation helpful? Give feedback.
{{actor}} deleted this content .
-
Thanks @mbaxamb3333 can you run the following and see what happens, in a fresh empty directory:
git clone https://github.com/ggerganov/whisper.cpp.git
cd whisper.cpp/bindings/go
make test
I'd be interested to see if that compiles. If it doesn't work, can you send me the output?
Beta Was this translation helpful? Give feedback.
-
On OSX, following those steps, I get
$ make all
Clean
Mkdir build
Mkdir models
Build whisper
make[1]: *** No rule to make target `libwhisper.a'. Stop.
make: *** [whisper] Error 2
Beta Was this translation helpful? Give feedback.
-
On OSX, following those steps, I get
$ make all Clean Mkdir build Mkdir models Build whisper make[1]: *** No rule to make target `libwhisper.a'. Stop. make: *** [whisper] Error 2
I encountered the same error, and it seems that the Go bindings are no longer being maintained.
Beta Was this translation helpful? Give feedback.
{{actor}} deleted this content .
-
When running the command from the makefile everything works fine.
Although If I try to run, within whisper_test.go, run package tests, I get the same error.
/var/folders/y0/dbsx2zd55lz31wt__nmdgm840000gn/T/go-build1048629891/b084/params.cover.go:11:10: fatal error: 'whisper.h' file not found
#include <whisper.h>
^~~~~~~~~~~
1 error generated.
FAIL github.com/ggerganov/whisper.cpp/bindings/go [build failed]
FAIL
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
1 reply
-
Great. In that case, I would take a look at the Makefile in the bindings/go directory and see what it is doing differently to make it work. The most important part of the makefile is as follows:
C_INCLUDE_PATH=${INCLUDE_PATH} LIBRARY_PATH=${LIBRARY_PATH} go build <folder>
It's really just important that INCLUDE_PATH contains the file whisper.h and the LIBRARY_PATH contains the file libwhisper.a - these should be on the same line as the "go build" command.
Beta Was this translation helpful? Give feedback.
-
Golang bindings are 45 times slower than the C++ binary when transcoding samples/jfk.wav using the ggml-tiny.en.bin model.
C++ and Golang examples are compiled following the readme. I haven't profiled the Golang bindings yet.
Raw results C++ binarytime ./main -m ./models/ggml-tiny.en.bin -f ./samples/jfk.wav whisper_init_from_file: loading model from './models/ggml-tiny.en.bin' whisper_model_load: loading model whisper_model_load: n_vocab = 51864 whisper_model_load: n_audio_ctx = 1500 whisper_model_load: n_audio_state = 384 whisper_model_load: n_audio_head = 6 whisper_model_load: n_audio_layer = 4 whisper_model_load: n_text_ctx = 448 whisper_model_load: n_text_state = 384 whisper_model_load: n_text_head = 6 whisper_model_load: n_text_layer = 4 whisper_model_load: n_mels = 80 whisper_model_load: f16 = 1 whisper_model_load: type = 1 whisper_model_load: mem required = 387.00 MB (+ 3.00 MB per decoder) whisper_model_load: kv self size = 2.62 MB whisper_model_load: kv cross size = 8.79 MB whisper_model_load: adding 1607 extra tokens whisper_model_load: model ctx = 73.58 MB whisper_model_load: model size = 73.54 MB system_info: n_threads = 4 / 16 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 | main: processing './samples/jfk.wav' (176000 samples, 11.0 sec), 4 threads, 1 processors, lang = en, task = transcribe, timestamps = 1 ... [00:00:00.000 --> 00:00:07.740] And so my fellow Americans ask not what your country can do for you [00:00:07.740 --> 00:00:10.740] ask what you can do for your country whisper_print_timings: load time = 186.77 ms whisper_print_timings: mel time = 83.82 ms whisper_print_timings: sample time = 16.98 ms whisper_print_timings: encode time = 791.23 ms / 197.81 ms per layer whisper_print_timings: decode time = 289.46 ms / 72.36 ms per layer whisper_print_timings: total time = 1374.19 ms real 0m1.428s user 0m4.208s sys 0m0.280sGolang bindings
time ./build/go-whisper -model ./models/ggml-tiny.en.bin samples/jfk.wav whisper_init_from_file: loading model from './models/ggml-tiny.en.bin' whisper_model_load: loading model whisper_model_load: n_vocab = 51864 whisper_model_load: n_audio_ctx = 1500 whisper_model_load: n_audio_state = 384 whisper_model_load: n_audio_head = 6 whisper_model_load: n_audio_layer = 4 whisper_model_load: n_text_ctx = 448 whisper_model_load: n_text_state = 384 whisper_model_load: n_text_head = 6 whisper_model_load: n_text_layer = 4 whisper_model_load: n_mels = 80 whisper_model_load: f16 = 1 whisper_model_load: type = 1 whisper_model_load: mem required = 387.00 MB (+ 3.00 MB per decoder) whisper_model_load: kv self size = 2.62 MB whisper_model_load: kv cross size = 8.79 MB whisper_model_load: adding 1607 extra tokens whisper_model_load: model ctx = 73.58 MB whisper_model_load: model size = 73.54 MB Loading "./samples/jfk.wav" ...processing "./samples/jfk.wav" [ 0s-> 7.74s] And so my fellow Americans ask not what your country can do for you [ 7.74s->10.74s] ask what you can do for your country real 1m3.919s user 4m7.851s sys 0m6.771s
Related issue: #421
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
4 replies
{{actor}} deleted this content .
-
Could this hard coded processors = 0 contributes to the slowness? Any reason we cannot use Whisper_full_parallel at this moment?
Beta Was this translation helpful? Give feedback.
{{actor}} deleted this content .
-
I made three samples and tested their performance on MLK's full "I have a dream" speech:
So there's a significant cost to using either of the official bindings. I'm not sure where the cost comes in on the low-level bindings, because my cgo looks extremely similar to what it's doing, but it's repeatably much faster.
All programs are available here (I've linked directly to the version I used for these tests)
The default example program gives 27s (!) when run with time ./whisper-cpp -m ~/.local/share/blisper/ggml-small.bin samples/mlk_ihaveadream_long.wav, which should be the roughly equivalent call. Not sure why my cgo program is running faster than it.
Beta Was this translation helpful? Give feedback.
{{actor}} deleted this content .
-
@jaybinks on a 53 minute gil strang lecture, with the large model this time, on my Mac OS laptop, plugged in:
whisper-cpp main (4-thread default): 14:15
whisper-cpp main (8 threads): 18:45
binding libwhisper.so directly, 2 CPUs (4-thread default): 12:38
binding libwhisper.so directly, 1 CPU (4-thread default): 14:25
low-level bindings: 62:28
high-level bindings: I cut it off because it didn't see to be going. very long.
whisper-cpp:
system_info: n_threads = 4 / 10 | AVX = 0 | AVX2 = 0 | AVX512 = 0 | FMA = 0 | NEON = 1 | ARM_FMA = 1 | F16C = 0 | FP16_VA = 1 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 0 | VSX = 0 | COREML = 0 |
main: processing 'samples/strang.wav' (51080185 samples, 3192.5 sec), 4 threads, 1 processors, lang = en, task = transcribe, timestamps = 1
whisper_print_timings: load time = 1080.02 ms
whisper_print_timings: fallbacks = 9 p / 5 h
whisper_print_timings: mel time = 3814.76 ms
whisper_print_timings: sample time = 4009.11 ms / 9769 runs ( 0.41 ms per run)
whisper_print_timings: encode time = 446511.75 ms / 126 runs ( 3543.74 ms per run)
whisper_print_timings: decode time = 390127.31 ms / 9760 runs ( 39.97 ms per run)
whisper_print_timings: total time = 845733.56 ms
./whisper-cpp -m ~/.local/share/blisper/ggml-large.bin samples/strang.wav 3471.29s user 36.83s system 414% cpu 14:05.80 total
I noticed that whisper-cpp spread the work across all my processors but kept them only about half-utilized. Possibly because it had only 4 threads on my 8-processor machine?
With 8 threads, it clearly does a better job of keeping the processors busy, but actually took longer (?):
whisper_print_timings: load time = 1570.18 ms
whisper_print_timings: fallbacks = 9 p / 5 h
whisper_print_timings: mel time = 2104.77 ms
whisper_print_timings: sample time = 4008.01 ms / 9769 runs ( 0.41 ms per run)
whisper_print_timings: encode time = 672592.88 ms / 126 runs ( 5338.04 ms per run)
whisper_print_timings: decode time = 445225.22 ms / 9760 runs ( 45.62 ms per run)
whisper_print_timings: total time = 1125743.62 ms
./whisper-cpp --threads 8 -m ~/.local/share/blisper/ggml-large.bin 8434.05s user 43.55s system 753% cpu 18:45.81 total
sotest (custom bindings): 2 CPUs
AVX = 0 | AVX2 = 0 | AVX512 = 0 | FMA = 0 | NEON = 1 | ARM_FMA = 1 | F16C = 0 | FP16_VA = 1 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 0 | VSX = 0 | COREML = 0 |
./sotest -model large ../samples/strang.wav /dev/stdout 5336.24s user 79.33s system 714% cpu 12:37.85 total
./sotest -model large ../samples/strang.wav /dev/stdout 3446.45s user 49.32s system 403% cpu 14:25.43 total
With the "low-level" bindings 62m 28s:./lltest -model large ../samples/strang.wav /dev/stdout 7388.36s user 90.48s system 199% cpu 1:02:27.94 total
$ sw_vers && system_profiler SPHardwareDataType 9:45AM
ProductName: macOS
ProductVersion: 13.4
BuildVersion: 22F66
Hardware:
Hardware Overview:
Model Name: MacBook Pro
Model Identifier: MacBookPro18,4
Model Number: Z15H00107LL/A
Chip: Apple M1 Max
Total Number of Cores: 10 (8 performance and 2 efficiency)
Memory: 32 GB
Beta Was this translation helpful? Give feedback.
-
For kicks, I edited the bindings to remove the progress callbacks on the theory that that caused frequent calls across the go -> C boundary, and that dropped the time for the low-level bindings to 14:19 from 62:28 - pretty conclusive evidence that they are a major issue.
Here's the diff:
--- whisper.go 2023-06-26 08:49:04
+++ whisper.2.go 2023-06-29 15:43:00
@@ -326,10 +326,10 @@
// It seems this approach can offer some speedup in some cases.
// However, the transcription accuracy can be worse at the beginning and end of each chunk.
func (ctx *Context) Whisper_full_parallel(params Params, samples []float32, processors int, encoderBeginCallback func() bool, newSegmentCallback func(int)) error {
- registerEncoderBeginCallback(ctx, encoderBeginCallback)
- registerNewSegmentCallback(ctx, newSegmentCallback)
- defer registerEncoderBeginCallback(ctx, nil)
- defer registerNewSegmentCallback(ctx, nil)
+ // registerEncoderBeginCallback(ctx, encoderBeginCallback)
+ // registerNewSegmentCallback(ctx, newSegmentCallback)
+ // defer registerEncoderBeginCallback(ctx, nil)
+ // defer registerNewSegmentCallback(ctx, nil)
if C.whisper_full_parallel((*C.struct_whisper_context)(ctx), (C.struct_whisper_full_params)(params), (*C.float)(&samples[0]), C.int(len(samples)), C.int(processors)) == 0 {
return nil
Beta Was this translation helpful? Give feedback.
{{actor}} deleted this content .
-
Can we reuse the same model instance to process multiple wav files which are in different languages?I have 20 wav files which are in different languages and totally unrelated. I create one model using a multi-lang model, and then create a new Context using model.NewContext() to process each audio file, with language auto-detection turned on for each context. The transcription for most of the files are sane, but 2 or 3 of them are just completely rambles, even though the language were correctly detected.
So my question is - are we supposedly to reuse one model and just create a new context for each new wav file? My understanding is that the model inference is somewhat stateful - would the first wav file / context influence the second wav file /context in some way?
Btw thanks for building such a great project!
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
2 replies
-
The C api does not have the concept of model. It only has whisper_context created by WHISPER_API struct whisper_context * whisper_init_from_file(const char * path_model);. However in Go, we create a Model instance by loading the model file, we can then use Model.NewContext() to create one or more contexts - this encourages reusing of the same model instance, which seem consistent with the C api?
Beta Was this translation helpful? Give feedback.
{{actor}} deleted this content .
-
I'm right that we can't do something like this
for i := 0; i < 10; i++{
go func(){ // new goroutine
ctx := whisper.NewContext()
ctx.Process(...)
}()
}
I'm having a data race error right now.
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
2 replies
-
2yrs in the future and i'm stuck on the same issue, according to the documentation (cpp) multi threading on one model should be possible if you use different contexts, and is unstable when doing it in one context.
But the above piece of code doesn't work, go crashes when two contexts are busy at the same time..
Beta Was this translation helpful? Give feedback.
-
Hi, the above code might work if there was a "WaitGroup" in there? What is the crash you're getting?
var wg sync.WaitGroup
for i := 0; i < 10; i++{
wg.Add(1)
go func(){ // new goroutine
defer wg.Done()
ctx := whisper.NewContext()
ctx.Process(...)
}()
}
// Wait for all goroutines to finish
wg.Wait()
Beta Was this translation helpful? Give feedback.
{{actor}} deleted this content .
-
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
2 replies
-
lol such good sense, I agree
Beta Was this translation helpful? Give feedback.
-
does make it a little more platform restricted...I can see why it'd be good to at least make sure the full compile works.
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
0 replies
{{actor}} deleted this content .
-
I've done similar to what your saying here. https://github.com/jaybinks/goConstmeWhisper And yea, I kinda agree. This allows you to simply download a binary .so of whisper without being required to build it yourself. That being said, it's also not super feasible since your going to get a very unoptimized whisper library
That doesn't seem to match my definition of 'infeasible' -- can you elaborate?
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
0 replies
-
It would be interesting to see how your performance observations scale with much longer audio durations. or, simply looping over the same audio multiple times after loading the model etc. My gut says the difference must be memory copies vs pointers or something like this.
…Beta Was this translation helpful? Give feedback.
You must be logged in to vote
0 replies
-
I had read that the go > boundary was a choke point , but never in my wildest dreams did I ever consider it was THIS bad. WOW . Just for kicks, how does your sotest / lltest compare against the native C CLI app on the same file / system / settings ??
…On Fri, 30 Jun 2023 at 05:44, Bill Mill ***@***.***> wrote: For kicks, I edited the bindings to remove the progress callbacks on the theory that that caused frequent calls across the go -> C boundary, and that dropped the time for the low-level bindings to 14:19 from 62:28 - pretty conclusive evidence that they are a major issue. Here's the diff: --- whisper.go 2023-06-26 08:49:04+++ whisper.2.go 2023-06-29 15:43:00@@ -326,10 +326,10 @@ // It seems this approach can offer some speedup in some cases. // However, the transcription accuracy can be worse at the beginning and end of each chunk. func (ctx *Context) Whisper_full_parallel(params Params, samples []float32, processors int, encoderBeginCallback func() bool, newSegmentCallback func(int)) error {- registerEncoderBeginCallback(ctx, encoderBeginCallback)- registerNewSegmentCallback(ctx, newSegmentCallback)- defer registerEncoderBeginCallback(ctx, nil)- defer registerNewSegmentCallback(ctx, nil)+ // registerEncoderBeginCallback(ctx, encoderBeginCallback)+ // registerNewSegmentCallback(ctx, newSegmentCallback)+ // defer registerEncoderBeginCallback(ctx, nil)+ // defer registerNewSegmentCallback(ctx, nil) if C.whisper_full_parallel((*C.struct_whisper_context)(ctx), (C.struct_whisper_full_params)(params), (*C.float)(&samples[0]), C.int(len(samples)), C.int(processors)) == 0 { return nil — Reply to this email directly, view it on GitHub <
#312 (reply in thread)>, or unsubscribe <
https://github.com/notifications/unsubscribe-auth/AALQR62BBVO76TAOQ4RFJ4DXNXLJZANCNFSM6AAAAAATG4MHKQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>
-- Sincerely Jay
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
1 reply
{{actor}} deleted this content .
-
Sorry I missed this message, github discussions doesn't thread correctly via email it seems, so I got no notification about it.
Using jfk.wav from the whisper.cpp repository, whisper-cpp built with make on commit 3998465, and blisper v0.0.1 from here, on a small file blisper is about 15% slower
$ hyperfine 'blisper -model large jfk.wav /tmp/a.srt'
Benchmark 1: blisper -model large jfk.wav /tmp/a.srt
Time (mean ± σ): 10.264 s ± 0.244 s [User: 36.698 s, System: 1.280 s]
Range (min … max): 9.911 s … 10.717 s 10 runs
$ hyperfine 'whisper-cpp --model ~/.local/share/blisper/ggml-large.bin -of /tmp/a.srt jfk.wav'
Benchmark 1: whisper-cpp --model ~/.local/share/blisper/ggml-large.bin -of /tmp/a.srt jfk.wav
Time (mean ± σ): 9.025 s ± 0.383 s [User: 26.350 s, System: 6.531 s]
Range (min … max): 8.329 s … 9.428 s 10 runs
Beta Was this translation helpful? Give feedback.
{{actor}} deleted this content .
-
When I use make whisper, the console always reports the following error.
Mkdir build
Mkdir models
Build whisper
sysctl: unknown oid 'hw.optional.arm64'
I whisper.cpp build info:
I UNAME_S: Darwin
I UNAME_P: i386
I UNAME_M: x86_64
I CFLAGS: -I. -O3 -DNDEBUG -std=c11 -fPIC -D_DARWIN_C_SOURCE -pthread -mfma -mavx -mavx2 -DGGML_USE_ACCELERATE
I CXXFLAGS: -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -D_DARWIN_C_SOURCE -pthread
I LDFLAGS: -framework Accelerate
I CC: Apple clang version 14.0.0 (clang-1400.0.29.202)
I CXX: Apple clang version 14.0.0 (clang-1400.0.29.202)
make[1]: `libwhisper.a' is up to date.
I have seen the same query raised by someone. issue#597.
I also tried removing CFLAGS += -mf16c. This did not work for me.
Can you please tell me how I should resolve this issue as I want to bind in my own Golang package. It asks me to compile libwhisper.a, that is, to run make whisper in the go/buildings folder.
Could this error be due to the low version of my Mac system? I'm using an Intel Core i7 processor and version 12.6.3.
If so, I'll update my system (it's slow to update and I want to make sure it's not the system version first)
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
4 replies
-
When I run make test in the go/buildings folder, it succeeds!
Beta Was this translation helpful? Give feedback.
-
I'm not an expert, but it seems like that's just a warning and the build is working
Beta Was this translation helpful? Give feedback.
-
Thanks for the reply, but make whisper doesn't generate a binary file, so I think there's something wrong with it
Beta Was this translation helpful? Give feedback.
-
The execution of make whisper reads :
whisper: mkdir
@echo Build whisper
@${MAKE} -C ... /... libwhisper.a
Then in the parent directory, it should generate a target file called libwhisper.a, but it doesn't do what I expect it to do.
Beta Was this translation helpful? Give feedback.
-
Could you please tell me how to use flags?
./build/go-whisper -model models/ggml-base.bin samples/speech.wav --language ru
`Loading "--language"
open --language: no such file or directory
system_info: n_threads = 8 / 8 | AVX = 0 | AVX2 = 0 | AVX512 = 0 | FMA = 0 | NEON = 1 | ARM_FMA = 1 | METAL = 0 | F16C = 0 | FP16_VA = 1 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 0 | SSSE3 = 0 | VSX = 0 | COREML = 0 | OPENVINO = 0 |
Loading "ru"
open ru: no such file or directory
`
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
1 reply
-
Try ./build/go-whisper -model models/ggml-base.bin -language ru samples/speech.wav
Beta Was this translation helpful? Give feedback.
{{actor}} deleted this content .
-
Hi, I am trying to execute make test but got this error:
# github.com/ggerganov/whisper.cpp/bindings/go.test
/usr/local/go/pkg/tool/darwin_arm64/link: running clang failed: exit status 1
ld: Undefined symbols:
_MTLCopyAllDevices, referenced from:
_ggml_metal_init in libwhisper.a[7](ggml-metal.o)
_MTLCreateSystemDefaultDevice, referenced from:
_ggml_metal_init in libwhisper.a[7](ggml-metal.o)
OBJC_CLASS$_MTLComputePassDescriptor, referenced from:
in libwhisper.a[7](ggml-metal.o)
OBJC_CLASS$_NSBundle, referenced from:
in libwhisper.a[7](ggml-metal.o)
OBJC_CLASS$_NSProcessInfo, referenced from:
in libwhisper.a[7](ggml-metal.o)
OBJC_CLASS$_NSString, referenced from:
in libwhisper.a[7](ggml-metal.o)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
FAIL github.com/ggerganov/whisper.cpp/bindings/go [build failed]
FAIL
make: *** [test] Error 1
OS: macOS Sonoma v.14
Chip: Apple Mac M1 MAX
Memory: 32 GB
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
2 replies
{{actor}} deleted this content .
-
I'm getting the same error in different scenario, I used the bindings and I'm running this command:
LIBRARY_PATH=./whisper.cpp C_INCLUDE_PATH=./whisper.cpp go build .
And I get this output:
/opt/homebrew/Cellar/go/1.21.0/libexec/pkg/tool/darwin_arm64/link: running cc failed: exit status 1
ld: Undefined symbols:
_MTLCopyAllDevices, referenced from:
_ggml_metal_init in libwhisper.a[7](ggml-metal.o)
_MTLCreateSystemDefaultDevice, referenced from:
_ggml_backend_metal_buffer_type_alloc_buffer in libwhisper.a[7](ggml-metal.o)
_ggml_backend_metal_buffer_type_get_max_size in libwhisper.a[7](ggml-metal.o)
_ggml_backend_metal_buffer_from_ptr in libwhisper.a[7](ggml-metal.o)
_ggml_metal_init in libwhisper.a[7](ggml-metal.o)
_OBJC_CLASS_$_MTLCaptureDescriptor, referenced from:
in libwhisper.a[7](ggml-metal.o)
_OBJC_CLASS_$_MTLCaptureManager, referenced from:
in libwhisper.a[7](ggml-metal.o)
_OBJC_CLASS_$_MTLCompileOptions, referenced from:
in libwhisper.a[7](ggml-metal.o)
_OBJC_CLASS_$_MTLComputePassDescriptor, referenced from:
in libwhisper.a[7](ggml-metal.o)
_OBJC_CLASS_$_NSBundle, referenced from:
in libwhisper.a[7](ggml-metal.o)
_OBJC_CLASS_$_NSProcessInfo, referenced from:
in libwhisper.a[7](ggml-metal.o)
_OBJC_CLASS_$_NSString, referenced from:
in libwhisper.a[7](ggml-metal.o)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [build] Error 1
Beta Was this translation helpful? Give feedback.
-
I've created a PR to setup Makefile env vars properly and got my binding running. #1530
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
0 replies
{{actor}} deleted this content .
-
Hi , does someone has similar issue? I got it run as a container in Kubernetes cluster (intel CPU) but it's very slow. I suspect no hardware acceleration enabled but not sure how to do that. below is my dockerfile. Thanks
############################
# STEP 1 build libwhisper.a
############################
FROM golang:alpine AS libBuilder
RUN apk add git build-base
WORKDIR /
RUN git clone https://github.com/ggerganov/whisper.cpp.git
WORKDIR /whisper.cpp/bindings/go
RUN make whisper
############################
# STEP 2 build executable binary
############################
FROM golang:alpine AS builder
RUN apk add git build-base
RUN adduser -S myuser -G root -s /bin/bash
COPY . /app/
COPY --from=libBuilder /whisper.cpp/libwhisper.a /app/whisper.cpp2/
COPY --from=libBuilder /whisper.cpp/*.h /app/whisper.cpp2/
WORKDIR /app
RUN C_INCLUDE_PATH="/app/whisper.cpp2/" LIBRARY_PATH="/app/whisper.cpp2/" go test -ldflags "-linkmode 'external' -extldflags '-static'" -v ./...
RUN C_INCLUDE_PATH="/app/whisper.cpp2/" LIBRARY_PATH="/app/whisper.cpp2/" go build -ldflags "-linkmode 'external' -extldflags '-static'" -v -o myapp
############################
# STEP 3 build image based on scratch
############################
FROM scratch
# Copy user
COPY --from=builder /etc/passwd /etc/passwd
# Copy our static executable
COPY --from=builder /app/myapp /bin/myapp
COPY ./models /models
USER myuser
ENTRYPOINT ["/bin/myapp"]
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
3 replies
-
Hey, that's a different thing - and very related to your OS and docker setup, not with metal and mac os + arm. In any case look at docker capabilities to enable gpu access - that's very dependent on drivers and so on: https://docs.docker.com/compose/gpu-support/
Beta Was this translation helpful? Give feedback.
{{actor}} deleted this content .
-
no GPU, just Intel CPU for the inference. I suspect the AVX or F16C capacity is not leveraged by the golang binary.
system_info: n_threads = 24 / 24 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | METAL = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | CUDA = 0 | COREML = 0 | OPENVINO = 0
Beta Was this translation helpful? Give feedback.
-
from the log, it looks like the AVX is already enabled when building libwhisper.a
Is any flag need to be set when building the final binary? Thanks.
RUN C_INCLUDE_PATH="/app/whisper.cpp2/" LIBRARY_PATH="/app/whisper.cpp2/" go build -ldflags "-linkmode 'external' -extldflags '-static'" -v -o myapp
#11 [libbuilder 5/6] RUN make whisper
#11 0.172 Mkdir build
#11 0.173 Mkdir models
#11 0.175 Build whisper
#11 0.176 make[1]: Entering directory '/whisper.cpp'
#11 0.191 I whisper.cpp build info:
#11 0.191 I UNAME_S: Linux
#11 0.191 I UNAME_P: unknown
#11 0.191 I UNAME_M: x86_64
#11 0.191 I CFLAGS: -I. -O3 -DNDEBUG -std=c11 -fPIC -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -pthread -mavx -mavx2 -mfma -mf16c -msse3 -mssse3
#11 0.191 I CXXFLAGS: -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -pthread -mavx -mavx2 -mfma -mf16c -msse3 -mssse3
#11 0.191 I LDFLAGS:
#11 0.191 I CC: cc (Alpine 13.2.1_git20231014) 13.2.1 20231014
#11 0.191 I CXX: g++ (Alpine 13.2.1_git20231014) 13.2.1 20231014
#11 0.191
#11 0.192 cc -I. -O3 -DNDEBUG -std=c11 -fPIC -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -pthread -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -c ggml.c -o ggml.o
#11 11.77 cc -I. -O3 -DNDEBUG -std=c11 -fPIC -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -pthread -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -c ggml-alloc.c -o ggml-alloc.o
#11 12.55 cc -I. -O3 -DNDEBUG -std=c11 -fPIC -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -pthread -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -c ggml-backend.c -o ggml-backend.o
#11 13.73 cc -I. -O3 -DNDEBUG -std=c11 -fPIC -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -pthread -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -c ggml-quants.c -o ggml-quants.o
#11 21.87 g++ -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -pthread -mavx -mavx2 -mfma -mf16c -msse3 -mssse3 -c whisper.cpp -o whisper.o
#11 37.16 ar rcs libwhisper.a ggml.o ggml-alloc.o ggml-backend.o ggml-quants.o whisper.o
#11 37.17 make[1]: Leaving directory '/whisper.cpp'
#11 DONE 37.3s
Beta Was this translation helpful? Give feedback.
-
I have some questions about this golang binding implementation.
First thank you, finaly something that is free and able to be used by selfhosting without any phone-home code or License requirement! :)
I made it so far, that I can use the if on the demo files that come with it. Also one other wav-file gets analysed properly, another wav-file is getting analysed as "Musik" :D even tho it i a normal speech.
So in general I have some questions:
ggml-small.en.bin (which is the smallest model which is presice enough for me. Is it currently beeing worked on this?system_info: n_threads = 10 / 10 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | METAL = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | CUDA = 0 | COREML = 0 | OPENVINO = 0
Does it benefit the tools performance, if this is getting run on an Intel CPU with an iGPU?
Thanks in advance!
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
3 replies
-
It would also be awesome if it could take all common audio codecs/formats (AC3, DTS, DTS-HD MA, TrueHD, TrueHD Atmos, OGG/OGA, MP4/M4A, ASF/WMA, MP3, FLAC, ALAC, DST, OPUS) of another process, that would be able to take in anything and convert it to WAV, so it can process it :)
At least it would be cool on the long run. So this can be used for analysing every Video/Playbook and eighter create subtitles, or just to index the words it contains.
Beta Was this translation helpful? Give feedback.
-
I'm not the author but based on my usage, live streaming would be possible if you buffered chunks (as in each 10s streaming you run the model on a goroutine). Converting from stream to chunks would be on your own. I think that even Context.Process taking a full buffer and its segment/progress callbacks can be wrapped with an io.Reader.
Running the model over the whole wav file takes a long time because it is expensive on CPU and GPU, with impressive improvements depending on your hardware (I got a 10x speed bump testing the same code on a mac M1 vs mac M3), but AFAIK it doesn't benefit of any live tuning/feedback so by my tests you get a pretty decent result chopping a wav file (which you can test with /bin/dd locally).
I figure that while you are streaming if you can detect silence or volume drops it would prevent mid-tokens.
About converting formats I think that the choice is to have the library understand the most basic/available format so we can overlay specialized libraries with their optimization to understand other formats and containers. I've been piping audio from ffmpeg. Hope that helps.
Beta Was this translation helpful? Give feedback.
{{actor}} deleted this content .
-
You make your Go code read float32 from standard input which is what the whisper model reads
Then feed it from ffmpeg -hide_banner -i i.mp4 -ac 1 -ar 16k -f f32le - | myGoCode
or any other float stream you can come up with
(little endian hardware)
use large-v3-turbo model of 2024-12: it’s 100× better
With Core ML on M, you be 2×25× at 20% CPU load
— the host can still be used normally or compile things
On Linux Intel cpu whisper will eat all cores 0%id, 2 GiB RAM and still only do 1.1×
— Quantize no faster
Core ML is 45× and 80% of the cpu better
Beta Was this translation helpful? Give feedback.
{{actor}} deleted this content .
-
Question: C++/C noobie here. Can we also expose the basic stream implementation in the bindings? Or is that not possible for some reason?
I saw djthorpe has his own Go implementation that's a bit buggy and hard to get running relative to this. Wondering if we can do without another implementation and only maintain this one stream here!
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
0 replies
-
Hello trying to follow the Readme but getting this error when I run make test on my m3 macbook air
"_ggml_view_tensor", referenced from:
whisper_build_graph_encoder(whisper_context&, whisper_state&) in libwhisper.a[2](whisper.o)
whisper_build_graph_cross(whisper_context&, whisper_state&) in libwhisper.a[2](whisper.o)
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [examples/go-whisper] Error 1
Any ideas?
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
12 replies
-
It looks like it's not linking with the CUDA-version of GGML - so you should build GGML with:
GGML_CUDA=1 make libggml.aAnd that might make the CUDA-enabled version of ggml
This didn't help, unfortunately.
Beta Was this translation helpful? Give feedback.
{{actor}} deleted this content .
-
Sorry to hear that. I do have an example of buiding with CUDA enabled libggml / libwhisper / go bindings in my project here, if that's at all useful?
https://github.com/mutablelogic/go-whisper
I use pkgconfig to bind to all the libraries needed and I start by generating all the pkgconfig files first.
Beta Was this translation helpful? Give feedback.
-
Thanks David, appreciate your taking the time to reply. Will check out your project to see if I can get some pointers.
Beta Was this translation helpful? Give feedback.
-
Hey! If anyone is still interested:
I submitted a patch which finally allows us to compile the go examples with cuda support in this pr: #2416
Beta Was this translation helpful? Give feedback.
-
Definitely still interested! Thank you very much! Will take a look.
Beta Was this translation helpful? Give feedback.
-
It seems that go bindings cannot be build using apple clang, but if I use brew version of llvm, I get errors as it is not able to use apple frameworks named in bindings. its M1 mac on macOS 14.4
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
1 reply
-
Hello, I am trying to use the Go bindings for Whisper.cpp, after following the documentation I am having an error with the make target 'libwhisper.a'
Anyone having this issue on Linux and have a fix?
I also tried with Docker like this, and I still have the error:
FROM golang:alpine AS libBuilder RUN apk add git build-base WORKDIR / RUN git clone https://github.com/ggerganov/whisper.cpp.git WORKDIR /whisper.cpp/bindings/go RUN make whisper
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
3 replies
-
Looked into it a bit and you need to replace the following in bindings/go/Makefile:33:@${MAKE} -C ../.. libwhisper.a -> @${MAKE} -C ../.. build
But now if I run make test I get the following error:/tmp/go-build2035026264/b001/go.test: error while loading shared libraries: libwhisper.so.1: cannot open shared object file: No such file or directory
Beta Was this translation helpful? Give feedback.
{{actor}} deleted this content .
-
I also took a look at this. Version 1.7.3 of whisper.cpp has deprecated building via Makefile, so the new way to do it is to update the Makefile targets is:
whisper: mkdir
@echo Build whisper
@cmake -S ../.. -B ${BUILD_DIR} -DBUILD_SHARED_LIBS=0
@cmake --build ${BUILD_DIR} -j --config Release
On my system, this places some libraries in the build folder, which are required to link (relative to ${BUILD_DIR}):
ggml/src/libggml.a, ggml/src/libggml-cpu.a, ggml/src/libggml-base.aggml/src/ggml-blas/libggml-blas.a, ggml/src/ggml-metal/libggml-metal.a,src/libwhisper.aI know from also compiling with CUDA that there are other libraries created when GPU-specific features are also included. The whisper process creates a file ${BUILD_DIR}/whisper.pc which can be used to bind the go bindings to whisper, but only if you install whisper first, or you could I guess fix-up this file in a new Makefile target.
The bigger issue is that the go bindings are out-of-date as the whisper API has changed. I have written some new bindings here but didn't test them enough to merge them back into the whisper.ccp repo.
Beta Was this translation helpful? Give feedback.
{{actor}} deleted this content .
-
With a bit more testing, a modified version of the whisper.pc file would look something like this (testing on my MacOS x86):
prefix=${PATH_TO}/whisper.cpp
exec_prefix=${prefix}/bindings/go
libdir=${exec_prefix}/build
Name: whisper
Description: Port of OpenAI's Whisper model in C/C++
Version: 1.7.3
Libs: -L${libdir}/src -lwhisper -L${libdir}/ggml/src -lggml -lggml-base -lggml-cpu -L${libdir}/ggml/src/ggml-metal -lggml-metal -L${libdir}/ggml/src/ggml-blas -lggml-blas
Cflags: -I${prefix}/include -I${prefix}/ggml/include
Not sure yet how you'd like to create this pkgconfig file. I have some examples in my own repo on using go generate to do this, but not sure it's the right long-term approach, given all the variations of GGML extensions.
Beta Was this translation helpful? Give feedback.
-
Help with Setting Up whisper.cpp in a Wails2 Project
Hello, I’m a beginner with both C++ and Go, and I am trying to integrate whisper.cpp into my application built using the Wails2 framework to recognize text from audio data. I followed the instructions in the Go bindings but encountered an issue while building the project.
Initially, I tried to compile the project using make whisper as instructed, expecting to generate the libwhisper.a file. However, I received the following error:
make[1]: Entering directory 'F:/proj/whisper/whisper.cpp'
make[1]: *** No rule to make target 'libwhisper.a'. Stop.
make[1]: Leaving directory 'F:/proj/whisper/whisper.cpp'
make: *** [Makefile:33: whisper] Error 2
Then, based on the main README, I proceeded with the cmake build process:
# build the project
cmake -B build
cmake --build build --config Release
This successfully generated the libwhisper.a file in the build directory. I then copied both libwhisper.a and whisper.h into my project, but when I tried to run wails dev to debug, I encountered the following error:
# github.com/ggerganov/whisper.cpp/bindings/go
In file included from D:\go\pkg\mod\github.com\ggerganov\whisper.cpp\bindings\go@v0.0.0-20241224073007-7d55637f0bc8\params.go:11:
F:\proj\echo\lib/whisper.h:4:10: fatal error: ggml.h: No such file or directory
4 | #include "ggml.h"
From the error message, I understand that it’s looking for the ggml.h file. However, I could not find this file anywhere in the entire whisper.cpp repository, nor is there any mention of how to build it.
Could someone kindly help me? My goal is to use whisper.cpp in my Wails2 project, and any advice or guidance would be greatly appreciated!
My environment:
Windows 11
CPU: 7800X3D
Go version: 1.23.2
Wails CLI v2.9.2
Wishing everyone a happy and prosperous New Year! May 2025 bring you success and happiness!
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
5 replies
-
The include directory flags are as follows, replacing ${prefix} with the repository path (whisper.cpp)
-I${prefix}/include -I${prefix}/ggml/include
Beta Was this translation helpful? Give feedback.
-
Thank you for your reply. I executed the following command:
$env:CGO_CFLAGS = "-I F:/proj/whisper/whisper.cpp/include -I F:/proj/whisper/whisper.cpp/ggml/include -I F:/proj/echo/lib"
With this setup, the error about missing ggml.h was resolved. However, now I am encountering a new issue: undefined references to various ggml methods. The error messages are as follows (there are many similar errors, so I am only providing a snippet):
。。。many similar error logs
F:/proj/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: F:\\proj\\echo\\lib/libwhisper.a(whisper.cpp.obj):whisper.cpp:(.text+0x21b23): undefined reference to `ggml_backend_dev_name'
F:/proj/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: F:\\proj\\echo\\lib/libwhisper.a(whisper.cpp.obj):whisper.cpp:(.text+0x21c0e): undefined reference to `ggml_backend_tensor_set'
F:/proj/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: F:\\proj\\echo\\lib/libwhisper.a(whisper.cpp.obj):whisper.cpp:(.text+0x21ccd): undefined reference to `ggml_backend_alloc_ctx_tensors'
F:/proj/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: F:\\proj\\echo\\lib/libwhisper.a(whisper.cpp.obj):whisper.cpp:(.text+0x2b12e): undefined reference to `ggml_backend_buffer_free'
F:/proj/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: F:\\proj\\echo\\lib/libwhisper.a(whisper.cpp.obj):whisper.cpp:(.text+0x2b258): undefined reference to `ggml_backend_buffer_clear'
F:/proj/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: F:\\proj\\echo\\lib/libwhisper.a(whisper.cpp.obj):whisper.cpp:(.text+0x2b455): undefined reference to `ggml_time_us'
F:/proj/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: F:\\proj\\echo\\lib/libwhisper.a(whisper.cpp.obj):whisper.cpp:(.text+0x2b646): undefined reference to `ggml_time_us'
F:/proj/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: F:\\proj\\echo\\lib/libwhisper.a(whisper.cpp.obj):whisper.cpp:(.text+0x2b6ca): undefined reference to `ggml_time_us'
F:/proj/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: F:\\proj\\echo\\lib/libwhisper.a(whisper.cpp.obj):whisper.cpp:(.text+0x2c33b): undefined reference to `ggml_time_us'
F:/proj/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: F:\\proj\\echo\\lib/libwhisper.a(whisper.cpp.obj):whisper.cpp:(.text+0x2c4ae): undefined reference to `ggml_time_us'
F:/proj/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: F:\\proj\\echo\\lib/libwhisper.a(whisper.cpp.obj):whisper.cpp:(.text+0x2c7b1): more undefined references to `ggml_time_us' follow
F:/proj/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: F:\\proj\\echo\\lib/libwhisper.a(whisper.cpp.obj):whisper.cpp:(.text+0x1108d): undefined reference to `ggml_log_set'
collect2.exe: error: ld returned 1 exit status
I have also tried copying all ggml-related files into my project's lib directory and linking them, but I encountered the same errors.
Do you have any suggestions on how I can resolve this issue? Thank you in advance for your help!
Beta Was this translation helpful? Give feedback.
-
Yes, you'll also need to link the GGML libraries. The library flags (CGO_ LDFLAGS) will be something like:
but it depends on the version of whisper.cpp (you can see the discussion above about the recent changes to building)
Beta Was this translation helpful? Give feedback.
-
Thanks for your willingness to help, even though it didn’t quite solve my issue. For me, looking into the Makefile is still a bit challenging, but I’ll try with ChatGPT’s assistance.
By the way, why is this Go building tool in such a nearly unmaintained state? I thought Go was supposed to be a pretty popular language!
Beta Was this translation helpful? Give feedback.
-
It's popular, but known to have a "difficult" time binding with third party libraries :-( Good luck and let me know if you need some more help.
Beta Was this translation helpful? Give feedback.
{{actor}} deleted this content .
-
large-v3-turbo whisper.cpp 1.7.4 Go Bindings macOS 15.2 Core ML
sustained transcription throughput:
M1 Max: 2×23× or 38×
2020 Mac mini: 2×24× or 36×
after 10+ hours, instances converge towards 23–24×
if running beyond main memory or GPU capabilities, one or all instances will exhibit periods ~0.1× or hang
longer streams = higher numbers, M1 Max: max 3 instances, mini: max 2. Main cpu load only 20%
GPU load 40% per instance
macOS compile commands: #2696 (comment)
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
0 replies
{{actor}} deleted this content .
-
go build macOS:CC="$(brew --prefix)/opt/llvm/bin/clang" go build…
// clang: error: unsupported option '-fopenmp'
// add: CC="$(brew --prefix)/opt/llvm/bin/clang"
// have brew llvm installed #2696 (comment)
go run macOS:W='…/whisper.cpp' LIBRARY_PATH="$W/build/src" CC="$(brew --prefix)/opt/llvm/bin/clang" DYLD_LIBRARY_PATH="$W/build/src" go run ./cmd/myGoMainDirectory
// whisper.h error: add: W='…/whisper.cpp' C_INCLUDE_PATH="$W/include:$W/ggml/include"
// link/clang error: add:W='…/whisper.cpp' LIBRARY_PATH="$W/build/src"
Go executable macOS:DYLD_LIBRARY_PATH='…/whisper.cpp/build/src' myGoExecutable
// dyld[13346]: Library not loaded: @rpath/libwhisper.1.dylib
// add DYLD_LIBRARY_PATH
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
2 replies
{{actor}} deleted this content .
-
Hello @resolutecacke, do you have something for
ework Metal -framework Foundation -framework CoreGraphics -O2 -g -framework CoreFoundation
clang: error: unsupported option '-fopenmp'
make: *** [run] Error 1
Maybe? I running on M1 Pro and I run using: C_INCLUDE_PATH="/.../projects/whisper.cpp/include" LIBRARY_PATH="/.../projects/whisper.cpp/bindings/go/build/src" CGO_CFLAGS="-I /.../projects/whisper.cpp/include -I /.../projects/whisper.cpp/ggml/include" go run ./cmd/serve
Beta Was this translation helpful? Give feedback.
-
I'm stupid, I'm using default clang, you need to use llvm ^
Beta Was this translation helpful? Give feedback.
{{actor}} deleted this content .
-
after 5+ days of transcribing, macOS may slow down to around 10×. Possibly macOS restart may be required to regain capabilities. If there is a period of high memory pressure or GPU load, macOS may not recover. It is unclear whether transcription quality is affected
It is necessary to constantly measure and monitor throughput per context-process method invocation, per audio file and per executable invocation. We know macOS is garbage and this is how that shows. It’s the processor that’s great
The speed multiple is fairly constant, but higher for non-speech noise or silence, or longer files
eg. if speech transcribes at 33×, noise may transcribe at 42× and average may be 38×
If it’s not 20×+, restart the executable or reboot macOS. Should be done to secure transcription quality
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
0 replies
{{actor}} deleted this content .
-
Hello all.
I'll does it for Darwin arm64.
You can use it if you need, or make PRs to support other platforms.
https://github.com/AlexSnet/whisper.go
The main idea is to store *.a with go code. Ok, it's not very good for main repo, but it is not very bad way for Golang binding, IMHO.
A tiny explanations how to get c-archive files:
cmake -B build -DBUILD_SHARED_LIBS=OFFcmake --build buildlibwhisper.a from build/src/libwhisper.aAlso you need to do something like above with ggml repo and get all the c-archive files from build directory.
After all you can put them into one directory and point them as libraries using CGO like this: #cgo LDFLAGS: -ldl -lm -lpthread -lcommon -lcommon-ggml -lggml -lggml-base -lggml-blas -lggml-cpu -lggml-metal -lwhisper -lstdc++
Maybe my method is not good and there is something to fix or remake.
Thank you.
PS> All what I did was rewriting a few lines of original whisper.cpp Golang bindings and gluing it together with builded c-archive files and headers.
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
0 replies
-
I have troubles in building for CUDA with Golang bindings in Docker:
-DGGML_CUDA 1 in cmake.I forked this project, made some changes, then it worked. But I don't know why...😂
Help of explanation will be very appreciated 😁.
And here is my Dockerfile when I encounter troubles above.
ARG UBUNTU_VERSION=22.04
ARG CUDA_VERSION=12.6.3
ARG BASE_CUDA_DEV_CONTAINER=nvidia/cuda:${CUDA_VERSION}-devel-ubuntu${UBUNTU_VERSION}
FROM ${BASE_CUDA_DEV_CONTAINER}
WORKDIR /app
ARG CUDA_DOCKER_ARCH=all
ENV CUDA_DOCKER_ARCH=${CUDA_DOCKER_ARCH}
ENV GGML_CUDA=1
RUN test -n "$http_proxy" && echo "Acquire::http::Proxy \"$http_proxy\";" || exit 0 >> /etc/apt/apt.conf.d/proxy.conf
RUN test -n "$https_proxy" && echo "Acquire::https::Proxy \"$https_proxy\";" || exit 0 >> /etc/apt/apt.conf.d/proxy.conf
RUN apt-get update \
&& apt-get install -y build-essential libsdl2-dev ffmpeg wget cmake git curl
RUN rm -f /etc/apt/apt.conf.d/proxy.conf
ENV CUDA_MAIN_VERSION=12.6
ENV LD_LIBRARY_PATH="/usr/local/cuda-${CUDA_MAIN_VERSION}/compat:$LD_LIBRARY_PATH"
RUN git clone --depth=1 "https://github.com/allape/whisper.cpp.git" .
RUN make build
WORKDIR /tmp
ENV GO_VERSION="1.24.1"
ENV GO_ARCH="linux-amd64"
ENV GO_ZIP_URL="https://go.dev/dl/go$GO_VERSION.$GO_ARCH.tar.gz"
RUN curl -L -o go.tar.gz "$GO_ZIP_URL" \
&& rm -rf /usr/local/go \
&& tar -C /usr/local -xzf go.tar.gz
ENV PATH="/usr/local/go/bin:$PATH"
WORKDIR /app/bindings/go
RUN make whisper
WORKDIR /app
COPY go.mod .
COPY go.sum .
RUN go mod download
COPY main.go .
RUN CGO_CFLAGS="-I/app/include -I/app/ggml/include" \
CGO_LDFLAGS="-L/app/build_go/src -L/app/build_go/ggml/src -L/app/build_go/ggml/src/ggml-cuda -L/usr/local/cuda/targets/x86_64-linux/lib -lggml-cuda -lcudart -lcuda -lcublas" \
/usr/local/go/bin/go build -o whisper main.go
EXPOSE 9090
CMD ["./whisper"]
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
0 replies
{{actor}} deleted this content .
-
If anyone needs the vulkan link, you can refer to the following patch:
diff --git a/bindings/go/Makefile b/bindings/go/Makefile
index edcc016..0e062cc 100644
--- a/bindings/go/Makefile
+++ b/bindings/go/Makefile
@@ -16,12 +16,18 @@ MODELS_DIR := models
EXAMPLES_DIR := $(wildcard examples/*)
INCLUDE_PATH := $(abspath ../../include):$(abspath ../../ggml/include)
LIBRARY_PATH := $(abspath ../../${BUILD_DIR}/src:$(abspath ../../${BUILD_DIR}/ggml/src))
+CMAKE_EXT :=
ifeq ($(GGML_CUDA),1)
LIBRARY_PATH := $(LIBRARY_PATH):$(CUDA_PATH)/targets/$(UNAME_M)-linux/lib/
BUILD_FLAGS := -ldflags "-extldflags '-lcudart -lcuda -lcublas'"
endif
+ifeq ($(GGML_VULKAN),1)
+ BUILD_FLAGS := -ldflags "-extldflags '-L $(abspath ../../${BUILD_DIR}/ggml/src/ggml-vulkan) -lvulkan -lggml-vulkan'"
+ CMAKE_EXT:= -DGGML_VULKAN=ON
+endif
+
ifeq ($(UNAME_S),Darwin)
EXT_LDFLAGS := -framework Foundation -framework Metal -framework MetalKit
endif
@@ -31,7 +37,7 @@ all: clean whisper examples
whisper: mkdir
cmake -S ../.. -B ../../${BUILD_DIR} \
-DCMAKE_BUILD_TYPE=Release \
- -DBUILD_SHARED_LIBS=OFF
+ -DBUILD_SHARED_LIBS=OFF $(CMAKE_EXT)
cmake --build ../../${BUILD_DIR} --target whisper
test: model-small whisper modtidy
then:
GGML_VULKAN=1 make examples
thanks: https://github.com/AlexSnet/whisper.go
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
0 replies
-
I have this working inside of a Docker container:
https://github.com/joaquindelcanto/go-whisper.cpp/tree/main
I was pretty new to many of the technologies involved, so It took me some days of learning and troubleshooting to get to this point. This has been a fun project to help me learn! I hope that this template repo will help others get started more easily. 🙏😎
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
0 replies
-
Beta Was this translation helpful? Give feedback.
You must be logged in to vote
0 replies
Heading
Bold
Italic
Quote
Code
Link
Numbered list
Unordered list
Task list
Attach files
Mention
Reference
Menu reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emojiYou can’t perform that action at this time.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4