Can't build

[olivierlaporte]

Hello I try to reproduce the work produced by arbdevml for issue #49.

I fork the repository.
I have docker, make installed.

I reproduce the described steps :

• on vosk-browser folder

make builder

I have the following output :

#7 [ 4/30] RUN git clone -b vosk --single-branch https://github.com/alphacep/kaldi . &&     git checkout 6417ac1dece94783e80dfbac0148604685d27579
#7 sha256:d72b762a9137ae3da9126377d52f4ac1e5fb4134afc31851f4a093636254bbdc
#7 0.474 Cloning into '.'...
#7 19.32 fatal: reference is not a tree: 6417ac1dece94783e80dfbac0148604685d27579
#7 ERROR: executor failed running [/bin/sh -c git clone -b vosk --single-branch https://github.com/alphacep/kaldi . &&     git checkout 6417ac1dece94783e80dfbac0148604685d27579]: exit code: 128
------
 > [ 4/30] RUN git clone -b vosk --single-branch https://github.com/alphacep/kaldi . &&     git checkout 6417ac1dece94783e80dfbac0148604685d27579:
------
executor failed running [/bin/sh -c git clone -b vosk --single-branch https://github.com/alphacep/kaldi . &&     git checkout 6417ac1dece94783e80dfbac0148604685d27579]: exit code: 128

I think the kaldi project was updated and the git hash does not exists anymore

=> I check the rest of the Dockerfile file and i see that a clone of an inria repository is needed... but this repository seems to be not accessible ?

Someone can help me because i really want to help and have the voice fingerprinting feature with spk model that i have already experiment on python distro ?

[simonfn]

Hello, I got the same issue. I tried to checkout to another commit by replacing "6417ac1dece94783e80dfbac0148604685d27579" to "9af2c5c16389e141f527ebde7ee432a0c1df9fb9", which is the commit done around the same date as the commit from "vosk/". But it fails to apply the "kaldi.patch".

#10 [ 6/30] RUN git apply -v /tmp/kaldi.patch
#10 1.862 /tmp/kaldi.patch:230: trailing whitespace.
#10 1.862
#10 1.862 /tmp/kaldi.patch:232: trailing whitespace.
#10 1.862
#10 1.862 /tmp/kaldi.patch:269: trailing whitespace.
#10 1.862
#10 1.862 Checking patch src/Makefile...
#10 1.862 error: while searching for:
#10 1.862
#10 1.862 # Reset the default goal, so that the all target will become default
#10 1.862 .DEFAULT_GOAL :=
#10 1.862 all: $(SUBDIRS) matrix/test
#10 1.862       -echo Done
#10 1.862
#10 1.862 mklibdir:
#10 1.862
#10 1.862 error: patch failed: src/Makefile:41
#10 1.862 error: src/Makefile: patch does not apply
#10 1.862 Checking patch src/configure...
#10 1.863 Hunk #1 succeeded at 1026 (offset 4 lines).
#10 1.863 Checking patch src/ivector/ivector-extractor.cc...
#10 1.863 Checking patch src/makefiles/default_rules.mk...
#10 1.863 Checking patch src/makefiles/wasm_clapack.mk...
#10 1.863 Checking patch tools/Makefile...
#10 1.864 error: while searching for:
#10 1.864   # This new OS path is confirmed working on Windows 10 / Cygwin64.
#10 1.864   openfst_add_CXXFLAGS = -g -O2 -Wa,-mbig-obj
#10 1.864 else
#10 1.864   openfst_add_CXXFLAGS = -g -O3 -msse -msse2
#10 1.864 endif
#10 1.864
#10 1.864 openfst-$(OPENFST_VERSION)/Makefile: openfst-$(OPENFST_VERSION)
#10 1.864       cd openfst-$(OPENFST_VERSION)/ && \
#10 1.864         autoreconf -i && \
#10 1.864       ./configure --prefix=`pwd` $(OPENFST_CONFIGURE) CXX="$(CXX)" \
#10 1.864               CXXFLAGS="$(openfst_add_CXXFLAGS) $(CXXFLAGS)" \
#10 1.864
#10 1.864 error: patch failed: tools/Makefile:78
#10 1.864 error: tools/Makefile: patch does not apply
#10 1.864 Checking patch tools/extras/check_dependencies.sh...
#10 1.864 Checking patch tools/openfst.patch...
#10 1.864 /tmp/kaldi.patch:269: new blank line at EOF.
#10 1.864 +
#10 ERROR: executor failed running [/bin/sh -c git apply -v /tmp/kaldi.patch]: exit code: 1
------
 > [ 6/30] RUN git apply -v /tmp/kaldi.patch:
#10 1.864         autoreconf -i && \
#10 1.864       ./configure --prefix=`pwd` $(OPENFST_CONFIGURE) CXX="$(CXX)" \
#10 1.864               CXXFLAGS="$(openfst_add_CXXFLAGS) $(CXXFLAGS)" \
#10 1.864
#10 1.864 error: patch failed: tools/Makefile:78
#10 1.864 error: tools/Makefile: patch does not apply
#10 1.864 Checking patch tools/extras/check_dependencies.sh...
#10 1.864 Checking patch tools/openfst.patch...
#10 1.864 /tmp/kaldi.patch:269: new blank line at EOF.
#10 1.864 +
------
ERROR: failed to solve: executor failed running [/bin/sh -c git apply -v /tmp/kaldi.patch]: exit code: 1
make: *** [Makefile:12: builder] Error 1

Any idea on which commit is the right one ?

[olivierlaporte]

simonfn I have done the same workaround and I have the same error.

I think repositories have changed since last summer and if you have not the builder installed, you cannot release.

[ccoreilly]

I guess I'll have to switch to forks of the repositories which I still have around.

In the meantime you should be able to pull the builder image:

docker pull ghcr.io/ccoreilly/vosk-wasm-builder:0.0.6

[simonfn]

I tried to build the binary from the v.0.0.8 with the docker image from the v.0.0.6, but it fails with :

94 warnings and 1 error generated.
em++: error: '/emsdk/upstream/bin/clang++ -target wasm32-unknown-emscripten -DEMSCRIPTEN -fignore-exceptions -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr -D__EMSCRIPTEN_major__=2 -D__EMSCRIPTEN_minor__=0 -D__EMSCRIPTEN_tiny__=29 -D_LIBCPP_ABI_VERSION=2 -Dunix -D__unix -D__unix__ -Werror=implicit-function-declaration -Xclang -iwithsysroot/include/SDL --sysroot=/emsdk/upstream/emscripten/cache/sysroot -Xclang -iwithsysroot/include/compat -std=c++17 -Wno-unused-function -I/opt/include -I/opt/kaldi/src -I/opt/kaldi/tools/openfst/include -I/io/src -O3 /io/src/../vosk/src/recognizer.cc -c -o /tmp/emscripten_temp_eh0ac_s_/recognizer_2.o' failed (returned 1)
make: *** [Makefile:90: /io/src/../build/release/vosk.bc] Error 1
make: Leaving directory '/io/src'
make: *** [Makefile:19: binary] Error 2

The v.0.0.7 and v.0.0.6 seems to work with the same build environnement with some warnings when compiling .

[olivierlaporte]

I will try on my side and give feedbacks. Thanks a lot @ccoreilly.

[ccoreilly]

@olivierlaporte @simonfn could you test the branch in #66 ?

[simonfn]

@ccoreilly #66 worked for me, although it took quite some times to build the image

[olivierlaporte]

It's also ok on my side. @ccoreilly Thanks you for your quick response and your very quick patch !

And like @simonfn building the image is quite long but it works.

git checkout -b make-builder-reproducible origin/make-builder-reproducible
time make builder

Step 31/32 : RUN rm -rf /tmp/libarchive
 ---> Running in 76862ebc8b05
Removing intermediate container 76862ebc8b05
 ---> ebe4d5360042
Step 32/32 : WORKDIR /opt
 ---> Running in 06b31cbed381
Removing intermediate container 06b31cbed381
 ---> f4353dc52c41
Successfully built f4353dc52c41
Successfully tagged ghcr.io/ccoreilly/vosk-wasm-builder:0.0.7
Successfully tagged ghcr.io/ccoreilly/vosk-wasm-builder:latest

time make binary

/emsdk/upstream/emscripten/cache/sysroot/include/c++/v1/__config:979:48: note: expanded from macro '_LIBCPP_DEPRECATED'
#    define _LIBCPP_DEPRECATED __attribute__ ((deprecated))
                                               ^
59 warnings generated.
em++: warning: generating an executable with an object extension (.bc).  If you meant to build an object file please use `-c, `-r`, or `-shared` [-Wemcc]
mkdir -p /io/src/../build/release/
cd /io/src/../build/release/; \
/emsdk/upstream/emscripten/em++ -o vosk.js /io/src/../build/release/vosk.bc --pre-js /io/src/vosk.pre.js --post-js /io/src/vosk.post.js --bind -std=c++17 -s ALLOW_MEMORY_GROWTH=1 -s ERROR_ON_UNDEFINED_SYMBOLS=0 -s MODULARIZE=1 -s EXPORT_NAME=LoadVosk -s ENVIRONMENT=worker -s EXPORTED_RUNTIME_METHODS=['FS','IDBFS'] -lidbfs.js -O3 -s SINGLE_FILE=1  /opt/kaldi/src/base/kaldi-base.bc /opt/kaldi/src/cudamatrix/kaldi-cudamatrix.bc /opt/kaldi/src/decoder/kaldi-decoder.bc /opt/kaldi/src/feat/kaldi-feat.bc /opt/kaldi/src/fstext/kaldi-fstext.bc /opt/kaldi/src/gmm/kaldi-gmm.bc /opt/kaldi/src/hmm/kaldi-hmm.bc /opt/kaldi/src/ivector/kaldi-ivector.bc /opt/kaldi/src/lat/kaldi-lat.bc /opt/kaldi/src/lm/kaldi-lm.bc /opt/kaldi/src/matrix/kaldi-matrix.bc /opt/kaldi/src/nnet3/kaldi-nnet3.bc /opt/kaldi/src/online2/kaldi-online2.bc /opt/kaldi/src/transform/kaldi-transform.bc /opt/kaldi/src/tree/kaldi-tree.bc /opt/kaldi/src/util/kaldi-util.bc /opt/kaldi/tools/clapack-wasm/CLAPACK-3.2.1/lapack.a /opt/kaldi/tools/clapack-wasm/CLAPACK-3.2.1/libcblaswr.a /opt/kaldi/tools/clapack-wasm/CBLAS/lib/cblas.a /opt/kaldi/tools/clapack-wasm/f2c_BLAS-3.8.0/blas.a /opt/kaldi/tools/openfst/lib/libfst.a /opt/kaldi/tools/openfst/lib/libfstngram.a /opt/lib/libarchive.so
warning: undefined symbol: _ZN5kaldi5rnnlm21RnnlmComputeStateInfoC1ERKNS0_35RnnlmComputeStateComputationOptionsERKNS_5nnet34NnetERKNS_8CuMatrixIfEE (referenced by top-level compiled C/C++ code)
warning: undefined symbol: _ZN5kaldi5rnnlm22ReadSparseWordFeaturesERNSt3__213basic_istreamIcNS1_11char_traitsIcEEEEiPNS_12SparseMatrixIfEE (referenced by top-level compiled C/C++ code)
warning: undefined symbol: _ZN5kaldi5rnnlm26KaldiRnnlmDeterministicFst5ClearEv (referenced by top-level compiled C/C++ code)
warning: undefined symbol: _ZN5kaldi5rnnlm26KaldiRnnlmDeterministicFstC1EiRKNS0_21RnnlmComputeStateInfoE (referenced by top-level compiled C/C++ code)
warning: undefined symbol: do_fio (referenced by top-level compiled C/C++ code)
warning: undefined symbol: e_wsfe (referenced by top-level compiled C/C++ code)
warning: undefined symbol: i_nint (referenced by top-level compiled C/C++ code)
warning: undefined symbol: popen (referenced by top-level compiled C/C++ code)
warning: undefined symbol: pow_dd (referenced by top-level compiled C/C++ code)
warning: undefined symbol: pow_ri (referenced by top-level compiled C/C++ code)
warning: undefined symbol: r_sign (referenced by top-level compiled C/C++ code)
warning: undefined symbol: s_cat (referenced by top-level compiled C/C++ code)
warning: undefined symbol: s_cmp (referenced by top-level compiled C/C++ code)
warning: undefined symbol: s_copy (referenced by top-level compiled C/C++ code)
warning: undefined symbol: s_wsfe (referenced by top-level compiled C/C++ code)
rsync -r /io/src/../build/release/vosk.js /io/src/../lib/src/vosk-wasm.js
rsync -r vosk.d.ts /io/src/../lib/src/vosk-wasm.d.ts
make: Leaving directory '/io/src'

[arbdevml]

Has anyone successfully implemented this functionality,
if yes, could someone share the experience, please?
Thank you for your answers.

[nshmyrev]

Has anyone successfully implemented this functionality, if yes, could someone share the experience, please? Thank you for your answers.

It works just fine. You'd better describe what is your trouble in detail.

[arbdevml]

Has anyone successfully implemented this functionality, if yes, could someone share the experience, please? Thank you for your answers.

It works just fine. You'd better describe what is your trouble in detail.

@nshmyrev, thank you for your answer.

First step.

git clone https://github.com/ccoreilly/vosk-browser.git vosk_xvector
cd vosk_xvector

git checkout -b make-builder-reproducible origin/make-builder-reproducible

git submodule update --init --recursive && git pull --recurse-submodules

time make builder

docker run --rm -it -v /home/vosk_xvector:/io -w /io ghcr.io/ccoreilly/vosk-wasm-builder:0.0.7 bash -c "apt-get update && apt-get install -y nodejs npm"

time make binary

echo $?

0

The build completed successfully, and the files appeared:
ls ./build/release
vosk.bc
vosk.js

Second step.
Build with file replacement: bindings.cc and vosk.d.ts

cd ../

git clone https://github.com/ccoreilly/vosk-browser.git vosk_xvector_updated
cd vosk_xvector_updated

git checkout -b make-builder-reproducible origin/make-builder-reproducible

git submodule update --init --recursive && git pull --recurse-submodules

Replace the files: bindings.cc and vosk.d.ts (#49)

Skip (this was already performed in the first step):
#time make builder

time make binary

unsuccessful make binary

Could you please advise on how to build with voice fingerprint (x-vector) support?
Do yo have any web js fingerprint (x-vector) example?
Thank you for the Vosk Kaldi project and for your response.