Example on using MLAS directly

[mmeterel]

@ymwangg
Hi,
I was trying to use MLAS directly without onnxruntime and found your repo. Do you have any example where I can use SGEMM (for example) directly using MlasGEMM API?
Thanks

[ymwangg]

Yes, this may be useful. https://github.com/ymwangg/mlas-old/blob/main/mlas_sgemm.cc

[mmeterel]

@ymwangg Thanks alot. It is very useful. I have written a similar code, but having hard time to link it. I see an error like this:

sgemm.cpp:(.text._Z13MlasGemmBatch15CBLAS_TRANSPOSES_mmmPK22MLAS_SGEMM_DATA_PARAMSmPN11onnxruntime11concurrency10ThreadPoolE+0xdf): undefined reference to `onnxruntime::concurrency::ThreadPool::DegreeOfParallelism(onnxruntime::concurrency::ThreadPool const*)'

I am building onnxruntime locally using the command below:
./build.sh --config Release --parallel --build_shared_lib

And building/linking my code as follows:

g++ -std=c++11 -c test.cpp -I/onnxruntime/onnxruntime/core/mlas/lib
g++ -std=c++11 -o test test.o -I/onnxruntime/onnxruntime/core/mlas/lib -L/onnxruntime/build/Linux/Release/ -lonnxruntime_mlas -lonnxruntime
/bin/ld: /onnxruntime/build/Linux/Release//libonnxruntime_mlas.a(sgemm.cpp.o): in function `MlasGemmBatch(CBLAS_TRANSPOSE, CBLAS_TRANSPOSE, unsigned long, unsigned long, unsigned long, MLAS_SGEMM_DATA_PARAMS const*, unsigned long, onnxruntime::concurrency::ThreadPool*)':

This is the first time I am working on onnxruntime, so I am not quite familiar with repo. Do you have any advice on the error above?

If the error is not familiar, I can also appreciate if you can share how you are building/linking the example you shared.
Thanks alot.

[mmeterel]

Attached is the sample code I am trying to build. Basically, I am trying to use MlasGemm API directly. Not sure if this is possible at all.

test.txt

[ymwangg]

To compile my example code, you can run:

git clone https://github.com/ymwangg/mlas-old.git
cd mlas-old
cmake .
make -j8
./test 10 10 10

undefined reference typically means you need to manually add the corresponding source files to your target similar to here

mlas/CMakeLists.txt

Lines 28 to 49 in 2fa74e6

	set(mlas_common_srcs
	${ONNXRUNTIME_ROOT}/lib/platform.cpp
	${ONNXRUNTIME_ROOT}/lib/threading.cpp
	${ONNXRUNTIME_ROOT}/lib/sgemm.cpp
	${ONNXRUNTIME_ROOT}/lib/qgemm.cpp
	${ONNXRUNTIME_ROOT}/lib/qdwconv.cpp
	${ONNXRUNTIME_ROOT}/lib/convolve.cpp
	${ONNXRUNTIME_ROOT}/lib/pooling.cpp
	${ONNXRUNTIME_ROOT}/lib/transpose.cpp
	${ONNXRUNTIME_ROOT}/lib/reorder.cpp
	${ONNXRUNTIME_ROOT}/lib/snchwc.cpp
	${ONNXRUNTIME_ROOT}/lib/activate.cpp
	${ONNXRUNTIME_ROOT}/lib/logistic.cpp
	${ONNXRUNTIME_ROOT}/lib/tanh.cpp
	${ONNXRUNTIME_ROOT}/lib/erf.cpp
	${ONNXRUNTIME_ROOT}/lib/compute.cpp
	${ONNXRUNTIME_ROOT}/lib/quantize.cpp
	${ONNXRUNTIME_ROOT}/lib/qladd.cpp
	${ONNXRUNTIME_ROOT}/lib/qlmul.cpp
	${ONNXRUNTIME_ROOT}/lib/qpostprocessor.cpp
	${ONNXRUNTIME_ROOT}/lib/qlgavgpool.cpp
	)