Use jkryl/spdk-sys

[jkozlowski]

No description provided.

[jkryl]

👍

[jkozlowski]

Status:

• I think I need the newest Ubuntu because of some packages required.
• CircleCI latest image is 16.04.

Because of that and because of being able to snapshot the compilation process to speed up the build, I'd like to run everything in docker. But ordinarily, the dpdk requirements are not available in vanilla docker.

Therefore, we run "machine" executor which gives us a docker enabled VM that we control. Currently, I managed to build everything inside of a docker container on an "ubuntu-1604:201903-01" VM.

Then I try to docker run the tests but having issues with binding devices (couldn't enable either of those: https://doc.dpdk.org/guides/linux_gsg/linux_drivers.html) and then hugepages.

It looks like running on top of circleci/classic:201710-01 image exposes the most kernel modules (looks like vfio-pci is available) and setup.sh script exits successfully, but then the test fails saying there are no hugepages available. When running on ubuntu-1604:201903-01, the setup.sh script fails.

On my laptop it looks like it works with uio_pci_generic and that's probably what I should try to get running.

Next steps:

• See if running setup.sh from the host and then docker running the test works.
• Generally try to read up on these things. It looks like this should be possible, but it's hard to get the right incantation.

Notes:

Successful output on my laptop:

app.c: 425:spdk_app_read_config_file_global_params: *ERROR*: ReactorMask config option is deprecated.  Use -m/--cpumask
command line parameter instead.
Starting SPDK v19.07 / DPDK 19.05.0 initialization...
[ DPDK EAL parameters: hello_blob --no-shconf -c 0x1 --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid5233 ]
EAL: No available hugepages reported in hugepages-1048576kB
EAL: VFIO support initialized
app.c: 627:spdk_app_start: *NOTICE*: Total cores available: 1
reactor.c: 251:_spdk_reactor_run: *NOTICE*: Reactor started on core 0
BDev { name: "AIO1", bdev: 0x7f772800c670 }
BlobStoreBDev { bs_dev: 0x7f7728174860 }
Page size: 4096
Blob created: BlobId { blob_id: 4294967296 }
Opened blob
blobstore has FREE clusters of 8
resized blob now has USED clusters of 8
metadata sync complete
Starting write
Finished writing
Starting read
Finished read
read SUCCESS and data matches!
Closed
Deleted
Successful
thread.c: 207:_free_thread: *WARNING*: poller 0x7f77281747f0 still registered at thread exit