In this post I will cover the process I used to get a program that uses CAPI to run within a docker container.
Base system config
The system I’m working on has been used inthe past to deploy my Hello AFU example project, which I will get to run in this Docker container. The host system already has the cxl support in the kernel, as well as my Hello AFU flashed onto my AlphaData KU3 device.
I first tested to ensure my AFU and CAPI application were still runing properly on the host system and installed Docker using Ubuntu’s docker package in apt.
Base docker image for PPC64LE
It’s been a while since I’ve used docker, so I’m a bit rusty. I decided to use the ppc64le/ubuntu from docker hub as my base image.
I started with a fairly minimal image, just to test that everything is working as expected. This simply pulls the dockerhub image, and runs a few apt commands.
FROM ppc64le/ubuntu
RUN apt update -y
RUN apt upgrade -y
RUN apt install -y git build-essential
CMD ["/bin/echo", "hellllooooooo!"]
Next I tag a new image, baseimage, with docker build, output below truncated for brevity.
root@Barreleye-15:~/docker-capi-test# docker build -t baseimage .
Sending build context to Docker daemon 2.048 kB
Step 1 : FROM ppc64le/ubuntu
latest: Pulling from ppc64le/ubuntu
0847857e6401: Pull complete
f8c18c152457: Pull complete
8643975d001d: Pull complete
d5802da4b3a0: Pull complete
fe172ed92137: Pull complete
Digest: sha256:5349f00594c719455f2c8e6f011b32758dcd326d8e225c737a55c15cf3d6948c
Status: Downloaded newer image for ppc64le/ubuntu:latest
---> 1967d889e07f
Step 2 : RUN apt update -y
---> Running in 8bb2d361d36c
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
Get:1 http://ports.ubuntu.com/ubuntu-ports xenial InRelease [247 kB]
[...]
Fetched 24.1 MB in 4s (4993 kB/s)
Reading package lists...
Building dependency tree...
Reading state information...
25 packages can be upgraded. Run 'apt list --upgradable' to see them.
---> e6e3775a3cd9
Removing intermediate container 8bb2d361d36c
Step 3 : RUN apt upgrade -y
---> Running in b617cbfa00f8
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
Reading package lists...
Building dependency tree...
Reading state information...
Calculating upgrade...
The following packages will be upgraded:
apt base-files bsdutils gcc-5-base libapt-pkg5.0 libblkid1 libc-bin libc6
[...]
Processing triggers for libc-bin (2.23-0ubuntu5) ...
---> 6cc12a86896f
Removing intermediate container b617cbfa00f8
Step 4 : RUN apt install -y git build-essential
---> Running in c42e2e2f5e52
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
binutils bzip2 ca-certificates cpp cpp-5 dpkg-dev fakeroot g++ g++-5 gcc
[...]
173 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
---> a3c8b863af74
Removing intermediate container c42e2e2f5e52
Step 5 : CMD /bin/bash echo hellllooooooo!
---> Running in e320ed315285
---> 7f633abdf66d
Removing intermediate container e320ed315285
Successfully built 7f633abdf66d
And I test to see if my image is working:
root@Barreleye-15:~/docker-capi-test# docker run -t baseimage
hellllooooooo!
Huzzah!
Building my CAPI application into the image
Next I will extend my Dockerfile to pull down my hello-afu code and build it. The first I’ll add an additional apt call to pull the necessary library and headers in from the libcxl-dev package. Then I use git and make commands to build my application the same as I would anywhere else. I also set my command line to automaticaly execute the test_afu application when running the container.
FROM ppc64le/ubuntu
RUN apt update -y
RUN apt upgrade -y
RUN apt install -y git build-essential
RUN apt install -y libcxl-dev
RUN git clone https://github.com/KennethWilke/hello-afu
RUN cd hello-afu && make
CMD ["/hello-afu/test_afu"]
I’ll build this image and tag it as hello-afu
root@Barreleye-15:~/docker-capi-test# docker build -t hello-afu .
Sending build context to Docker daemon 2.048 kB
Step 1 : FROM ppc64le/ubuntu
---> 1967d889e07f
Step 2 : RUN apt update -y
---> Using cache
---> daf3a9437751
Step 3 : RUN apt upgrade -y
---> Using cache
---> 3882f8f83b78
Step 4 : RUN apt install -y git build-essential
---> Using cache
---> 2cf498f74f15
Step 5 : RUN apt install -y libcxl-dev
---> Running in dbeb87ec38fd
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
libcxl1
The following NEW packages will be installed:
libcxl-dev libcxl1
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 58.5 kB of archives.
After this operation, 178 kB of additional disk space will be used.
Get:1 http://ports.ubuntu.com/ubuntu-ports xenial/universe ppc64el libcxl1 ppc64el 1.3-0ubuntu2 [12.6 kB]
Get:2 http://ports.ubuntu.com/ubuntu-ports xenial/universe ppc64el libcxl-dev ppc64el 1.3-0ubuntu2 [45.9 kB]
debconf: delaying package configuration, since apt-utils is not installed
Fetched 58.5 kB in 0s (124 kB/s)
Selecting previously unselected package libcxl1.
(Reading database ... 16348 files and directories currently installed.)
Preparing to unpack .../libcxl1_1.3-0ubuntu2_ppc64el.deb ...
Unpacking libcxl1 (1.3-0ubuntu2) ...
Selecting previously unselected package libcxl-dev.
Preparing to unpack .../libcxl-dev_1.3-0ubuntu2_ppc64el.deb ...
Unpacking libcxl-dev (1.3-0ubuntu2) ...
Processing triggers for libc-bin (2.23-0ubuntu5) ...
Setting up libcxl1 (1.3-0ubuntu2) ...
Setting up libcxl-dev (1.3-0ubuntu2) ...
Processing triggers for libc-bin (2.23-0ubuntu5) ...
---> db6ef39a11d3
Removing intermediate container dbeb87ec38fd
Step 6 : RUN git clone https://github.com/KennethWilke/hello-afu
---> Running in ca75aa96477f
Cloning into 'hello-afu'...
---> 4471e2887fad
Removing intermediate container ca75aa96477f
Step 7 : RUN cd hello-afu && make
---> Running in e9cd952ef56c
gcc -Wall -o test_afu test_afu.c -I ~/workprojects/pslse/libcxl -L ~/workprojects/pslse/libcxl -lcxl -lpthread
test_afu.c: In function 'main':
test_afu.c:56:9: warning: format '%llu' expects argument of type 'long long unsigned int', but argument 2 has type '__u64 {aka long unsigned int}' [-Wformat=]
printf(" example->size: %llu\n", example->size);
^
---> 75346e99c3a7
Removing intermediate container e9cd952ef56c
Step 8 : CMD /hello-afu/test_afu
---> Running in 53c0cb116657
---> dd2ea88f3a11
Removing intermediate container 53c0cb116657
Successfully built dd2ea88f3a11
And I’ll test out this new image:
root@Barreleye-15:~/docker-capi-test# docker run -t hello-afu
Failed to open AFU: No such file or directory
So far my docker file seems setup properly, but the test_afu application is not finding the capi device, which is expected as it’s not yet within the containers view of the world.
Sharing the CXL device
In my first attempt, I tried to directly mount my /dev/cxl from host to container, this resulted in a different error regarding the permissions
root@Barreleye-15:~/docker-capi-test# docker run -v /dev/cxl/:/dev/cxl -t hello-afu
Failed to open AFU: Operation not permitted
After some documentation perusing, I found the --privileged flag that allows me to use this type of device sharing.
root@Barreleye-15:~/docker-capi-test# docker run -v /dev/cxl/:/dev/cxl --privileged -t hello-afu
[example structure
example: 0x1002c4d0200
example->size: 128
example->stripe1: 0x1002c4d0300
example->stripe2: 0x1002c4d0400
example->parity: 0x1002c4d0580
&(example->done): 0x1002c4d0220
Attached to AFU
Waiting for completion by AFU
PARITY:
That is some proper parity! This is exactly what I'm expecting to see. I'd also like to see this running on some real gear soon
Releasing AFU
Success! I now have my application from my docker container that can interface with the host CAPI device. I hope this post proves helpful, please leave any feedback you may have in the comments!
Thank you for the great post