Running nvidia cuopt server on WSL2 from scratch

On a Windows Subsystem for Linux (WSL2) environment there are several steps to be done in order to get cuopt server container running — this might be helpful, thanks Ramakrishna Prabhu for helping me out on this :-)

1. Follow these instructions to get Ubuntu with WSL2

How To Live Without Docker Desktop - A Developer's Perspective | Objectivity Blog

2. Follow instructions to install nvidia-container-toolkit

Installing Docker and The Docker Utility Engine for NVIDIA GPUs — NVIDIA AI Enterprise…

3. Follow instructions to install cuopt server container

https://ngc.nvidia.com/containers/ea-reopt-member-zone:ea-cuopt-server

4. Run container & fix errors

sudo docker run -it --gpus all --rm --network=bridge nvcr.io/ea-reopt-member-zone/ea-cuopt-server

This might lead to an error message like this:

docker: Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: nvidia-container-cli: mount error: file creation failed: /var/lib/docker/overlay2/539c88be4ef911f3886406a0d84829121eca7de952f700fcf378d273c3783852/merged/usr/lib/x86_64-linux-gnu/libnvidia-ml.so.1: file exists: unknown.

4a. Patch docker image like this:

sudo docker run --privileged -it --rm --entrypoint /bin/bash nvcr.io/ea-reopt-member-zone/unrestricted/cuopt-server
# umount /usr/lib/x86_64-linux-gnu/libnvidia-ml.so.1
# umount usr/lib/x86_64-linux-gnu/libcuda.so.1
# rm -rf usr/lib/x86_64-linux-gnu/libcuda.so.1
# rm -rf /usr/lib/x86_64-linux-gnu/libnvidia-ml.so.1

4b. Then in another bash terminal commit changes:

# docker container list 
CONTAINER ID   IMAGE 
b52e37fe5592   nvcr.io/ea-reopt-member-zone/ea-cuopt-server
# docker commit b52e37fe5592 nvcr.io/ea-reopt-member-zone/ea-cuopt-server
# docker images 
sudo docker images
REPOSITORY                                     TAG       IMAGE ID       CREATED          SIZE
nvcr.io/ea-reopt-member-zone/ea-cuopt-server   latest    36228ad5936a   11 seconds ago   14.4GB
nvcr.io/ea-reopt-member-zone/ea-cuopt-server   <none>    21e508bdd168   5 weeks ago      14.4GB

5. Startup container
startup container in bridge mode, exposing explicitly local port 5000 to host port 5000 ( -p “publish” option)

sudo docker run -it --gpus all --rm --network=bridge -p 5000:5000 nvcr.io/ea-reopt-member-zone/ea-cuopt-server

6. Startup server

# conda activate cuopt
# python cuopt-service/service.py

7. Display API documentation

If everything has worked out we can display the API documentation on the windows host with this in the browser, nice thing with OpenAPI :

http://localhost:5000/docs

you should see something like this:

FastAPI documentation page