I installed datalore with podman by using
docker-compose.yaml file provided and did very few changes to make it work.
I’m also able to attach external agents since configuration of external agents accepts a parameter called:
additionalOptions where you can pass devices and environment variables.
First of all your documentation for
Set up external agents is wrong!
podman run -it --rm -e NVIDIA_VISIBLE_DEVICES=all jetbrains/datalore-agent:2023.3 nvidia-smi Error: runc: runc create failed: unable to start container process: error during container init: exec: "nvidia-smi": executable file not found in $PATH: OCI runtime attempted to invoke a command that was not found
--device nvidia.com/gpu=all in the podman command it will fail and it’s failing because devices are not getting attached to the container.
Second problem is that: It looks like docker-base agent, which is datalore server itself I believe, does not have
additionalOptions fields where you can put
--device nvidia.com/gpu=all and since I use podman for docker-base agent as well, it sends
--gpus all which podman does not understand from the looks of it. We should have had something similar to external agents have, so that we would pass
--device nvidia.com/gpu=all on docker-base agents as well, then GPUs would be usable on podman installations too.