A shim driver allows in-docker nvidia-smi showing correct process list without modify anything
A shim driver allows in-docker nvidia-smi showing correct process list without modify anything.
The NVIDIA driver is not aware of the PID namespace and nvidia-smi has no capability to map global pid to virtual pid, thus it shows nothing. What's more, The NVIDIA driver is proprietary and we have no idea what's going on inside even small part of the Linux NVIDIA driver is open sourced.
NOTE: kernel 5.7.7 build routines don't export kallsyms kernel functions any longer, which means this module may not work properly.
sudo apt install linux-headers-$(uname -r)
. run sudo apt-get install build-essential
to get make
toolset installed.cd
and make
sudo make install
to install the module--gpu
enabled instance and run several cases and check process list via nvidia-smi
to see if all associated processes have been correctly shown0x20
ioctl command with 0xee4
flag to getting the global PID list (under init_pid_ns
) ①
0x20
ioctl command with 0x1f48
flag with previous returned pids as input arguments to getting the process GPU memory consumptions ②
195
and minor is 255
(control dev) which is defined in NVIDIA header file)init_pid_ns
)tested on
Afterwords, we'd like to maintain the project with fully tested and more kernels and NVIDIA drivers supported. However we sincerely hope NVIDIA will fix this with simplicity and professionalism. Thx.