You use the nvidia-smi command to look at GPU memory usage. I’m doing some tests with TensorFlow.

$ nvidia-smi

Tue May 17 00:01:46 2016
+------------------------------------------------------+
| NVIDIA-SMI 352.93 Driver Version: 352.93 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GRID K520 Off | 0000:00:03.0 Off | N/A |
| N/A 38C P0 68W / 125W | 4095MiB / 4095MiB | 98% Default |
+-------------------------------+----------------------+----------------------+
| 1 GRID K520 Off | 0000:00:04.0 Off | N/A |
| N/A 22C P8 17W / 125W | 90MiB / 4095MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 2 GRID K520 Off | 0000:00:05.0 Off | N/A |
| N/A 27C P8 17W / 125W | 90MiB / 4095MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 3 GRID K520 Off | 0000:00:06.0 Off | N/A |
| N/A 23C P8 17W / 125W | 90MiB / 4095MiB | 0% Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 2198 C /home/ubuntu/anaconda3/bin/python 3710MiB |
| 0 2457 C /home/ubuntu/anaconda3/bin/python 368MiB |
| 1 2198 C /home/ubuntu/anaconda3/bin/python 37MiB |
| 1 2457 C /home/ubuntu/anaconda3/bin/python 37MiB |
| 2 2198 C /home/ubuntu/anaconda3/bin/python 37MiB |
| 2 2457 C /home/ubuntu/anaconda3/bin/python 37MiB |
| 3 2198 C /home/ubuntu/anaconda3/bin/python 37MiB |
| 3 2457 C /home/ubuntu/anaconda3/bin/python 37MiB |
+-----------------------------------------------------------------------------+