Performance Analysis on Multi-Gpus

  • fix global batch_size
resnet152 resnet152 resnet152
global batch_size 128 128 128
local batch_size x num_gpus 64x2 32x4 16x8
performance (imgs/sec) OOM 73.62 112.23
ratio - 1 1.524449878
vgg16 vgg16 vgg16
global batch_size 128 128 128
local batch_size x num_gpus 64x2 32x4 16x8
performance (imgs/sec) 74.72 132.34 181.97
ratio 1 1.7711 2.4353
  • fix local batch_size
vgg16 vgg16 vgg16 vgg16
global batch_size 64 128 256 512
local batch_size x num_gpus 64x1 64x2 64x4 64x8
performance (imgs/sec) 38.79 74.94 144.52 250.44
ratio 1 1.9319 3.7257 6.4563

In [ ]: