Benchmark Mar_26_2018

Transcription

1 Benchmark Mar_26_2018 Referens Tensorflow Official Benchmarks (May 2017, GitHub sour): IBM Power9 benchmark results (Nov 2017, 1.4.0): Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour, Fabook (Jun 2017): /imagenet1kin1h5.pdf Benchmark Sour Code Official TF Benchmark System Characteristics Instan type: NVIDIA DGX-1 GPU: 8x NVIDIA Tesla P100 OS: Ubuntu LTS with tests run via Docker CUDA / cudnn: 8.0 / 5.1 TensorFlow GitHub hash: b1e174e Benchmark GitHub hash: 9165a70 Build Command: bazel build -c opt --copt=-march="haswell" --config=cuda //tensorflow/tools/pip_package :build_pip_package Disk: Local SSD DataSet: ImageNet Test Date: May 2017 Our System Characteristics (more details in GitHub Repo) Instan type: IBM Power9, 8335-GTG AC922 server CPU: 2x 20-core IBM POWER9 2.00GHz SDRAM: 512G DDR4 GPU: 4x NVIDIA Tesla V100, 5120 cores, 16 GB HBM 2 Disk: Local SSD OS: Red Hat Enterprise Linux Server release 7.4 Python Distribution: Anaconda python CUDA / cudnn: 9.1/7.0.5 TensorFLow Version: DataSet: ImageNet (synthetic) Precision: floating point 32 and 16 Test Date: Mar The following table is the result of running with the same configurations as the official Tensorflow benchmark mentioned in "Referen" section above: base_model FP_type batch_size n_gpus variable_update local_parameter _devi image_per_sec 0 alexnet replicated N/A alexnet replicated N/A alexnet replicated N/A alexnet replicated N/A alexnet replicated N/A alexnet replicated N/A

2 6 inption parameter_server cpu_parameterdevi 7 inption parameter_server cpu_parameterdevi 8 inption parameter_server cpu_parameterdevi 9 inption parameter_server cpu_parameterdevi 10 inption parameter_server cpu_parameterdevi 11 inption parameter_server cpu_parameterdevi 12 resnet parameter_server cpu_parameterdevi 13 resnet parameter_server cpu_parameterdevi 14 resnet parameter_server cpu_parameterdevi 15 resnet parameter_server cpu_parameterdevi 16 resnet parameter_server cpu_parameterdevi 17 resnet parameter_server cpu_parameterdevi 18 resnet parameter_server cpu_parameterdevi 19 resnet parameter_server cpu_parameterdevi 20 resnet parameter_server cpu_parameterdevi 21 resnet parameter_server cpu_parameterdevi 22 resnet parameter_server cpu_parameterdevi 23 resnet parameter_server cpu_parameterdevi vgg replicated N/A vgg replicated N/A vgg replicated N/A vgg replicated N/A vgg replicated N/A vgg replicated N/A This figure compares the result we get with Tensorflow official ones. Green bars stand for our benchmark results using floating point 16. Red bars are the official Tensorflow result. Blue bars stand for our benchmark results using floating point 32.

3 This figure shows the performan ratio of our floating point 16 and 32 benchmarks with respect to Tensorflow official results: The following table provides a more comprehensive benchmark results on our system: base_model FP_type batch_size n_gpus variable_update local_parameter _devi image_per_sec 0 alexnet parameter_server N/A alexnet replicated N/A alexnet parameter_server N/A alexnet replicated N/A alexnet parameter_server N/A alexnet replicated N/A alexnet parameter_server N/A alexnet replicated N/A alexnet parameter_server N/A alexnet replicated N/A alexnet parameter_server N/A alexnet replicated N/A

4 12 inption parameter_server gpu_parameterdevi 13 inption parameter_server cpu_parameterdevi 14 inption replicated cpu_parameterdevi 15 inption replicated gpu_parameterdevi 16 inption parameter_server gpu_parameterdevi 17 inption parameter_server cpu_parameterdevi 18 inption replicated cpu_parameterdevi 19 inption replicated gpu_parameterdevi 20 inption parameter_server gpu_parameterdevi 21 inption parameter_server cpu_parameterdevi 22 inption replicated gpu_parameterdevi 23 inption replicated cpu_parameterdevi 24 inption parameter_server cpu_parameterdevi 25 inption parameter_server gpu_parameterdevi 26 inption replicated gpu_parameterdevi 27 inption replicated cpu_parameterdevi 28 inption parameter_server cpu_parameterdevi 29 inption parameter_server gpu_parameterdevi 30 inption replicated cpu_parameterdevi 31 inption replicated gpu_parameterdevi 32 inption parameter_server cpu_parameterdevi 33 inption parameter_server gpu_parameterdevi 34 inption replicated cpu_parameterdevi 35 inption replicated gpu_parameterdevi inption replicated no_parameterdevi inption replicated no_parameterdevi inption replicated no_parameterdevi inption replicated no_parameterdevi inption replicated no_parameterdevi inption replicated no_parameterdevi inption parameter_server gpu_parameterdevi 43 inption parameter_server cpu_parameterdevi 44 inption replicated cpu_parameterdevi 45 inption replicated gpu_parameterdevi 46 inption parameter_server gpu_parameterdevi

5 47 inption parameter_server cpu_parameterdevi 48 inption replicated cpu_parameterdevi 49 inption replicated gpu_parameterdevi 50 inption parameter_server cpu_parameterdevi 51 inption parameter_server gpu_parameterdevi 52 inption replicated cpu_parameterdevi 53 inption replicated gpu_parameterdevi 54 inption parameter_server gpu_parameterdevi 55 inption parameter_server cpu_parameterdevi 56 inption replicated cpu_parameterdevi 57 inption replicated gpu_parameterdevi 58 inption parameter_server cpu_parameterdevi 59 inption parameter_server gpu_parameterdevi 60 inption replicated gpu_parameterdevi 61 inption replicated cpu_parameterdevi 62 inption parameter_server gpu_parameterdevi 63 inption parameter_server cpu_parameterdevi 64 inption replicated gpu_parameterdevi 65 inption replicated cpu_parameterdevi 66 resnet parameter_server cpu_parameterdevi 67 resnet parameter_server gpu_parameterdevi 68 resnet replicated gpu_parameterdevi 69 resnet replicated cpu_parameterdevi 70 resnet parameter_server cpu_parameterdevi 71 resnet parameter_server gpu_parameterdevi 72 resnet replicated cpu_parameterdevi 73 resnet replicated gpu_parameterdevi 74 resnet parameter_server gpu_parameterdevi 75 resnet parameter_server cpu_parameterdevi 76 resnet replicated cpu_parameterdevi 77 resnet replicated gpu_parameterdevi 78 resnet parameter_server gpu_parameterdevi 79 resnet parameter_server cpu_parameterdevi

6 80 resnet replicated cpu_parameterdevi 81 resnet replicated gpu_parameterdevi 82 resnet parameter_server cpu_parameterdevi 83 resnet parameter_server gpu_parameterdevi 84 resnet replicated cpu_parameterdevi 85 resnet replicated gpu_parameterdevi 86 resnet parameter_server cpu_parameterdevi 87 resnet parameter_server gpu_parameterdevi 88 resnet replicated cpu_parameterdevi 89 resnet replicated gpu_parameterdevi resnet replicated no_parameterdevi resnet replicated no_parameterdevi resnet replicated no_parameterdevi resnet parameter_server cpu_parameterdevi 94 resnet parameter_server gpu_parameterdevi 95 resnet replicated gpu_parameterdevi 96 resnet replicated cpu_parameterdevi 97 resnet parameter_server cpu_parameterdevi 98 resnet parameter_server gpu_parameterdevi 99 resnet replicated cpu_parameterdevi 100 resnet replicated gpu_parameterdevi 101 resnet parameter_server cpu_parameterdevi 102 resnet parameter_server gpu_parameterdevi 103 resnet replicated cpu_parameterdevi 104 resnet replicated gpu_parameterdevi 105 resnet parameter_server cpu_parameterdevi 106 resnet parameter_server gpu_parameterdevi 107 resnet replicated cpu_parameterdevi 108 resnet replicated gpu_parameterdevi 109 resnet parameter_server gpu_parameterdevi 110 resnet parameter_server cpu_parameterdevi 111 resnet replicated gpu_parameterdevi 112 resnet replicated cpu_parameterdevi 113 resnet parameter_server cpu_parameterdevi

7 114 resnet parameter_server gpu_parameterdevi 115 resnet replicated cpu_parameterdevi 116 resnet replicated gpu_parameterdevi 117 resnet152_v parameter_server cpu_parameterdevi 118 resnet152_v parameter_server gpu_parameterdevi 119 resnet152_v replicated cpu_parameterdevi 120 resnet152_v replicated gpu_parameterdevi 121 resnet152_v parameter_server cpu_parameterdevi 122 resnet152_v parameter_server gpu_parameterdevi 123 resnet152_v replicated gpu_parameterdevi 124 resnet152_v replicated cpu_parameterdevi 125 resnet152_v parameter_server cpu_parameterdevi 126 resnet152_v parameter_server gpu_parameterdevi 127 resnet152_v replicated cpu_parameterdevi 128 resnet152_v replicated gpu_parameterdevi 129 resnet152_v parameter_server gpu_parameterdevi 130 resnet152_v parameter_server cpu_parameterdevi 131 resnet152_v replicated gpu_parameterdevi 132 resnet152_v replicated cpu_parameterdevi 133 resnet152_v parameter_server gpu_parameterdevi 134 resnet152_v parameter_server cpu_parameterdevi 135 resnet152_v replicated gpu_parameterdevi 136 resnet152_v replicated cpu_parameterdevi 137 resnet152_v parameter_server cpu_parameterdevi 138 resnet152_v parameter_server gpu_parameterdevi 139 resnet152_v replicated gpu_parameterdevi 140 resnet152_v replicated cpu_parameterdevi resnet152_v replicated no_parameterdevi resnet152_v replicated no_parameterdevi resnet152_v replicated no_parameterdevi resnet152_v parameter_server cpu_parameterdevi 145 resnet152_v parameter_server gpu_parameterdevi 146 resnet152_v replicated gpu_parameterdevi 147 resnet152_v replicated cpu_parameterdevi

8 148 resnet152_v parameter_server gpu_parameterdevi 149 resnet152_v parameter_server cpu_parameterdevi 150 resnet152_v replicated gpu_parameterdevi 151 resnet152_v replicated cpu_parameterdevi 152 resnet152_v parameter_server gpu_parameterdevi 153 resnet152_v parameter_server cpu_parameterdevi 154 resnet152_v replicated gpu_parameterdevi 155 resnet152_v replicated cpu_parameterdevi 156 resnet152_v parameter_server gpu_parameterdevi 157 resnet152_v parameter_server cpu_parameterdevi 158 resnet152_v replicated gpu_parameterdevi 159 resnet152_v replicated cpu_parameterdevi 160 resnet152_v parameter_server cpu_parameterdevi 161 resnet152_v parameter_server gpu_parameterdevi 162 resnet152_v replicated gpu_parameterdevi 163 resnet152_v replicated cpu_parameterdevi 164 resnet152_v parameter_server cpu_parameterdevi 165 resnet152_v parameter_server gpu_parameterdevi 166 resnet152_v replicated cpu_parameterdevi 167 resnet152_v replicated gpu_parameterdevi 168 resnet parameter_server cpu_parameterdevi 169 resnet parameter_server gpu_parameterdevi 170 resnet replicated cpu_parameterdevi 171 resnet replicated gpu_parameterdevi 172 resnet parameter_server gpu_parameterdevi 173 resnet parameter_server cpu_parameterdevi 174 resnet replicated cpu_parameterdevi 175 resnet replicated gpu_parameterdevi 176 resnet parameter_server cpu_parameterdevi 177 resnet parameter_server gpu_parameterdevi 178 resnet replicated cpu_parameterdevi 179 resnet replicated gpu_parameterdevi 180 resnet parameter_server cpu_parameterdevi

9 181 resnet parameter_server gpu_parameterdevi 182 resnet replicated cpu_parameterdevi 183 resnet replicated gpu_parameterdevi 184 resnet parameter_server cpu_parameterdevi 185 resnet parameter_server gpu_parameterdevi 186 resnet replicated cpu_parameterdevi 187 resnet replicated gpu_parameterdevi 188 resnet parameter_server cpu_parameterdevi 189 resnet parameter_server gpu_parameterdevi 190 resnet replicated cpu_parameterdevi 191 resnet replicated gpu_parameterdevi resnet replicated no_parameterdevi resnet replicated no_parameterdevi resnet replicated no_parameterdevi resnet replicated no_parameterdevi resnet replicated no_parameterdevi resnet replicated no_parameterdevi resnet parameter_server cpu_parameterdevi 199 resnet parameter_server gpu_parameterdevi 200 resnet replicated cpu_parameterdevi 201 resnet replicated gpu_parameterdevi 202 resnet parameter_server gpu_parameterdevi 203 resnet parameter_server cpu_parameterdevi 204 resnet replicated cpu_parameterdevi 205 resnet replicated gpu_parameterdevi 206 resnet parameter_server gpu_parameterdevi 207 resnet parameter_server cpu_parameterdevi 208 resnet replicated gpu_parameterdevi 209 resnet replicated cpu_parameterdevi 210 resnet parameter_server cpu_parameterdevi 211 resnet parameter_server gpu_parameterdevi 212 resnet replicated cpu_parameterdevi 213 resnet replicated gpu_parameterdevi 214 resnet parameter_server cpu_parameterdevi 215 resnet parameter_server gpu_parameterdevi

10 216 resnet replicated cpu_parameterdevi 217 resnet replicated gpu_parameterdevi 218 resnet parameter_server gpu_parameterdevi 219 resnet parameter_server cpu_parameterdevi 220 resnet replicated cpu_parameterdevi 221 resnet replicated gpu_parameterdevi 222 resnet50_v parameter_server cpu_parameterdevi 223 resnet50_v parameter_server gpu_parameterdevi 224 resnet50_v replicated gpu_parameterdevi 225 resnet50_v replicated cpu_parameterdevi 226 resnet50_v parameter_server gpu_parameterdevi 227 resnet50_v parameter_server cpu_parameterdevi 228 resnet50_v replicated gpu_parameterdevi 229 resnet50_v replicated cpu_parameterdevi 230 resnet50_v parameter_server cpu_parameterdevi 231 resnet50_v parameter_server gpu_parameterdevi 232 resnet50_v replicated gpu_parameterdevi 233 resnet50_v replicated cpu_parameterdevi 234 resnet50_v parameter_server cpu_parameterdevi 235 resnet50_v parameter_server gpu_parameterdevi 236 resnet50_v replicated gpu_parameterdevi 237 resnet50_v replicated cpu_parameterdevi 238 resnet50_v parameter_server cpu_parameterdevi 239 resnet50_v parameter_server gpu_parameterdevi 240 resnet50_v replicated cpu_parameterdevi 241 resnet50_v replicated gpu_parameterdevi 242 resnet50_v parameter_server gpu_parameterdevi 243 resnet50_v parameter_server cpu_parameterdevi 244 resnet50_v replicated cpu_parameterdevi 245 resnet50_v replicated gpu_parameterdevi resnet50_v replicated no_parameterdevi resnet50_v replicated no_parameterdevi resnet50_v replicated no_parameterdevi resnet50_v replicated no_parameterdevi resnet50_v replicated no_parameterdevi

11 251 resnet50_v replicated no_parameterdevi resnet50_v parameter_server cpu_parameterdevi 253 resnet50_v parameter_server gpu_parameterdevi 254 resnet50_v replicated cpu_parameterdevi 255 resnet50_v replicated gpu_parameterdevi 256 resnet50_v parameter_server gpu_parameterdevi 257 resnet50_v parameter_server cpu_parameterdevi 258 resnet50_v replicated cpu_parameterdevi 259 resnet50_v replicated gpu_parameterdevi 260 resnet50_v parameter_server cpu_parameterdevi 261 resnet50_v parameter_server gpu_parameterdevi 262 resnet50_v replicated gpu_parameterdevi 263 resnet50_v replicated cpu_parameterdevi 264 resnet50_v parameter_server cpu_parameterdevi 265 resnet50_v parameter_server gpu_parameterdevi 266 resnet50_v replicated gpu_parameterdevi 267 resnet50_v replicated cpu_parameterdevi 268 resnet50_v parameter_server cpu_parameterdevi 269 resnet50_v parameter_server gpu_parameterdevi 270 resnet50_v replicated gpu_parameterdevi 271 resnet50_v replicated cpu_parameterdevi 272 resnet50_v parameter_server gpu_parameterdevi 273 resnet50_v parameter_server cpu_parameterdevi 274 resnet50_v replicated gpu_parameterdevi 275 resnet50_v replicated cpu_parameterdevi vgg parameter_server N/A vgg replicated N/A vgg parameter_server N/A vgg replicated N/A vgg parameter_server N/A vgg replicated N/A vgg parameter_server N/A vgg replicated N/A vgg parameter_server N/A vgg replicated N/A vgg parameter_server N/A vgg replicated N/A vgg replicated N/A vgg replicated N/A

12 290 vgg replicated N/A vgg replicated N/A vgg replicated N/A vgg replicated N/A vgg parameter_server N/A vgg replicated N/A vgg parameter_server N/A vgg replicated N/A vgg parameter_server N/A vgg replicated N/A vgg parameter_server N/A vgg replicated N/A vgg parameter_server N/A vgg replicated N/A vgg parameter_server N/A vgg replicated N/A POWER8 (p8)