Hi,
I am performing somehow similar benchmark, but on caffenet128 (and moving to ResNets now) on ImageNet.
One thing, that I have found - the best position of BN in non-res net is after ReLU and without scale+bias layer (https://github.com/ducha-aiki/caffenet-benchmark/blob/master/batchnorm.md):
May be, it is worth testing too.
Second, results on CIFAR-10 often contradicts results on ImageNet. I.e., leaky ReLU > ReLU on CIFAR, but worse on ImageNet.
P.S. We could cooperate in ImageNet testing, if you agree.
Viliami, sahilg06, giussepi and aromanro
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4