Adam

AdaGrad

AdaDelta


参考：

http://blog.csdn.net/qq_21460525/article/details/70146665