Stochastic Gradient Descent optimizer class. More...
#include <dnn_opt.h>
Inheritance diagram for dnn::opt_SGD:
Public Member Functions | |
| opt_SGD (DNN_Dtype s, DNN_Dtype l=0.0, DNN_Dtype a=0.0) | |
| SGD constructor. More... | |
| ~opt_SGD () | |
| void | apply (arma::Cube< DNN_Dtype > &W, arma::Mat< DNN_Dtype > &B, const arma::Cube< DNN_Dtype > &Wgrad, const arma::Mat< DNN_Dtype > &Bgrad) |
| Apply the optimizer to the layer parameters. More... | |
| std::string | get_algorithm (void) |
| Get the optimizer algorithm information. More... | |
 Public Member Functions inherited from dnn::opt | |
| opt () | |
| ~opt () | |
| void | set_learn_rate_alg (LR_ALG alg, DNN_Dtype a=0.0, DNN_Dtype b=10.0) |
| Set learning rate algorithm. More... | |
| void | update_learn_rate (void) |
| Update learning rate. More... | |
| DNN_Dtype | get_learn_rate (void) |
| Get the learning rate. More... | |
Additional Inherited Members | |
| Protected Attributes inherited from dnn::opt | |
| std::string | alg |
| DNN_Dtype | lr |
| Learning rate. More... | |
| DNN_Dtype | reg_lambda |
| Regularisation parameter lambda. More... | |
| DNN_Dtype | reg_alpha |
| Elastic net mix parameter - 0=ridge (L2) .. 1=LASSO (L1) More... | |
| LR_ALG | lr_alg |
| Learning rate schedule algorithm. More... | |
| DNN_Dtype | lr_0 |
| Init value for lr. More... | |
| DNN_Dtype | lr_a |
| Internal parameter a. More... | |
| DNN_Dtype | lr_b |
| Internal parameter b. More... | |
| arma::uword | it |
| Iteration counter. More... | |
Detailed Description
Stochastic Gradient Descent optimizer class.
Vanilla SGD algorithm, see https://arxiv.org/pdf/1609.04747.pdf
Constructor & Destructor Documentation
◆ opt_SGD()
◆ ~opt_SGD()
Member Function Documentation
◆ apply()
◆ get_algorithm()
|
inlinevirtual |
The documentation for this class was generated from the following file:
