dnn::opt_adam Class Reference

ADAM optimizer class. More...

#include <dnn_opt.h>

Inheritance diagram for dnn::opt_adam:

Public Member Functions
	opt_adam (DNN_Dtype s, DNN_Dtype l=0.0, DNN_Dtype a=0.0, DNN_Dtype b1=0.9, DNN_Dtype b2=0.999, DNN_Dtype e=1e-8)
	ADAM constructor. More...
	~opt_adam ()
void	apply (arma::Cube< DNN_Dtype > &W, arma::Mat< DNN_Dtype > &B, const arma::Cube< DNN_Dtype > &Wgrad, const arma::Mat< DNN_Dtype > &Bgrad)
	Apply the optimizer to the layer parameters. More...
std::string	get_algorithm (void)
	Get the optimizer algorithm information. More...
Public Member Functions inherited from dnn::opt
	opt ()
	~opt ()
void	set_learn_rate_alg (LR_ALG alg, DNN_Dtype a=0.0, DNN_Dtype b=10.0)
	Set learning rate algorithm. More...
void	update_learn_rate (void)
	Update learning rate. More...
DNN_Dtype	get_learn_rate (void)
	Get the learning rate. More...

Private Attributes
arma::Cube< DNN_Dtype >	v
arma::Cube< DNN_Dtype >	m
arma::Mat< DNN_Dtype >	vB
arma::Mat< DNN_Dtype >	mB
DNN_Dtype	beta1
DNN_Dtype	beta2
DNN_Dtype	eps

Additional Inherited Members
Protected Attributes inherited from dnn::opt
std::string	alg
DNN_Dtype	lr
	Learning rate. More...
DNN_Dtype	reg_lambda
	Regularisation parameter lambda. More...
DNN_Dtype	reg_alpha
	Elastic net mix parameter - 0=ridge (L2) .. 1=LASSO (L1) More...
LR_ALG	lr_alg
	Learning rate schedule algorithm. More...
DNN_Dtype	lr_0
	Init value for lr. More...
DNN_Dtype	lr_a
	Internal parameter a. More...
DNN_Dtype	lr_b
	Internal parameter b. More...
arma::uword	it
	Iteration counter. More...

Detailed Description

ADAM optimizer class.

ADAM algorithm, see https://arxiv.org/pdf/1609.04747.pdf

Definition at line 356 of file dnn_opt.h.

Constructor & Destructor Documentation

opt_adam()

dnn::opt_adam::opt_adam ( DNN_Dtype  s, DNN_Dtype  l = 0.0, DNN_Dtype  a = 0.0, DNN_Dtype  b1 = 0.9, DNN_Dtype  b2 = 0.999, DNN_Dtype  e = 1e-8  )

inline

ADAM constructor.

Parameters

[in]	s	Step size - learning rate
[in]	l	Regularisation param lambda
[in]	a	Regularisation param alpha
[in]	b1	beta 1 parameter
[in]	b2	beta 2 parameter
[in]	e	eps parameter

Definition at line 377 of file dnn_opt.h.

~opt_adam()

dnn::opt_adam::~opt_adam ( )

inline

Definition at line 389 of file dnn_opt.h.

Member Function Documentation

apply()

void dnn::opt_adam::apply ( arma::Cube< DNN_Dtype > &  W, arma::Mat< DNN_Dtype > &  B, const arma::Cube< DNN_Dtype > &  Wgrad, const arma::Mat< DNN_Dtype > &  Bgrad  )

inlinevirtual

Apply the optimizer to the layer parameters.

Parameters

[in,out]	W,B	Learnable parameters
[in]	Wgrad,Bgrad	Gradient of the learnable parameters

Implements dnn::opt.

Definition at line 396 of file dnn_opt.h.

get_algorithm()

std::string dnn::opt_adam::get_algorithm ( void  )

inlinevirtual

Get the optimizer algorithm information.

Returns

Algorithm information string

Reimplemented from dnn::opt.

Definition at line 438 of file dnn_opt.h.

Member Data Documentation

beta1

DNN_Dtype dnn::opt_adam::beta1

private

Definition at line 363 of file dnn_opt.h.

beta2

DNN_Dtype dnn::opt_adam::beta2

private

Definition at line 364 of file dnn_opt.h.

eps

DNN_Dtype dnn::opt_adam::eps

private

Definition at line 365 of file dnn_opt.h.

m

arma::Cube<DNN_Dtype> dnn::opt_adam::m

private

Definition at line 360 of file dnn_opt.h.

mB

arma::Mat<DNN_Dtype> dnn::opt_adam::mB

private

Definition at line 362 of file dnn_opt.h.

v

arma::Cube<DNN_Dtype> dnn::opt_adam::v

private

Definition at line 359 of file dnn_opt.h.

vB

arma::Mat<DNN_Dtype> dnn::opt_adam::vB

private

Definition at line 361 of file dnn_opt.h.

---

The documentation for this class was generated from the following file:

• dnn_opt.h