Struct engram::optimizer::Adagrad

pub struct Adagrad {
    pub learning_rate: f64,
    pub accumulators: Tensor,
    pub weight_decay: Option<f64>,
    pub epsilon: Option<f64>,
}

Adaptive Gradient (Adagrad):

Adapts the learning rate based on the history of gradients. Divides the learning rate by a running average of the magnitude of the gradients. This allows the learning rate to decrease for parameters that have consistently large gradients and increase for parameters that have consistently small gradients. Includes an option to apply weight decay regularization to the gradients.

Fields

learning_rate: f64

accumulators: Tensor

weight_decay: Option<f64>

epsilon: Option<f64>

Implementations

impl Adagrad

pub fn new( learning_rate: f64, shape: (usize, usize), weight_decay: Option<f64>, epsilon: Option<f64> ) -> Adagrad

Creates a new Adagrad optimizer with the specified parameters.

Trait Implementations

impl Clone for Adagrad

fn clone(&self) -> Adagrad

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Adagrad

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for Adagrad

fn default() -> Self

Returns the “default value” for a type.

impl Optimizer for Adagrad

fn step(&mut self, weights: &mut Tensor, gradients: &Tensor)