7.2. Node

class fhez.nn.graph.node.Node

Abstract class for neural network nodes for traversal/ computation.

abstract backward(gradient)

Calculate backward pass for singular example.

backwards(gradients)

Calculate backward pass for multiple examples simultaneously.

property cache

Get caching dictionary of auxilary data.

abstract property cost

Get the computational cost per forward example of the node.

disable_cache()

Disable caching.

enable_cache()

Enable caching.

abstract forward(x)

Calculate forward pass for singular example.

forwards(xs)

Calculate forward pass for multiple examples simultaneously.

property gradients

Get cached input stack.

For neural networks to calculate any given weight update, it needs to remember atleast the last gradient in the case of stocastic descent, or multiple gradients if implementing batch normalised gradient descent. This is a helper method that initialises a stack so that implementation can be offloaded and made-uniform between all subclasses

property inputs

Get cached input stack.

Neural networks backpropogation requires cached inputs to calculate the gradient with respect to x and the weights. This is a utility method that initialises a stack and allows you to easily append or pop off of it so that the computation can occur in FILO.

property is_cache_enabled

Get status of whether or not caching is enabled.

property optimiser

Get optimiser object, E.G Stocastic Gradient Descent.

probe_shape(lst: list)

Get the shape of a list, assuming each sublist is the same length.

This function is recursive, sending the sublists down and terminating once a type error is thrown by the final point being a non-list

abstract update()

Update node state/ weights for a single example.

updater(parm_names: list, it=None)

Private function to universaly update any Node instance.

To simplify the process of updating so and to reduce code duplication, this function serves to derive all the important information given a parameter dictionary. It will then infer from the dictionary the attributes with which to modify.

abstract updates()

Update node state/ weights for multiple examples simultaneously.