NeuralNet v1.0.0 NeuralNet.Helpers
These helper functions provide a DSL for specifying network architectures. Look at lib/gru.ex for an example implementation of the Gated Recurrent Unit architecture.
The functions mult, mult_const, add, add_const, custom_net_layer, sigmoid, tanh, pointwise_tanh, tanh_given_weights, and custom_net_layer all take input(s), and create a resulting output vector. A random uid will be generated and returned, for use as reference. A custom uid for important neural network vectors (like for the network’s input and output), can be provided as the final argument for any of these functions.
tanh and sigmoid are network layers. Network layers are train-able connections (that can later be developed through training algorithms like backpropogation) that take into account 1 or more input vectors in creating the values for an output vector. For a sigmoid network layer, output vector components will have values from 0 to 1. For a tanh network layer, output vector components will have values from -1 to 1.
input() returns a reference to the neural network’s input, and output() to its output.
uid() returns a unique identifier (which is used for vector names if a name is not supplied).
previous(vector) returns a reference to the given vector from the previous time frame. This is how recurrent neural networks are built.
mult, mult_const, add, add_const, and pointwise_tanh are all considered pointwise operations.
All vectors must have component definitions, that is, a list of their component_names. Input and output vectors on either side of a pointwise operation must all have the same component definitions. Because of this, vectors are automatically put into groups. These groups of vectors must all share the same definition, so the user need only define 1 vector per group, and the rest of the vectors will automatically obtain their definitions. If creating a new architecture, try running it once done, and an error message will print out the vector groupings that require definitions. Use def_vec and def_vec_by_size to provide vector definitions.
Because of this feature, a lot of network architectures only need the network’s ultimate input and output to be define, and the rest of the vectors will recieve their definitions automatically.
Summary
Functions
Adds together the corresponding components from each of the input vectors
Adds the provided constant to each component of the single input vector. Note, this only takes 1 input vector
This allows for custom network layers using a provided activation_function (instead of tanh or a sigmoid). A function for caluclating the derivative at any value x must also be supplied as activation_function_prime
Specifies a vector component definition given the vec_name, and ids, which is a list of component names
Specifies a vector component definition given the vec_name, and a size. Based on the size, a list of component names will be generated of size size using UIDs
Use this to reference the input vector
Multiplies together the corresponding components from each of the input vectors
Multiplies each component of the single input vector by the provided constant. Note, this only takes 1 input vector
Use this to reference the output vector
Applies the function tanh to all components of the vector, compressing values to be between -1 and 1. Note, this only takes 1 input vector
Use this to reference a vector from a previous time frame
A sigmoid network layer. The output vector’s components will have values between 0 and 1
A tanh network layer. The output vector’s components will have values between -1 and 1
This provides experimental functionality. This is sort of like a network layer, only values for weights are supplied as a vector, instead of developed through training algorithms
Generates a uid (using :erlang.unique_integer([:monotonic]))
Functions
Adds together the corresponding components from each of the input vectors.
Adds the provided constant to each component of the single input vector. Note, this only takes 1 input vector.
This allows for custom network layers using a provided activation_function (instead of tanh or a sigmoid). A function for caluclating the derivative at any value x must also be supplied as activation_function_prime.
Specifies a vector component definition given the vec_name, and ids, which is a list of component names.
Specifies a vector component definition given the vec_name, and a size. Based on the size, a list of component names will be generated of size size using UIDs.
Multiplies together the corresponding components from each of the input vectors.
Multiplies each component of the single input vector by the provided constant. Note, this only takes 1 input vector.
Applies the function tanh to all components of the vector, compressing values to be between -1 and 1. Note, this only takes 1 input vector.
A sigmoid network layer. The output vector’s components will have values between 0 and 1.
A tanh network layer. The output vector’s components will have values between -1 and 1.
This provides experimental functionality. This is sort of like a network layer, only values for weights are supplied as a vector, instead of developed through training algorithms.