NeuralNet v1.0.0 NeuralNet
This module allows you to define and train neural networks with complex architectures. See lib/gru.ex for an example implementation of the Gated Recurrent Unit architecture. For making a new architecture, use the template at lib/.template.ex. Just copy the file and change the module name to get started.
For a great post on LSTMs, GRUs, and other recurrent neural network architectures, see here: http://colah.github.io/posts/2015-08-Understanding-LSTMs/.
This module of course allows for the creation of non recurrent neural networks as well. An architecture that makes no reference to vectors from previous time frames is not recurrent.
Networks are evaluated and trained using “vectors across time”. A vector-across-time is a list of time_frames, where each element is a vector, which is a map of {component_name, value} key-val-pairs.
Example
[%{x: 1.0, y: -1.0}, %{x: 0.5, -0.5}, %{x: 0.0, y: 0.0}]Summary
Functions
Breaks down a vector name into its core name. When given {:next, vec} or {:previous, vec}, it will just return vec. :next and :previous are used in the fashion to keep track of references to vectors from future or past time frames
Evaluates a network. The input should be a vector-across-time. The function returns a {output_values, time_frames} tuple. time_frames holds all the data for the networks state across its time frames of execution. If you want to continue evaluation with more inputs, run the eval function again passing in the previous data as given_time_frames
Given a list of component names for a vector, returns a complete vector where each component has value value (default is 0)
Given a vector, returns the component name with the greatest value
Retrieves the list of the named components that make up the given vector
train uses backpropogation to train any neural_net. training_data should be a list of {inputs, expected_output} tuples, where inputs is a vector-across-time. expected_output can be either a vector representing final expected output, or it can be a list of expected outputs for every time frame. learn_val should be a positive constant which controls the effect of each batch. Higher values can cause faster learning, but also may introduce trouble finding a minimum error. batch_size specifies the number of training pairs to be run in parallel. At each training iteration, batch_size number of training sessions run in parallel, and the results are averaged together. Small batch sizes of 1-3 tend to work best. The training_complete?(info) function should take 1 argument of miscellaneous info, and return true when training should be stopped. This function can also be used for debugging or monitoring, and can print out information at regular intervals. The time (in seconds) between each call to training_complete?(info) is specified by the argument completion_checking_interval
Functions
Breaks down a vector name into its core name. When given {:next, vec} or {:previous, vec}, it will just return vec. :next and :previous are used in the fashion to keep track of references to vectors from future or past time frames.
Evaluates a network. The input should be a vector-across-time. The function returns a {output_values, time_frames} tuple. time_frames holds all the data for the networks state across its time frames of execution. If you want to continue evaluation with more inputs, run the eval function again passing in the previous data as given_time_frames.
Given a list of component names for a vector, returns a complete vector where each component has value value (default is 0).
train uses backpropogation to train any neural_net. training_data should be a list of {inputs, expected_output} tuples, where inputs is a vector-across-time. expected_output can be either a vector representing final expected output, or it can be a list of expected outputs for every time frame. learn_val should be a positive constant which controls the effect of each batch. Higher values can cause faster learning, but also may introduce trouble finding a minimum error. batch_size specifies the number of training pairs to be run in parallel. At each training iteration, batch_size number of training sessions run in parallel, and the results are averaged together. Small batch sizes of 1-3 tend to work best. The training_complete?(info) function should take 1 argument of miscellaneous info, and return true when training should be stopped. This function can also be used for debugging or monitoring, and can print out information at regular intervals. The time (in seconds) between each call to training_complete?(info) is specified by the argument completion_checking_interval.