NeuLab

NeuLab is a complete laboratory for learning about neural networks through hands on usage. * Each screen has a Help button to give usage help for all controls on that screen. * Configure, train, test, and optimize your network. * Use AutoPrune to increase network accuracy. * Use Prune and Link to build convolution layers. * Define the input-output examples used for training and testing. Use NeuLab's built in editor or define these externally for import. * Save and export your trained networks for study and reuse. * Save and export your training data and test results. * Step the network training to see how internal weights change with each presentation of training data. * Step the network testing to see how internal nodes activate in response to the inputs and produce the network outputs. Background: Neural networks process data via highly interconnected layers of simple processing nodes. These networks are being employed across all areas of pattern recognition and classification: handwritten character recognition, voice recognition, face detection/recognition, image classification, biometric security, credit risk assessment, financial markets analysis, sports analysis, language processing, game playing, the list goes on and on. A neural net learns by observation and doesn’t need problem specific rules or algorithms programmed in. The key factors for success are the network architecture, the training parameters, and most importantly the training data representation and the data examples themselves. NeuLab is primarily an educational app to give you familiarity with how a neural network operates and to see its inner workings. You configure the network and provide the example data for training and testing the network. Your trained network hopefully will have learned the relationships necessary for good results when presented with your test data. Testing the network is critical to assessing its usefulness on as yet unseen data. Examining the test failures will give you insight into how the network operates, the importance of broad representation in the training data, and the importance of the data representation itself. NeuLab lets you define a network up to 5 layers deep with up to 100 nodes per layer. The networks here are feed forward where signals propagate from an Input layer of nodes through one or more hidden layers and finally to an output layer. Each layer is initially fully connected to the next layer (i.e. each node in one layer is connected to every node in the next layer). Each layer includes a bias node also fully connected to the next layer. Non-input nodes use a sigmoid activation function with outputs constrained to 0.0-1.0. Example data values are similarly constrained and should be scaled if needed. A network is trained by multiple presentations of the training examples. The network learns by adjusting its internal connection weights until the outputs produced are sufficiently close to those given in the training examples. The learning algorithm is gradient descent backpropagation using the generalized delta rule as described in the seminal 1986 PDP books by Rumelhart, McClelland, et al. You can vary network topology, initialization, training parameters, and the training data to see how training speed and training success is affected. Once trained, network accuracy can be tested using examples not included in the training set. If test results are not good enough, more experimentation is needed. Perhaps more training data is needed. Perhaps the data representation is a problem. Perhaps pruning will help. Perhaps the input data is missing a key attribute. You will develop some understanding for how these networks work, to what uses they can be employed, and how to solve training and usage difficulties that may arise. Please send problems, questions, requests, or comments to me at tim@timdyes.com. See the case study and the FAQ there as well. Enjoy! - Tim Dyes