Neural Networks, Step 1: Where to Begin with Neural Nets & Deep Learning

This is a short post for beginners learning neural networks, covering several essential neural networks concepts.
By Matthew Mayo, KDnuggets.

This is a short supplementary post for beginners learning neural networks. It does not intend to provide a complete learning roadmap, but the contents included should give a short introduction to several essential neural networks concepts.

The first resource covers defining some key neural network terminology.

Deep Learning Key Terms, Explained

As defined above, deep learning is the process of applying deep neural network technologies to solve problems. Deep neural networks are neural networks with one hidden layer minimum. Like

This pair of posts covers a few of the most important foundational concepts of neural networks at a very introductory level, without any of the math. If you can understand the high level concepts contained within these posts, you should be ready for the resources that follow.

Neural Network Foundations, Explained: Activation Function

Forward propagation is the the process of multiplying the various input values of a particular neuron by their associated weights, summing the results, and scaling or "squashing" the values back between a given range before passing these signals on to the next layer of neurons. This, in turn, affects the weighted input value sums of the following layer, and so on, which then affects the computation of new weights and their distribution backward through the network. Ultimately, of course, this all affects the final output value(s) of the neural network. The activation function keeps values forward to subsequent layers within an acceptable and useful range, and forwards the output.

Neural Network Foundations, Explained: Updating Weights with Gradient Descent & Backpropagation

Recall that in order for a neural networks to learn, weights associated with neuron connections must be updated after forward passes of