But what is a neural network? | Deep learning chapter 1

This is a 3.

It's sloppily written and rendered at an extremely low resolution of 28x28 pixels,

but your brain has no trouble recognizing it as a 3.

And I want you to take a moment to appreciate how

crazy it is that brains can do this so effortlessly.

I mean, this, this and this are also recognizable as 3s,

even though the specific values of each pixel is very different from one

image to the next.

The particular light-sensitive cells in your eye that are firing when you

see this 3 are very different from the ones firing when you see this 3.

But something in that crazy-smart visual cortex of yours resolves these as representing

the same idea, while at the same time recognizing other images as their own distinct

ideas.

But if I told you, hey, sit down and write for me a program that takes in a grid of

28x28 pixels like this and outputs a single number between 0 and 10,

telling you what it thinks the digit is, well the task goes from comically trivial to

dauntingly difficult.

Unless you've been living under a rock, I think I hardly need to motivate the relevance

and importance of machine learning and neural networks to the present and to the future.

But what I want to do here is show you what a neural network actually is,

assuming no background, and to help visualize what it's doing,

not as a buzzword but as a piece of math.

My hope is that you come away feeling like the structure itself is motivated,

and to feel like you know what it means when you read,

or you hear about a neural network quote-unquote learning.

This video is just going to be devoted to the structure component of that,

and the following one is going to tackle learning.

What we're going to do is put together a neural

network that can learn to recognize handwritten digits.

This is a somewhat classic example for introducing the topic,

and I'm happy to stick with the status quo here,

because at the end of the two videos I want to point you to a couple good

resources where you can learn more, and where you can download the code that

does this and play with it on your own computer.

There are many many variants of neural networks,

and in recent years there's been sort of a boom in research towards these variants,

but in these two introductory videos you and I are just going to look at the simplest

plain vanilla form with no added frills.

This is kind of a necessary prerequisite for understanding any of the more powerful

modern variants, and trust me it still has plenty of complexity for us to wrap our minds

around.

But even in this simplest form it can learn to recognize handwritten digits,

which is a pretty cool thing for a computer to be able to do.

And at the same time you'll see how it does fall

short of a couple hopes that we might have for it.

As the name suggests neural networks are inspired by the brain, but let's break that down.

What are the neurons, and in what sense are they linked together?

Right now when I say neuron all I want you to think about is a thing that holds a number,

specifically a number between 0 and 1.

It's really not more than that.

For example the network starts with a bunch of neurons corresponding to

each of the 28x28 pixels of the input image, which is 784 neurons in total.

Each one of these holds a number that represents the grayscale value of the

corresponding pixel, ranging from 0 for black pixels up to 1 for white pixels.

This number inside the neuron is called its activation,

and the image you might have in mind here is that each neuron is lit up when its

activation is a high number.

So all of these 784 neurons make up the first layer of our network.