What is a neural network?
A neural network is a computer program built from layers of small calculators called neurons, and it learns by changing the numbers that connect them. Wikipedia defines it as "a computational model inspired by the structure and functions of biological neural networks". Nobody writes the rules. The network finds them from examples.

| Part | What it is | In the example below |
|---|---|---|
| Input | A number the network reads | 1 and 2 |
| Weight | How strongly one link counts | 0.4, 0.3, 0.8 and others |
| Bias | A number added to shift the result | 0.1, minus 0.1 and minus 0.3 |
| Activation | A rule that reshapes the total (sigmoid, ReLU and tanh are common) | ReLU, then sigmoid |
| Layer | A row of neurons | 1 hidden layer, 1 output layer |
The phrase covers two things. A biological neural network is the web of nerve cells in a brain. An artificial one is software, and it is the meaning in nearly every search for the term. The software borrowed the idea of many simple units wired together. It didn't copy how a brain works.
How does a neural network work?
It passes numbers forward through layers, and each neuron does the same three steps: multiply, add, reshape. Here is a network small enough to check with a calculator. It has two inputs (1 and 2), two hidden neurons and one output, so 9 adjustable numbers in all: 6 weights and 3 biases. The example was written for this page.
Hidden neuron A has weights 0.4 and 0.3 and a bias of 0.1. It computes 0.4 x 1 + 0.3 x 2 + 0.1 = 1.1. ReLU keeps positive numbers, so A outputs 1.1. Hidden neuron B has weights minus 0.5 and 0.2 and a bias of minus 0.1. It computes minus 0.5 + 0.4 minus 0.1 = minus 0.2. ReLU turns that into 0, so B stays silent.
The output neuron has weights 0.8 and 0.6 and a bias of minus 0.3. It computes 0.8 x 1.1 + 0.6 x 0 minus 0.3 = 0.58. A sigmoid turns 0.58 into 0.64, a score between 0 and 1. Read it as a 64% chance that an email is spam, say. The first input was 1 and the second was 2, and the network has turned them into one useful number.
Real networks do exactly this, only wider, and you'll see the same three steps at every size. GPT-3 has 175 billion weights (Brown et al., 2020) where this one has 6. A network is called deep when it has at least two hidden layers. The example has one, so it isn't deep.
How does a neural network learn?
It guesses, measures how wrong it was, and nudges every weight a little in the direction that shrinks the error. Take the same network. Suppose the email really was spam, so the right output is 1. The network said 0.64. The squared error is (0.64 minus 1) squared, about 0.129. That error number is the loss, the gap between the guess and the right answer. Learning from examples that each carry a right answer is called supervised learning.
Backpropagation works out how much each weight contributed to that error. For the output neuron's weight on hidden neuron A, the push is about minus 0.18. With a learning rate of 0.5, which is the size of the step, the weight moves from 0.8 to about 0.89. The output bias moves from minus 0.3 to about minus 0.22.
Run the network again with the new numbers. Output: 0.68. Squared error: about 0.101, down from 0.129. One step made the network a little better. The method of stepping the weights downhill on the error is called gradient descent (IBM). Training repeats that step over millions of examples, and one full pass through the data is an epoch, until the error stops falling. Seppo Linnainmaa published the modern form of backpropagation in his 1970 master's thesis, and David Rumelhart and colleagues popularized it in 1986 (Wikipedia).
A network only knows what its examples taught it, and that's the catch. Show it spam written in a style it never saw and it may score that email 0.2. It learns patterns, not meanings.
What are the types of neural networks?
The four main types are feedforward, convolutional, recurrent and transformer networks, and they differ in how data moves through them. Two well known variants, LSTM and GAN, sit in the same table. The tiny network above is a feedforward network: numbers go one way, from input to output.
| Type | Reads best | How it passes data | Landmark |
|---|---|---|---|
| Feedforward | Rows of numbers, such as a spam score | One direction, input to output | Rosenblatt's perceptron, 1958 |
| Convolutional (CNN) | Images and other grids | Small filters slide over the picture to find edges and shapes | LeNet read handwritten ZIP codes, 1989 |
| Recurrent (RNN) | Sequences, such as text or time series | Feedback loops carry information from step to step | Speech and sequence work until transformers (IBM) |
| LSTM (a recurrent variant) | Long sequences | Memory cells that fight the vanishing gradient problem | Hochreiter and Schmidhuber, early version 1995 (Wikipedia) |
| GAN | Making new images and other samples | A generator and a discriminator compete in a two player game | Goodfellow and colleagues, June 2014 (arXiv) |
| Transformer | Text, code, images as tokens | Attention lets every token look at every other token at once | Attention Is All You Need, June 2017 |
The transformer is the one behind the chatbots. GPT stands for generative pretrained transformer, and Claude and Gemini use the same design. What is a transformer in AI covers attention in depth, and what is an LLM covers what a transformer does with text.
Another article calls recurrent networks the engine of speech and language systems. That's dated. Modern language systems run on transformers, which replaced recurrent networks for most sequence work.
Is a neural network the same as deep learning or AI?
No. They nest inside each other. Artificial intelligence is the widest field, machine learning is the part where programs learn from data, and neural networks are one kind of machine learning model. Deep learning means using neural networks with many layers.
So every deep learning system is a neural network, and every neural network is machine learning. Not every machine learning model is a neural network, because a decision tree or a straight line fitted to data is not one. The date the whole field began is covered in when was AI invented.
Neural network history in dates
The idea is older than most people expect: the first mathematical neuron dates to 1943, and the breakthroughs since came when data and computers caught up. Every date below comes from the Wikipedia history of neural networks unless marked.
| Year | What happened | Who |
|---|---|---|
| 1943 | First mathematical models of artificial neurons that could represent logical functions | Warren McCulloch and Walter Pitts |
| 1958 | The perceptron, one of the first implemented neural networks, funded by the US Office of Naval Research | Frank Rosenblatt |
| 1970 | The modern form of backpropagation, in a master's thesis | Seppo Linnainmaa |
| 1986 | Backpropagation popularized for training networks | David Rumelhart and colleagues |
| 1989 | LeNet, a CNN that read handwritten ZIP codes and took 3 days to train | Yann LeCun and colleagues |
| 1998 | LeNet-5, a 7 level CNN that banks used to read check numbers | Yann LeCun and colleagues |
| October 2012 | AlexNet won the ImageNet competition by a significant margin over shallow methods | Krizhevsky, Sutskever and Hinton |
| 12 June 2017 | The transformer, in the paper Attention Is All You Need | Vaswani and seven coauthors (arXiv) |
The gap between 1958 and 2012 is the lesson. The ideas existed. What changed wasn't the idea. It was the amount of training data and the speed of the chips that run the sums.
Where are neural networks used?
They run in anything that has to find a pattern in images, sound, text or numbers. IBM lists computer vision (including medical imaging and autonomous vehicles), translation and chatbots, speech recognition, forecasting, game playing agents such as AlphaGo, and fraud detection as the main uses. Three real cases show the range.
- Reading handwriting. In 1989 LeNet read handwritten ZIP codes on mail, and by 1998 LeNet-5 read check numbers for banks (Wikipedia). It was a CNN, and it is the same family that now sorts your phone photos.
- Scoring an email. A feedforward network like the one above takes numbers such as word counts and outputs one score. That is how a spam filter can decide.
- Writing a reply. A transformer reads your message as tokens and predicts the next token, over and over. That is what happens when you type into a chatbot.
The pattern across all three is the same. Show the network many examples and it learns the rule. That is also why it fails outside them.
Are neural networks like the brain?
Only loosely. An artificial neuron is a weighted sum and a reshaping rule, a few lines of arithmetic. A brain cell is a living cell with far richer behavior. The name stuck because the first designers drew on the brain for ideas.
There are honest limits to know. A network can be confidently wrong, because the score of 0.64 is not a probability anyone checked. It can memorize its training examples and then fail on new ones, which is called overfitting. And it doesn't keep learning after you deploy it: standard models improve only when someone retrains them. Don't treat any single output as more than a strong guess.
What questions can you ask an AI about neural networks?
A chatbot's a good tutor for this topic if you make it show numbers. Paste any of these into Claude, ChatGPT, Gemini or Whizi and fill the brackets.
- "Walk me through one forward pass of a network with [number] inputs and [number] hidden neurons using small whole numbers. Show every multiplication."
- "Explain backpropagation to a [grade level] student using one weight and one example. Don't use calculus words until the end."
- "Here is my attempt at the forward pass: [paste]. Find the first arithmetic mistake and stop there."
- "I want to recognize [handwriting, spam or customer photos]. Which type of neural network fits, and why not the other three?"
- "Give me five questions that test whether I understand overfitting, one at a time. Wait for my answer."
- "What did neural networks look like in [1958, 1989 or 2012], and what changed to make the next step possible? Cite a source for each date."
Ask for the arithmetic, not a summary. A model that shows each multiplication lets you catch an error. One that gives a paragraph hides it.
Where does Whizi fit when you are learning neural networks?
Whizi is one app for asking Claude, GPT and Gemini the same question, which suits a topic where the first explanation often misses. Web search is on by default in each chat, so you can ask for sources on a date. On Pro, Compare puts two models side by side on one prompt, and you can read only where they disagree.
The free plan gives you 10 messages a day on Whizi AI, Auto and Claude Haiku, with no card. Pro carries Claude, GPT and Gemini in one app, and its first week is $1.99. See Whizi pricing for the plans.
- Know the three steps every neuron repeats: multiply by weights, add a bias, apply an activation
- Work one tiny network by hand before reading about big ones
- Pick the type by the data: grids for CNNs, sequences for RNNs, tokens for transformers
- Treat an output score as a strong guess, not a checked probability
- Check every date and figure an AI gives you against a source you can open
Frequently asked questions
What is a neural network in simple terms?
It is a program made of layers of small calculators that learns a task from examples. Each calculator multiplies its inputs by weights, adds a bias and reshapes the total. Training adjusts the weights until the answers are right more often. The example on this page has 9 such numbers.
What is a neural network in the brain?
A biological neural network is the web of connected nerve cells in a brain. The artificial kind in software was inspired by it but is far simpler: each artificial neuron is a weighted sum and a rule, not a living cell. Wikipedia describes the artificial kind as inspired by the structure and functions of biological ones.
Is ChatGPT a neural network?
Yes. ChatGPT runs on GPT models, and GPT stands for generative pretrained transformer, a type of neural network. The transformer design came from a paper posted on 12 June 2017, and Claude and Gemini use it too.
How many layers make a neural network deep?
At least two hidden layers, according to Wikipedia. A network with that many is called a deep neural network. A hidden layer is any layer between the input and the output. The tiny network on this page has one hidden layer, so it counts as shallow.
Is a neural network supervised or unsupervised?
Often supervised. Wikipedia lists neural networks among the most widely used supervised learning algorithms, which learn from example input and output pairs, like the spam example here where each email has a right answer. Other training setups exist, such as the generator against discriminator game in a GAN.