Neural Network: How It Works, With Real Numbers

In brief

A neural network is a program that learns a task by adjusting numbers called weights, and a tiny one with two inputs, two hidden neurons and one output has just 9 of them. Warren McCulloch and Walter Pitts modeled the first artificial neuron in 1943. As of November 2026, the transformer type powers ChatGPT.

What is a neural network?

A neural network is a computer program built from layers of small calculators called neurons, and it learns by changing the numbers that connect them. Wikipedia defines it as "a computational model inspired by the structure and functions of biological neural networks". Nobody writes the rules. The network finds them from examples.

Three columns of small rainbow glass spheres joined by fine copper threads on a black plinth, beside an open notebook, in a charcoal studio under white rim light.
PartWhat it isIn the example below
InputA number the network reads1 and 2
WeightHow strongly one link counts0.4, 0.3, 0.8 and others
BiasA number added to shift the result0.1, minus 0.1 and minus 0.3
ActivationA rule that reshapes the total (sigmoid, ReLU and tanh are common)ReLU, then sigmoid
LayerA row of neurons1 hidden layer, 1 output layer

The phrase covers two things. A biological neural network is the web of nerve cells in a brain. An artificial one is software, and it is the meaning in nearly every search for the term. The software borrowed the idea of many simple units wired together. It didn't copy how a brain works.

How does a neural network work?

It passes numbers forward through layers, and each neuron does the same three steps: multiply, add, reshape. Here is a network small enough to check with a calculator. It has two inputs (1 and 2), two hidden neurons and one output, so 9 adjustable numbers in all: 6 weights and 3 biases. The example was written for this page.

Hidden neuron A has weights 0.4 and 0.3 and a bias of 0.1. It computes 0.4 x 1 + 0.3 x 2 + 0.1 = 1.1. ReLU keeps positive numbers, so A outputs 1.1. Hidden neuron B has weights minus 0.5 and 0.2 and a bias of minus 0.1. It computes minus 0.5 + 0.4 minus 0.1 = minus 0.2. ReLU turns that into 0, so B stays silent.

The output neuron has weights 0.8 and 0.6 and a bias of minus 0.3. It computes 0.8 x 1.1 + 0.6 x 0 minus 0.3 = 0.58. A sigmoid turns 0.58 into 0.64, a score between 0 and 1. Read it as a 64% chance that an email is spam, say. The first input was 1 and the second was 2, and the network has turned them into one useful number.

Real networks do exactly this, only wider, and you'll see the same three steps at every size. GPT-3 has 175 billion weights (Brown et al., 2020) where this one has 6. A network is called deep when it has at least two hidden layers. The example has one, so it isn't deep.

How does a neural network learn?

It guesses, measures how wrong it was, and nudges every weight a little in the direction that shrinks the error. Take the same network. Suppose the email really was spam, so the right output is 1. The network said 0.64. The squared error is (0.64 minus 1) squared, about 0.129. That error number is the loss, the gap between the guess and the right answer. Learning from examples that each carry a right answer is called supervised learning.

Backpropagation works out how much each weight contributed to that error. For the output neuron's weight on hidden neuron A, the push is about minus 0.18. With a learning rate of 0.5, which is the size of the step, the weight moves from 0.8 to about 0.89. The output bias moves from minus 0.3 to about minus 0.22.

Run the network again with the new numbers. Output: 0.68. Squared error: about 0.101, down from 0.129. One step made the network a little better. The method of stepping the weights downhill on the error is called gradient descent (IBM). Training repeats that step over millions of examples, and one full pass through the data is an epoch, until the error stops falling. Seppo Linnainmaa published the modern form of backpropagation in his 1970 master's thesis, and David Rumelhart and colleagues popularized it in 1986 (Wikipedia).

A network only knows what its examples taught it, and that's the catch. Show it spam written in a style it never saw and it may score that email 0.2. It learns patterns, not meanings.

What are the types of neural networks?

The four main types are feedforward, convolutional, recurrent and transformer networks, and they differ in how data moves through them. Two well known variants, LSTM and GAN, sit in the same table. The tiny network above is a feedforward network: numbers go one way, from input to output.

TypeReads bestHow it passes dataLandmark
FeedforwardRows of numbers, such as a spam scoreOne direction, input to outputRosenblatt's perceptron, 1958
Convolutional (CNN)Images and other gridsSmall filters slide over the picture to find edges and shapesLeNet read handwritten ZIP codes, 1989
Recurrent (RNN)Sequences, such as text or time seriesFeedback loops carry information from step to stepSpeech and sequence work until transformers (IBM)
LSTM (a recurrent variant)Long sequencesMemory cells that fight the vanishing gradient problemHochreiter and Schmidhuber, early version 1995 (Wikipedia)
GANMaking new images and other samplesA generator and a discriminator compete in a two player gameGoodfellow and colleagues, June 2014 (arXiv)
TransformerText, code, images as tokensAttention lets every token look at every other token at onceAttention Is All You Need, June 2017

The transformer is the one behind the chatbots. GPT stands for generative pretrained transformer, and Claude and Gemini use the same design. What is a transformer in AI covers attention in depth, and what is an LLM covers what a transformer does with text.

Another article calls recurrent networks the engine of speech and language systems. That's dated. Modern language systems run on transformers, which replaced recurrent networks for most sequence work.

Is a neural network the same as deep learning or AI?

No. They nest inside each other. Artificial intelligence is the widest field, machine learning is the part where programs learn from data, and neural networks are one kind of machine learning model. Deep learning means using neural networks with many layers.

So every deep learning system is a neural network, and every neural network is machine learning. Not every machine learning model is a neural network, because a decision tree or a straight line fitted to data is not one. The date the whole field began is covered in when was AI invented.

Neural network history in dates

The idea is older than most people expect: the first mathematical neuron dates to 1943, and the breakthroughs since came when data and computers caught up. Every date below comes from the Wikipedia history of neural networks unless marked.

YearWhat happenedWho
1943First mathematical models of artificial neurons that could represent logical functionsWarren McCulloch and Walter Pitts
1958The perceptron, one of the first implemented neural networks, funded by the US Office of Naval ResearchFrank Rosenblatt
1970The modern form of backpropagation, in a master's thesisSeppo Linnainmaa
1986Backpropagation popularized for training networksDavid Rumelhart and colleagues
1989LeNet, a CNN that read handwritten ZIP codes and took 3 days to trainYann LeCun and colleagues
1998LeNet-5, a 7 level CNN that banks used to read check numbersYann LeCun and colleagues
October 2012AlexNet won the ImageNet competition by a significant margin over shallow methodsKrizhevsky, Sutskever and Hinton
12 June 2017The transformer, in the paper Attention Is All You NeedVaswani and seven coauthors (arXiv)

The gap between 1958 and 2012 is the lesson. The ideas existed. What changed wasn't the idea. It was the amount of training data and the speed of the chips that run the sums.

Where are neural networks used?

They run in anything that has to find a pattern in images, sound, text or numbers. IBM lists computer vision (including medical imaging and autonomous vehicles), translation and chatbots, speech recognition, forecasting, game playing agents such as AlphaGo, and fraud detection as the main uses. Three real cases show the range.

  1. Reading handwriting. In 1989 LeNet read handwritten ZIP codes on mail, and by 1998 LeNet-5 read check numbers for banks (Wikipedia). It was a CNN, and it is the same family that now sorts your phone photos.
  2. Scoring an email. A feedforward network like the one above takes numbers such as word counts and outputs one score. That is how a spam filter can decide.
  3. Writing a reply. A transformer reads your message as tokens and predicts the next token, over and over. That is what happens when you type into a chatbot.

The pattern across all three is the same. Show the network many examples and it learns the rule. That is also why it fails outside them.

Are neural networks like the brain?

Only loosely. An artificial neuron is a weighted sum and a reshaping rule, a few lines of arithmetic. A brain cell is a living cell with far richer behavior. The name stuck because the first designers drew on the brain for ideas.

There are honest limits to know. A network can be confidently wrong, because the score of 0.64 is not a probability anyone checked. It can memorize its training examples and then fail on new ones, which is called overfitting. And it doesn't keep learning after you deploy it: standard models improve only when someone retrains them. Don't treat any single output as more than a strong guess.

What questions can you ask an AI about neural networks?

A chatbot's a good tutor for this topic if you make it show numbers. Paste any of these into Claude, ChatGPT, Gemini or Whizi and fill the brackets.

  1. "Walk me through one forward pass of a network with [number] inputs and [number] hidden neurons using small whole numbers. Show every multiplication."
  2. "Explain backpropagation to a [grade level] student using one weight and one example. Don't use calculus words until the end."
  3. "Here is my attempt at the forward pass: [paste]. Find the first arithmetic mistake and stop there."
  4. "I want to recognize [handwriting, spam or customer photos]. Which type of neural network fits, and why not the other three?"
  5. "Give me five questions that test whether I understand overfitting, one at a time. Wait for my answer."
  6. "What did neural networks look like in [1958, 1989 or 2012], and what changed to make the next step possible? Cite a source for each date."

Ask for the arithmetic, not a summary. A model that shows each multiplication lets you catch an error. One that gives a paragraph hides it.

Where does Whizi fit when you are learning neural networks?

Whizi is one app for asking Claude, GPT and Gemini the same question, which suits a topic where the first explanation often misses. Web search is on by default in each chat, so you can ask for sources on a date. On Pro, Compare puts two models side by side on one prompt, and you can read only where they disagree.

The free plan gives you 10 messages a day on Whizi AI, Auto and Claude Haiku, with no card. Pro carries Claude, GPT and Gemini in one app, and its first week is $1.99. See Whizi pricing for the plans.

Workflow checklist
  • Know the three steps every neuron repeats: multiply by weights, add a bias, apply an activation
  • Work one tiny network by hand before reading about big ones
  • Pick the type by the data: grids for CNNs, sequences for RNNs, tokens for transformers
  • Treat an output score as a strong guess, not a checked probability
  • Check every date and figure an AI gives you against a source you can open
Common questions

Frequently asked questions

What is a neural network in simple terms?

It is a program made of layers of small calculators that learns a task from examples. Each calculator multiplies its inputs by weights, adds a bias and reshapes the total. Training adjusts the weights until the answers are right more often. The example on this page has 9 such numbers.

What is a neural network in the brain?

A biological neural network is the web of connected nerve cells in a brain. The artificial kind in software was inspired by it but is far simpler: each artificial neuron is a weighted sum and a rule, not a living cell. Wikipedia describes the artificial kind as inspired by the structure and functions of biological ones.

Is ChatGPT a neural network?

Yes. ChatGPT runs on GPT models, and GPT stands for generative pretrained transformer, a type of neural network. The transformer design came from a paper posted on 12 June 2017, and Claude and Gemini use it too.

How many layers make a neural network deep?

At least two hidden layers, according to Wikipedia. A network with that many is called a deep neural network. A hidden layer is any layer between the input and the output. The tiny network on this page has one hidden layer, so it counts as shallow.

Is a neural network supervised or unsupervised?

Often supervised. Wikipedia lists neural networks among the most widely used supervised learning algorithms, which learn from example input and output pairs, like the spam example here where each email has a right answer. Other training setups exist, such as the generator against discriminator game in a GAN.