Neural network

An artificial neuron is a small calculation: it takes in several numbers, weights them, adds them up and decides how strongly to pass a signal on. Many such neurons, arranged in layers and connected to each other, form a neural network. What the network can do lives in the weights of its connections. Nobody writes those numbers in by hand; training sets them from examples. To distinguish them from nerve cells in the brain, they are also called artificial neural networks (ANNs).

Neural networks are a machine learning method. Networks with many layers are the basis of deep learning, and with it of image and speech recognition, machine translation and large language models. The idea is old: Warren McCulloch and Walter Pitts described a formal model of a neuron in 1943, and in 1958 Frank Rosenblatt introduced the perceptron, a model that learns its weights from examples. John Hopfield and Geoffrey Hinton received the 2024 Nobel Prize in Physics for foundational discoveries and inventions that enable machine learning with artificial neural networks.

For executives, the term has a practical side. A neural network delivers results, but no justification you can read like a rule. Wherever decisions have to be explained, audited or defended, that shapes what a network is used for and who checks its output before it is passed on.

How an artificial neuron computes.

An artificial neuron receives inputs, such as the brightness of individual pixels or the values from a previous layer. Each input is multiplied by a weight that sets its importance. The weighted values are added up, plus a fixed offset called the bias. The result passes through an activation function that determines what the neuron passes on.

Without the activation function, an entire network, however large, would amount to a single linear calculation and could not capture complex relationships. Common choices are the sigmoid function, with values between 0 and 1, the hyperbolic tangent, with values between minus 1 and 1, and the now widespread ReLU, which sets negative values to zero and passes positive ones through unchanged.

The model comes from biology: a nerve cell receives signals through synapses and fires when enough of them add up. The resemblance ends there. An artificial neuron is a formula with a few numbers, with no metabolism and none of the timing behavior of real nerve cells. A network of such neurons does not replicate the brain, and it does not think.

From neuron to network: layers and weights.

In a network, neurons are arranged in layers. The input layer takes in the data, the output layer delivers the result, such as a probability for “scratched” or “OK.” In between are hidden layers whose values nobody observes directly. Each neuron in one layer connects to many neurons in the next, and each connection has its own weight.

Weights and biases together are called parameters. Their count is the usual measure of a network’s size. A small forecasting network can get by with a few thousand parameters. GPT-3, the language model OpenAI introduced in 2020, had 175 billion. For many newer models, the vendors no longer disclose the number (as of November 2026).

There is a mathematical reason networks are so versatile. In 1989, Kurt Hornik, Maxwell Stinchcombe and Halbert White showed that a network with just one hidden layer can approximate almost any relationship between input and output to any desired accuracy, given enough neurons. The theorem does not say how to find the right weights or how much data that takes. That is the job of training.

How a network learns from examples.

A freshly initialized network has random weights and produces random results. Training changes that in a loop. The network receives an example, computes an output through all its layers and compares it with the correct result. A loss function sums up the error in a single number.

Next comes the step that made multi-layer networks practical. Backpropagation passes the error backward through the network and determines how much each weight contributed to it. In 1986, David Rumelhart, Geoffrey Hinton and Ronald Williams showed in Nature that networks with hidden layers learn useful internal representations this way. Gradient descent then nudges every weight a small step in the direction that reduces the error. This repeats over many examples.

A network with many parameters can memorize its training examples instead of finding general patterns. This overfitting shows when the network shines in training and fails on new cases. That is why part of the data is held back and only used for testing. A network learns noisy or skewed examples just as thoroughly as good ones; more data only helps if its quality holds up.

The main types of networks.

Networks differ mainly in how their neurons are connected, and that design follows the type of data.

In a feedforward network, signals flow in one direction only, from input to output. Such networks suit fixed lists of features, such as readings from a machine.

Convolutional neural networks (CNNs) slide small filters across an image and recognize patterns regardless of their position. In 1998, Yann LeCun and colleagues described such networks for recognizing handwritten characters, which were already reading bank checks commercially. Today, CNNs power visual inspection, document capture and medical imaging.

Recurrent networks process sequences step by step and carry an intermediate state along, for example with speech or sensor data. Since 2017, Transformers have replaced them in many areas: in a Transformer, every element of an input can relate directly to every other. It is the architecture behind large language models.

Some networks deliver no verdict but a list of numbers that describes the content of a text or image. These embeddings make similarity in meaning computable and are the basis of semantic search.

The black box and what follows from it.

A decision tree can be read: if revenue is below a threshold and the payment deadline has passed, the customer counts as a risk. In a neural network, what it has learned is spread across thousands to billions of weights, none of which means anything on its own. Why a network reaches a result can therefore only be explained to a limited degree. Practitioners call it a black box.

Explanation methods show after the fact which parts of an input most influenced a result, such as which image region or which words. They help with debugging but provide an approximation, not a full justification.

This has consequences in practice. Under Article 22 of the EU General Data Protection Regulation (GDPR), people have the right not to be subject to a decision based solely on automated processing that has legal or similarly significant effects, with some exceptions. Where such decisions are made, Article 15 entitles those affected to meaningful information about the logic involved. A network that alone decides on loan applications or job candidates is therefore risky. The usual setup is different: the network pre-sorts or suggests, and a person decides and can justify the decision.

What good implementation looks like.

Good implementation starts by asking whether a neural network is needed at all. For small, clean tabular data, decision trees and other classic methods often deliver similar results, need less data and make their rules visible. Networks have the edge with images, speech, text and very large datasets.

If a network is the right choice, a pretrained model is usually adapted rather than one trained from scratch. A credible vendor can explain which model it is based on, which of your examples were used to adapt it and which held-out data it was tested on. The vendor reports the error rate separately for the cases that are expensive for the business and explains how errors surface in production.

Part of this is a plan for the network’s limits. Results below a confidence threshold go to a person, model versions are stored, and there is a way back if a new version performs worse. Being told a system is “AI-powered” or “based on neural networks” says nothing yet about its quality.

Where the term stops working

A neural network is a specific type of machine learning model, not a synonym for AI. Rule-based systems, search algorithms and classic statistics are not neural networks, and neither are decision trees, even though they also learn from data. Deep learning is learning with multi-layer networks; large language models and foundation models are especially large networks that were pretrained on broad data.