What are AI Weights?

Weights are a specific type of parameter in an AI system that determine the strength of connections between different parts of the system. They are numbers that control how much influence one piece of information has on another.

Understanding weights through analogy

Imagine a group of people deciding where to go for dinner. Some people's opinions matter more than others. Your closest friend's suggestion might have high weight, meaning you consider it carefully. A stranger's opinion might have low weight, meaning you barely notice it.

Weights in AI work similarly. They determine how much attention the system pays to different pieces of information when making a decision.

Weights in neural networks

AI systems, particularly neural networks, are made of layers of interconnected nodes. These nodes pass information to each other.

Weights determine the strength of the connection between nodes. A high weight means that signal passes strongly from one node to the next. A low weight means that signal is dampened or weakened.

How weights are used

When you type a question into an AI system, your text is first converted into embeddings. Your text then passes through many layers of nodes. Each time information passes from one node to the next, it is multiplied by a weight.

If a weight is 2, the signal doubles. If a weight is 0.1, the signal becomes one-tenth as strong. If a weight is negative, the signal is reversed.

By the time your question has passed through all these layers, it has been transformed by millions or billions of weight multiplications.

Weights determine behavior

The specific pattern of weights in an AI system is what makes it behave the way it does.

Different systems have different weight patterns because they were trained on different data or with different methods. These different weight patterns produce different answers to the same question.

Weights are learned during training

When an AI system is first created, its weights are set to random values.

During training, the system makes predictions and receives feedback on whether those predictions are correct. Based on this feedback, the weights are adjusted slightly.

This process repeats many billions of times. Small adjustments add up, gradually shaping the weights into patterns that help the system make accurate predictions.

Weights cannot be inspected easily

Although weights are just numbers, modern AI systems have so many weights that humans cannot easily understand them.

A system with 100 billion weights would take an extremely long time to examine manually. Even if you could look at them, you probably would not understand why each individual weight has its particular value.

This opacity is one reason why AI systems are sometimes called "black boxes," even though they work through mathematical processes we can describe in principle.

The relationship between weights and parameters

All weights are parameters, but not all parameters are weights. Weights specifically refer to the numbers that determine connection strength in neural networks. Parameters is the broader term for all settings in an AI system.

The simple explanation

Weights are numbers in an AI system that determine the strength of connections between different parts of the system. They control how much different pieces of information influence the final output. Weights are learned during training and are then fixed. A system's behaviour is largely determined by its specific pattern of weights. Modern AI systems have so many weights that humans cannot easily examine or understand them individually.

To see how billions of weights work together to process your input and generate responses, read How Does AI Actually Work On The Inside?