What are AI Parameters?

Parameters are the numbers and settings inside an AI system that determine how it works. They are the system's internal configuration, created during training and fixed once the system is ready to use.

What do parameters do?

Think of parameters as the dials and switches on a very complex machine. Each dial controls a different aspect of how the system operates. The most important type of parameters are weights, which control how strongly different pieces of information influence the system's decisions.

When you ask ChatGPT a question, the answer is generated by processing your text through thousands of billions of parameters. These parameters have been adjusted during training to recognize patterns in language.

How many parameters does an AI system have?

Large language models have enormous numbers of parameters.

GPT-4 is reported to have hundreds of billions of parameters. This huge number is one reason why large AI systems require powerful computers to run and are expensive to build.

Smaller AI models might have millions or billions of parameters instead of hundreds of billions. Fewer parameters means the system is simpler and faster, but it may not be as capable.

Parameters and performance

In general, more parameters allow an AI system to learn more complex patterns and give more sophisticated answers.

However, more parameters do not automatically mean better results. A well-trained system with fewer parameters can sometimes outperform a poorly-trained system with more parameters.

The training data and the quality of the training process matter as much as the number of parameters.

Parameters are set during training

Parameters are not fixed when a system is designed. They are learned during the training process.

A newly designed AI system starts with random parameter values. As the system is trained on examples, these parameters are gradually adjusted. When training is complete, the parameters are locked in place.

This is why training a large AI system can take weeks or months and require enormous computing resources.

You cannot directly change parameters

As a user, you cannot adjust an AI system's parameters directly. The parameters are internal to the system.

What you can control is your input, the prompts you give the system. The system then uses its fixed parameters to generate answers based on what it has learned.

Parameters versus weights

In AI, the terms "parameters" and "weights" are often used interchangeably, but they have slightly different meanings.

Parameters is the broader term for all the numbers that define how an AI system works. Weights are a specific type of parameter that represent the importance or influence of different connections in a neural network.

The simple explanation

Parameters are the numbers inside an AI system that determine how it works. They are learned during training and then locked in place. Large AI systems have hundreds of billions of parameters. More parameters generally allow more complex learning, but the quality of training matters as much as the quantity of parameters. As a user, you cannot change parameters directly, but you can influence results by changing your prompts.

To see how parameters work within the complete architecture of how AI systems actually function, read How Does AI Actually Work On The Inside?