Twenty years ago, a saying made the rounds in the scientific community: “It is only Artificial Intelligence until we understand how it works; then it becomes ‘Machine Learning.’”
Most people don’t care about the semantic gymnastics, and to be fair, the terms overlap significantly. But let’s be a bit pedantic for three minutes to clarify what Machine Learning actually is. It might not shock you, but having this clarity will definitely help you win your next “AI-at-the-water-cooler” conversation.
What is it, exactly?
At its core, Machine Learning (ML) is a field of computer science focused on using algorithms to generate new insights or “knowledge” from a given dataset without being explicitly programmed for every scenario.
To a modern reader, that might sound obvious. But forty years ago, no one in their right mind believed a machine could generate “true” knowledge. Back then, the dominant theory was the Feigenbaum Bottleneck (named after Edward Feigenbaum). He argued that AI could only be built by interviewing human experts and distilling their wisdom into a massive sequence of “if-then” statements.
Spoiler alert: He was wrong. It turns out that humans are actually quite bad at explaining how they know what they know.
Why “Machine” and not “Computer”?
Why do we call it “Machine” learning? It’s likely a nod to Alan Turing, who was writing about “intelligent machinery” long before the term “computer” became a household word. This is a crucial reminder: AI science is as old as the digital computer itself. It didn’t just appear out of thin air in 2017, no matter what your LinkedIn feed tells you.
The Magic of “New Knowledge”
When we talk about “learning” or “knowledge generation,” we are talking about a machine discovering rules that weren’t in the original input. One of the earliest “Aha!” moments for ML involved a computer figuring out a strategy for a notoriously difficult chess endgame. Instead of just calculating every move, the computer generated its own simple rules, like “Keep the king toward the center” and “Use the rook to push the opponent toward the edge.” At the time, it felt like magic. Today, we’re so used to probabilistic thinking that we brush it off as “the machine just spotted patterns in the data.” While that’s true, the real breakthrough was the machine’s ability to create a strategy that a human could then read, understand, and evaluate.
The LLM Twist
Here is the catch: This “explainable” knowledge is almost the opposite of how Large Language Models (LLMs) work today.
While an LLM is trained using machine learning, it doesn’t output a set of neat, human-readable rules. It trains billions of “weights”—numbers that don’t mean anything in isolation. We get the right answer, but we lose the ability to inspect the “why” behind it. The “learning” is hidden in a black box.
Leave a Reply