IBM Research scientists extended information theory to account for reasoning, creating a new measure called logical semantic entropy.
If humanity’s scientific knowledge were about to be wiped out, and you could save only one sentence, what would you write? Physicist Richard Feynman once answered that question by trying to pack in as much inferable knowledge as possible: that everything is made of atoms in constant motion, attracting and repelling each other.
Feynman’s sentence wasn’t valuable because of its word count. It was valuable because of everything a future scientist could deduce from it. That distinction — between what a message says and what it lets someone figure out — is exactly what a new IBM Research paper tackles.
The 78-Year-Old Gap in Information Theory
In 1948, Claude Shannon reshaped communication with one powerful idea: you could separate a message’s meaning from the problem of transmitting it efficiently. That insight became the foundation of modern information theory and everything built on it since, from phone calls to the training data behind large language models.
Shannon’s framework works because communication networks don’t need to understand what they’re carrying. The internet moves a medical diagnosis and a photo of a cat with equal indifference. It just moves bits.
But humans instinctively know not all bits carry equal value. A bit from a broken sensor might be worthless. A single bit from a reliable collision-avoidance system telling a self-driving car to brake could be critical. The difference lies entirely in what can be inferred from it, something Shannon’s original framework never measured.
Giving the Receiver the Ability to Reason
A new paper published in the Proceedings of the National Academy of Sciences by IBM Research scientists Luis Lastras, Jonathan Lenchner, Barry Trager, Mark Squillante, Chai Wah Wu, and Ronald Fagin, alongside collaborators Wojciech Szpankowski and Alexander Gray, extends Shannon’s classic sender-receiver model by giving the receiver a reasoning capability.
Instead of only counting the information transmitted directly, the new model also accounts for everything a receiver can logically derive from what they received. A sender doesn’t have to spell out every fact individually if the receiver can infer many of them from a smaller set of clues.
To measure this mathematically, the team introduced a new quantity called “logical semantic entropy.” It defines the fundamental limits of communication once reasoning is factored in.
Three Findings That Challenge Intuition
The framework produced results the researchers didn’t expect going in.
The first is what they call the “No Need to Know” result. Normally, knowing exactly what a receiver already understands helps a sender compress a message further. Surprisingly, the team found that when reasoning is allowed, not knowing what the receiver already knows barely changes the fundamental communication limit at all.
The second is the “Less Is More” paradox, illustrated using the classic cryptography characters Alice and Bob. If Alice wants Bob to learn only a specific subset of her knowledge while using as few bits as possible, her most efficient strategy can end up teaching him more than she intended.
Here’s why. Broad, reusable shorthand patterns cut down on total data sent. But because those patterns cover extra ground, they end up revealing more background context than necessary — a real security consideration.
The third finding concerns correcting mistaken beliefs. The researchers modeled what happens when a receiver holds an incorrect belief that directly contradicts the sender’s facts. They found that correcting a wrong belief can cost dramatically more data than simply filling a gap in someone’s knowledge. As the receiver’s incorrect belief becomes more specific and strongly held, the cost of correcting it grows toward infinity.
A Side Project That Grew Into Something Bigger
For lead researcher Luis Lastras, this wasn’t a short-term assignment. The work started as a personal research effort he pursued largely in the background, devoting time to it each week over several years. A small team gradually formed around the question, eventually reaching deeper into mathematical logic to work through the problem in full.
Why This Matters for AI Going Forward
Modern AI systems increasingly depend on reasoning over information, not just processing it. Shannon’s original framework still underlies all of digital communication today, but Lastras and his team argue that future intelligent systems may need a richer theory — one that accounts for what can be inferred from a message, not just how efficiently it can be sent.
The paper extends a conversation that began in 1948 and asks a new question suited to the age of intelligent systems: what if communication theory measured not just the data sent, but the background knowledge required to make sense of it?