Gambling Against the Second Law

Jim Crutchfield is no stranger to random systems, or to the idea of trying to wring utility out of tiny bits of information. Back in the late 1970s, he was part of the so-called ’Eudaemonic Enterprises’—a group of physicists who tried to win at roulette by using a computer to track the wheel’s movements and predict roughly where the ball would land.

That, as it turned out, gave them about a 20% boost over house odds in Vegas, but it was a hard grind for the money. "You have to work at it…it’s an actual job," says Crutchfield. To win $100,000, say, they’d need to be bankrolled with about $100,000 to start, and play 24/7 for about 4 months out of the year. "This is not that interesting, at least for us," he laughs. "At some point I decided to go finish my PhD."

Fast forward 45 years, and you’ll find Crutchfield trying to beat the odds in a different way. He and his colleagues at Caltech have been playing with the world’s highest resolution thermometer, this time wringing work out of ’information fuel’ and trying to break the second law of thermodynamics.

"We will try to break it, but I don’t think we will," laughs Crutchfield, now director of the Complexity Sciences Center at the University of California, Davis. "I’m not advocating that we can bust it. But maybe we can temporarily violate it."

Crutchfield’s experiments are helping physicists to better understand the limits on the efficiency of computation in everything from modern digital computers and even in biochemical processes. "This is the frontier, where physics, biology, computation all meet," says Paul Davies, a physicist at Arizona State University, in Phoenix.

The Second Law of thermodynamics basically says that disorder always increases—in other words, a kid’s bedroom always gets messier, unless you put a lot of energy into cleaning it. There are many different ways of expressing the Second Law. Some people call it ’time’s arrow,’ since a measure of disorder—formally an increase in ’entropy’—can reveal which way time is flowing. It also means that in a closed system, like a container of gas, the molecules cannot spontaneously organize themselves into cold molecules on one side and hot on the other; instead they all eventually bounce together into one even-temperatured mush. This is important for people building heat engines: devices that take advantage of a temperature difference to do some useful work. As French scientist Sadi Carnot said back in the 1820s, the Second Law means that there’s a theoretical upper limit to the amount of work you can get a heat engine to do.

Busting the Limit?

Crutchfield and his Caltech experimental colleagues have been using all the tricks of modern science—including nanoscale manufacturing and precision thermometry—to make and investigate smarter, better heat engines, that drive right up to Carnot’s limit, and maybe even, for a moment, bust through it.

The key to it all is understanding how information can be traded in for energy.

Back in the 1870s, Scottish physicist James Clerk Maxwell devised a clever thought experiment regarding the Second Law. What if, he proposed, you have a container of gas divided into two compartments, and a tiny, clever little being who can open and shut a gate between them. What if he opens the gate whenever a hot, fast molecule is whizzing from left to right, or a slow, cold molecule is heading right to left? After a time, the being will separate out the cold from the hot, making order out of disorder. Maxwell’s ’demon,’ as it was later called, would seemingly break the Second Law.

Some physicists have actually made working versions of Maxwell’s demon. In 2007, for example, physicists reported making a nanoscale ratchet system that could create order from disorder: a tiny cog wiggled and jiggled thanks to the random motions introduced by heat, and when it wiggled in one direction (say clockwise) a ratchet would lock it one click along the cogwheel. But every demon has created order at the expense of energy and disorder elsewhere—like an influx of light or heat, for example. Overall, the Second Law isn’t broken.

One of the most interesting recent developments has been to show how, just as with Maxwell’s envisioned demon, the energy used to drive such a system can be wrung from information; in other words, information can be used as fuel. In 2016, for example, researchers used single-electron transistors to make a tiny "information-powered refrigerator." But physicists are still working on describing exactly how much energy the demon gets from information, how much entropy he creates by gathering knowledge, by holding the memory of that knowledge, or erasing his memory to make room for new information.

In the 1960s, German-American physicist Rolf Landauer calculated the theoretical minimum amount of energy used by computation—or heat made by information erasure. But this is hard to test. One of Crutchfield’s goals has been to see if different kinds of memories are better for different kinds of computations (Boyd et al, arXiv:2104.12072 (2021)). But tracking energy in an experimental setting means having some very precise instrumentation, in particular for measuring the tiny amounts of heat. "Measuring directly the heat dissipated in the erasure of one bit, for instance, is really difficult," says Juan Parrondo, who works on these topics at Complutense University of Madrid, Spain.

Stripped-Down Demon

Crutchfield’s team aims to simultaneously measure both a demon’s information processing and the resulting heat flows. Their experimental system is designed to be the simplest, smallest, stripped-down version of a demon you can get. They use a Josephson junction, a device in which a thin insulating layer is sandwiched between two superconducting layers; under certain conditions pairs of electrons can tunnel from one superconductor side to the other. This acts as their ’box of hot gas.’ All the demon has to do is decide whether or not to ’open the door’ (allow the electron to tunnel) based on an aspect of its quantum properties. "This would be an unimpressive engine," Crutchfield laughs, "but it forms a great start for probing the fundamental physics of information."

The tricky bit is then to measure the energy changes that result from this information use. Crutchfield and co used a new thermometer built by Caltech colleague Michael Roukes and his team. Back in the mid-1990s, Roukes, an old college buddy of Crutchfield’s, proposed a way to detect energy changes on the ’yocto’ scale, or 10^-24 Joules. "At the time everyone thought he was nuts," says Crutchfield. The thermometer is made out of a nanoscale sandwich of normal and superconducting metals, capable of detecting individual heat-carrying oscillations, or phonons. Realizing the device allows physicists "to look at simple physics with fresh eyes," says Michael de Podesta, who studies temperature measurement at the National Physical Laboratory in Middlesex, UK, and is not involved with the work.

University of Michigan physicist Jordan Horowitz, who helped to establish the field of information thermodynamics, is impressed with Crutchfield’s approach. "I have seen Jim and his students take these ideas to whole new places," he says. "This particular measurement is incredibly important," he adds. Remarkably, "no one, to date, has been able to measure heat on such tiny scales and therefore confirm this theory."

The entire experimental collaboration, with about 20 people working on it, was funded by a Department of Defense initial grant of $6.5 million, with a $1.5 million top up; FQXi grants of over $165,000 helped to link several of the sub-projects together with a theoretical backbone.

The project helped to experimentally confirm how much energy you have to invest to do information processing; or, to put it another way, how much work an observing demon can do with information fuel. The team discovered that logically reversible computations are thermodynamically efficient (Ray et al, arXiv: 2202.07122 (2022)), but logical irreversibility is much less efficient than had previously been thought (Boyd et al, arXiv:2104.12072 (2021)). "We feel like we’re very close to understanding what the cost is to observing," Crutchfield says.

The new results should yield more practical applications, Crutchfield says. "We have some wild ideas about how to make engines more efficient, with structures and materials that are ’intelligent’ so they can harvest energy or, say, move heat off your computer chip," he says. Horowitz agrees: "Such insights, I believe, will be quite important as we scale down the size of computers."

When it comes to demons, the smarter the better.