Eric Betzig shared the 2014 Nobel Prize in Chemistry for super-resolution microscopy. I had no idea who he was, but after watching his recent interview on the 632nm podcast I’m totally hooked:

Live-Cell Imaging and the Limits of Structural Biology | Eric Betzig on Super-Resolution Microscopy.

Much of the conversation, which runs almost three hours, covers in great detail his career in physics and engineering. He loves building things. And he’s always looking around to find and use new technology to create his stuff. He did near-field microscopy at Bell Labs, but then he quit science in utter frustration in 1995 and spent six years building advanced machine tools at his father’s company in Michigan. Then during two stretches of unemployment in the early 2000s, he and his physicist friend Harald Hess built the first PALM (Photoactivated Localization Microscopy) in Hess’s living room using their own cash. That work later earned Betzig the Nobel Prize. That itself is wild. I guess it pays to take some time off to hack on some side projects, eh? “By far the best periods of my life were my two periods of unemployment,” Betzig says.

When Betzig talks about machine tools he’s not just just poking around in the factory floor banging on sheet metal. He was designing and building totally new machines and concepts to solve specific problems others hadn’t even attempted to crack. It’s also clear from his language that he sees obvious connections between disciplines that others can’t imagine. It just makes sense to him to draw from as many disparate pools of expertise as possible.

What he did with the method he implemented with his friend matters much more to him than the Nobel Prize. In fact, he hates the awards and prize culture that’s pervasive throughout science. He says it’s crazy that one person gets the credit for some bit that resulted from the work of many other people over decades of time. And it also probably helps explain why a physicist who says he never wanted anything to do with AI is now trying to build the largest vision model in the history of biology using AI.

“It’s my firm belief that you cannot understand life without looking at it live,” Betzig says. “And it drives me insane because I keep making this this point over and over again that we need to go back to a holistic understanding of this complexity that exists in the cell instead of being focused on just little parts.” He’s certainly bold in his assertions. But if he succeeds, we may discover that our current understanding of the cell has been rudimentary at best. I guess the science is never settled.

The Cell we See in Textbooks

Betzig and his colleagues used PALM-style visualization technologies to watch transcription factors in live cells. These are the proteins that gather at the start of a gene before the cell reads it to initiate some process. The accepted scientific model at present says that transcription factors form stable complexes that lasted minutes or hours. But that’s not what Betzig saw when he looked with his new kit. “None of them were binding to the DNA for more than a second or two,” Betzig said. “And so it was like, holy fuck, our whole model of how transcription works is completely wrong.”

That’s another thing about Betzig. He curses. A lot. I like it. We need more Nobel laureates like that. It’s refreshing to hear deeply technical language mixed in with talk you’d hear in a garage. Anyway, the results he observed shaped the rest of his career. And it led him to a wild claim:

“This is the take-home story of this whole talk. Almost everything you learn in biology textbooks is a hallucination. It is. Or at least cell biology because what they’re doing is they’re taking three reductionist tools — biochemistry, molecular biology, and structural biology — and then hypothesizing how those little pieces, tiny tiny little bits, come together both structurally, stoichiometrically, and dynamically to create the cell. They have no direct knowledge of the stoichiometry or the spatial arrangements or the dynamics. All of that is hallucination. I’m exaggerating, but largely speaking.”

For that example he was pointing to the animations that show up online and in classrooms. “You guys have probably seen on the web those beautiful things of all these molecules coming together. Here’s a cargo on a kinesin walking along a microtubule like this. And it’s all in this vast empty space. I don’t know any cell that’s a bunch of vast empty space. I’m sorry. It’s crowded as fuck in there.”

The fix for this, he says, is to look at cells while they’re alive and moving. “When you start to fucking look at the dynamics, not just the structure, you realize that you had it all wrong. And you realize that so many of the things that they thought they knew, you can’t be sure that they know. We have to reinvestigate all of it. So, that’s why I pivoted to live imaging.”

More Complex than a Neutron Star

Betzig keeps coming back to the scale of what’s inside a single cell. And he gets animated real quick. “There’s 100 trillion water molecules in every cell. There’s 10 billion protein molecules. There’s 10 billion carbohydrates. There’s 10 billion lipids. There’s metabolites. It’s by far the most complex matter in the known universe. We understand the interiors of neutron stars far better than we understand the interior of cells. It’s crazy complex.”

Betzig says that most of the order in a cell comes from what we perceive as random motion. Molecules bounce off one another trillions of times around the cell until they happen to land somewhere they stick. Entire structures build up from there, and he says the same pattern repeats at every level of life. “Living matter involves emergence from many different levels, starting from the stochastic motion of single molecules to macromolecular assemblies to membrane-bound organelles to cells to tissues to populations to the whole fucking biosphere. Everything about life is emergence from single molecules to that level.”

He offered a practical comparison from his years in the auto industry to illustrate the point that we just don’t know how the cell really functions. “Imagine you try to reverse engineer an internal combustion engine. If all you have is the reductionist tools that rule biology today and ruled in the 20th century … trying to reverse engineer life from that would be many, many orders of magnitude harder than having a random pile of internal combustion engine parts and trying to figure out how internal combustion engines work. That’s where we are today.” If you’e ever broke down and worked on an engine, you get a small sense of what he’s talking about.

Why so Many Drugs Fail

Betzig ties this gap in observable knowledge directly to the cost of drug development today. He works with Eikon Therapeutics, which he helped found. Roger Perlmutter, the CEO, is the former head of research at Merck and one of the most prolific and successful drug discoverers in the 20th century. Betzig quoted Perlmutter: “It’s a miracle if we ever find a drug that works because we have no idea what we’re doing. And that is accurate because of this focus on such a crazy level of reductionism instead of understanding the system holistically.”

Betzig concludes: “There’s a reason why only 9% of the drugs that enter phase one come out of phase three. Because we don’t know what the fuck we’re doing. We don’t know the real mechanisms that are going on.”

A drug might bind its target protein perfectly but still fail. The protein may never reach the right place in the cell, or the drug may affect some of the other proteins in the cell. Those problems often don’t show up until clinical trials or after the drug is on the market. He’s especially skeptical of startups that design drugs from protein structures predicted by AlphaFold. “Those proteins and that stuff is such an infinitesimal part of the whole dynamic complex system that creates life. They’re burning money for nothing.”

How we engineer cells today “is to put fluorescent tags on specific proteins so they’ll light up. And that’s one of the most limiting parts of optical microscopy still because in visible wavelengths there’s a limited number of colors that you can do. There’s 20,000 different types of proteins in the cell. It would be great if we could see them all at once, but that technology does not exist. And that is a Nobel waiting to happen if somebody can directly interrogate proteins in some way and deduce them without having to put those tags on.”

His answer is to watch living systems directly at every scale from single molecules to whole organisms. His lab now has microscopes and the technology that can capture that data. The challenge is what to do with the data afterward. That’s the missing opportunity he sees.

Too Much Data and no Way to Read it

The instruments Betzig has cover time scales from milliseconds to days and sizes from nanometers to centimeters. And they work for as far as they go. But the analysis of the data can’t keep up with the collection of the data. “We evolved to see in 2D plus time,” he says. “Life happens in five dimensions. XYZT and molecular species, the 20,000 proteins, all the lipids, the carbohydrates, all the rest. Even with the microscopes that take our petabytes of data at all of those length scales, it’s fucking bits on a drive and it does nothing. It’s so fucking frustrating to have petabytes of data on drives that are completely worthless because there is no scalable way to look and understand that data.”

So, he needs a new kind of model to dive into all this data.

“What we need is to build a five-dimensional mind that can look and see in five dimensions. A five-dimensional vision transformer. That’s what we need to be able to crack this nut.” He needs AI. But he didn’t come to this conclusion willingly. “I was the last human on earth who ever wanted to have anything to do with AI because I hate doing what everybody else is doing. But I am forced in this direction because I think it is the only possible scalable way to really extract meaning at scale from the data that we can take.”

Segmentation Comes First

The hosts asked what the model would actually be trained to do. Betzig says the first job is basic. The model has to clearly see where things are and where things begin and end. It has to segment. It has to see the edges of everything in the cell.

“The first task that you have to do is robust 4D segmentation. The fundamental unit of life is the cell. You would like to be able to see the cells individually. Beyond that, you would like to see the organelles inside of the cell.”

He points out that even flat images are still a problem. “In this modern age with everything that we have, even 2D segmentation is imperfect.” Tools like Meta’s Segment Anything work on 2D images, and researchers often apply them one slice at a time to 3D data. He thinks that’s a mistake. “That’s not the way to do it because you’re missing a prior. The prior is there’s reasonable continuity between successive planes. And then likewise, there’s reasonable continuity in time. The cell doesn’t jump to this right away. It moves continuously. So you need to build an inherently native 4D model.”

Once an LLM model is taught and understands where cells and organelles begin and end, he wants to talk to it directly. “I want to be able to ask through an LLM interface what happens when a T cell is, through immuno-oncology, engaging with the tumor. What particular proteins are expressed at the surface? What is the course of its motility in order to get to the tumor? I have all of that data on fucking drives, but I can’t access it because I can’t find out exactly where it is. I need something to recognize that.”

Training Without an Army of Annotators

One host noted that most AI successes depend on properly labeled data. Tesla, for example, learns from drivers who take over the wheel. Betzig says the fluorescent labels already carry much of that information, and he can teach the LLM all it needs to know about the cell. If a T cell and its target glow in different colors, the model can see when they meet. The rest has to come from the data itself. “This is why you have to do something through a transformer, because it has to be self-supervised with minimal annotation thereafter. That’s the only way it’s going to work because you just can’t use human annotation at scale.”

The hosts raised self-driving cars as a comparison, and he quickly agreed that it’s the best one. “You have hit on the closest analogy to what we need exactly.” He added, though, that Tesla’s cameras still only estimate the third dimension. He tried to interest xAI, along with many other AI companies, in his project but without success yet.

Is this Even Possible?

Betzig isn’t saying the problem is easy. “Even I, and I’m not an AI guy, feel like this is at the very bleeding edge of what’s tractable. It is a big scale. It would make AlphaFold look like a picnic.”

He’d like to test his idea with smaller ablation studies first. But compute resources are scarce, and his funding is limited. “I would kill to have 128 B200s.” His lab recently bought 32. It took nine months to get them, and then they had to “fight like hell” for enough power and cooling to run them.

Hiring the right engineering talent is also hard. He works at Berkeley, a few miles from companies across the bay in Silicon Valley that pay AI engineers far more than a university can. “At fucking Cal, how much do you think I can hire an AI engineer for compared to what he can get five miles away from here? You have to find the crazies. The crazies like me and Harold who don’t give a fuck about the money. They give a fuck about the problem. They’re hard to find.”

Even viewing the data is slow. “A 10-second movie can take days,” he says. His team is looking at Gaussian splatting as a lighter way to represent the data, although he’d prefer that someone else build it. “The last thing I want to do is reinvent the wheel if there’s people better at this type of shit than us neophytes.”

He already uses AI in his own daily work. He used to design optical systems in Zemax. Now he describes the lens he needs to Grok, and the design comes back in about five minutes. He estimates the full project would cost about $50 million. So far no funders are interested, including the Howard Hughes Medical Institute, which still supports his lab. Betzig also admits that he’s no businessman. He’s an engineer. A scientist. He could also use some business types who can think about these problems like he does. They, too, are rare.

How it Could be Used

Besides basic biology, Betzig sees a practical use in drug testing, which brings him back to the failure rate he complained about earlier. His lab uses zebrafish because they’re transparent vertebrates that share much of their genetics with humans. With a trained model and a baseline of normal activity across organs and cell types, a company could screen drugs for off-target effects before human trials. “If they got something they’re about to put in phase one, they bring it to us, we put it into the fish, and we see if bad shit happens.” That could revolutionize the pharmaceutical industry by solving problems they’ve had for decades. He didn’t mention this in the interview, but could you imagine the images you’d get in future MRI and CT scans if the resolution could be increased to the point that Betzig imagines?

No Virtual Cell Yet

He has little patience for efforts in the field to simulate a whole cell from partial data. Most of those projects rely on spatial transcriptomics or protein structure. “It’s one part in 10 to the tenth of what’s going on in the cell, and they’re going to recapitulate all of cellular behavior on the basis of that. It’s naive to the point of craziness, in my opinion.”

He has a broader complaint too. “Everybody’s creating huge atlases, and so we’re getting all sorts of data and no understanding. Just collect, collect, collect.” But at least having the data can lead to the perfect opportunity for AI to use the data in a way that’s never been possible before.

When the hosts kept coming back to modeling, Betzig stopped them cold. “I haven’t gotten through to you guys how complex it is. We’re not ready to model. We have to observe.” He compared the field to astronomy before Kepler. “We have to be Tycho Brahe and guys like that. We have to be looking at the thing and seeing these epicycles. We don’t fucking know the inverse square law yet, but we have to see the epicycles first.”

And he’s clear that just storing piles of data on disks doesn’t count as observing it. “Putting it as bits on drive is not observation. Having understanding either through a machine or through humans, or best, humans and machines working together, is the only path forward.”

It’s Mystery Everywhere

Near the end of the interview, he described why he spends his last working years on this work.

“It’s the last frontier. It’s mystery everywhere. We understand so little. We’re in the pre-Kepler era. We’re in the phlogiston era of understanding biology. There are so many things we think we know that are just bullshit. It really is the final frontier because it is the most complex matter in the known universe. And it’s going to take generations, if not eons, if ever, to have enough of a mechanistic understanding to really do a virtual cell like they say they want to do now. People just don’t understand exactly what big of a mystery it is.”

And More

The interview goes on and on to explore many other subjects such as peer review, the culture of science, politics, funding, grant writing, engineering, rockets, space exploration, and many bits I can’t even remember. I only really covered the AI section. Betzig thinks as he talks so he can bounce around a lot. But it’s that free-form thinking and his enthusiasm that’s addictive because he makes such interesting connections. In fact, he ends the interview by saying that the best place for young people to go work right now is SpacexAI. He says they are doing the best science out there at the moment.

Betzig is quite the character. He reminds me of Kary Mullis, who won the 1993 Nobel Prize in Chemistry for his work on the polymerase chain reaction (PCR). It’s a shame Mullis died so early. We need guys like Betzig and Mullis in science who speak so freely about what we don’t know now and the possibilities we all have to create new realities in the future.