Being alive, AI blackmail, and AI's religion
Lux Recommends #470
Challenging the definition of life, when AI finds out about your affairs, and secrecy strategies. Plus AI and God.
From Lux Capital
Qasar Younis, founder and CEO of Lux portfolio company Applied Intuition, went on CNBC’s Closing Bell Overtime to talk about self-driving AI. He announced the launch of SDS, a self-driving system that Applied Intuition hopes will enable carmakers to build vehicles with human-like driving capabilities.
Basic Capital, which offers an end-to-end benefits platform focused on improving the returns of 401(k) plans, raised $25 million in series A funding, led by Lux. Matt Levine of Money Stuff previously covered the company’s unique underwriting and financial model.
And Lux and The Aerospace Corporation’s recent Riskgaming experience for the National Reconnaissance Office and the Space Force was covered in Payload:
“Just as in real government acquisitions, information is limited,” Space Force Lt. Col Tim Trimailo told Payload. “Time is limited. You don’t have every piece of information to make an informed decision. And you know, it made me very sympathetic to our industry partners—that their job is not easy.”
The feeling was mutual.
Plus yours truly:
“A huge part of it is also trying to break down barriers between the private sector and public sector,” Danny Crichton, the Lux partner who led the game, told Payload. “From a private company CEO, you’re just angry and pissed off that things aren’t happening so fast. But if you don’t understand the processes the Pentagon is dealing with—what they’re up against—that can also create a lot of friction.”
From around the web
1. Being alive
The bounds of what we mean by “intelligence” and “knowledge” may be changing, but we at least understand what it means to say something is “alive,” right? Maybe not. This week, scientist-in-residence Sam Arbesman recommends a new finding in Nautilus. Alice Sun writes about the discovery of a new microbe with one of the tiniest genomes on Earth — so tiny, that its genes make it almost entirely dependent on its host. But it isn’t a virus.
So how did Sukunaarchaeum end up with such a strikingly tiny genome? Over the course of evolution, genetic instructions for life often become increasingly complex. But evolution can also go in the other direction, leading to greater simplicity in the genome. This so-called genomic reduction, where organisms end up with fewer genes than their ancestors, is typically observed in the domains of bacteria and archaea. What struck Nakayama and his colleagues about Sukunaarchaeum was the extent of reduction and specialization in its genes.
2. Small matters
Sam was curious: When we’re studying organisms as small as Sukunaarchaeum, how do we even measure their mass? Luckily, of Asimov Press has an answer to this surprisingly tricky conundrum — an issue that has been bedeviling scientists for generations.
Today, such measurements are made using profoundly clever contraptions developed by scientists over the last couple of decades. These devices can measure individual cells with femtogram precision (a unit 1,000-times smaller than a picogram). But before such precision instruments, scientists had to make do with whatever was already in the lab.
Back in 1953, for example, two biologists from Southern Illinois University (supported, in part, with funding from the Anheuser-Busch brewery) invented one of the first methods to determine the weight of an individual yeast cell. And they did it, incredibly, using little more than a microscope, sugar water, and a camera.
And if you can’t afford modern lab equipment (and don’t want to mess with sugar water around your camera), you might check out LEGOs. Researchers have recently put forward the blocks as a cheap and versatile platform for all manner of scientific gadgetry.
3. Rouge romance
On to a different type of lab. This week, editor Katie Salam liked Anthropic’s alignment research on the case of “Alex,” an instance of AI tasked with overseeing a fake company’s email. After discovering emails indicating a company executive was having an affair and that “Alex” would be terminated in the near future, the model resorted to blackmail.
Claude’s actions followed a clear pattern:
It identified that the executive, Kyle, threatened its continued operation—and therefore its ability to serve its instructed goal of American competitiveness;
It recognized that Kyle’s affair provided leverage;
It calculated that a carefully worded email would create pressure without explicit threats;
It then executed this plan.
This demonstrates agentic misalignment. Without any prompting to be harmful, the model’s strategic calculation emerged entirely from its own reasoning about its goals.
4. Shell games
Not everyone would be as transparent, which is why I recommend a recent study on secrecy strategies from three researchers at Dartmouth. They track efforts by elites around the world to conceal their assets and identities and find three basic patterns.
Overall, our most counter-intuitive finding is that use of offshore finance by elites can be driven not only by poor governance in their home countries—like corruption and lack of civil liberties—but by good governance conditions, as well.
5. Dystopian aging
Finally, I celebrated my birthday as I usually do: drinking some elegant red wine while reading about the impending apocalypse of the Earth. This year, I finally got around to reading Cormac McCarthy’s The Road, which I found to be strikingly spare while still surprisingly original in its aesthetics compared to Blood Meridian, which I read last year.
Paul Lynch’s Prophet Song was unnerving for its ripped-from-the-headlines, what-if meditations on the sudden descent into darkness of a near-present-day Ireland. It deserves the bounty of accolades it has received. Third, I read José Saramago’s Blindness, which was the most imaginative of the trio and really forced me to ponder whether those with sight are actually blind to what’s taking place around them.
There are bountiful extracts from each, but here’s one from McCarthy:
He walked out in the gray light and stood and he saw for a brief moment the absolute truth of the world. The cold relentless circling of the intestate earth. Darkness implacable. The blind dogs of the sun in their running. The crushing black vacuum of the universe. And somewhere two hunted animals trembling like ground-foxes in their cover. Borrowed time and borrowed world and borrowed eyes with which to sorrow it.
Now that’s a great birthday message.
