The Case for Using AI to Log Your Every Living Moment

People know Otter.ai as one of the AI-driven transcription services that have popped up over the past few years, automatically converting spoken words in interviews and meetings into text. The service can even distinguish between individual speakers. But its CEO, Sam Liang, sees this handy functionality as just a beachhead into a more sweeping and provocative project: Capturing everything you hear into a master dataset where you can search and reexperience every conversation you’ve ever had.

Liang began thinking about this a decade ago, after he left a job at Google to cofound a startup that monitored people’s behavior on mobile devices to provide services like automatically tracking mileage expenses. “I’m obsessed with getting data and understanding data,” he confesses. “In my first startup, we used a lot of iPhone sensors: location, GPS, Wi-Fi, motion. The one sensor we didn’t use was the microphone.” Fixing that would be transformational, he thought. “I was frustrated that with Gmail I could search for something from 10 years ago, but there was no way to search for something I heard three hours ago,” he says. “So I did a thought experiment. What if I keep my microphone on the whole day?” Liang then raised the stakes still further. “What if I do it even better—what if I kept the mic on all the time, my entire life—from the day I started talking till the day I die?” He calculated how much data that would be and figured out that you could store a lifetime of audio on a 2-terabyte USD drive. “Then I can search for everything I heard in my whole life,” he says. “My parents have already died. I really wish I can just retrieve their speech.”

Liang isn’t the only one chasing the dream—or perhaps nightmare—of total AI-powered recall. As I wrote in February 2021, a startup called Rewind has already launched with the promise of life-capture, and it has since tapped the latest AI advances to build out that vision. Founder Dan Siroker recently announced a wearable pendant to more nimbly snare everything within electronic earshot. And just this month a much ballyhooed new startup called Humane announced a replacement for the smartphone in the form of a “pin” that can also capture voice.

These products join countless devices like Amazon’s Alexa with microphones always at the ready, potentially fertile ground for apps that can passively record. Maybe the rise of generative AI marks the inflection point for this idea. Using that technology, recording corpuses can become datasets by which people can search through and summarize their life events, and literally dialog with the minutiae of their existence. It might be like having your personal Robert Caro–level biographer on hand.

As you might expect, civil libertarians have some issues with this concept. Jay Stanley, a senior policy analyst for the American Civil Liberties Union, says that the rise of always-on audio capture raises tensions between personal privacy and the right to record. But he mostly worries about how all that data might be used against people, even if originally intended to enhance their memories. “The prospect raises questions about whether the data will be protected, whether it will be vulnerable to hacking, and whether it can be vulnerable to access by the government,” he says. Overall, he thinks services that record all your conversations are a bad idea. “People might feel like it’s empowering to have a record of everything they’ve ever heard, like a super memory or something like that. But it could actually be disempowering and turn against you.”

Not surprisingly, Liang and Siroker both insist that privacy is built into their systems. Both say that they discourage recording anyone without consent. And of course they vouch for the security of their systems.