An AI-Curated Library of Our Shared Past
Andrej Karpathy's HackerTube project uses AI to curate a library of enduringly popular YouTube videos from two decades of Hacker News comments.
A link shared in a quiet corner of the internet in 2008. A brief comment: "You have to see this." Years pass. The video, a grainy lecture or a flash of unexpected animation, is forgotten. Andrej Karpathy's 'HackerTube' project uses AI not to invent a new future, but to recover these shared moments from our digital past. His work demonstrates how we can task these powerful models with being our cultural archivists, sifting through the noise of millions of conversations to find the signals we collectively deemed worth saving.
HackerTube is a website born from a weekend project, a curated list of YouTube videos that have been most consistently recommended by the Hacker News community over the last two decades. Instead of a feed optimized for immediate engagement, the site presents a collection built on longevity. Karpathy took on the monumental task of analyzing 43 million comments to distill this essence of shared value. The process, detailed in **how to build a YouTube alternative**, serves as a blueprint for using AI to unearth cultural artifacts from vast datasets.
How does an AI decide what has value?
The genius of Karpathy's method lies in its pragmatism. He didn't just throw an expensive AI at the entire 43-million-comment dataset. The process began with a simple, computationally cheap script to perform a first pass, filtering for YouTube links that were recommended by at least three different people in three different months. This initial step narrowed the field from millions of comments down to a manageable 40,000. This two-stage approach—brute-force filtering followed by nuanced analysis—is a crucial insight, offering a sustainable model for how humans can collaborate with AI, using simple code to handle scale and reserving the sophisticated models for tasks that require a semblance of judgment.
Only after the initial culling did an AI get involved. Karpathy used Anthropic's Claude to read the remaining 40,000 comments and judge their sentiment. The model's job was to act as a discerning reader, ensuring that a link was part of a genuine recommendation, not a critique or a casual mention. In this role, the AI wasn't creating content; it was acting as a tireless research assistant with a specific, human-centric directive: find the things people truly loved and wanted to share.
To browse HackerTube is to step into a different kind of internet. It is a time capsule of a community’s intellectual and creative heart. Here, a classic lecture on physics by Richard Feynman might sit alongside a pioneering piece of computer animation from the early 2000s. These are not videos algorithmically selected to hold your attention for another thirty seconds. They are artifacts that earned their place through years of quiet, human-to-human endorsement. The site showcases not the power of the AI, but the enduring quality of the human creativity the AI was tasked to find.
Projects like HackerTube hint at a more compelling partnership with artificial intelligence. Rather than using these systems to generate endless streams of novel, often soulless content, we can use them as lenses to look back at ourselves. They can become our archivists, our librarians, our curators, helping us make sense of the digital mountains of text and media we have accumulated. Andrej Karpathy's project built a mirror out of code, and in it, we can see a reflection of a community's shared memory, carefully polished and presented anew.