Let’s just say it’s 1995 and you’re looking for information on the Web. Where do you go?

Well, history tells us a lot of people went to the Yahoo Directory, a cutting edge tool to manage the ever-expanding tsunami of information inundating the Web every day. The Yahoo Directory: the most ambitious attempt at cataloging the sum total of human knowledge since the ancient Library of Alexandria. Yes, the mighty Yahoo Directory: compiled, sorted, arranged and cataloged by a crack team of 100-plus experts that enabled the …

Wait a minute. You don’t know what the hell I’m talking about, do you?

This thing!

Screen shot 2014-10-06 at 11.15.14 AMWhat, you  don’t remember that?

You kids, with your “apps” and your “streaming” and your this and your that.

Get off my lawn!

I guess I’m cranky because Yahoo said last month that come New Year’s, it would put an end to the directory , a product I used to work on.

Not that anyone was using the thing anymore, except maybe by accident. Where once it took up the entire homepage of Yahoo, the most-trafficked and celebrated site on the Web (that’s how far back I’m talking), you can only find it today if you can somehow manage to get to this page then look under D. Here’s what it looks like now (if “now” is before Jan. 1, 2015).

Pretty lame, huh?

Well, it wasn’t always like that. Believe it or not, browsing a directory — categories broken up into subcategories and then even more subcategories and then even more subcategories — was actually a rational way to present Web content in the mid-to-late ’90s. That’s because the search engines of the time pretty much stunk, and you often had to sift through a lot of irrelevant results when you typed something into now-forgotten sites like Excite, Infoseek and Lycos.

So Yahoo drove a ton of separation between it and those competitors by organizing websites by subject, which it did by having a human being review and sort every single thing submitted. Yahoo called such an employee a “surfer” (which is what it said on my business card). And it had to hire a lot of them, because the websites, well, they just kept coming and coming. …

“It was the original idea,” says Harold Check, who worked on the directory from 1997 to 2002.

“Dave and Jerry [Yahoo founders David Filo and Jerry Yang] sat down and said, ‘Hey, the Internet’s cool, let’s put it in categories and make it browsable,’ Check says. “Yahoo was the first thing when you hooked people up to the Internet in 1995 or ’96, and you said, ‘This is the front door, this is the Internet, look around here and your mind will be blown.’ In that sense [the directory] was great, probably because the Internet was great.”

Christian du Lac, who worked on the directory from 1999 to 2004, says he was “bummed out” when he read the news it was shutting down.

“It had a lot of influence,” he says. “Yahoo was the directory.”

Du Lac thinks the directory’s ethos — a zealous dedication to the most logical and useful arrangement of information — has been lost on the Internet.

“Right now there’s a lot of talk about doing the right thing for the users, but it’s really about the advertisers and how you can monetize,” he says. “But when Yahoo started it really was about doing the right thing for the users. You see that culture kind of go away with the directory.”

Yahoo surfers, many of whom had worked at bookstores and libraries, had a lot of unexpected power back then.

“I just remember people begging me to get their site listed because the perception was that would be the difference between success and failure of a website and of a company,” Du Lac says.

Yet, I can corroborate the employees who worked on the directory had very little on their minds except the integrity of the massive and intricate edifice of information they were building. I’m talking thousands upon thousands of employee hours spent on considering and arguing the nuances of where and how Web content should be categorized. They weren’t necessarily trivial debates, either. For instance, in the abortion category, do you call sites that expressed a negative opinion “pro-life” or “anti-abortion?” How about whether to call it Myanmar or Burma?

“A country name is changing — that was one we always used to talk about,” Check says. “If the country wants to call itself something different, the people want to call it something different, but the United Nations isn’t recognizing it, what’s our responsibility within the world to kind of rename something, to choose what to call it and where  to put it? I thought the people that worked there took great responsibility in getting things right. You can’t imagine that sort of thing existing right now, outside of an academic setting. Maybe it does, but it’s just not endemic.”

The G Word

For several years, the directory not only worked as an information tool, but as a business model, and Yahoo became the darling of Wall Street.

So what happened?

“I hate to say it,” says Steve Berlin, the very first surfer to be hired full time in 1995, “but when Google came around, the directory was basically redundant.”

Google! Around the time of the millennium, a dirty word at Yahoo.

“The biggest problem we had was everybody wanted their site added, but it’s only possible to add so many sites per day, no matter how many people you throw at it,” Berlin says. “So more was falling through the cracks than there was getting on the directory. And with Google, that’s not a problem. Because of their algorithm, you can get exactly what you want, you don’t have to drill down through various categories.”

So the years went by and Yahoo kept demoting the directory on its home page, supplanting it with an ever-expanding stable of homegrown products — Yahoo Finance, Yahoo Sports, Yahoo News, Yahoo Movies … you get the idea. Because the directory was filled with websites that sent people away from Yahoo — not exactly what the biz dev and product managers liked to see —  the surfers, once the most important gatekeepers on the Internet, became increasingly marginalized, until, like the directory, they were virtually extinct.

Come 2015, the patient will finally be put out of its misery. But that won’t keep some of us from nostalgically partying like it’s 1999.

“You wouldn’t think of a list of websites as having a soul,” Du Lac says. “But the Yahoo directory did have a soul.”

And so it goes. …

  • O Boy

    I’m shocked there’s no mention of DMOZ (dmoz.org) which I believe was a precursor and possibly the original source for the Yahoo directory (dir.yahoo.com). In fact their “About” page says “DMOZ powers the core directory services for the Web’s largest
    and most popular search engines and portals, including Netscape Search,
    AOL Search, Google, Lycos, HotBot, DirectHit, and hundreds of others.” It looks like the last DMOZ site update was March of this year.

  • Chris OConnell

    Nice story. I am surprised the directory is still there.

Author

Jon Brooks

Jon Brooks is the host and editor of KQED’s health and technology blog, Future of You. He is the former editor of KQED’s daily news blog, News Fix. A veteran blogger, he previously worked for Yahoo! in various news writing and editing roles. He was also the editor of EconomyBeat.org, which documented user-generated content about the financial crisis and recession. Jon is also a playwright whose work has been produced in San Francisco, New York, Italy, and around the U.S. He has written about film for his own blog and studied film at Boston University. He has an MFA in Creative Writing from Brooklyn College.

Sponsored by

Become a KQED sponsor