Deepnet, DarkNet and other, similar terms, have been in the media a lot lately. Most famously, the hacker group Anonymous took down some sites on the DarkNet that were distributing illegal content. This has made quite a few people understandably interested in what the Deepnet really is. It’s not USENET, which will become obvious as you start to understand how and why the Deepnet or DarkNet exist.

Getting Indexed

You might have heard terms such as “search engine optimization”, “SEO” and “search engine marketing in your travels across the web. These are fields that relate to getting search engines to notice them and, thus, to add those sites to the search engine indexes. It’s actually quite a bit of work to get a search engine to notice you; it’s hard to stand out among billions of sites! One of the ways that search engines index a site is by following links from other sites that lead to it.

On the USENET system, the entire point of having a newsgroup is to have it added to as many servers as possible or, at least, to as many servers to which the newsgroup is relevant. USENET doesn’t require search engine indexing, though Google has an extensive archive of historical USENET posts.

Sometimes, sites don’t get indexed at all, and that’s where the DarkNet starts.

Not All Sinister

When sites don’t get indexed, it’s usually because the webmaster was incompetent in some regard, because they didn’t put any effort into SEO or because there was simply no need to have the page indexed at all. For example, some research projects have websites dedicated to them that are just bibliographies or other material that no one but participants would be interested in, so there’s no point in having those sites indexed at all. The sites end up floating around in the Internet ether, being of little interest to anyone and are never really picked up by the search engine crawlers. These sites become part of the DarkNet.

There are also millions of sites that are started and abandoned by designers and webmasters, usually amateurs. These sites end up becoming part of the DarkNet, particularly when they’re on free hosting where they’re never removed and where they just sit forever. Sometimes people stumble upon them and end up finding interesting resources, sometimes not.

Some DarkNet sites are used for illegal purposes, but there is little chance that you’re going to stumble on these. The search engines simply don’t have them in their indexes so, without typing the URL right into your browser bar, you’re not going to find them.

USENET is not part of the DarkNet. USENET is transparent and is designed around sharing information, not hiding it. It’s also something to which you buy a subscription, so finding it is obviously not all that hard. The USENET, however, has a huge backlog of archived articles and other information that makes it as fascinating as any hidden part of the Internet.