<<< Back to Overview

The basics of Usenet

How Articles travel through Usenet

The Usenet consists of thousands of servers all over the world. While every single server is either owned by a person, an organization or a company, the network as a whole isn't (articles posted to a Usenet server will travel beyond the borders of companies and organizations). The owners of Usenet servers have full power over their server(s) and can make decissions regarding their server(s). These are decisions like:

• which Newsgroups are offered on that server,
• who is allowed to post to this server,
• how much space can articles waste on that server,
• which articles aren't allowed on that server,

and so on. But their power always ends exactly where articles leave their server(s) and enter the server(s) of someone else. That's why the network as whole isn't controlled by anyone.

Peers, Infeeds, Outfeeds

Every Usenet server has neighbor servers, called Peers. It will exchange articles with those peers and ONLY with those peers. Not every server exchanges articles with every other server on Usenet, thus articles sometimes must travel over a large number of servers to get from server A to server B. You must distinguish two types of peers: Infeeds and Outfeeds. Infeeds are peers used for receiving new articles, outfeeds are peers used for sending new articles. Of course a peer can be both at once, infeed and outfeed, and often it is both at once.

Articles are pushed on Usenet

Pushing means that whenever one of your infeeds receives a new article, it will offer your server to upload the just received article, by telling your server:

I have <message-id>

And your server can only reply in two possible ways:

1. Send me that article.
2. Do not send me that article.

In case your server accepts, it will upload (push) the article to you. Otherwise nothing will happen. Pushing is used as it distributes articles much faster than if a server would have to regularly scan another server and "grab" each wanted new articles of the server. The message-id is included so your server does not get duplicate copies of the same article (another peer may already have pushed this article onto your server before).

Travel Rules

In theory, an article is posted to one server will sooner or later reach every other server on the Usenet. Unfortunately that's only theory. Articles may not get pass a certain server for various reasons, you could say they might get stuck. One would expect that an article will sooner or later reach every outfeed of a server, but there's no guarantee that those outfeeds will really receive or accept the article. Let me show you some reasons why an article may never be found on a server:

1. The first reason is simple to explain: Not ever server accepts traffic for every hierarchy. When two servers start peering they exchange a filter string regarding what kind of traffic they want to accept and if an article doesn't meet that filter string, it's never offered to the other server in the first place. Unlike most people think, this is not directly related to what groups a server carries (a server often gets traffic for groups it doesn't carry, just like it may carry groups and never get traffic for it). Since the filter is stored on your peers and not on your own server, you can't easily update it.

2. If the server gets an article because it accepts the hierarchy it is posted to, but it doesn't offer the specific Newsgroup it is posted to, the article is not stored anywhere on that server. But it can still be forwarded to other servers, it will just not show up on this specific server.

3. Not every server accepts every article, even though it might accept traffic for a certain hierarchy and even carry the Newsgroup the article is posted to. Some examples for filtering include (but are not limited to):

   • articles with a specific content.
     (E.g. to prevent adult content on a school server.)
   • articles with a specific size.
     (E.g. to prevent binary files on servers with little disc space.)
   • articles from a specific person.
     (E.g. someone who is known to abuse the Usenet for illegal actions.)
   • articles posted through a specific service.
     (E.g. if a service that doesn't play fair and causes trouble for other services, the service might get punished by "article rejection".)
   • or articles of a certain age.
     (E.g. an article needs one week to reach a certain server and articles of that age might be considered out-of-date there.)

4. Sometimes articles simply get lost, either caused by transfer problems or a buggy server software. In both cases the article will never reach a peer although it should have got sent there.

5. Sometimes articles expire early. That means an Usenet server received the article and stored it to disk, but then it was expired before anybody had a chance to ever see it.

If an article never makes it to a certain outfeed, this server will also not be able to further forward the article. In some cases that's no problem, because since Usenet is very chaotic network, there are often plenty of ways an article can take to get from one server to another one. But sometimes there's only one way and if that way doesn't work, the article will simply never get there. Even worse, if you post an article to a server and all of its outfeeds reject this article, it will not travel beyond the scope of this server. That means only people using the same server as you will be able to see this article, for everyone else it simply doesn't exist.

<<< Back to Overview