Some interesting information about Usenet

What are Usenet Newsgroups
Newsgroups, E-mail, Web-Pages and FTP are separate, incompatible protocols (or languages) on the Internet. The difference between them is the port number you connect to on the server and the rules for communication when exchanging information and requests. The protocols specify a set of commands and response formats required to exchange information between a client (your computer) and a server (the destination computer). The latest Web browsers (Netscape, Internet Explorer, FireFox, Mosaic, etc.) incorporate these different protocols in one application making them almost transparent to the user. The actual protocols are:
HTTP - Hyper-Text-Transfer-Protocol (http://www.nomadnews.com)
SMTP - Simple-Mail-Transfer-Protocol (E-mail)
NNTP - Network-News-Transfer-Protocol (Newsgroups)
FTP - File-Transfer-Protocol (disk-to-Internet-to-disk file copying)

The NNTP protocol is similar to SMTP (E-mail). When you post to a Newsgroup it's very similar to E-mailing someone, except the destination is a Newsgroup. The one-line header you see when viewing Newsgroups corresponds to the Subject line in an E-mail message.

Newsgroups are like bulletin boards. When you post something, it's visible to anyone who chooses to view it, and on the Internet, that means anyone in the world. News Servers (see list of newsgroup providers here) usually keep posts for about 30 days then discard them (although some companies specialize in archiving posts and letting users search them - for example, Google Groups (was Dejanews)).

To prevent total chaos, the 'bulletin board' is separated by subjects (Newsgroups). At the end of this year there were 121,000 different Newsgroups. Some examples are:

rec.crafts.textiles
rec.skiing.snowboard
soc.culture.indian.american
alt.support.alzheimers
alt.travel.new-orleans
alt.sport.basketball.nba
alt.binaries.games.quake
alt.binaries.pictures.erotica.female
alt.binaries.sounds.1990s.mp3
alt.binaries.multimedia.3-stooges
News applications allow you to post and retrieve articles and retrieve a list of all the Newsgroups your Newsgroup Service Provider carries.

****************************

How do Newsgroups Work?
You might think that since anyone in the world can view articles posted by anyone else, there must be one huge central computer that stores all the articles posted anywhere. Actually, there are thousands of simple PCs (and Servers) running new server software that exchange articles constantly.

A list of providers of Usenet newsgroup services is listed here. This is what you connect to when you view Newsgroups. This Server could simply be a PC running a NNTP Server application. When you post to a Newsgroup the article is stored on your Newsgroup Service Provider's server. Constantly the server will exchange articles with other servers. The servers are connected in a hierarchical fashion so eventually every server around the world should get every article.

(DNews and Tornado, are examples of NNTP server applications which can be run on many operating systems and platforms)

****************************

What are 'binary' groups and what is UU, yEnc and MIME encoding?
Binary groups (alt.binaries.pictures) are like all the others except the articles posted to them are not text (readable). Instead they are anything else (programs, pictures, sounds, videos, ...). The articles found in binary newsgroups are a lot like E-mails with attachments. The attachment is a file that can be anything.

Because Newsgroups were created with only text in mind (like E-mail), users had to get creative in finding ways to post non-text. Text is unique because all the readable characters can be represented with 6 bits (64 combinations), which means the 7th and 8th bit in every byte can be discarded. Non-text requires all 8 bits (256 combinations). So how can you post non-text? You could take every 3 consecutive bytes (24-bits) and translate them into 4 characters (4 6-bit characters = 24-bits too). The resulting file is 33% larger (3 bytes to 4), but can now be represented entirely with 6-bit characters.

This 4-for-3 encoding is exactly what UU and MIME/Base64 Encoding do. The difference between the older UU Encoding and the newer MIME/Base64 encoding are the characters chosen to represent each of the 64 possibilities.

UUE represents 0 through 63 with the following characters:

!\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[]^_
MIME/Base64 represents 0 through 63 with the following characters:

ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/
Notice that UUE uses many punctuation characters. This causes problems with some text filters/processors. For instance multiple spaces can be reduced to one, spaces can be removed if at the end of a line, a '-' can be interpreted as a hyphen and allow a new line to be created.

yEnc has been the latest encoding method used; it supposedly is not as large (when converting binary to ASCII) when uploaded to newsservers. yEnc-encoded files can be found usually by looking at the SUBJECT of the posting (it will say yEnc). Nomad News decodes yenc; download it today.

Originally users had to download the encoded file from the newsgroup, then run a separate application to decode it! Luckily, the latest versions of viewers do the translation automatically while the file is being loaded, transparently to the user.

****************************

Why are there multi-part files ([0/3], [1/3], [2/3], [3/3])?
Since Newsgroups were intended for text articles, the size of posts were expected to remain small. Some older News servers and posting software limited the size of an individual post to an arbitrary size (64K for example). Binary files (pictures, programs) as we all know, can be huge. In some multimedia newsgroups it wouldn't be uncommon to see a 150 part post for a file over 100 Mbytes in size (welcome cable modems).

To post these larger files they had to be broken into multiple parts. The accepted convention is to use the same subject line for all parts except for the part number:

My great post - filename.jpg (0/2)
My great post - filename.jpg (1/2)
My great post - filename.jpg (2/2)
Originally it was up to the user to identify all the parts, download them one-at-a-time, then use a separate application to combine the parts into one file! Now, some browsers (like Nomad News) automatically identify the parts and download them as one automatically.

Parts numbered 0 (0/2) are text descriptions of what the file is. Usually part 0 is not included, but some older encoding software requires the article to have a part 0 when multi-part posts are created.

You can imagine how browsing these groups would be difficult because of all the clutter. You'd see 150 headers that are identical except for the part number. Nomad News automatically identifies and hides the subpart headers and shows just the first part with the total filesize.

The UUE, yEnc and MIME binary encoding schemes support multi-part files.

****************************

What's the difference between GIF and JPEG images?
GIF and JPEG are compression schemes used for storing images as files. GIF is a 'lossless' compression scheme while JPEG is 'lossy'. Lossless means that when the file is decompressed an exact copy of the original is obtained. Lossy, on the other hand, means that the decompressed file is almost an exact copy of the original.

GIF files are usually much larger than the corresponding JPEG would be. An estimation for the size of a JPEG file is 1 bit per pixel of the original image!

The GIF compression algorithm uses 'run-length encoding' which replaces successive copies of the same byte with a byte and a count. Therefore, if the original image has large areas of the same color, the resulting GIF file may be smaller than the JPEG counterpart. This is why GIF compression is usually chosen for drawings or cartoon pictures.

The JPEG compression scheme combines many different encoding methods. The algorithms take advantage of the way the human eye interprets colors and transitions between different colors and shades. Although the overall process is lossy, for all practical purposes, it's impossible to tell the difference between the original and its JPEG representation. For this reason JPEG is almost always used for photographed images.