USENET notes

From Driscollwiki

Jump to: navigation, search


Contents

Resources

Tech history

"A" News

  • 1979
  • Initially two grad students at Duke in NC
  • Utilities written by others, connected to UNC
    • Dennis Rockwell (Duke)
    • Steve Bellovin (UNC) wrote the first program
      • Fred Brooks (Mythical man month) was a prof at UNC, provided leased line between UNC + Duke (see Truscott interview)
    • Steve Daniel (brought on board in 1980 after USENIX conference) wrote first widely used version
  • Connecting machines via UUCP for info exchange
  • Scaling issues

"B" News ("Bnews")

  • 1982
  • Rewrite
    • Mark Horton
    • Matt Glickman

"C" News ("C-News")

  • 1987
  • Second rewrite
    • Geoff Collyer
    • Henry Spencer

Network News Transfer Protocol (NNTP)

User testimonials from echo

USENET culture

Strange

RFC 850: Standard for Interchange of USENET Messages, 1983

Horton, M. R. (1983). "RFC 850 Standard for interchange of usenet messages." June. Retrieved from: http://www.rfc-editor.org/rfc/rfc850.txt

  • Describes the "B news" format

Introduction

News transmission is not entirely standardized

  • Flexible implementation on individual hosts' hardware/software needs
    • e.g. whether to batch news or not

Article format

All articles must be formatted as valid ARPANET mail messages

  • RFC 822
  • Takes advantage of extensible nature of ARPANET message
  • Demands certain additional information
  • Justification: tools designed to handle ARPANET messages will be able to handle USENET messages
  • Assumes a certain honesty on part of users
    • Sender, From, Reply-To explicitly set by user/user agent

Control messages

"Implementors and administrators may choose to allow control messages to be automatically carried out, or to queue them for manual processing. However, manually processed messages should be dealt with promptly."
  • Did "Cancel" command for sender or superuser remain or was it deprecated?

ihave/sendme protocol

Optional protocol for reducing redundant traffic

  • Machine A prompts with "ihave ..."
  • If Machine B wants it, it responds, "sendme ..."
  • Machine A then sends the article
  • However, there is "high overhead" in sending ANY uucp message so the savings may be quite small
    • Optional batch announcement

Any machine can initiate a newgroup

  • Must be done in advance of posts to that group

Any admin can rmgroup

  • "Since the newsgroup is removed from every site on the network, this command should be used carefully by a responsible administrator."

sys file

Contains list of all neighbors and to which groups they subscribe

  • Requested via sendsys
  • All USENET machines must provide

L.sys file

UUCP network map

  • Contains passwords to access adjacent machines
  • Optional senduuname maybe issued remotely to build UUCP routing maps

Transmission methods

"USENET is not a physical network, but rather a logical network resting on top of several existing physical networks."

Not important WHICH method is used

  • But that A method is available

ARPA Internet mail client not required

  • But very difficult to reply without one
  • Sites must have an internet address
  • In some cases, using email is the best way to propagate news posts
    • This method adds (even) more overhead to the transmission process
    • Makes it harder to prioritize mail over news

USENET news is also batched to reduce overhead

News Propagation Algorithm

USENET is a directed graph

  • Each node is a host
  • Each arc may be bidirectional (often are)

USENET is made up of many subnetworks

  • Subnets are connected graphs
    • Every node is connected to every other node

Lifecycle of a post

  • Article posted on one machine with a list of groups
  • That machine accepts it, assigns unique message ID
    • message_id_123123@foo.bar
  • That machine fwds it to all neighbors interested in those groups
    • According to a local file with all the neighbors and neighbors' interests
  • Receiving sites follow the same process
  • Loops avoided via stored message ID history
  • Optimization req'd to avoid sending messages only to have them thrown away
    • Don't send to anyone in the Path or Posting-Version
  • This process assumes that the local machine carries the group to which the user is posting
    • Otherwise the message is rejected immediately, right?

RFC 1036

RFC 5536, RFC 5537

Peter da Silva and USENET II, 1998

see: http://wired-vig.wired.com/science/discoveries/news/1998/04/11425

"Use rules, not tools" (da Silva)

  • Similar to Fidonet breakdown
  • Lots of voting and committees
  • Similar to Wikipedia?
  • Is there something common to Fidonet, USENET II, and Wikipedia bureaucratic entropy?
    • When code becomes secondary?

Were the problems USENET II was designed to address the result of the trust I see in RFC 850?

Internet Robustness Principle

"Be liberal in what you accept, and conservative in what you send."

"In the case of Netnews, there is an even more important principle, derived from a much older code of practice, the Hippocratic Oath (we may thus call this the Hippocratic Principle): First, do no harm. It is vital to realize that decisions that might be merely suboptimal in a smaller context can become devastating mistakes when amplified by the actions of thousands of hosts within a few minutes." -- (RFC 5537, p 6)

RFC 977, Network News Transfer Protocol (nntp), 1986

  • Brian Kantor (UC/SD)
  • Phil Lapsley (UC/Berkeley)
  • Designed to address the need for connecting PCs to USENET
    • uucp only available between expensive time-sharing minicomputers (DECs, etc.)

History

USENET superior distribution method to mailing lists

  • Fewer admin issues
  • Easier for subscribers
  • Scalable to many 100s of readers
  • Reduce storage by using db of messages on a server
    • Workstations have limited disk space
    • Client-server arrangement among subnets connected via fast ethernet

NNTP

Few constraints on structure of articles

  • Useful beyond USENET to many diff news services
  • Runs as daemon on one server
  • Works well in a network of small computer systems
    • Workstations with small number of users + cnetral server
    • Still not imagining home computers
    • "university or large industrial environment" (1.5)

More efficient system than "traditional transfer methods (such as UUCP)" (1.6)

  • UUCP is typically a once-a-day blast with sorting on the receiver end
  • NNTP is interactive exchange

NNTP cycle

Designed to reduce duplicates

  • Host connects to a neighbor over NNTP
  • Checks if there are any new groups
    • If they meet local rules, new groups are created locally
  • NEWNEWS command checks for new articles in desirged groups
    • Receives list of new articles
    • Selects articles it does not already have
    • Neighbor sends new articles
  • Host indicates other new articles it has recently received (perhaps from local users)
    • Neighbor requests any it does not already have
    • Host sends those new articles

NNTP assumptions

Connection is more frequently than once-a-day (uucp, fido)

  • Stream connection (Internet: TCP on port 119)
  • "SMTP-like" commands/ responses

Server does no user-facing work

  • This is handled by localized client softwares

4.8 Brief world about the USENET News System

USENET is efficient distribution mechanism for users "on UNIX and other hoses worldwide"

  • "traditionally ... linked by 1200 baud dial-up telephone lines"
  • UUCP is the primary weakness, addressed by NNTP
  • USENET runs on "most versions of UNIX and on many other operating systems"

What is the spirit of UseNet?

Bush, R. (1989). "What is the spirit of usenet?". Posted to news.misc on 14 Mar 1980. Retrieved from: http://www.textfiles.com/fidonet-on-the-internet/878889/fido0624.txt

Bush, a veteran user ("before there even was a usenet.")

  • The spirit is what the readers, admis wish to read, transmit, and pay for

"The net is not a commune"

"People own and control property, both physical and intellectual. This comes from outside the net, not within it. Because the net is subject to outside rules it is also not an anarchy, not strictly speaking. It is a propertarian minarchy, to get technical."
"Usenet is built by people who *do*, not by people who argue endlessly about undoing."
"Those who dictate what others should do are the ones violating the spirit of usenet -- that much I do know."


UUCP

"UUCP": Unix-to-Unix Copy Program

  • Re: common shell command cp
  • Originally used for serial connections
    • Including dial up
  • Used in the UUCP project to route email messages among computers using store and forward
  • Initially used for USENET but replaced by NNTP
  • Later implemented over TCP/IP internet links
  • Today used in POS retail applications

Note: most of this info from Wikipedia, http://en.wikipedia.org/wiki/Uucp

History

"The UUCP Project: History". Retrieved from: http://www.uucp.org/history/

Implementations

Horton, M. R. (1986). "UUCP Mail Interchange Format Standard". Retrieved from: http://tools.ietf.org/html/rfc976

  • Attempt to standardize UUCP for mail message transmission among Unix machines outside of ARPA net

Documentation of uucp from BSD


Interesting stuff not relevant to project at hand

Retrieved from "http://www.kevindriscoll.org/wiki/USENET_notes"
Personal tools