NOTE: These notes are of historical value only!

Please note that Grex has changed drastically since these notes were written. In particular, it is no longer running on Sun hardware, nor is it running SunOS. As of July 2008, Grex is an AMD Athlon based machine running OpenBSD. Much of the software listed below is installed, but much more recent versions, and most of it through the OpenBSD ports collection. Notable exceptions are things written by people close to Grex itself, such as the Backtalk conferencing software.

[index]

This note describes briefly the hardware and software used on Grex. Cyberspace Communications has very little money to spend, and we have a very large number of users on our systems, so, although we don't have much, we work it very hard.

Hardware

The inventory database, gives a bare-bones list of all the equipment we owned by Grex. This document describes the systems and their configurations in more detail.

Main Grex Computer

Our main computer is currently a Sun SPARCserver 4/670MP. This is a pretty decent machine by our standards. We actually bought our first 4/670 system and had it running at least a year before it got fashionable for companies to stop using them and start pushing them off their loading docks. Lately though, we've been lucky enough to catch a few that were being dumped.

Sun's SPARC machines are all RISC (Reduced Instruction Set Computer) architectures. This is sort of the opposite approach to chip design from things like Intel's Pentium chips. The basic idea behind RISC is to start with a very simple processor that handles a small, orthogonal instruction set, and then work like heck to make it do the simple things it does very fast. There are several RISC architectures around, but Sun's SPARC architecture is probably the most commercially successful.

Grex's actual 4/670 system consists of the following components:

1 Sun 12-slot chassis with power supply and fans.
1 Sun 501-1439 backplane with 12 VME card connectors.
1 Sun 501-1686
(Rev 06)
4/6xx VME motherboard with boot proms (version 2.10), two Mbus connectors, three SBus connectors, a SCSI controller, an ethernet interface, two serial ports and 32 sockets for 30-pin SIMMs.
1 Sun 501-1901
(Rev 06)
4/6xx VME memory expansion card with 64 sockets for 30-pin SIMMs.
2 Sun 370-1388 SM100 Mbus module with two 40MHz Ross CPUs.
1 Performance Technologies PT-SBS430 S-Bus SCSI Controller Daughterboard
96 Various 4-Meg (80ns) SIMMs (30-pin).
2 HP 3010 2.1 Gig Full Height SCSI Disks.
3 Seagate ST42400N 2.1 Gig Full Height SCSI Disks.
2 Seagate ST15150N 4.1 Gig Full Height SCSI Disks.
1 Toshiba XM3401B CD-ROM Drive.
1 Exabyte Mammoth 8mm Tape Drive.

We are currently running on a two SM100 modules, so it is a four processor system. We will probably experiment with removing one module, giving us just two processors, because we aren't convinced that the system wouldn't be faster that way. This is because of the limitations of SunOS 4.1.4, which we will discuss more below.

Our motherboard is fully populated with 32 4-Meg SIMMs, and there are 64 more in the memory expansion board, so Grex has 384 Megs of total memory. (You could put in 16-Meg SIMMs instead, which is actually more cost effective on a dollars per megabyte basis, but we didn't have the money for them.) Unfortunately 30-pin SIMMS are no longer made, and are thus a bit scarce, so the cost on these isn't coming down as much as other memory.

The 4/670 was designed around 1991 as an enterprise server machine, one that will be simultaneously used by many users. It is solidly over-engineered for robustness and stability. This is in contrast to most Intel systems which are designed primarily to work as single-user desktop workstations and are built as cheaply as humanly possible. The difference is not so much in the speed of the processor (probably most modern Pentium systems easily could outrun the 4/670 at a single task), but in hardware features that make it better able to juggle many tasks. For example, Sun's servers have "hardware contexts" where the states of many different running jobs can be stored in the processor, so that the processor can jump almost instantaneously between them. Pentium systems have only one hardware context - whatever is being run at the moment. Our previous computer, a Sun 4/260, had 16 hardware contexts. The Sun 4/670 (with the SM100 processors that we have) supports 4,096 hardware contexts in each processor. Features like this mean that the 4/670 can deliver a lot better performance in our application than you'd expect from four 40 MHz processors.

We purchased this machine piecemeal, with money donated by our users in a fundraiser. Since then, many spare parts and add-on have been donated by various people. In January of 1997, we bought a second-hand Sun 4/370 from the University of Michigan property disposition for $212. We used only the chassis, backplane and power supply from this for the new system. A chassis is expensive to ship, so buying it locally was advantagous. In Februrary of 1997, we purchased a second-hand 4/600MP board and a memory board from CRA, Inc for $718, an SM100 module from Piece by Piece for $40 (plus $10 shipping) and 20 (two spares) 4-Meg memory chips from QuickerCAD and Ken Olsen for $440. In December of 1997 we bought 16 additional SIMMs for $248. Thus the total cost of the machine, not including disk drives, was $1718. The CD-ROM was donated to Grex by Tod Plesco.

In January of 1999, two different donors gave us parts of two different Sun 4/690 systems. A 4/690 is exactly like a 4/670, but in a different box. It comes in a rack mount chassis that uses 220-volt power instead of 110-volt. One donor gave us a spare motherboard, a rack, a VME card bay, and a drive bay. The other donor gave us a motherboard, three more SM100 processor boards, 32 4-Meg SIMMs, 10 2.1 Gig differential SCSI drives, plus another rack-mount drive bay, and various SBus cards. A pair of processors and 16 memory chips were added. Then in February we were about to collect another 16 4-Meg SIMMs from a number of different donors, and brought Grex up to 256M of memory. Since then, as the 4/670 continues it's slide toward total obsolesence, more and more components have been appearing on our doorstep. In July 2001, we added the SBus SCSI controller, so that a second SCSI chain could be supported. We are building a second 4/670 for use as a backup and development system.

Routers

Grex is currently connected to the internet by a 144K DSL connection. The router is FlowPoint/2200-12 SDSL Router. Very conventional and even vaguely modern.

Before we had DSL, we had a 128K ISDN link and a pair of Pipeline P50s. Before we had the ISDN link, Grex was connected to the internet through a 28K dial-up PPP connection. Our router at that time was a machine called "Gryps." It started as a 386 system running FreeBSD, then got upgraded to a 100 MHz Pentium running OpenBSD. It still serves a few functions, like supporting the terminal servers, and acting as a gateway machine for staff.

Modems and Terminal Server

Grex maintains 7 of its own dial-in modems. The number has been steadily decreasing as more and more of our users change over to using the net connection instead of dialing direct. All are 14.4 GVC modems that look identical, but since they were bought at two different dates, they have two different chipsets in them. All modems connect to a Chase IOLan terminal server called "groupie", which interfaces up to 16 modems to our local ethernet. It is configured to be transparent, so that dial-in users are automatically given telnet connections to Grex, without going through any kind of menu. We have a spare Chase IOLan called "groovy" which is not currently being used. Groupie and Groovy were donated to Grex by Rob Argy.

System Software

Operating System

Grex runs SunOS version 4.1.4. Sun has changed over to a new operating system called Solaris 2 (and retroactively renamed SunOS 4.1.4 to Solaris 1.1.2), and is rumored to be close to ending their support of SunOS 4. At the time that we were setting up Grex, Solaris 2 had a somewhat poor reputation among computer gurus - it was generally thought to run slower and less reliably then SunOS 4 did. Since we had a very heavily loaded machine, we felt that we couldn't afford the slower speed or lower reliability, and thus choose SunOS. (Actually, since we were on a Sun 2 at the time, I suspect Solaris wasn't even available.)

At this point Solaris has probably improved enough so that it would be possible to change over to using it, but changing over from SunOS to Solaris is a very big job. We've made many modifications to our system, including modifications to the kernel, and converting them over to a new operating system would be difficult.

As it is, SunOS 4.1.4 has worked very well for us. We've been placing our machines under extremely high loads for many years, and the robustness and performance of the system have been genuinely impressive (though to the casual user who doesn't appreciate the capacity of the hardware and the size of the load it has sometimes seemed impressively pathetic). Much as we'd like to have access to source code for the operating system, we feel that none of the free Unix versions currently available would perform as well under these conditions as SunOS.

Sun still regularly releases new patches for SunOS 4.1.4. We regularly check the current patch report and try to keep Grex current. A list of installed patches is in the /etc/patchlist file.

Unfortunately, SunOS 4.1.4 wasn't really designed to work with multiprocessor systems like the Sun 4/670. Many kernel functions depend on the assumption that only one processor is executing them at a time. To make SunOS 4.1.4 work at all on multiprocessor systems, Sun took a drastic approach - they allow only one processor in the kernel at a time. This is simple, but it has a big performance penalty. If one user's process has done a kernel call to read a file from the disk, and another user's process wants to call the kernel to send a packet out on the net, then the second process must wait for the first one to get out of the kernel. This is obnoxious. On Grex most processes spend almost 50% of their time in the kernel, so if you have two processors and the system is fully loaded, then there is a processor in the kernel almost 100% of the time. If we added any more processors, they would probably spend most of their time waiting around, and, since interprocessor communication isn't done all that efficiently under SunOS 4.1.4, it's quite possible that adding more processors to communicate between would actually slow the system down. These problems are fixed in Solaris, which has a true multi-threaded kernel so processors spend much less time waiting for each other.

The Sun 4/670 appears to be the last machine Sun made which could run SunOS 4.1.4 at all. So if Grex ever wants to move to a more powerful machine, we will have to move away from SunOS. However, we are currently thinking that the future of Grex lies not in building faster and faster machines, but in going to a distributed system consisting of multiple machines.

Kernel Modifications - Internet Blocks

Most Unix system administrators are very interested in keeping strangers off of their computers, but have no problem letting users of their machines access the internet. At Grex, our requirements are different. We let anyone at all onto our computer, but since we don't want it used as just a stepping stone for people who want to attack other servers on the net, we limit the ways in which people can connect to the net from our machine.

To enforce this, we have made some modifications to our SunOS kernel that prevent most users from making most kinds of outbound internet connections from our host. The kernel of a Unix operating system is the part that schedules all processes and organizes and controls access to all devices (disk, memory, CPU, ethernet, serial ports, etc), so putting these blocks there means that they cannot be bypassed. For detailed information on how this works, see the kernel blocks page.

Kernel Modifications - Fork Killer

We also have occasional problems with users running fork bombs on Grex. A fork bomb is simple a program that endlessly launches copies of itself, each of which also endlessly launches copies of itself. Fork bombs can slow the system down badly, and can be hard to kill because new processes are created faster than anyone can type kill commands.

SunOS has the ability to limit the maximum number of simultaneous processes any one user can have running. Setting this limit to a low number (32 processes on Grex) reduces the annoyance of fork bombs, but doesn't eliminate it. So we modified the kernel so that if any user tries to use more than 32 processes then that user is instantly logged out and all his processes killed. We also modified the kernel's kill logic so that processes can be killed while they are forking. This pretty much eliminates all problems with fork bombs.

Free Software

There is a huge amount of public domain or Gnu-license software available for Unix systems, and a lot of it is installed on Grex. The Gnu project especially is steadily working away at creating a complete free Unix system. They have developed very good versions of most of the Unix utilities, and these are widely used by all the free Unix versions (Linux, FreeBSD, etc). We have installed most of these in the /usr/local/bin directory on Grex, so that most of the commands you'll find there are actually Gnu versions rather than SunOS versions, though the Sun versions are still available.

The following is an incomplete (and probably soon to be obsolete) list of the freely distributed software installed on Grex:

Program Version Description
apache 1.3.26 Apache HTTP Server Daemon
bash 2.05 Gnu Borne-Again Shell (enhanced sh shell)
bison 1.25 Gnu Yacc clone (yet another compiler compiler)
cvs 1.10 Gnu Concurrent Versions System
diffutils 2.7 Gnu File Comparison Tools
elm 2.5.2 Elm Electronic Mail Program
emacs 20.3 Gnu Emacs Text Editor
fileutils 3.16 Gnu File Utilities (ls, chmod, mv, rm, etc)
findutils 4.1 Gnu Find Utility
gawk 3.0.3 Gnu Awk
gcc 2.95.3 Gnu Compiler Collection (C, C++, Fortran 77, Objective C, CHILL, but not Java)
gdb 5.0 Gnu Debugger
grep 2.4.2 Gnu Grep
gzip 1.2.4 Gnu File Compresser
ispell 3.2.04 Gnu International Spell Checker
joe 2.8 Joe's Own Editor
less 332 Gnu Pager
libg++ 2.7.2 Gnu C++ Library
lynx 2.8.4 Text Based Web Browser
make 3.76.1 Gnu Make Program
msql 2.0.3 Hughes Technologies Mini SQL
mysql 3.23.39 mySQL SQL
ncurses 5.2 New Curses screen management package
nethack 3.3.0 Net-Created Dungeon Game
openssl 0.9.6b Secure Sockets Layer Toolkit
patch 2.5 Gnu Patch Program
perl 5.004_004 Perl Interpreter
pine 3.96 Pine Mailer
procmail 3.10 Mail Processor
screen 3.7.4 Gnu Screen Manager
sh-utils 1.16 Gnu Shell Utilities (date, echo, who, stty, etc)
ssh 1.2.20 Secure Shell Remote Login Program
tar 1.12 Gnu Tape Archiver
tcl 8.3.3 Tcl Programming language
tcsh 6.10.00 Extended C Shell
texinfo 3.9 Gnu Documentation System
textutils 1.22 Gnu Text Utilities (cat, head, wc, sum, etc)
top 3.5beta7 Top CPU Usage Display
vim 5.8 Improved VI editor
w3m 0.1.10 Text-based Web Browser
wget 1.5.3 Noninteractive Downloading Utility
zsh 3.0.5 Z shell

We have both the Gnu gcc compiler and the Sun C compiler, but most of our software is built with the Gnu compiler, which conforms with the ANSI standards and generally produces better code.

In a few cases we've found the Gnu versions of commands unsatisfactory. Their "df" command behaved oddly on unmounted disks, reporting partitions that didn't exist, so we pulled it out. Newer versions of their "who" command do DNS lookups on each IP address of each user. Since we have users from many different domains, and many are not resolvable, this is very slow, so we fell back to an older version of the Gnu "who" program.

Password Database

SunOS 4.1.4 uses a flat password file with encrypted passwords stored in the publicly readable file. This is totally inadequate for our needs. With over 28,000 lines in the password file, searching it is much too slow. Also, since untrusted users can easily get onto our system, and password cracking technology is pretty good these days, leaving encrypted passwords in a publicly readable password file would be a very large security problem. We have installed a much-modified version of the JFH password system to give us a hashed password database for quick look-ups and a protected shadow file to keep encrypted passwords in. For more information about this, see the Grex staff notes on the password database. To support this, we have rebuilt many programs with our own shadow library, and have installed our own versions of such programs like passwd, chfn, chsh, etc.

Queuing Telnet Daemon

The vast majority of Grex users connect into the system via telnet connections. To keep the system from being too slow to use, we limit the number of simultaneous telnet sessions that can be established. Originally this was done by limiting the number of psuedo-tty devices (ptys) installed, but this approach had many problems. Telnetting into the system became a lottery: when you tried to telnet in, you either succeeded or failed. If you failed, you tried again and kept trying again until you happened to hit it when there was a psuedo-tty available. Rejecting all these repeated attempts slowed down the system, especially as users started developing "attack telnet scripts" to telnet repeatedly until a connection succeeded. People could only get on by being lucky or persistant. Furthermore, we wanted to use the terminal server for dial-in users, but dial-in users effectively telnet in from the terminal server, and we didn't have any way to reserve connections for those dial-in users. Finally, when the system is full of users, there were no psuedo-ttys left available, so programs like "screen" could not be used.

Marcus Watts largely solved these problems with the development of his queuing telnet daemon. With this, users telnetting in from the terminal server always get a connection, but users telnetting in from anywhere else must pass through the queue. If the maximum allowed number of users is already on, then additional users are placed on a queue, where they see a count-down showing their progress through the queue. Actual telnet connections are given to users as space is available, in the order that they telnetted in.

The queuing telnet daemon solves most of our telnet problems. Users can get connections in a fair and predictable fashion, "attack telnet" programs are useless, and plenty of pseudo-ttys are left available for dial-in and "screen" users. Users on the queue do still cost some processing to maintain, so having a long queue can contribute to slowing down the system.

We are also running a queuing version of the rlogin daemon. Unfortunately, there was no good way to build queuing into the ssh daemon, but we have modified it so that it can't be used as a way around the queue. This means that it mostly can't be used at all when the system is full.

For more information about the queuing telnet daemon, contact Marcus Watts.

Idle Zapper

Because telnet connections are a scarce quantity on Grex, we run an idle daemon to automatically log out users who aren't doing anything on their connections. We use Michael Crider's freeware "idled" program, with a few small modifications and bug fixes. We have set it to warn idle users after 15 minutes and then log them out if they are still idle 5 minutes later. If a user logs into Grex more than once and Grex has more than a certain number of users logged in, idled sends warnings to their oldest connections, asking the users to please log out. Some programs (including some versions of "less") like to read and write from /dev/tty instead of from the user's actual tty device. This means that activity in those programs may not be noticed by "idled". We have fixed some of those programs on Grex. Valerie Mates has been the main person responsible for maintaining the idled on Grex.

Process Monitor

Since we primarily regulate load by limiting the number of simultaneous users, we try to prevent people from leaving processes running after they log off, because otherwise you can accumulate an awful lot of running processes even if you keep the number of users low.

To enforce this, and to generally keep a watch on the process table, we run a program called "robocop". This was originally written as a shell script named "kill_orphans" by Greg Cronau, and has evolved into a much faster and more sophisticated C program written by Jan Wolter.

"Robocop" regularly scans the list of users currently logged on and the process table, and then kills any process left behind by a user who has logged off. That includes all processes which fail all the following tests:

"Robocop" also does some simple tests to try to detect users using excess resources (mainly people running fork bombs and other denial of service attacks). If a user has too many processes, or the processes are using too much memory, it automatically kills all the user's processes. This function has been partly obsoleted by the kernel modifications which kill fork bombs more quickly than robocop does.

"Robocop" also cleans up "ftpd" processes that are more than 6 hours old. These are often left behind when people's ftp connections are broken, and we don't want them cluttering up the process table forever.

In all its tasks, "robocop" is designed to err on the side of not killing processes that might be OK, so it is not by any means impossible to fool. However, Grex staff monitor things pretty closely, so what "robocop" misses is usually cleaned up by a human pretty fast.

Note also that "robocop" is very tuned toward Grex and SunOS 4.1. Porting it to another system would be non-trivial.

Hierarchical Home Directories

On most Unix systems, all users are given home directories under a single parent directory. Typically, the home directory for user "bubbles" would be someplace like "/home/bubbles." But Grex has over 28,000 users, so our "/home" directory would have more than 28,000 entries in it. Any time anyone accesses their home directory by its full path, the operating system would have to search the 28,000 entry parent directory. On average, it would have to look through 14,000 directory entries. This is seriously slow.

We have restructured our home directories into a two level hierarchy. Instead of the directory for "bubbles" being "/a/bubbles" it is now "/a/b/u/bubbles", where "b" and "u" are the first and second letters of the user's login name respectively. This way the operating system has to search the top level directory (with 26 entries), the second-level directory (another 26 directories) and the the third-level directory (with an average of 28,000/(26*26) = 41 directories). On average it should have to look at about 47 directory entries (that is 26/2 + 26/2 + 41/2), which is a big improvement on 14,000. For further optimization, we carefully created the intermediate directories for the most common letters first, so that the average search times are actually even lower than the estimate above.

Some users' accounts are now being created on the new /c disk instead of /a. These users have home directories like /c/b/u/bubbles instead.

Hierarchical Mail Directories

Unix normally stores the unread mail for all users in a single directory, such as /var/spool/mail. On Grex roughly two-thirds of our users have mail files in that directory, so this was another huge directory that needs to be searched every time a piece of mail was delivered or a mail reader was started. So, as another trick to speed up the system, we set up the same two-level directory hierarchy on the mail spool directory as we set up for users' home directories.

Making this work required modifications to our "mail.local" program, to make it deliver to the correct place ("mail.local" is the program that "sendmail" invokes to do local mail delivery). The configuration file /etc/mailfmt tells where mail should be delivered by "mail.local".

Modifications were also made to the "login" program to correctly set the MAIL environment variable to the location of the user's mail file, and to all the mail reading programs that did not respect the MAIL environment variable (notably "pine").

Mailbox Directory Permissions

On standard SunOS systems, the /var/spool/mail directory is permitted 2777, which allows anyone to create or delete their own mailboxes, but not to delete other people's mailboxes. Unfortunately, it's also possible for vandals to create someone else's mailbox, which gives you access to their mail. So when we switched to heirarchical mail directories we also changed the directory permissions to 755 - so people can't create or delete their own mailboxes, though they can empty them. Mailboxes are created by "mail.local" and we don't delete them until the account is deleted. Changes were also made to the kind of locking used when reading or writing mailboxes (flock is now used).

This was a problem for some mail programs. Sun's "Mail" mail reader bombs if it can't delete mail files. Since we didn't have source to it, Marcus Watts extended "mailx" to give it the same features as "Mail" and we now use that instead. If you run "pine" before you have ever received mail, it complains that you haven't got a mail box and that it can't create one. This is harmless, but the message confuses some users. Someday we need to kill this error message, but figuring out "pine" source is amazingly painful. We have modified the error message to be less alarming.

Mailbox Size Quotas

Grex has had problems with users who take out an account, sign up for high traffic mailing lists, and then never log in again. Huge quantities of mail pile up, jamming our net connection as it comes in and filling our mail spool partition once it is here. Once the mail partition is full, nobody can receive mail. Similar problems are caused by people mailing in huge files, often gif images or executable programs.

To limit these problems, Grex has set limits on mail. Messages that are longer than 600K are refused so that they bounce back to the sender. If a user's mailbox grows larger than one megabyte, then we bounce all new messages for that user in a similar manner.

This was implemented by Marcus Watts. The "mail.local" program which does local mail delivery detects when a mailbox is too full, and adds that user's login name to the file /var/adm/badmail. "Sendmail" was taught to reject all mail to people who are listed in that file. A new program called "umailck" deletes users from the badmail file if they are no longer over quota. It is run hourly by "cron". "Login" warns users if they are near or over quota, and runs "umailck" on them if they are in the /var/adm/badmail file.

Marcus's original idea was that we would reject mail sent to accounts which were overquota with a "temporary" error. This tells the machine sending the mail to Grex to try again later, by which time the user might have emptied out the mailbox enough so that the message could be delivered. This was nice in theory because this way no mail would be lost. In practice we discovered that some mail hosts try again immediately so Grex would end up rejecting the same mail message many times a minute. We couldn't afford this, so we changed it to reject over-quota mail with a "permanent" error, so it would be bounced immediately to the sender.

Spam Filters

"Spammers" are people who send huge amounts of unsolicited E-mail to tens of thousands of people, usually advertising various forms of worthless garbage. Spamming is an attractive way to advertise because you can reach a lot of people very cheaply. However, most people don't like receiving this kind of mail, so in the early days it was common for people who had received unwanted spam to apply poetic justice by sending back thousands of E-mail messages in reply, thus flooding the spammer's systems with junk E-mail.

Because of this, modern spammers try very hard to send their E-mail in such a way that nobody can reply to it, or otherwise easily trace it back to them. They use forged return addresses, so that replies go to the wrong place, or they relay their messages through machines belonging to other people, thus using some unknowing person's machine to send out their spam.

We at Grex, like many other people on the net, are philosophically opposed to people doing nefarious things so they can send E-mail without being held accountable for it, and we are even more strongly opposed to letting people use our already overburdened machine for this purpose. Like many other systems on the net, we've been steadily instituting new measures to try to block spam mail while still passing through legitimate mail. Meanwhile, of course, spammers have been steadily inventing new ways around the new defenses.

Recent versions of "sendmail " contain many new features designed to limit spam. Unfortunately, we've made so many modifications to our "sendmail" for various other purposes that upgrading to the newest version is rather difficult. So Grex staff member Marcus Watts has been waging his own war against spam, monitoring the spam that passes through the system, building defenses against common spammer tricks into our version of "sendmail", and blocking all mail from sites that regularly seem to be originating spam.

Un-POP Server

Grex doesn't run a POP server. Providing mail service to the universe is not our primary goal, even if it is much of what we end up doing. We want people to get involved in our virtual community, so we want them actually get on the system to get their mail.

But lots of people try accessing their mail via a POP client and then send us mail asking why it doesn't work. So we set up a fake POP server that simply rejects all POP connections with an error message that explains that Grex does not support POP. This little program was written for Grex by Jared Mauch.

Disk Space Quotas

Grex policy sets a limit on 1 megabyte of disk space per user. SunOS 4.1.4 contains software to enforce such quotas, but we do not use it here. It would slow down the system considerably to have to be constantly checking quotas. Instead we police disk usage manually.

Application Software

Newuser Program

The "newuser" program creates new accounts for people. Grex runs two versions of this. The one that you run by telnetting or dialing Grex and typing "newuser" at the login prompt was originally written by Marcus Watts for M-Net in 1983, and has been slowly and steadily evolving ever since. The web-based one that you run by pointing your browser at http://www.grex.org/nu/newuser.html is a new interface to Marcus's same old account creation code, that was developed by Jan Wolter. More than two hundred people run one or the other newuser program on Grex every day.

PicoSpan Conferencing System

Grex's main reason for existing is to host a virtual community, based primarily on the PicoSpan conferencing system. We have over 100 active conferences on Grex, with a wide range of subject areas. The PicoSpan program, which is accessed by the Unix "bbs" command, allows user to join any of these conferences (except the "staff" conference which is restricted to Grex's staff members and used only for technical discussions and coordination of efforts among staff). Users can read, post new responses to existing discussion items, or start new items. It keeps track of what you have read in files saved in your home directory and called "participation files", so you only see new material when you come by again.

PicoSpan was written by grex staff member Marcus Watts around 1983. It is the same conferencing system made famous on "The Well".

Backtalk Conferencing System

More recently Grex staff members Jan Wolter and Steve Weiss have developed the web-based Backtalk conferencing system. On Grex, Backtalk is set up to access the same conferences as PicoSpan and use the same participation files, so that it doesn't really matter which interface you use, or if you mix and match them. Backtalk is available free to other systems.

Users log into Backtalk using their usual Grex logins and passwords. This is a bit tricky, because neither Backtalk nor the httpd run as root and the encrypted passwords are stored in the shadow file which is readable only by root. To get around this, we have the http daemon run an external authentication module called "pwauth". Since this does run as root, it can check passwords. This authentication system is available for free.

Party Program

Grex's party program is real-time chat program where you can hold live conversations with whatever other users are currently running it. It's vaguely like IRC, except it only has Grex users on it and there are no "channel operators". Party was originally written by Marcus Watts around 1983. Jan Wolter started modifying it around 1985, and eventually completely rewrote it, adding many features.

Orville Write Program

Orville Write is a much extended version of the Unix write program written by Jan Wolter. Among other things, it supports the sending of "telegrams" between users, and allows users to designate themselves as "helpers". Doing "write help" connects you to any available user who has designated themselves as a "helper."

Vote Program

Grex is run democratically, with members electing board members, and voting on referenda. Since we run lots of elections, we need a vote program. Ours was written by Grex staff member John Remmers. It has both a command-line interface and a web interface.

Menu Shell

Since Unix commands are often difficult for new users to learn, Dave Parks developed a menu system for M-Net long ago. This system is designed to give users an easy interface to commonly used Unix commands, and also to teach the users what those commands are, so that one day they will be able to switch from menu shell to a more standard Unix shell. Grex's menu shell was a version of Dave Parks's menu shell, installed on Grex by Dave Parks and modified by Valerie Mates in the years since.

The whole thing was implemented as one giant sh script. We weren't entirely happy with the maintainability and efficiency of this implementation.

Michelangelo Giansiracusa re-wrote the shell scripts into a C program template design, which means configuring menu templates is easy and can be done by some menu-admin person who is programming-illiterate. Some new features, back-end scripts, and improved help messages were also added to the newmenu system in this project undertaken by Mic. A template format debugger program was also written by Mic, which may be more helpful to Grex users if we someday decide they can construct their own customizable menu template files.

Most of our users are not any kind of Unix gurus, and being able to offer an interface that is more approachable for novice users is important.

Lynx Shell

Grex staffers Steve Weiss, Rob Henderson, and Carl Miller have also developed a shell based on the lynx that gives a more hypertext-flavored menu interface. This is a good concept, but Steve says it probably needs more work.

Help System

When you type "help" at a shell prompt on Grex, this runs Grex's help system. The help system is an archive of various files of Grex documentation. You can also go from the help system into either of Grex's menu systems, or you can request a chat with a volunteer helper. Valerie Mates wrote and maintains the help system; it is a shell script.

The Change Program

When you type change at a shell prompt, or select "Change password, etc" from a menu, this runs Grex's Change program. This program is a user friendly collection of shell scripts, awk programs, Perl programs, and C code that users can use to change settings on their accounts without needing to memorize cryptic acronyms like chfn (change full name), or needing to know that if you switch to a shell that uses different startup files than your current shell then you must also create the startup files for your new shell or your account will be broken. The change program was written and is maintained by Valerie Mates.

Document History:

Apr 4, 1998: Jan Wolter (janc) - Initial revision.
Apr 10, 1998: Valerie Mates (valerie) - Added sections about the help and change programs; split the section about the menu programs into two sections and expanded each section; many minor corrections and tweaks.
Jun 19, 1998: Jan Wolter (janc) - Minor updates. Deleted "DRAFT" disclaimor.
Feb 4, 1999: Jan Wolter (janc) - Replace section about "kill_orphans" with section about "robocop".
Feb 11, 1999: Jan Wolter (janc) - Update description of 4/670 to reflect donated parts.
Feb 18, 1999: Jan Wolter (janc) - Added short section on kernel blocks.
Mar 25, 1999: Jan Wolter (janc) - Updated robocop info, added Orville Write info.
Sep 19, 1999: Jan Wolter (janc) - Added info about fork killer kernel modifiations.
Oct 1, 1999: Jan Wolter (janc) - Expanded and corrected info on mail directories and quotas.
Jul 7, 2000: Michelangelo Giansiracusa (spooked) - Added note on re-design of menu system.
May 28, 2001: Jan Wolter (janc) - General update of hardware information.
Jul 16, 2001: Jan Wolter (janc) - More updates of hardware information.
Jul 20, 2008: Dan Cross (cross) - Convert to XHTML and deprecate.