PersonalINN.Manager contains a schematic flow diagram of PersonalINN for a better overview of its components. To open it in a window, choose Flow Diagram from the Info menu.
Below you'll find a detailed commentary on the flow diagram to the following topics:
Conventions in the flow diagramFor more detailed information about a particular program or configuration file of PersonalINN, consult the corresponding UNIX manual pages.
All binaries are located in /usr/local/inn/bin. Exceptions are PersonalINN.Manager (installed in an application directory), ip-up and ip-down (not belonging to PersonalINN and installed in /etc/ppp), Alexandra (not belonging to PersonalINN and representing the newsreader of the user's choice) and all binaries belonging to NEXTSTEP (as marked by the underlying gray area on the bottom of the diagram with the NeXT logo on the right).
All text files are located in /usr/local/inn/etc. Exceptions are the files inside a directory (as marked by a gray outline), the files with an explicit path (which is relative to /usr/local/inn), and all files belonging to NEXTSTEP (as marked by the underlying gray area on the bottom of the diagram with the NeXT logo on the right).
All directories are located in /usr/spool/news. The only exception is the run directory, which is pictured as a gear in the diagram and is located in /usr/local/inn).
The types of connection between the files are indicated by different strokes:
Note that for clarity unimportant connections are not shown.
When you use PersonalINN.Manager to configure PersonalINN, it makes entries in some of PersonalINN's configuration files and NEXTSTEP's system files. These connections are not shown, either. The files are:
newsfeedsA lot of basic configuration settings to adapt INN to the operating system on which it is running are hardcoded into the binary programs; for use in the sh and perl scripts, these settings are also listed in innshellvars and innshellvars.pl, respectively. Further basic configuration values that innd reads only at start-up are contained in inn.conf; innshellvars and innshellvars.pl import these values by running innconfval.
When the computer is booted, rc.news will be run (as user news) due to an entry in rc.local. rc.news checks the integrity of the news system (and will send mail to the newsmaster if it detects something wrong), performs clean-ups if necessary, and starts inndstart. inndstart is run su root to be able to prepare the environment for running innd (open ports, sockets etc.); then it starts innd (as user news).
innd is PersonalINN's central program, which runs all the time. For security reasons, it can be contacted only through sockets which reside in the run directory. The run directory is only accessible by members of the group news and therefore acts as a firewall for innd. Besides the sockets, the run directory contains several PID and lock files to assure the integrity of the news system. PersonalINN shell scripts that need to write lock files into the run directory do so using the shlock program.
When a TCP/IP connection to the Internet is brought up via PPP, the ip-up script is executed, which starts newsxstart. As long as it is not blocked by a BLOCK.autostart file in the run directory (which is written by PersonalINN.Manager to block the start of news.daily and newsxstart), newsxstart starts newsx periodically according to the settings of PersonalINN.Manager, which are stored in pinn.conf. If the TCP/IP connection is closed, newsxstart will be killed by ip-down. As long as the TCP/IP connection is up, newsx can also be started manually by PersonalINN.Manager.
newsx locks itself in the run directory and connects to the NNTP news server specified in newsfeeds, pretending to be a newsreader. If a password was specified during configuration, password.provider will exist, and newsx will use the password listed there for the connection. newsx fetches all groups listed in active (apart from control, junk and newsgroups with the top-level hierarchy for local newsgroups that is stored in newsfeeds). For each group, newsx writes the highest article number of the articles it has fetched into incoming/provider.name (with provider.name the news server specified in newsfeeds) and will begin with the first article not yet fetched the next time it is started. newsx also searches the local history database (consisting of history, history.hash, history.dir and history.index) for message IDs of articles already stored locally. If it does find a message ID of an article which is offered to it as new now, it will not fetch the article a second time to avoid duplicates. (Note that this will always be the case for locally written articles that were posted to the specified news server by newsx before, because the news server will always list these articles as new articles, too.)
newsx inserts newsx in the Path: header of the articles it fetches. Then it pipes the articles it has fetched into rnews which converts the data so that they look to innd as if they were directly fed by another news server; then, it sends them to innd. If innd is paused or throttled at this time, newsx and rnews will wait until innd is no longer paused or throttled. If innd is not running at this moment, rnews saves the articles in one or more batch files instead (each of 1 MB maximum size) that it writes into the articles directory (but not into the subdirectories of articles). The next time the news.daily maintenance routine is running, rnews will be called again, look for batch files still stored in articles, and, if present, feed them to innd. If the header of an article is corrupted or bad, rnews will store the article in the tmp directory, where it will stay for a few days.
innd adds the Xref: header to the articles it receives, stores them in the corresponding subdirectories (each newsgroup hierarchy corresponds to one subdirectory) of the articles directory in text files with continuous article numbers as file names (this is the so-called traditional spool method; INN 2.2 also provides alternative methods that target very large news servers), stores the message IDs, the arrival time of the articles and the paths of the corresponding article files in the history database and saves the highest of the article numbers of a newsgroup in active. As controlled by the entries in newsfeeds, it also sends the articles to overchan which stores the articles' header data in the overview database for faster access (using the data format described in overview.fmt). Since newsfeeds will not send to filechan any article which has a newsx entry in its Path: header (which newsx inserts in all articles it has fetched), none of the articles received by newsx will go there.
In contrast, articles that are written locally (see below) will also be sent to filechan when innd receives them. filechan writes the local paths of the corresponding article files (and the file sizes) into outgoing/provider.name (apart from articles from control, junk and groups with the top-level hierarchy for local newsgroups that is stored in newsfeeds). Thus, all locally written articles (respectively the path names of the corresponding article files) that are intended to be sent to the Internet will be listed in outgoing/provider.name.
newsx reads outgoing/provider.name and posts the listed articles to the news server specified in newsfeeds, again pretending to be a newsreader. After articles have been posted successfully, newsx will delete them from outgoing/provider.name.
When you start a newsreader application (such as Alexandra) on your system and open a connection to PersonalINN, innd will spawn an instance of nnrpd which handles all further requests of this newsreader. This delegation is configured by the files incoming.conf (lists all hosts that are intended to be news servers, which means they need to connect to innd rather than nnrpd) and nnrp.access (sets permissions and restrictions for newsreader access). In PersonalINN, incoming.conf lists no hosts, so that all connections are intended to be newsreader connections to nnrpd (innd in PersonalINN is fed by newsx rather than directly by another news server), while nnrp.access allows general access for newsreaders (since PersonalINN won't typically be accessible from the Internet).
nnrpd learns about available newsgroups and the number of articles in them from active, about the available articles in the newsgroups (as determined by their message ID) from the history database, and about articles headers (subject, author etc.) from the overview database. On request (via NNTP), it provides the newsreader with all this information. When the newsreader requests an article by message ID, nnrpd queries the history database for the path of the article file that corresponds to the article with the requested message ID, opens this file and sends its content to the newsreader; when the newsreader requests an article by number, nnrpd can open the file and send its content immediately.
When the newsreader posts an article, nnrpd generates a message ID and some other header lines for it and sends it to innd the same way that rnews does (see above). innd adds the Xref: header (which nnrpd can't set because it doesn't know about the article number that innd will provide), stores the article in the corresponding subdirectory of the articles directory, updates active and the history database and stores the article's header in the overview database (see above). Since the article is locally written and therefore does not contain a newsx entry in its Path: header, innd (unless the article is posted to control, junk or a group with the top-level hierarchy for local newsgroups that is stored in newsfeeds) also sends the article to filechan which lists it in outgoing/provider.name (see above), so that newsx will post it to the Internet.
If nnrpd learns from active that the newsgroup to which the article is posted is moderated, it emails the article to newsgroup@moderators.isc.org instead, from where it is forwarded to the moderator of the newsgroup.
In PersonalINN, PersonalINN.Manager is the central program for administering and maintaining the system. It stores its own configuration data in pinn.conf and locks itself with the lock file LOCK.pinn in the run directory.
To fetch a list of newsgroups available from the NNTP news server specified in newsfeeds, PersonalINN.Manager uses getlist, which fetches the active and newsgroups files (which contain the list of available groups and descriptions of these groups, respectively) from this news server. While newsgroups is stored `as is' locally, PersonalINN.Manager merges the content of newsgroups and active to feed and feed.new, which contain the names and descriptions of all available and of all new newsgroups, respectively, in a format suitable for PersonalINN.Manager.
To add newsgroups to and remove newsgroups from PersonalINN, PersonalINN.Manager edits active accordingly and reloads it into innd afterwards.
To change expiration periods, PersonalINN.Manager edits expire.ctl.
When local newsgroups are created, PersonalINN.Manager stores them in local in the same format as feed and feed.new and adds them to active; the top-level hierarchy for local newsgroups is written to newsfeeds.
To change the Organization: header, PersonalINN.Manager edits inn.conf. innd must be restarted after this for the new setting to become valid.
For checking and repairing PersonalINN, PersonalINN.Manager uses the tool programs inncheck, makehistory and makeactive.
For the files that PersonalINN.Manager edits when configuring PersonalINN, see above.
ctlinnd is the classic command line interface to INN and INN's equivalent to PersonalINN's PersonalINN.Manager. It is also used by several scripts that need to communicate with innd; news.daily, newsx and scanlogs use ctlinnd to pause innd when they need to do so.
Log files are written into the log directory. While some programs write directly into the log directory, innd, nnrpd, rnews and newsx use syslogd according to the entries in syslog.conf (facilities local0 and local1). nnrpd reads pinn.conf to decide whether it should log the host names or the IP addresses of the hosts that connect to it . (This connection is not shown in the flow diagram.)
Email that PersonalINN's programs want to send to the newsmaster(s) is sent via innmail; the real email addresses for newsmaster are stored in the aliases directory of NEXTSTEP's NetInfo database.
When rc.news starts up innd, it also starts innwatch which, as set by the entries in innwatch.ctl, checks PersonalINN every 10 minutes. It inspects the space remaining on the hard disk(s) where PersonalINN is installed (using inndf), looks for new critical error messages in the log file news.crit, and tests if innd is still running (using ctlinnd). innwatch locks itself in the run directory with the files LOCK.innwatch and innwatch.pid; it also saves the last checking time in innwatch.time so that it is can report only errors since the last checking. When innwatch discovers an error, it immediately sends an email to the newsmaster(s), and throttles innd via ctlinnd if necessary because of too little hard disk space (this should not happen with PersonalINN fetching articles, though, since newsx stops fetching articles if free space drops below 20 MB, and innd will be throttled when less than 18 MB are available); if space becomes sufficient again, innwatch can make innd resume, too.
Once a day, according to the setting in crontab.local, cron starts news.daily. If blocked by a BLOCK.autostart file in the run directory (which is written by PersonalINN.Manager to block the start of news.daily and newsxstart), news.daily exits immediately; otherwise, it locks itself in the run directory, cleans up the tmp directory and removes stale files from outgoing and incoming; finally, it starts rnews to feed any remaining batch files (see above) to innd.
news.daily calls innstat for a status report of PersonalINN. innstat queries ctlinnd for the status of innd, uses inndf to report the space remaining on the disk(s) where PersonalINN is installed, and checks for the existence and size of files in the incoming, outgoing and log directories, as well as for the existence of lock files in the run directory.
After that, news.daily starts scanlogs. scanlogs locks itself in the run directory, summarizes important messages from the log files using innreport (which is configured by innreport.conf and innreport_inn.pm), and then rotates the content of the log files and of the active file into the backup files in the OLD subdirectory of the log directory.
Next, news.daily starts expire, which checks the history database for articles that are older than expire.ctl permits for the respective newsgroup. Using a temporary file called expire.rm, expire pipes the paths of these articles to expireover, which deletes the respective entries in the overview database, and to expirerm, which checks for errors and then uses fastrm to actually remove the article files from the articles directory. If everything went fine, expire.rm will be renamed expire.list and stored in the log directory. expire itself deletes the respective entries in the history database; however, it will keep the entries for 30 days longer than the article files themselves to allow newsx to prevent articles from being fetched twice after they have expired.
After the expiration, news.daily renumbers active via ctlinnd and calls innstat a second time to provide an after-expiration status report.
Finally, news.daily will email the result of the expiration and the reports from innstat and innreport to the newsmaster(s) (if configured to do so in the crontab.local entry). The time at which news.daily finishes is saved in .news.daily; this allows rc.news to check for proper expiration when PersonalINN is started.