Support@MWat.DeQuite a lot of small LAN installations (e.g. at lawyers, physicians, SOHOs, travel agencies etc.) I've seen over the years where the users don't have an internal accesspoint for their emails incoming from the Internet; almost always a consequence of their (on demand) dialup internet connection. While there might exist an internal mailserver (for intranet mail), their emails from "outside" (i.e. somewhere in the Internet) end up somewhere else (their ISP, a freemail provider). To actually read their mails each of them has to setup her/his own email program to pick up the mails from whereever they happen to be, which in turn means, that the firewall / Internet gateway must be set up to allow POP/IMAP traffic from all intranet hosts to the outside world – which might turn out as a potential security risk at least as long as there are still some WinDOS hosts around.
A better way for both the users and the LAN admin would be, if the mail fetching could be centralized on one machine (possibly the gateway system). All users would pick up their mails at a host within the LAN. And the firewall could be set up in a more restrictive way. Although this problem seemed quite common to me I couldn't find a ready-to-use solution on the net (newsgroups, web-pages). So I came up with my own. – But please note that we're talking here on about the receiving part of email communications. This script has nothing to with the sending of emails; this is completely up to the LANs postmaster to setup so I won't discuss that here any more.
[up to Table of Contents]
From the users point of view the cmf script is just a little web-interface which allows them to enter, change or delete a mail fetching job. "Mail fetching" means, that "someone" fetches their incoming mails from their ISP (or freemailer or ...) and forwards them to a mailserver within their own LAN. Here they can read the mail locally with their preferred email-program.
The visual appearance of the scripts web-interface (i.e. the HTML pages and forms) is completely configurable: Both the design and the wording can be customized by the administrator, the latter be editing – or adding – a language file holding all text/strings used by the pages, the former by editing a CSS files to be used by the browser when rendering the pages.
Internally the cmf script uses a private database where it stores all
the user-related data. Every time this database is changed by use of the
web-interface the script checks whether there are active jobs. If this is the case
the relevant database fields are converted to an other file called
.fetchmailrc, the latter being a configuration file for the
fetchmail utility by Eric S. Raymond, and a cronjob is set up to run
that tool to look for new emails on a regular base and forwarding them to the
respective intranet user accounts.
What I called a "private database" is in fact just a textfile which is read from and written to by help of the systems awk utility; so there's no need to install additional software to run this script. And since the users are changing their settings by means of a Web-form there (on the users machine) as well isn't any additional software needed apart from a web-browser (which may be a graphical one or a text-based). Actually, one of my goals was to keep the requirements as low as possible. – The flow of data would look like this:
User "visible" "internal" processing
---------------- -----------------------
user (browser) ---> cmf (intranet-webserver) <--> database file
^ ^ | |
| | | | + <--> ISP-1
| +--- web-form ---+ + ---> .fetchmailrc | <--> ISP-2
| | | ...
+--> LAN mailhost <--- fetchmail (crond) <--> + <--> ISP-n
So the user has to setup – once – her mail fetching using this scripts web-interface and can from then on read her incoming emails at the "LAN mailhost". The latter may be a POP/IMAP server within the LAN or just the host where the she happens to have her local account. It would be even possible to skip the users step of registering her mail fetching by delegating that job to the LANs administrator.
[up to Table of Contents]
There are several issues to deal with when implementing and running this script, most of which related to privacy/security in one way or another:
.fetchmailrc file must have mode 0600, a security
constraint by the fetchmail utility, which is enforced by this script
for all other files (see the Files section below) as well./bin:/sbin:/usr/bin:/usr/sbin prior to calling any other program.To address these points I came up with a solution internally using two scripts, a
small CGI stub (which is called by the users web-browser) and this script which is
called by that CGI script using the systems sudo facility (see section
Installation / Setup below). A separate account with no special
privileges and no login shell at all is used as the owner of all files involved (see
section Files below). Also on behalf of this account the
fetchmail utility is run.
The database file used internally does not require any database management system (DBMS) to be installed. Generally it's designed as a flat ASCII file consisting of several lines each of which representing a data record holding a number of data fields. The GNU/Linux awk utility is used to read/write that file. – Surely such a solution would become kind of slow if there were tens of thousands of data records to handle. But a company or organization using this script for that many users should reconsider its point of view: forwarding mail in such a setup is not really a good idea. A central mail-server (a combination of SMTP and POP/IMAP servers) would be a better solution for all involved (users, admins and programs/scripts). This script, in contrast, may be used in LAN environments with some tens of users.
[up to Table of Contents]
The web-interface (i.e. the web-pages sent to the user) consists of only some two or three forms. The first form requests the user to enter three values:
The user input is then looked up by the script in its database. It's also checked whether an entry already exists that should be added or does not exist although the user selected to change it. The user then gets a second form (possibly with some err-texts pointing out missing or invalid fields) with three additional fields:
fetchmail utility;fetchmail for use.Below that is shown for informational purposes when and from what host the
settings were last changed. Finally selecting the update button causes
the script to update its database accordingly and – if necessary –
rewrite the .fechmailrc as well and start or stop the cronjob which
periodically runs the fetchmail utiltity. – The user finally
receives a good-bye page listing her current settings.
That's it! Everthing else is done automatically by cron and the
fetchmail utility.
[up to Table of Contents]
This script is not a "standalone" solution doing all and everthing but
kind of fetchmail addon providing a web-interface for configuring a
LAN-wide multiuser mail fetching and forwarding facility as discussed above. As such
besides the fetchmail program a web-server (such as Apache)
or -proxy (such as WWWoffle) is needed, configured to allow for delivery
of CGI generated pages. While not required by this script as such, the localhost
(i.e. the host where the script is supposed to run) needs an active SMTP server (such
as PostFix) capable to handle all the mails picked up and forwarded by
fetchmail. To sum up, you'll need a standard GNU/Linux installation
(such as Debian, Fedora or Redhat)
including
Presuming that your machine meets this requirements, to successfully run this
script perform (as user root) the following steps:
/bin/false and no valid password. Its only purpose is
to provide a UID/GID for use by the fetchmail utility and this very
script. – An easy way to create such a new account:
#
# useradd -s /bin/false cmf-user
# _
where cmf-user is the name you'd like to use; for more info
refer to man useradd.
#
# cd ~cmf-user/
# mv /wherever/you/downloaded/the/script/cmf.gz .
# gzip -d cmf.gz
# chown cmf-user:cmf-user cmf
# chmod 0700 cmf
# _
This step is quite important since the script uses its own directory to store its
files and makes the owner of that directory the owner of those files. This is necessary
not only for security reasons but also in order to run fetchmail
successfully on behalf of that account.
#
# cmf --test
# _
The output of the --test option should look like this:
+++ "awk" found as "/bin/awk"
+++ "cat" found as "/bin/cat"
+++ "chmod" found as "/bin/chmod"
+++ "chown" found as "/bin/chown"
+++ "crontab" found as "/usr/bin/crontab"
+++ "date" found as "/bin/date"
+++ "fetchmail" found as "/usr/bin/fetchmail"
+++ "hostname" found as "/bin/hostname"
+++ "ls" found as "/bin/ls"
+++ "mv" found as "/bin/mv"
+++ "pwd" found as "/bin/pwd"
+++ "ps" found as "/bin/ps"
+++ "rm" found as "/bin/rm"
+++ "sed" found as "/bin/sed"
+++ "sudo" found as "/usr/bin/sudo"
+++ "cmf" found as "/home/cmf-user/cmf"
+++ "/home/cmf-user/.cmf.conf" created
+++ "/home/cmf-user/.cmf.lang.de" created
+++ "/home/cmf-user/.cmf.lang.en" created
+++ no user crontab installed for "cmf-user"
+++ file permissions/ownership OK
If any needed files are missing, check what happened to them and install (see
also the Files section below). As far I'm aware of all of them should
be installed by default, except perhaps the sudo utility which might need
installing it seperately. Also, if you see a message like
--- there's already a user crontab installed for "cmf-user"!
you know that you're running into trouble because we need this user
completely for our mail fetching job (see also the last point, 7, below). –
Repeat this step as long as there are any lines with leading "---" (instead
of "+++") in the output.
cmf.conf file and check, whether all settings there
seem reasonable to you. – Note, that the RC_URILOCAL entry will
be automatically updated by the following step, so don't worry about it
yet.
#
# cmf --cgi /path/to/httpd/cgi-bin /cgi-bin
# _
which creates the starter CGI script /path/to/httpd/cgi-bin/cmf.cgi
(see also section Commandline Options below) and should give you a line
like:
+++ updated "/home/cmf-user/.cmf.conf"
/etc/sudoers
file (allowing the web-server/proxy user to call this script) like this one:
wwwoffle ALL= NOPASSWD: /home/cmf-user/cmf
You might create such an entry be either using the visudo utility
(which is part of the sudo package) or by calling this script:
#
# cmf --sudo httpd-account
# _
where httpd-account is the user account used by your web-server
(wwwoffle in the example entry above). – For more information about
sudo see man sudo.
fetchmail utility periodically for actually fetching the mails. But
don't worry: the whole magic of maintaining the users crontab is dealt with by
this script. It dynamically removes the users crontab (if there are no active users
to fetch mail for) and installs it again whenever needed (e.g. a user becomes added
or switches her/his setting from disabled to enabled). – You can see the
default cronjob definition in the ~/.cmf.config file, entry
RC_CRON. – Please note, that it is possible, but not
recommended, to use an already existing user account to run this script. If,
however, that user happens to have its own cronjobs you'll run into problems:
As mentioned above the crontab is dynamically enabled/disabled according to the
current user settings. In consequence all other cronjobs of this user are affected
(i.e. deleted) as well.Now you should be able to test the setup by pointing your favourite web-browser
(such as e.g. Lynx, Mozilla or Opera) to an
address like http://www.your.webserver/cgi-bin/cmf.cgi. This should
result in a web-form to show up in your browsers window requesting your
authentification/login. Otherwise – i.e. in case of errors – you should
check the web-servers logfiles for hints what might have gone wrong. You could try a
su -l httpd-user at the commandline prompt and call the CGI stub
/path/to/httpd/cgi-bin/cmf.cgi manually to see whether you get a HTML
page this way. If that results in an error you probably missed the 6. step
(sudo) above.
For information about how to setup fetchmail and your favourite
web-server/-proxy please consult their respective documentation and man
pages.
[up to Table of Contents]
Usually – that is in CGI mode – there are no commandline
options to worry about: all data to process is streamed via stdin to
this script (for details refer to your web-servers CGI documentation). There are,
however, some options to print out some information about this script and how to use
it as well as two used for initial setup. Please note that
intentionally there's no loop for commandline parsing, so you can always use only
one option per call. They are in alphabetically order:
httpd used you may or may not have to adjust the
ownership of the created CGI script in order to have it run by the web-server.
Consult your web-server manual for possible constraints when running CGI scripts.
– The localpart refers to the URI the users are supposed to use:
so if the full URI would look like e.g.
http://your.proxy.domain:8080/cgi-bin/cmf.cgi then the
localpart would be /cgi-bin, while the
cmf.cgi will be appended automatically. – Please
note that no attempt is made to check whether the
directory and localpart arguments match since that would
involve parsing your web-server or proxy configuration files as well, so just use
your administrative knowledge to make sure, they're fitting. Please note as
well, that this option modifys the scripts config file ~/.cmf.conf in
order to store the localpart which is used/needed to generate all the
forms the users get presented when they access this script./etc/sudoers file allowing the
given account – which must be the one running the web-server – to call
this script.[up to Table of Contents]
Since this script might be running on an exposed host – the internet gateway/firewall – it requires no additional software to be installed apart from the usual system tools, neither JAVA, PERL or PHP etc. are needed, nor any database servers like MySQL; everything is done by GNU/Linux tools available with every recent distro I'm aware of. In detail the following utilities are used by this script and are assumed to be reachable in PATH:
awk, bash, cat, chmod, chown, crontab, date,
hostname, mv, pwd, ps, rm, sed, sudo
Although not needed by this script directly, Eric S. Raymond's
fetchmail utility has to be installed on the system (see the original
fetchmail homepage for details). To
check whether all tools are available use the --test switch (see section
Commandline Options above)
[up to Table of Contents]
Apart from the system tools listed above the following files are used, all of which are stored in the selected unprivileged users home directory and owned by that user with no access rights for either group or others. Some of this files are used [i]nternally by this script, some are [o]ptional, some [m]andatory; all this files are to be owned and read-/writeable by the user you created for this script (see section Installation / Setup above):
~/.fetchmailrc [i]fetchmail utility. This file is
automatically (over-)written whenever a user changes (adds/updates) her/his data,
so do not modify!~/cmf [m]~/.cmf.conf [o]fetchmail utility). – If you happen to
change the file ownership or permissions, or if this file is removed altogether
it will be re-created with the default settings thus reverting all changes you
might have done, including the --cgi setting you made during setup!
So be carefull ;-)fetchmail utility (see
man crontab for details).domain" option; it defaults to the domain of the local host
(i.e. the one running this script).hosts" option i.e. the host where the fetched mail is
forwarded to; it defaults to the name of the local host (i.e. the one running
this script). fetchmail allows a comma delimited list of hosts
here, see man fetchmail for details.filename" option. – Note that you must
leave this variable empty or unset (by commenting out the respective line) if
you want to use I/O-redirection with the fetchmail cronjob.name" option; it defaults to postmaster, which
must be a working alias on your local host (i.e. the one running this script);
see man fetchmail for more information.fetchmail related settings are written to its config file
~/.fetchmailrc which is automatically generated/maintained by this
script.
~/.cmf.css [o]~/.cmf.css.default [i]~/.cmf.css.default is created (but not
used otherwise) to show you the default CSS settings used for all HTML pages. In
case you don't like these you may create the file ~/.cmf.css with
your own favourite styles. If such a file exists
and its filemode is 0600 (i.e.
"-rw-------") and it is owned by "our" user
(see section Installation / Setup above) it will be used instead of
the builtin defaults for designing the HTML pages. – Please
note that the styles in ~/.cmf.css will get inserted
as is (in the HTML "head" section) not used as an
external stylesheet; so just write styles to that file but no e.g.
"media" selectors or other fancy stuff. – And if you don't have
a clue what I'm talking about here, just forget this whole paragraph ;-)~/.cmf.db [i]~/.fetchmailrc file.~/.cmf.dec [i]sed script used to decode URL-encoded form-data submitted by the
user. You should not modify it, since it's rather important
(incl. its internal order) for dealing with the user data; it's automatically
(re-)created if you deleted it by accident.~/.cmf.lang.de [o]~/.cmf.lang.en [o]~/.cmf.lang.en which is created
automatically by the --test option (see the sections
Installation / Setup and Commandline Options above). The
same applies for the german language file ~/.cmf.lang.de. At runtime
the script checks whether the user sent a preferred language setting. If that user
setting matches one of the installed language files ("en" aka english and
"de" aka german by default) the respective strings are used when
generating the HMTL pages sent to the user. If no matching language file exists,
the default builtin english strings are used.~/.cmf.MenAtWork [i][up to Table of Contents]
$Log: cmf,v $
Revision 1.6 2004/08/05 18:51:12 matthias
* made private files "hidden" dot-files (again);
* implemented error-page (for serious errors only);
* completed string localization;
* added more documentation;
Revision 1.5 2004/07/31 21:59:09 matthias
* changed name of script (and all depended files) to cmf to avoid
confusion with esr's fetchmailconf;
* form-data encoding is set to application/x-www-form-urlencoded
explicitely to not rely on browser implementation;
* renamed internal vars for binaries (OS tools) with a prefix of EXE;
+ implemented localization for all strings in HTML pages
(files for de and en are created by --test option);
Revision 1.4 2004/07/29 20:29:02 matthias
+ made CSS used for generated pages user configurable;
+ added handling (creating/removing) of user crontab;
+ added configuration file to allow users tweaking some settings;
* added stronger permission/ownership tests for our files;
Revision 1.3 2004/07/28 19:12:54 matthias
* replaced various global variables by three arrays holding form-data,
error-messages and missing-flags respectivily;
+ added --test switch, enhanced --cgi option;
+ added lock file handling for concurrent access;
* added more internal checks for better state- and error-handling;
* login form now allows for selecting Add/Change/Delete a record;
* added explicit "tabindex" attribute for navigation to form fields and
reworked overall HTML structure for use with no-CSS browsers;
Revision 1.2 2004/07/25 20:25:39 matthias
+ added commandline options and a lot of documentation;
* implemented error array and seterr() function;
Revision 1.1 2004/07/17 19:19:33 matthias
+ initial CVS checkin;
[up to Table of Contents]
Disclaimer: No bits or bytes were harmed
and no harddisk destroyed in order to create this page.
All letters and digits on this page are strictly virtual and
any resemblance to real letters or digits – monospaced, serif or sans-serif
– is purely coincidental.