I have made these programs publicly available, and you are free to copy,
modify and or re-distribute them. Please do not remove my name from
any of them. If you make modifications, please mail them back to me so that
others may share them.
- encrypt: My File Encrypt/Decrypt Program (Perl Script)
- Perl program to encrypt/decrypt pipelined file streams using the perl
cryptography module Crypt::CBC to do the task with the derived key and IV
parameters, much like the "aespipe" C program.
However this version uses the PBKDF v2 password to derive the encryption
key from user passphrase. This is a lot safer than simply using the
OpenSSL "enc" option to do a 'Salted' file encryption, which does only
a single iteration to derive the encryption key from the user provided
password. By using PBKDFv2 to iterated the encryion key derivation, you
effectively slow down brute force dictionary attacks to impossible speeds,
without sacrificing the normal usage of the pass-phrase to encryption key
I use this for almost all my encryption needs, and have even set up my VIM
text editor to call this program when editing encrypted files. Which make
it especially useful for personal password files (web passwords), as well
as editing "key_files" (see the "ks" script below)
See the script comments for the encrypted file format.
- pbkdf2: Access SSL Library PBKDF2 sub-routine (C Program)
- A trival program to make the OpenSSL library function
PKCS5_PBKDF2_HMAC_SHA1() available on the command line. This is the
function that implements the Password Based Key Derivation Function
version 2 (PBKDF v2) as per RFC 2898 or PKCS #5.
It was used by the previous "encrypt" script, but is no longer required,
as a perl equivelent is now built into the above script.
- pbkdf2: Perl Equivelent to PBKDF2 algorithm (Perl Script)
- A direct perl implementation of the same program and the C code above.
Can be used either as a loadable module, or directly as a perl program.
Of course it is slightly slower than the equivalent C program.
I verified that it does indeed generate the same results as the OpenSSL
This is built into the "encrypt" script above.
Filesystems Mounting and Data Store Encryption
- ks: An Encryption Keystore (Perl Script)
- A perl script implementing a 'key_store' of multiple 'key_files'
containing the master keys and configuration data used to decrypt
The 'key_files' are encrypted using a user supplied password, and contain
the actual randomly generated master keys of the real data, as well
as the commands, and configuration data needed for the decryption. That
is even the 'public' details of the encryption the real data is secured.
I don't believe in giving a cracker any hint at all.
The 'key_store' can be physically separated from the actual encrypted data
(on USB sticks, or network mounts), making it more secure (two factor).
Also fake 'key_files' (name/password pairs) can be added, to the
'key_store'. These fake keys can be made to decrypt other data, or even be
made to destory the access to the real data, to further confuse would be
attackers, and prevent rubber hose attacks. Basically they can run any
This is an in-house key management system that can handle the binary
encryption keys needed for ANY encryption scheme. Though it is currently
specifically aimed at EncFS data stores, it can be set up (and has been)
to handle any encrypted data store, that uses command line controls, and
can do so right now.
It is a simple perl script and can be expanded to provide options for the
mangement of other types of encrypted data stores, and methods.
- mount_encrypted: User mount of DM-crypt filesystems (Shell)
- With the system "/etc/fstab" setup correctly, you do not need to need to
become root to mount a dm_crypt filesystem. This script lets you mount
a dm-crypt or luk-crypt encrypted block file systems, with appropriate
with password handling for either TTY and X-Windows, as well as good error
handling and reporting of any problem encountered.
I previously used this script extensivally from command line, shell
scripts, and GUI application launchers, menus, and filesystem mount
programs, to mount encrypted filesystems given a user password, without
needing root or sudo access.
However its use has since been superseeded by encfs and the "ks" script
above. EncFS allows me to directly back up and/or file syncronize the
encrypted data between machines without requiring decryption, unlike
a disk encrypt method like this.
Files and Paths
- cmdout: Label Command Output (Shell Script)
- A wrapper around a command which marks the commands output as being from
standard output or standard error. Also reports the actual command given
and the final exit status of the command.
A useful information gathering about the wrapper program. Especially when
you plan to later use that command in a shell script, or for
Note getting the exit status of a command while piping its output is
generally difficult in older shells. This script is a demonstartion on how
this can be achieved.
- home_backup: Home Directory Backup (Tar file of Scripts)
- A push backup scheme that creates snapshot "rsync" backup of your home
directory into a "current" sub-directory directory, on directory or remote
account directory given. After a backup (update) is complete, a number of
rolling 'cycles' of hardlinked snapshots are created, generating per month,
week, day and even hourly snapshots.
A simple "recover" script can be used to list, and restore specific files
and directories, from any of the backup 'cycles'.
For more information see Rsync Backups, and Snapshoting
- linkdups: Hardlink duplicate files (Perl Script)
- Quite a complex perl script that very quickly finds large files that
are exact copies of each other. It then hardlinks them together to save
Files are only size tested initially, with full comparison
be performed when a posible match has been found, making this
re-hardlinking program very fast. Its complexity is its algorithm for
attempting to merge two separate hardlink groups of the same file. Only
when all the files of two hardlink groups finally merged together as
a single hardlinked group, is disk space saved, so it goes to great effort
to find all such files.
The primary purpose of this program is to attempt to re-link files that
were moved or renamed in "rsync" backups. This program can thus make
incredible disk space savings by restoring the hardlinking between
duplicate files. This commonly happens if a directory is renamed,
causing the hardlinkes in a later rsync backups to not be made, even
though the file itself is untouched (just the directory path).
- unlinkdups: Break All Hardlinks (Shell Script)
- Recursively look at the given files and the given directories and break
any hardlink that is found. Basically the reverse of the previous
"linkdups" script, in that it will un-merge duplicate files again, while
preserving any dates and permissions.
This was needed to remove the hardlinks from files that should not have
been hardlinked together. Specifically files in my working home directory
that are temporary backups or revisions, configuration files, or SVN
copies. This allows the 'separated' files to be able to be edited
independently from each other, without a 'vi' or 'cp' modifying ALL the
backup copies (revisions).
- mv_renum: Rename numbered files (Perl Script)
- A simple perl script which will find the largest number in the given
filenames, and prepend zeros to the number found in the other filenames
so they list in the correct order. An option will let you specify the
number of digits to make all the numbers instead.
If this script is renamed to be "
mv_reseq", it can then be
used to re-sequence all the numbers, so as to remove any gaps, or spread
out the numbers so as to add gaps to the sequence. This can be useful to
insert and re-arrange the numbered order of the files. It is a VERY handy
- mv_perl: Perform complex file renaming (Perl Script)
- Rename files based on either a given perl substitution expression, OR
using one of the many pre-prepared expressions, based on the scripts
If the script is linked/copied to the filename "
ln_perl", then it will copy or symbolically link files to
the new filename rather than move or rename them.
Built in perl expressions have been included to rename files to: all
lowercase, all uppercase, capitalise words, remove punctuation, replace
spaces with underscores, and visa-versa, and many more common file
These can accessed by linking the script to appropriate "mv_*" names (see
internal documentation). For example if the script is linked/copied to
the command name "
mv_lcase" it will rename the given file to
This script was originally based on a common perl renaming script, the core
of which was originally created by Larry Wall, the creator of perl. Many
variants exist including "mmv" and under Debian Linux "rename".
- merge: Rename or Copy files safely (Shell Script)
- A replacement for the 'mv' and 'cp' command to use in the previous scripts
which adds numbers to the filenames to prevent files overwriting and
destroying other files of the same name.
Text File Filters
- randomize: Randomize lines in pipeline (Perl Script)
- A simple perl script that can be used as a filter. It basically
will randomize all the input lines. Essentially the opposite of the
- randomline: Extract one random line (Perl Script)
- A perl script similar to the previous one, but only outputs a single
randomly picked line from the input list. Sort of a 'pick any one' type
This was designed so it does not need to read in the whole input list into
memory, instead only holding the 'current' selection from the list that it
has already read. That is it has a very small memory footprint. Of course
it will not output the final single random selection until it has finished
reading all the input lines.
- xwin_find: Wait for a window and print its WindowID
- Wait for a specified x-window client window to appear, (or timeout), and
return that windows WindowID. This ID can then be used to modify the
application window such as resize, move, iconize it. The script is
basically a simple looped wrapper around the standard "xwininfo" command.
Currently it could use a re-write at this point in time to make better
use of newer x window control tools.
- jiggle_window: Window shake or bounce (Shell Script)
- Jiggle (move around) a window in some specific way so as to highlight the
window or some condition to the user.
By default it will cause the window specified to 'bounce' like a ball to
highlight its existance. Other actions include 'shake' left and right,
which is commonly used to indicate some error condition (bad password).
A xwit version and a xdotool version is available.
- edit_textbuf: Edit a Web text buffer in VI (Shell Script)
- Grab the text from current text buffer input field, pop up your prefered
editor with that text. When Editor ends, paste the text back into the
original text buffer.
This program should be linked to a key event macro sequence such as your
window manager (or other) such as 'Window-E'
- ping_monitor: monitor machine network status. (Perl_Tk script)
- Display a table of hosts and indicate if the hosts are currently
pingable. used to set up a monitor of what machines in a lab are up and
currently in use. Machines automatically power off when not in use.
- Gif trans (.c) and
Man Page (.man) and
Text Manual (.txt)
- This is a patched version of the "giftrans" program. That fixes some
reported GIF file setting numbers (bad rolls) and comment handling of the
more modern "rgb.txt" files. It should be compiled with a RGBTXT define
giving the location of the "rgb.txt" file, though it is not strictly
necessary for its correct working.
I use this program in some of my icon library scripts to gather
information about GIF files, such as the exact colormap, disposal, and
delay settings from GIF animations.
Also see ImageMagick
Examples, Helper Scripts, and especially the "gif2anim" script. This
script originally used this program for information gathering to create
a ImageMagick command that can re-build the animation from its de-composed
frames. However no it uses ImageMagick itself to gather this data.
- XbmBrowser v5.1b.tar.gz. (X Windows C program)
- This is an interactive X bitmap and pixmap browser.
The program is a must, for exploring and viewing my icon library, and
without which the library would have been an impossibility.
- Percent: Generate a Percentage Bar (Shell Script)
- Generate a ASCII percentage bar, that can be printed to a terminal
or included in plain test emails.
- Homespace: report home disk or quota left (Shell Script)
- Look up disk or quota space, used and free, of the users home, and
report it in a nice way using either the previous 'percentage' script,
or using a 'zenity' percentage bar.
- Timeout: kill long running commands (Shell Script)
- A complex script that runs the given command, but will then kill it,
if the command has not completed in the time specified.
This is useful to prevent network commands taking too long waiting for
slow remote servers when the information is not that important. For
example when getting a hostname from a network IP, or disk quota when
the file system is on a remote NFS server that is down.
The script is completely 'Bourne Shell' based, and uses some very complex
scripting tricks to allow it to, exit immeditatally the command does,
without any 'sleep interval' pauses, or leaving behind a long running
sleep command. For more details of its developemnt see my notes in "Shell Script Hints, and the
section "Command Timeout".
Linux machine often have a C version, also called 'timeout', but that is
not always available on non-linux machines, and that is where this program
fills the gap.
- Countdown: A test command for timeout scripts (Shell Script)
- This is just a simple script outputing a countdown reminisent of a
NASA rocket launch. It is used as a test command for testing timeout
programs and scripts to see how well they work.
See my notes in "Shell Script Hints, and the section "Command Timeout".
- shell_select.pl: select system call for shell (Perl Script)
- A small simple perl script to allow access to the UNIX select() system
from a shell script which is handling multiple data pipelines. This is
typically required in a complex shell co-process programming technqiue.
Also see shell_select_example.sh,
which is a demonstration program using "shell_select.pl" to handle both normal result output, and error
output form a "
For more information see my own notes on running
"Co-Processes in Shell Scripts, and the section on
"Multiple Output Streams".
- shell_expect: simple static co-processing script (Shell Script)
- A generic but simple co-processing scripting method feeding static
requests after waiting for specific prompts from the process being feed
Based on a similar script by Steve Parker,
"Simple Expect Replacement". See my own notes on running
"Co-Processes in Shell Scripts, in the section on
"Timed Data Pipelines".
WWW Programs (Client)
- www_ctrl: Web Browser Remote Control. (Shell Script)
- Read the currently selected string, and direct the Firefox web browser
When configured with hotkeys such as from a Window Manager, users can
simply highlight a link, word, or other text, and press some button to
find out more information. Makes the lookup of more. information from
emails, web pages, manuals, or terminal output very fast and efficent.
For example I have Ctrl-F12 set to go to the selected link, Ctrl-F11 for
Google search, and Shift-F11 for Wikipedia Lookup. For more details with
its use with OpenBox Window manager see this CrunchBang
Linux Forum Discussion
- Go to the selected link
- Google Search (page or image) on selected text
- Wikipedia looking on selected text
WWW Programs (Server)
- User Links: Generate User Home Page List. (CGI Perl Script)
- A Simple program to search the users home directories for a wide world
web public directory and output a "User
Home Pages" document at the time of the WWW request.
The program is designed for use with the NCSA webserver, or the Apache Web
server. It may however be modified to work with other servers. Or for
other needs if required.
The program does assume that the password file contains user's real
names and not some sort of alias such a 'Death Stalker'. This is the
only source of a users real name unless modified to use some other
source particular to the machine using it.
A modified version is a Sort by Surname,
assuming that the name in the password file is their real name. This trys
to sort the users by their last name (which is assumed to be the last word
in their password files full name).
Guestbook: Simple CGI Guestbook. (CGI Perl Script)
- Another perl script which allows any user on a Web Server to have
a 'Guestbook' on their home page.
The Guestbook itself must be writable by 'nobody', and called
"guestbook.html", and can contain a opening comment and final
address. I used this as my own guestbook for about 6 years before
I discontinued it. Web users today are so much more apathetic about
things like this.
A "QUERY_STRING" included in the forms ACTION call to this program
contains the login name of person who's guestbook is to be updated, and an
optional flag allowing a particular guestbook to be 'reversed' so that the
newest entries are first. Any entry added is also mailed to that
guestbook owner so that he can see new entries that were added, and make
a correction if necessary.
The Form input accepts four fields, containing the users 'real name',
'email address', 'home page' and a single line 'comment'. The
last two are optional and can be left blank. From this a guestbook
entry is created and inserted. For an example form look at my own
guestbook input form.
The program is designed for use with the NCSA Http Server, but may be
modified to work with other servers. Or for other needs if required.
- Counter: generate counter image. (CGI Perl Script)
- Counter Images Archive. (Gziped Tar file)
- A very simple counter script written in perl. This script features
- Call looks like a normal ".gif" image to WWW client
IE: no `query sting arguments' or other strange needs.
- Handle any number of counters on a server or page!
- Modify the counter format (images used, number format)
- Counter images can selected for each counter and can be
any size, and almost any image type.
- Internal defaults for a 8 digit `odometer'.
This is a very simple perl program which extracts the counter location
from the "Extra URL path information", when the script is called. The
counter image output by the program is handled by an external image filter
package such a ImageMagick or PbmPlus filters, under the script s control.
That is the script itself does not handle the images!
Due to this use of external image filters, more complex counters can
be created using almost any set of `digit images' and can be used
The counter is also easily expanded to handle, and may have other
capabilities added to it in the future
- Hexadecimal Images (if images for digits a-f are provided)
- Output text strings by concatenating images (given the images)
- Generate graphical text around counter (given images)
For more information have a look in my WWW Laboratory at the information about
- A counter surrounded by larger wrapper image
- Pre-generated Complex and time consuming images.
For example a raytraced 3 dimensional image of the count!
- Animated counter images.
- Whatever image you can create automatically via a program!
- Other information, such as current time, or the number of
items left, etc..
- test-cgi: Check input arguments to CGI. (CGI Perl Script)
- This is a simple script I use for testing HTML forms and links
to see what the current server and client would supply to a CGI
script. All it does is output the following information...
Basically all the information available to a cgi script when run.
- Current working directory of the script
- Environment Passed (Includes the CGI input, like QUERY_STRING)
- Decoded Query String
- Decoded Path Info
- Decoded Standard Input (CGI POST Information)
This script is used in "Sorcery of Form Input" in my Castle's WWW Laboratory.
- Talk: WWW Adventure Interaction. (CGI Perl Script)
- This is the program I use to handle the "talk to person" type
forms I use in my Castle Adventure Home Page. But I make this
available for others to use.
A very simple program which is given a "QUERY_STRING", used to
locate a talk data file, and a simple form 'message'. The message
is parsed so as to remove extra spaces, punctuation, and converted to
lowercase. It is then compared against the contents of the talk
data file and when a match is found the user's client is jumped
to the URL associated with that response.
If the users response or message is not known they are jumped to a
default URL also given in the data file. Also the unknown response
is (optionally) mailed to a particular person (me) so as to allow
unknown responses to be added later if appropriate, or to find
abusers of the adventure game.
For example this program is used to communicate with Castle's
resident Dragon, which can be seen at Balcony over the Dragon's Lair deep in the Castle Dungeons.
- Blackboard. (CGI Perl Script)
- I created this program to use in the classroom of the castle. It
provides a way for passing WWW users to leave a message for the next
user. I have also linked the program to `figlet' to allow a small
phrase to be 'graffiti' onto the blackboard.
The script takes great pains to ensure that a HTML program will parse the
output correctly. It does this by replacing the special characters such as
"<" with the html sequence "<" so that the characters will be
This program is used to implement the blackboard in the Castle's
- MH Send Filter (Shell Script)
- This script pretends to be the MH send program and collect the out
going mail. Which is passed to a mail filter of your choice, before
being passed to the real MH mail system `send' command.
This allows you to append signatures, add mail faces or just edit
outgoing mail in some fashion (PGP encryption?).
The script is installed by modifing your personal
.mh_profile" to use this command for the mh send,
as well as the real mh send command to use, and the mail filter
this script is to call.
- Unix Sendmail Filter (Shell Script)
- This script complements the above MH send filter but replaced the
normal system sendmail command.
To use this filter you specify this program in your personal
.mailrc". Unfortunately the filter called by this script
must be specified inside the script itself. The same output mail
filter can however be used for both MH and BSD mail systems.
- Outgoing Mail Filter (Shell Script)
- This script is my personal outgoing mail filter, which the above scripts
call. This script actually adds signatures, mail face, and other mail
header modifications to my outgoing mail, automatically.
ANSI Graphics Scripts
- graphics (Shell Script)
- graphics2 (Shell Script)
- graphics_colors (Shell Script)
- graphics_utf (Shell Script)
- Various scripts I wrote a long time ago (1990's before X windows became
common) to check on the ANSI graphics capabilities of my current terminal.
The exact results vary greatly depending on the terminal program
(especially for colors and attributes) and on the font you are using. It
seems that many of the special ANSI graphic character modes are no longer
functional with the more modren UTF fonts, but then they have other
methods of use the vast UTF characters now available.