But usually just
mirrordir [--excludepath] control mirror copydirsrc [src ...] dest recursdir src [src ...] [-Cprogram]
pslogin
[username@]hostname[:portnumber]
DESCRIPTION
mirrordir is a set of useful utilities for manipulating and
mirroring directories. Included is also the command pslogin - an
alternative to ssh(1), and forward(1) for forwarding
arbitrary TCP socket connections over encrypted secure channels.
mirrordir copies files that are different between the directories
control and mirror to the directory mirror. Files
whose modification times or sizes differ are copied. File permissions,
ownerships, modification times, access times (only if --access-times is
used), sticky bits, and device types are duplicated. Symlinks are
duplicated without any translation. Symlink modification and access times
(of the symlink itself, not the file it points to) are not
preserved. Hard linked files are merely copied. Creation times cannot be
set with Unix as far as I can see.
mirrordir is a DANGEROUS command because files or
directories that exist in mirror that don't exist in control
are deleted. If control is entirely empty, then all files and
directories in mirror will be deleted. If mirror is entirely
empty, then all files and directories in control will be copied.
In short, mirrordir forces mirror to be an exact replica of
the directory tree control in every possible detail suitable for
purposes of timed backup. It naturally descends into subdirectories to
all their depths. mirrordir tries to be as efficient as possible
by making the minimal set of changes necessary to mirror the directory.
Access time duplication is not usually required and creates unnecessary
load. Hence it is given as an option.
The directory control is left untouched. If --restore-access is
given then access times are reset to their original with each read.
If the the --strict-locking option is on, files in control that
are copied are locked for `shared reading'. This will ensure, if
another process is busy writing to that file, that the file is not
copied in its incomplete or corrupted state.
Usually mirrordir will not exit, but will give error messages to
stderr to report any problems, and then will continue.
The directory mirror or dest must exist, even if it is
empty.
Before erasing all the files in a directory, mirrordir checks for
the file *--keep-me (where * is zero or one characters). If
this file is present it will abort with an error message. Hence such a
file can be created in all directories that you are fearful of being
recursively erased.
copydir is equivalent to mirrordir-ck
--no-erase-directories ...
(although -c implies -k anyway), so copydir is very much like a
rigorous version of cp(1) where filenames can also be URLs, and
only outdated files are replaced. Use copydir instead of
mirrordir for most file transfers. Only use mirrordir, when
you really want to delete things.
recursdir is a further program that does nothing but descend into
the directories on the command line. It is equivalent to mirrordir--recurs-mode ...
It was born after the -C option was added, and can be used as a
more rigorous version of find(1) and can also pack all the files
it finds into a tar file.
pslogin is yet a further program which has almost nothing to
do with the previous three. It envokes a secure login session
using secure-mcserv. It is equivalent to mirrordir--login-mode--secure ...
pslogin should be called logindir. See --login-mode
below.
forward is yet a further program which has almost nothing to
do with first three. It can do forwarding of arbitrary services over a
secure channel. See forward(1) for details.
The importance of this package is that you can use URL's instead of
normal filenames, and hence manipulate files over a network. The URL
types currently supported are ftp:// and mc://
(http:// is not a filesystem and therefore is not supported).
mc:// is the Midnight Commander filesystem and is served by
the secure-mcserv daemon. It has the advantage of serving
cryptographically strong secure file transfers and logins.
You can also use glob expressions in filenames for the recursdir
and copydir commands. These will be recursively expanded.
SECURITY AND ENCRYPTION
mirrordir supports strong stream cipher encryption and
Diffie-Hellman key exchanges with several possible key sizes. Secure
connections work with mc:// type connections. See the options
--secure, --key-size, --download-scripts. See the
EXAMPLES section for demo's and the FILES section for where
public/private keys are stored.
OPTIONS
--help
Print out detailed help, then exit.
--verbose
Specifies verbose output of file modifications made to mirror.
This option can be given multiple times for greater verbosity. Output is
written to stdout.
--restore-access
Restore the access times of control with each
read.
--access-times
Duplicate even the access times of control.
--always-write
Force rewrite of every file regardless of whether they appear to
be identical or not.
--recurs-mode
This is set by default with recursdir. Listed directories are
read recursively and nothing is done to them. This option is useful with
-C in order to execute shell commands and search for files. Note
the system(), exec() and popen() functions available to
the C interpretor.
--login-mode
This option is set by default with pslogin. This approximates
rlogin(1) using secure-mcserv as a server. It is a
secure encrypted connection intended as a replacement to ssh(1).
pslogin implies --secure as well. When using this option
or when using pslogin, only one path must be present on the
command-line. The path is of the form
[mc://][username@]hostname[:portnumber][/path].
Immediately after logging in, a cd/path will be
executed at the shell prompt. To determine if a prompt is
available, pslogin searches for a #, $ or
> character. Should these not be found, pslogin
blocks indefinately. To avoid this behaviour, specify /path
as exactly /. This will leave you in your login directory.
Alternatively, change your shell prompt to include one of these
characters.
--copy-mode
This is set by default with copydir. Approximate the behaviour of
cp(1) as regards source and destination files. This option implies
--keep-files. It overwrites any existing file or directory with the same
name as a source file or directory, but does not delete files or
directories unnecessarily. Multiple source files or directories may be
given. The destination path must be a directory.
--no-erase-directories
With this option, if a target directory conflicts with a source file,
and the directory is not empty, then an error message will be printed
and the program will abort. This is the default behaviour for
copydir.
--erase-directories
With this option, if a target directory conflicts with a source file,
then the target directory will be recursively deleted. This is the
default behaviour for mirrordir.
--allow-empty-ftp-dirs
Some ftp servers do not produce a . or .. directory. This makes it appear
as though you did not have permission to read the directory. This
option overrides this by assuming that such completely empty directories
are merely empty and do not have errors. If you get
unable to open directory: *: Permission denied errors, then you can
use this option. This option is enabled by default (see next).
--no-allow-empty-ftp-dirs
Because we now cd to the directory to check for permission to access it,
the default behaviour is to allow empty directories.
--only-delete
Do not make any changes to mirror that will cause the total space
occupied by mirror to enlarge. This is a useful option when backing
up onto a drive that has limited space, where changes have been made to
control that might cause mirror to grow larger during the
duration of the transfer. Running mirrordir once with this option
and then once normally will ensure that the available space is not
overrun.
-i, --ignore-next-exclude
This option dictates that the next --exclude- type option must
completely overlook those files regardless of their existence or
non-existence in the mirror directory. It has the effect of a
return value of IGNORE for the --exclude-script
option below. This can be used to cause certain files to never be
modified, for example if you would like /etc/named.boot to
never be modified, use mirrordir /mnt/1 /mnt/2 -i --exclude
/mnt/1/etc/named.boot. Note that all paths must be specified in
the control directory and not the mirror directory, hence
--exclude /mnt/2/etc/named.boot won't work. This has the
idiotic behaviour that if you want to avoid erasing a file, you
have to have that file present in the control directory, even if
it has zero length.
--excludepath
Exclude file or directory path. A large number of excluded paths
on the command-line will slow performance. An excluded path will be
removed from the mirror tree if it already exists, in the same way as
any absent directory or file. Use the --ignore option to ignore keep
rather than delete these files.
If you have a long list of files to exclude, use the
--exclude-from option.
--exclude-globglob
Exclude file or directory names matching glob style expression
glob. Matches the file without its full path. For shells, the
expression should be enclosed in appropriate quotes to prevent
substitutions.
--exclude-regexpregex
Exclude full pathnames of files or directories matching regex. For
shells, the expression should be enclosed in appropriate quotes to
prevent substitutions.
--exclude-script [expr|file]
Cause execution of the script expr for each file before
doing anything with that file. The script is a C style statement
block, terminating with a `returnexpression;'. The
value of expression can be one of INCLUDE,
EXCLUDE, UNKNOWN or IGNORE (see -i
above) to explain what is to be done with that file. If
expr does not contain a semi-colon (;) then it is assumed
to be the name of a file - which is then loaded. In either case,
the text is byte compiled into reverse-polish notation for fast
execution. This option can be specified multiple times and the
scripts will be executed in order until a script returns
something other than UNKNOWN. A return value of
UNKNOWN allows the remaining --exclude- options on
the command-line to take effect.
If you find that the interpretor incorrectly reports errors or
segfaults, please report the breaking script to me.
The scripting language itself is a subset of the full C
programming language. For example, the following is a valid
script:
/* PATH is the full name of the file including
its path, DIR is the directory, excluding
the trailing slash (/), CWD is the current
directory, and depth() returns the number
of forward slashes (/) less one. */
if (depth (DIR) - depth (CWD) > 3) {
printf ("%s: excluded\n", PATH);
return EXCLUDE;
} else
return INCLUDE;
The scripting language does not support the assignment operator,
and hence does not support user defined variables.
The following predefined macros are available. Note that the
expansion of the macros applies equally well to directories as to
files.
FILE
current file without its path
NAME
file name without its path or extension or trailing dot (.)
EXTENSION
file extension without its leading dot (.)
DIR
directory without file-name or trailing slash (/)
PATH
full file name with path
CWD
current working directory
TIME
current time in seconds
All of the logical, arithmetic and bitwise C operators are
supported. These are ()>=<=><!===&&||!-+*/%&^ and have the same meanings and precedences as in C.
The following further predefined macros are available. Each
returns an integer (type long int in C). These are based on
a C lstat (or stat if --follow-symlinks is
used) on the file. See stat(2) for a detailed explanation.
stat.st_dev - device
stat.st_ino - inode
stat.st_mode - permissions
stat.st_nlink - number of hard links
stat.st_uid - user id of owner
stat.st_gid - group id of owner
stat.st_rdev - device type
stat.st_size - file size in bytes
stat.st_blksize - block-size for file-system I/O
stat.st_blocks - number of blocks allocate
stat.st_atime - time of last accessed in seconds
stat.st_mtime - time of last modification in seconds
stat.st_ctime - time of creation
The following functions return boolean values:
strncmp(string1, string2, integer);
returns an integer less than, equal to, or greater than zero
if string1 is found, respectively, to be less than, to match,
or be greater than string2.
glob(glob, string);
returns zero if string matches glob expression glob.
Try to use only one glob expression in your code for
efficiency of the underlying implementation.
regexp(regexp, string);
returns zero if string matches regular expression
regexp. Try to use only one regular expression in your code
for efficiency of the underlying implementation.
strstr(string1, string2);
returns the first occurance of string2 in string1
up to the length of string1, or zero if it did not occur.
The following functions also return a boolean value and are
analogous to the corresponding macros explained in stat(2).
They return non-zero if the specified condition is true.
S_ISLNK(integer); - file is a sym-link
S_ISREG(integer); - file is a regular file
S_ISDIR(integer); - file is a directory
S_ISCHR(integer); - file is a character device
S_ISBLK(integer); - file is a block device
S_ISFIFO(integer); - file is an fifo
S_ISSOCK(integer); - file is a socket
The following functions manipulate strings:
strcat(string1, string2);
returns the concatenation of string1 with string2. Note that
the + operator also concatenates strings.
depth(string);
returns one less than the number of forward slashes (/) in
string.
printf(format, ...);
behaves like printf(3) with an important exception: only
long int format specifiers should be used. The behaviour
of anything that results in conversion of other than a long
int is undefined. For example, use "%ld" instead of
"%d". This function prints to stdout.
The following functions do system calls:
system(command);
executes /bin/sh -ccommand, but unlike the C version,
it returns the exit code of the command. I.e. it executes a single
line of shell script, command.
exec(argv0, argv1, ...);
executes process argv0 with arguments argv1....
argv0 must be a full path name. This is faster than system
because it need not envoke sh.
popen([string, ] shell_command);
like system, but returns the output of shell_command as a string.
If string is given, this writes string into the standard input
of shell_command and return zero on success.
The following further integer constants are available and are
analogous to the macros defined in stat.h and explained in
stat(2).
One of the the following constants should be returned using the
return keyword, and imply to the caller as follows. If
nothing is returned, the return value is assumed to be
UNKNOWN.
UNKNOWN
didn't know what to do, continue with other --exclude- options
INCLUDE
include the file
IGNORE
do nothing with the file regardless of its existence or
non-existence in the mirror directory
EXCLUDE
consider the file to be non-existent in the control directory and
hence must be removed from the mirror directory (this does not
override the option --keep-files)
The following perform flow control analogous to C:
The if clause causes statement1, statement2,
etc. to be executed if integer is true (i.e. non-zero), or
otherwise causes statement1, statement2, etc. to be
executed. The else {...} part is optional.
The return clause gives a value back to mirrordir and
causes the script to exit:
returnexpression;
The exit function cause mirrordir to exit with the
specified exit code.
exit(integer);
C scripts would typically be used to exclude types of files. Note
that this is an excessive implementation of a scripting language,
and all the features are not meant for general use. A typical
script will, for example, do nothing more than cause device files
to be excluded:
C scripts can also be used to search for files with the
--recurs-mode option (same as the recursdir
command):
/* removes all core files */
/* This example has been moved to the EXAMPLES section. */
--exclude-fromfile
Exclude from a list of paths listed in the file file. Empty lines and
comment lines (with a # as the first character of the line) are ignored.
This list of files is sorted and binary searched, so if you have lots of
filenames to exclude, it is best to include them here for performance.
This option can be specified multiple times with different files. BUG:
the last path in file must end with a newline.
--backup-extensionlevel
Create backups of files before deleting or replacing them.
extension is a C style format string e.g. .ORIG.%d (be careful of
shell substitutions with the %). level is the highest number of
revision to keep. extension is appended to the filename, the
oldest file having the highest number.
--backup-outdate sec
Delete backup files older than sec seconds.
--nicenum
Be nice to other processes by sleeping occasionally. num is a
small integer. --nice causes the process to sleep for as long as it
is active, times a factor of num. Hence a value of 1 will (very
roughly) double the time it takes to do a copy, and a value of 3 will
quadruple the time. This can be used where one would like timed
backups to place less load on the CPU. --nice may not be available
on your system.
--no-chmod
Normally the permissions of files are set. If you have restricted access
and cannot change permissions, than this can be used to disable setting
of permissions.
--no-chown
Normally the ownerships of files are set. If you have restricted access
and cannot change ownerships, than this can be used to disable setting
of ownerships.
--mtime-thresholdsec
This is the deviation in mtime that is allowed for a file before it is
overwritten. If you have mirrored an ftp site, the mtimes are accurate
to within a minute only, thereafter an nfs mirror will cause every file
to be copied: you can then use --mtime-threshold 60 to fix this.
--time-offset [[+]|-][H]H[:MM]
This sets the time offset of any vfs (i.e. non-local) directory. For
instance, I am 8 hours east of New York, so when I mirror from New York,
I use --time-offset -08:00.
--test-only, --dry-run
Do not make any changes. If used with --verbose, this will show what
changes would be made. This is an effective way of comparing directory
trees. This is untested - i.e. I don't know whether this optionactuallymakes any changes or not!
--skip-symlinks
Symlinks are treated as though they were not read - hence if they are
found in the mirror directory then they are deleted.
--keep-files
Don't remove files from mirror, even if they don't exist in
control. This makes mirrordir somewhat like cp(1).
--no-hard-links
mirrordir mirrors hard-links properly unless this option is set, in
which case hard-links are copied as regular files.
--follow-symlinks
mirrordir mirrors symlinks properly unless this option is set, in
which case symlinks are copied as regular files. Useful for mirroring
the Debian tree. Note that this follows symlinks in both the control and
mirror directories, hence if symlinks already exist in the mirror
directory, these will be left as symlinks. Note that
--follow-symlinks implies --no-hard-links also.
--strict-locking
Create shared read locks on files as they are read. This prevents
clashes especially when copying files from the mail directory - mail
programs will be trying to write to those files at the same time as
mirrordir is trying to to read from them. This option has
no effect with virtual file systems.
--max-bytes [[num[k|M|G]]|num]
Maximum number of bytes to write before giving the message, `filled up
all blocks - first file/dir not mirrored: path', which is written
to stdout. The remaining files are deleted from mirror, but in
their listed order - hence it is possible that the archive will grow
greater than num while mirrordir is still running. You
should make allowance for this eventuality by making num smaller
than the maximum available space. Also note that some filesystems will
give an error message `No space left on device' before the device is
completely full. You can restart mirrordir with the option
--starting-filepath and hence continue with this file on
another volume. This enables mirrordir to back up across different
devices. num can be appended by k, M or
G (case insensitive) to specify kilobytes, megabytes or gigabytes
respectively. If any single file is greater than this number, then an
error message will be given. See also --block-size.
--password password
Specify the password for FTP and mc:// connections. The anonymous password
defaults to your login name @ your local machine name. You will be
prompted for other login passwords. As usual, you are warned that
including passwords in scripts is a security risk. It is much better
to put the password in your
~/.netrc
file and then not use the --no-netrc option; see
man ftp
for details.
--password-exactpassword]
Don't prepend a - to the anonymous password. With ftp
anonymous passwords, a - is usually prepended to the
password string. I don't know why the Midnight Commander vfs did
this, but one user had problems with it, hence this option is
given to send the password exactly as specified by
password.
--test-login
When using --login-mode or pslogin, you may want to
test access non-interactively (eg. in a shell script). To do
this you can run pslogin with this option and then check
its exit status. secure-mcserv uses this to verify if the
user can login on the password server.
--no-warn-first-login
The first time you try a secure connection to a machine, no
public key exists on the local machine. There is hence no
security against a man-in-the-middle attack. A warning to this
effect is printed and the user is prompted if they want to
continue. This option disables this warning and goes ahead
regardless.
--read-password-from-stdin
Instead of specifying the password on the command line, you can
write it into the command via stdin. This is not the same as
typing the invisible password as one normally does, because it
can be used even if there is no terminal. This is useful
for usage within other programs using, say popen(3).
secure-mcserv uses this to verify if the user can login
on the password server.
--netrc
Scan the ~/.netrc file. By default, this option is on.
--no-netrc
Turn off reading of ~/.netrc file.
--proxy-hosthost
Set the proxy for ftp downloads. Don't know how or if this works. Consult the
mc(1) man page for info in proxy support.
--secure
(This feature is BETA) I have implemented a secure socket layer
for mirrordir. It is enabled with this option and applies to
connections to secure-mcserv (i.e. using mc:// type URLs).
The secure socket library consists of libdiffie.a and a
header file diffie-socket.h. If you include diffie-socket.h
after you include sys/socket.h you can recompile the resulting
program with all normal sockets turned into secure sockets. (This
applies to any program that uses Unix socket calls, but is
untested). The first service supporting this is secure-mcserv
which compiles and installs by default. Hence you can use the
mc:// file system with the --secure option, provided the
remote host is running secure-mcserv (try secure-mcserv -h
to for help). --secure uses a stream cipher (much more secure and
faster than block ciphers like DES) with discrete logarithm key
exchanges with public key server authentication (Diffie-Hellman and
p-NEW schemes). For the full details, see the file diffie-socket.h
in the source distribution. The default key size is 512 bits. Note that
if you are using gcc you should compile mirrordir with the
options -O3 -fomit-frame-pointer -s -Wall to speed key generation.
--key-size bits
The default key size is 512 bits. Sizes require primes to be
generated and compiled into the file field.c, hence only those
primes listed in field.c are supported, which are at the moment
512, 768, 1024 and 1536. 768 is the recommended value for medium
security if you have slow computer. Otherwise 1536 is not an
unreasonably paranoid size for long term security. 512 can be used
if you are not worried about large corporations, well connected
hackers, or governments snooping your connections. Note that the
the stream cipher has a length of bits/2, which means that
you have more chance of being hit by a meteor while cashing in at
the state lottery, than cracking it. Be aware that a person can sniff
your connection and then keep the output for as long as it takes to
break it. In twenty years time a 1536 bit key will be considered
small. Note also that the discrete logarithm problem (used here)
is considered to be more difficult to solve than the factorisation
problem, hence the key is effectively a little larger than for RSA.
This is all my (rather uneducated) opinion.
--download-scripts
Mirrordir comes in two versions, an International version and
a US version (see --version). The US version contains
no encryption code whatsoever. Instead it downloads the needed
algorithms from encrypt.obsidian.co.za (in South Africa).
These are written in a fast, native, C-style interpreted language. There
are four scripts: one for the server Diffie-Hellman key exchange, one
for the client Diffie-Hellman key exchange, one for initialising the
stream cipher and one for actually performing the encryption using the
stream cipher. Mirrordir automatically downloads these scripts
when you try to use the security features. The option
--download-scripts however can be used to force a download at any
time. The International version contains compiled-in versions of
the stream cipher, hence only two scripts are used, which need never be
downloaded since they come with the distribution. No speed penalty is
incurred by having the Diffie-Hellman exchanges in scripts, however, you
will notice a significant speed different when using the
encryption scripts compared to compiled-in encryption.
--version
Prints out the version number as well as whether this is an
International or US version of mirrordir. See
--download-scripts.
-z, --gzip
Enable compression for mc:// connections. This actually envokes a
compressed socket layer at a lower level than the encryption.
Compression uses the libz library of gzip(1). The degree of
compression is dynamically set to minimise transfer time. It will drop
to no compression for fast ethernet connections, and will go to maximum
compression for slow modem connections. The algorithm adjusts the
compression level so that the time for a TCP write call is between 2%
and 5% of the time for the same amount of data to be deflated
(i.e. compressed).
--gzip-backups
Backups are usually just a copy of the file. With this option they are
compressed and a .gz is added to the default extension. Note that
if you specify your own extension using --backup-extension, then
it must have a .gz at the end for compression to work.
--case-insensitive, --for-Robert-Seese
Ignores case in comparison of filenames and linknames. This option is
useful when interacting with certain brain-dead operating systems. I am
not sure if this option behaves correctly under all circumstances.
--to-lower
--to-upper
Convert all new filenames to upper or lower case. When used with
--case-insensitive, it will apply only when creating new files.
When used without --case-insensitive, all files, existing or not,
will be converted to uppercase in the inefficient way of deleting the
old file and then copying the new file. This option is useful when
interacting with certain brain-dead operating systems. I am not sure
if this options behave correctly under all circumstances.
--no-use-passive-connections
If you get an error message could not setup passive mode it is likely
because you need to enable this option. I don't really understand what `passive'
means, so don't ask me.
--tar-filefilename
This is to be used only with recursdir. It creates a tar archive in
the same format as GNU tar(1) and stores it in filename.
Leading special prefixes and leading slashes are removed, i.e.
ftp://machine/dir/file becomes dir/file. If the first
character of the filename is a | character, the remaining text is
considered as a command through which the output is to be piped. Hence
a gzipped archive can be created for example with
recursdir ftp://machine/dir --tar-file '| gzip -d > foo.tar.gz'
--tar-block-sizeN
Sets the block size for tar output to 512 * N. These are the units in
which data is written to the archive. The default is 20. This is
significant only when writing to block devices. This must not be
confused with --block-size.
--block-size bytes
The default block size is 1024 bytes. File sizes are rounded up to the
nearest block when calculating the total number of blocks consumed. If
the actual block size is greater than assumed then it is possible for
fewer blocks to be counted than are actually consumed during writing.
Hence it is essential to specify a block size greater than or equal to
the actual block size when using the option --max-bytes.
--strict-mtimes
When copying regular files, mirrordir normally only overwrites the
mirror file if it is older than the control file. This option forces a
copy if there is any discrepancy at all in the modified times of
the files.
--no-mtimes
Copy files only if their sizes differ. Ignore the file modification
time.
--ignore-size
Copy files depending on mtime, but ignore size differences.
--starting-file path
path can be a file or directory. Until path is read, files
or directories will be processed as though they were excluded files
(i.e. if they exist in mirror they will be deleted). Directories
that contain path will be created if they don't exist. After
path is read, files are mirrored as usual. path itself will
also be mirrored. If path or any of its subdirectories are not
present, then mirrordir exits immediately. This is the only time
mirrordir exits prematurely. This is to prevent the entire
filesystem from being erased on account of path never being found.
FTP SUPPORT
Ftp transfers are supported using the Midnight Commander's Virtual
File System (VFS), see mc(1). In short, this means that full
URLs as well as local directories are supported. The following example
demonstrates:
also works, but will prompt me for a password first. If you are
uploading to an ftp server, you should not have the
--strict-mtimes options on, because modification times cannot
be set via ftp, and hence everything will be copied.
You will notice (with --verbose) that mirrordir repeatedly tries
to set modification and access times over ftp. I am going to leave these
messages in to remind users of the limitations of the VFS type in use.
These attempts do not appreciably detract from performance. You can use
mc:// instead although performance is poor when uploading with
this protocol. Downloading is always preferable to uploading.
In general you should NOT have an ftp upload in your cron jobs
or use ftp upload to keep directories in sync. Always download from the
other end when trying to keep directories in sync. Uploading is good
for once off uploads only.
EXAMPLES
Here are some nifty things you can do with mirrordir.
Pedantic minimalist copy
I have two source trees. I like to keep an old copy before I apply
a patch. I just do mkdir tree.OLD and then
mirrordir -v tree tree.OLD
If I run mirrordir again, then only a minimal change is
made, i.e. only the updated files are copied. (cp(1) can in
fact do this).
System backups
Some systems used timed backups onto tape archives. Others still use RAID
devices which constantly maintain an identical copy of a partition.
mirrordir provides a further alternative. You can install two
drives in a system - one for use and one for backups - and place
mirrordir in your cron(8) tables. The various backup options
can be set to make backups of files that have changed. The backup
directory might also be readable by users so that they can see their
backed up files. Previous versions of files would also be available for
users that might like to retrieve any older instances of the file.
Because mirrordir executes only the absolutely minimal set of
changes, it is extremely fast. It can be run several times a day or even
continuously with the --nice option.
Backups can also be made by FTP onto a remote machine for additional
security against the destruction of the machine.
Backing up hourly between two machines:
On one machine called dar2 I have a cron job that runs every six hours:
#!/bin/sh
# (this is just in case of any bugs I don't know about,
# but I don't think it is necessary)
killall -9 tee
killall mirrordir >& /dev/null
sleep 2
killall -9 mirrordir >& /dev/null
( \
date ; \
echo "mirrordir says (if it said nothing it is bad):" ; \
mirrordir mc://dar1:12346/ -p abcdefg /mnt/dar1/ \
-i --exclude-regexp '^mc://dar1:12346/var/lock/subsys/atd' \
--exclude-regexp '^mc://dar1:12346/proc/' \
--exclude-regexp '^mc://dar1:12346/mnt/[^/]*/.*$' \
-i --exclude-regexp '^mc://dar1:12346/boot/' \
-i --exclude-regexp '^mc://dar1:12346/etc/lilo.conf' \
-C \
'
if (S_ISDIR (stat.st_mode)) {
if (!regexp ("^mc://dar1:12346/[^/]*$", PATH))
printf ("Backing up: %s, PATH);
}
' ; \
date ; \
echo "Done" ; \
) 2>&1 \
| tee --ignore-interrupts --append /var/log/mirrordir.log \
| mail -s 'dar1 backup results' psheer@obsidian.co.za
Typical FTP transfers can be made easily with a single command using the
--copy-mode option. You can copy multiple files in both directions and
even between two ftp sites (albeit indirectly), just like with
cp(1). I use
Passwords for non-anonymous ftp transfers are best placed in the
~/.netrc
file according to standard ftp convention, and the option --netrc used.
Alternatively use ftp://myname@machine/ instead.
Finding files
recursdir / -C 'if (!glob (*.c, FILE)) printf (%s\\n, PATH);'
will print out all the C files on your system.
recursdir / -C 'if (S_ISCHR(stat.st_mode)) printf (%s\\n, PATH);'
will print out all character devices on your system.
recursdir / -C '
long l;
if (strncmp (PATH, "/proc", 5)) {
if (S_ISREG (stat.st_mode) && !strcmp ("core", FILE)) {
if (strstr (popen ("file " + PATH), "ELF 32-bit LSB core")) {
l = l + stat.st_size;
printf ("removing: %s, cumu. total = %ldkB\n", PATH, l >> 10);
exec ("rm", "-f", PATH); /* could also use system() */
}
}
}
'
ENVIRONMENT VARIABLES
TMPDIR
The directory where you would like temporary files to be stored. The ftp
filesystem downloads files first into the this temporary directory, and
then copies the file into its correct place. See BUGS below.
If TMPDIR is not specified, then it defaults to the directory of the
current file in progress.
RETURN VALUE
mirrordir returns the following:
0
Success.
1
Some kind of error occurred like a write error, a permissions error or
the like. In this case, the precise error would have been written to
stderr.
2
A file was in use and therefore could not be copied, but otherwise a
success. In this case the error `unable to open control file for writing'
would have been written to stderr. If you grep for these in stderr, you
can then run mirrordir with just these files. See grep(1).
BUGS
Some versions of the atd daemon create a lock (or pid?) file with
a lock on the file that cause secure-mcserv to block indefinately.
You will have to kill atd or exclude (?) this file before hand to
prevent this.
Mirroring over ftp where there is some daylight savings correction, seems
to produce a one hour time offset. Use --time-offset as a temporary
work-around. I don't know if this is mirrordir's fault though.
A bug that caused mirrordir to eat CPU and appear to halt has been
fixed.
A large number of command-line --exclude expressions will slow
performance. This is noted for the purposes of those wanting to
exclude a large list of files. This list should be placed in a
text file and excluded with the --exclude-from option.
Modification and access times of symlinks (not the files they point to)
are not duplicated.
Note that the previous limitation that hardlinked files are treated as
regular files has now been alleviated. The option --no-hard-links is
provided to emulate the behaviour of versions prior to 0.9.8.
No check is made to see if a hard-link would be created across devices,
in which case an appropriate error will be reported.
The ftp file-system downloads files first into the temporary directory,
as a file with the name *ftpfs*. This is a waste of space but is
the default behavior of the vfs library. If there is not enough space in
this directory, then mirrordir will probably hang. See
ENVIRONMENT VARIABLES above.
The C scripting language used to exclude files is an excessive
implementation.
FILES
~/.netrc
List of machines and corresponding passwords. See the option --netrc.
/etc/ssocket/accept.cs
This script performs the key exchange and signature generation on the
server side of the connection.
/etc/ssocket/connect.cs
This script performs the key exchange and signature authentication on the
client side of the connection.
/etc/ssocket/arcinit.cs
This initialises the stream cipher encryption. (Not present in International versions.)
/etc/ssocket/arcencrypt.cs
This performs the stream cipher encryption. (Not present in International versions.)
/etc/ssocket/private/
This directory contains private keys for the host. Each key will be held
in a file: filenames are 512, 1024 etc. This is a preferable alternative to
databases of keys as it obviates the need for a key management utility. Soon
the Reiser (spelling) filesystem will be standard and database files will
become obsolete anyway.
/etc/ssocket/public/
Analogous to /etc/ssocket/private/ for public keys.
STANDARDS
mirrordir is an invention of its author and does not belong to any
operating system standard (although it should!).
