Rtbackup(1)

From BRTT

Jump to: navigation, search


Contents

NAME

rtbackup - real-time disk archive program - second generation

SYNOPSIS

rtbackup [-v] [-V] [-n] [-A] [-E] [-R]
        [-p parameter_file]
        [-m mail_to]
        [-r reject_stas]
        [-s sta_match]
        [-t start_time]
        [-e end_time]
        rt_db

DESCRIPTION

rtbackup replicates and heals database named rt_db containing waveforms to a database on a disk directory. The output disk directory is preferably on a separate disk from the operational real time data acquisition system. rtbackup is in its second generation, the original version archived day volumes to a tar tape. With the advent of large cheap disk drives which have a much longer shelf life than tape storage, is it now advantageous to archive to external network attached storage. The original version of rtbackup is now called rtbackup2tape. The other development which necessitated a rewrite of the original rtbackup is the unfolding capability of long-term (days to months) local storage at seismic stations. This can cause a large asynchronicity between stations with good communications and those with marginal, intermittent, or down communications. rtbackup now makes liberal use of trexcerpt(1) build databases which have a duration of a month or a year depending on the parameter file specification. Each day of data is organized into a station-channel-day of data split precisely on day boundaries. The use of trexcerpt(1) has the advantage of removing any overlaps in the data which might have occurred. Data are not archived unless the lddate on the wfdisc row is older than a number of days specified in the parameter file. The question of what to do with gapped data from a station which reconnects later is a bit tricky. The most conservative approach was taken with the default action of the program which is to not remove data from the archive. This has the problem in some pathological cases to allow for duplicate or overlapped waveform data in a database. If you are feeling trusting and bold, you can use the -E and the -R options which will attempt to remove overlapping waveforms in the archive database and produce the cleanest ( dbverify(1) ) database.

OPTIONS

-v
verbose
-V
Extremely verbose. Used for debugging purposes only.
-n
No operation. Shows list of actions, but does not archive any data.
-A
Archive all. Skip lddate checks.
-E
Expert mode. Supresses some warnings. Required to remove archive waveforms.
-R
Remove archive waveforms if they can be replaced from original database. This is useful when data overlaps occur in station data where a long communcation outage has occured. Can only be used with the -E option.
-p parameter_file
Name of parameter file to use. $PFPATH is searched to find parameter file. The default parameter file is rtbackup.
-m mail_to
Email address(es) for error notification, should be quoted if using multiple addresses.
-r reject_stas
Regular expression for stations to be rejected. Default is "-".
-s sta_match
Regular expression for stations to be matched. Default is ".*".
-t start_time
Start time for processing data. The default start time is 1 January 1970.
-e end_time
End time for processing data. The default end time is now().

ENVIRONMENT

needs to be called from rtexec or have sourced $ANTELOPE/setup.csh. Need environment variable $PFPATH to be set.

PARAMETER FILE

rtbackup parameter file elements are:

dirbase
directory base name for building waveform databases
dbbase
base name for building database names
period
Period of time for database segmentation. Can be "year", "month", or "day".
lag
Number of complete days before present which lddate has to have in wfdisc before processing.
dbidserver
Name of idserver to be written into descriptor file for output database to use.
dbpath
dbpath to be written into descriptor file for output database to use.
dblocks
dblocking type for database
backup_dirs
Table list of directories which are backed up to a tar file
backup_ev_tables
Table list of database tables which are backed up to a tar file

EXAMPLE PARAMETER FILE

dirbase		/anf/TA/build/clean/rt			# should be on a different disk than the rtdb
dbbase		usarray						#
period		year							# allowed values are "year" or "month"
#period		month						# allowed values are "year" or "month"
lag			2 							# number of days of lag for lddate before processing row
dbpath		/anf/TA/dbs/dbmaster/{usarray}	# default dbpath
dbidserver	anfops:2498					# default dbidserver
dblocks		nfs							# default dblocks
backup_dirs &Tbl{							# directories to backup to tar file
logs
pf
ttgrid
}
backup_ev_tables &Tbl{						# directories to backup to tar file
arrival
assoc
emodel
event
origerr
origin
predarr
netmag
stamag
}

DIAGNOSTICS

rtbackup is normally run as a cron job under rtexec, so the errors will be written to the log directory or to the email log if the -m option is used.

SEE ALSO

pf(3)
pfecho(1)
rtexec(1)
dbverify(1)
trexcerpt(1)

BUGS AND CAVEATS

Each day of data is backed up individually. Arrivals are subsetted on day boundaries, and are then joined to the assoc, origin, event, and origerr tables. If the arrivals for an event span a day boundary, then the origin and event rows will appear once in each days database. This could cause a corrupt database if two days of data are extracted from the archive and the tables from each day are merged improperly.


rtbackup will check to see if files named .rtdbclean or .rtbackup exist and are locked. If either of these files are locked then rtbackup will not execute. This is to avoid potential situations which can either corrupt the rt_db database and/or corrupt the execution of rtbackup. Other programs which can lock .rtdbclean are rtdbclean, mk_dmc_seed, dmc_nrtwf, rtbackup, event_archive.

AUTHOR

Frank Vernon


Boulder Real Time Technologies, Inc.

Antelope 5.2-64, Boulder Real Time Technologies, Inc.
Last Modified 2010-06-02