ASTER high-rate data analysis users' guide

Introduction

This is a quick tutorial example of how to read and work with high-rate data in S+ using our code. "%" are unix command-line prompts and ">" is the S+ prompt. A general diagram of the data flow is below. Note that the following discussion starts with the binary archive files. "prep" knows how to read these and convert to real values. "S+" calls "prep" to preform this conversion.

Example

% set_project CASES99 
[among other things, this sets the unix environment variable $RAWDATADIR,
 if the archive files are not in this directory:
 % setenv RAWDATADIR /scr/sci2/gsp/CASES99/raw_data
 or copy the files into the location specified by $RAWDATADIR
 (usually a long process).  The "copy" could be a transfer from
 the NCAR MSS.]
% Splus
> dpar(start="1999 oct 4 10:59",lenmin=20)
# variable names must be determined from prep.config:
# % more $ASTER/projects/$PROJECT/$OPS/prep.config
# look for entries of the form: 
# d=(u,v,w,...).xyz
# the complete variable names are then
# u.xyz, v.xyz, w.xyz 
> iod = prep(c("w.5m","tc.5m","w.50m"))
> x = readts(iod)
> close(iod)
> dim(x)
[1] 71998     3
> dimnames(x)
[[1]]:
NULL
[[2]]:
[1] "w.5m"   "tc.20m" "w.50m"
> units(x)
[1] "m/s"  "degC" "m/s"
> xf = fftw(x)
> xs = specsmooth(pspectra(xf),bpd=100)
> xc = specsmooth(cospectra(xspectra(xf)))
> cmotif()
> par(mfrow=c(2,2))
# note that the following plots are multiplied by frequency
# also note the automatic logarithmic scaling
> plot(xs[,c(1,3)],type="l")
> plot(xs[,2],type="l")
> plot(xc[,1],type="l")
> q()

One subtle note is that prep() synchronizes the data to the first variable requested. In the above example, this is "w.5m". Since "tc.5m" is from the same anemometer, resampling would have no effect on it. However, "w.50m" is from a different sensor and thus would be resampled -- it might even have been reporting at 10 sps rather than 20 sps (or vice versa). Using "rate=10" option to prep() synchronizes all variables to even 100ms times. The type of resampling can be changed, but defaults to using the nearest sample in time. Note that because of this resampling, prep(c("a","b")) in general will not return the same times as prep(c("b","a"))!

Other software

It is possible to run "prep" manually from the (unix) command line to generate data either in ASCII or in Fortran-style fixed-length binary records. Type "prep" at a unix prompt for usage help.

A Java equivalent to "prep" is available for non-NCAR/EOL users. See page 1 and page 2 of our user instructions to get started (or look for a link to high-rate data directly from a project's WWW page.