RAF's primary datasets consist of synchronous time series data. Data are written to the netCDF file one record per second, even for data at higher rates.
Data acquisition of numerous instruments at high-rate (from 1 to 5K samples per second) for 10 to 11 hours can produce large data files. Typical, raw packed data files from the NCAR C-130 and GV are 2-3 GB, and unpacked with no data reduction would be twice that size. Files reduced to 1 sample per second (sps) are typically 100-150 MB.
Note: For all size-distributions (PMS1D, PMS2D, RDMA) the zeroth or first (depending on how you count) bin is undefined, do not use this value. Attributes for the various size distribution variables should include FirstBin and LastBin.
There is also an occasional need to look at data at the sample rate.
RAF uses a naming convention consisting of a project designator followed by the flight type, followed by a 2 digit flight number, followed by an indicator of sample rate ( 'h' to indicate high-rate data, low-rate data has no indicator), followed by ".nc". Prior to about 2005/2006, field projects were designated with a 3 digit number. With the adoption of ADS-3 / nidas in 2005/2006, a switch was made to upper-case project names.
e.g.
Flight types used by RAF are "ff", "tf", and "rf".
e.g.,
In our files RPO stands for "Right Pod Outboard".
Time is the unlimited dimension, and every variable (except base_time) has the Time dimension. No more than one record is to be written per second; all high-rate data are to be contained "per record", though data averaged down slower than 1 sps may be written (e.g., when you have 1 sample every 10 seconds).
int Time(Time) ; Time:long_name = "time of measurement" ; Time:standard_name = "time" ; Time:units = "seconds since 2004-08-02 16:46:11 +0000" ; Time:strptime_format = "seconds since %F %T %z" ;
// global attributes: :institution = "NCAR Research Aviation Facility" ; :Address = "P.O. Box 3000, Boulder, CO 80307-3000" ; :Phone = "(303) 497-1030" ; :creator_url = "http://www.eol.ucar.edu" ; :creator_email = "codiac at ucar.edu" ; :RepositoryURL = "https://github.com/NCAR/nimbus" ; :RepositoryBranch = "master" ; :RepositoryRevision = "v5.0-80M" ; :RepositoryDate = "Sat Apr 24 12:26:46 2021 -0600" ; :RepositoryHASH = "8fd13287c0ce7a4cdbc387e355e3fa306fbb3f6b" ; :RepositoryStatus = "?? tests/__pycache__/" ; :ProjectDirectoryRepoURL = "https://topher800@github.com/ncar/aircraft_projects" ; :ProjectDirectoryRevision = "da43da3aeb11dcc9abde6a10bf00cdb707a6763" ; :Conventions = "NCAR-RAF/nimbus" ; :ConventionsURL = "http://www.eol.ucar.edu/raf/Software/netCDF.html" ; :Metadata_Conventions = "Unidata Dataset Discovery v1.0" ; :ConventionsVersion = "1.3" ; :standard_name_vocabulary = "CF-1.0" ; :NIDASrevision = "v1.2-1469" ; :WARNING = "This file contains PRELIMINARY DATA that are NOT to be used for critical analysis." ; :date_created = "2021-06-06T23:41:31 +0000" ; :ProjectName = "SPICULE" ; :project = "SPICULE" ; :Platform = "N677F" ; :FlightNumber = "rf04" ; :FlightDate = "06/05/2021" ; :TimeInterval = "11:54:29-22:09:16" ; :InterpolationMethod = "Linear" ; :latitude_coordinate = "LATC" ; :longitude_coordinate = "LONC" ; :zaxis_coordinate = "GGALT" ; :time_coordinate = "Time" ; :geospatial_lat_min = 33.74531f ; :geospatial_lat_max = 40.17661f ; :geospatial_lon_min = -105.1467f ; :geospatial_lon_max = -96.44314f ; :geospatial_vertical_min = 304.9131f ; :geospatial_vertical_max = 13012.9f ; :geospatial_vertical_positive = "up" ; :geospatial_vertical_units = "m" ; :wind_field = "WSC WDC WIC" ; :landmarks = "39.9088 -105.117 jeffco" ; :Categories = "Position,Thermodynamic,Aircraft State,Atmos. State,Liquid Water,Uncorr\'d Raw,Wind,PMS Probe,Housekeeping,Chemistry,Radiation,Non-Standard" ; :time_coverage_start = "2021-06-05T11:54:29 +0000" ; :time_coverage_end = "2021-06-05T22:09:16 +0000" ;
*_coordinate is required starting with version 1.3. These
are the aircrafts "best" position variables.
All variables contain the basic attributes units, long_name, and _FillValue (missing_value is being deprecated). In addition we provide some other attributes; not all are used for every variable:
standard_name we have adopted the CF CF conventions section 3.3 use of standard_name where applicable.
SampledRate is the rate (sps) at which the variable was sampled onboard the aircraft. This does not apply to derived variables.
DataQuality attempts to inform the user of the quality of these data. Some values are Bad, Preliminary and Good
Category Since a file can contain hundreds of variables with not very descriptive names, this is provided to help create sub-lists of variables. Comma separated list.
CalibrationCoefficients are the values used to produce engineering units from a measurement's DC voltage. It is used by the analog/digital group. These values have already been applied! They are present for documentation.
Dependencies are the input variables that were used to produce this derived value. Like CalibrationCoefficients , these are present for documentation.
For processing purposes, variables in our files are organized into two sorted lists. The first sorted list consists of raw sampled variables. These only have calibration coefficients applied to them. The second group consists of derived variables. I have included one example from each of the 3 groups (which also show different dimension schemes):
float PITCH(Time) ; PITCH:_FillValue = -32767.f ; PITCH:units = "degree" ; PITCH:long_name = "IRS Aircraft Pitch Angle" ; PITCH:standard_name = "platform_pitch_angle" ; PITCH:valid_range = -180.f, 180.f ; PITCH:actual_range = -3.31724 15.88532f ; PITCH:Category = "Analog" ; PITCH:SampledRate = 50 ; PITCH:TimeLag = -180 ; PITCH:TimeLagUnits = "milliseconds" ; PITCH:DataQuality = "Preliminary" ; PITCH:CalibrationCoefficients = 0.26f, 1.f ; float CCDP_LWOO(Time, sps1, Vector30) ; CCDP_LWOO:_FillValue = -32767.f ; CCDP_LWOO:units = "#/cm3" ; CCDP_LWOO:long_name = "CDP Concentration (per cell)" ; CCDP_LWOO:actual_range = 0.f, 538.7375f ; CCDP_LWOO:Category = "PMS Probe" ; CCDP_LWOO:SerialNumber = "CDP016" ; CCDP_LWOO:DataQuality = "Preliminary" ; CCDP_LWOO:Dependencies = "2 ACDP_LWOO TASX" ; CCDP_LWOO:FirstBin = 0 ; CCDP_LWOO:LastBin = 29 ; CCDP_LWOO:CellSizes = 2.f, 3.f, 4.f, 5.f, 6.f, 7.f, 8.f, 9.f, 10.f, 11.f, 12.f, 13.f, 14.f, 16.f, 18.f, 20.f, 22.f, 24.f, 26.f, 28.f, 30.f, 32.f, 34.f, 36.f, 38.f, 40.f, 42.f, 44.f, 46.f, 48.f, 50.f ; CCDP_LWOO:CellSizeUnits = "micrometers" ; CCDP_LWOO:CellSizeNote = "CellSizes are lower bin limits as particle " ; CCDP_LWOO:DepthOfField = 1.2f ; CCDP_LWOO:BeamDiameter = 0.2f ; CCDP_LWOO:Density = 1.f ; CCDP_LWOO:PLWfactor = 1.e-06f ; CCDP_LWOO:DBZfactor = 1000000.f ; float WIC(Time, sps25) ; WIC:_FillValue = -32767.f ; WIC:units = "m/s" ; WIC:long_name = "GPS-Corrected Wind Vector, Vertical Gust Component" ; WIC:standard_name = "upward_air_velocity" ; WIC:actual_range = -3.393437 5.762867f ; WIC:Category = "Wind" ; WIC:DataQuality = "Good" ; WIC:Dependencies = "9 TASX VEWC VNSC PITCH ROLL THDG ATTACK SSLIP VSPD" ;
PMS1D | # bins | Counts name | Concentration name | DMT Converted | # bins | Counts name | Concentration name |
---|---|---|---|---|---|---|---|
FSSP | 16 | AFSSP_xxx | CFSSP_xxx | S100 | 30 | AS100_xxx | CS100_xxx |
PCASP | 16 | APCAS_xxx | CPCAS_xxx | S200 | 30 | AS200_xxx | CS200_xxx |
F300 | 32 | AF300_xxx | CF300_xxx | S300 | 30 | AS300_xxx | CS300_xxx |
CDP | 30 | ACDP_xxx | CCDP_xxx | ||||
UHSAS | 99 | AUHSAS_xxx | CUHSAS_xxx | ||||
260X | 64 | A260X_xxx | C260X_xxx | ---- | -- | ||
2D-C (entire-in algo) | 32 | A1DC_xxx | C1DC_xxx | Fast2DC | 64 | A1DC_xxx | C1DC_xxx |
2D-C (center-in algo) | 64 | A2DC_xxx | C2DC_xxx | Fast2DC | 128 | A2DC_xxx | C2DC_xxx |
e.g. The bin limits for CS100[n] are CellSizes[n] and CellSizes[n+1], lower and upper respectivily.
float ACDP_LWOO(Time, sps1, Vector30) ; ACDP_LWOO:_FillValue = -32767.f ; ACDP_LWOO:units = "count" ; ACDP_LWOO:long_name = "CDP Raw Accumulation (per cell)" ; ACDP_LWOO:actual_range = 0.f, 8328.f ; ACDP_LWOO:Category = "PMS Probe" ; ACDP_LWOO:SerialNumber = "CDP016" ; ACDP_LWOO:SampledRate = 10 ; ACDP_LWOO:DataQuality = "Preliminary" ; float CCDP_LWOO(Time, sps1, Vector30) ; CCDP_LWOO:_FillValue = -32767.f ; CCDP_LWOO:units = "#/cm3" ; CCDP_LWOO:long_name = "CDP Concentration (per cell)" ; CCDP_LWOO:actual_range = 0.f, 273.5868f ; CCDP_LWOO:Category = "PMS Probe" ; CCDP_LWOO:SerialNumber = "CDP016" ; CCDP_LWOO:DataQuality = "Preliminary" ; CCDP_LWOO:Dependencies = "2 ACDP_LWOO TASX" ; CCDP_LWOO:FirstBin = 0 ; CCDP_LWOO:LastBin = 29 ; CCDP_LWOO:CellSizes = 2.12f, 3.14f, 4.16f, 5.18f, 6.2f, 7.22f, 8.24f, 9.26f, 10.29f, 11.31f, 12.33f, 13.35f, 14.37f, 16.41f, 18.46f, 20.5f, 22.54f, 24.58f, 26.63f, 28.67f, 30.71f, 32.75f, 34.8f, 36.84f, 38.88f, 40.92f, 42.97f, 45.01f, 47.05f, 49.1f, 51.14f ; CCDP_LWOO:CellSizeUnits = "micrometers" ; CCDP_LWOO:CellSizeNote = "CellSizes are lower bin limits as particle " ; CCDP_LWOO:DepthOfField = 1.2f ; CCDP_LWOO:BeamDiameter = 0.2f ; CCDP_LWOO:Density = 1.f ; CCDP_LWOO:PLWfactor = 1.e-06f ; CCDP_LWOO:DBZfactor = 1000000.f ; float CONCD_LWOO(Time) ; CONCD_LWOO:_FillValue = -32767.f ; CONCD_LWOO:units = "#/cm3" ; CONCD_LWOO:long_name = "CDP Concentration (all cells)" ; CONCD_LWOO:actual_range = 0.f, 963.0727f ; CONCD_LWOO:Category = "PMS Probe" ; CONCD_LWOO:SerialNumber = "CDP016" ; CONCD_LWOO:DataQuality = "Preliminary" ; CONCD_LWOO:Dependencies = "1 CCDP_LWOO" ;
Older version of RAF netCDF files have a "legacy zeroth bin" in the histogram. Old PMS probes had a rotating housekeeping value in the first bin, and we wanted to preserve the raw data, and have a one-to-one correspondance between the counts array and the size-distribution array. Previously:
float ACDP_LWOO(Time, sps1, Vector31) ; ACDP_LWOO:_FillValue = -32767.f ; ACDP_LWOO:units = "count" ; ACDP_LWOO:long_name = "CDP Raw Accumulation (per cell)" ; ACDP_LWOO:actual_range = 0.f, 8328.f ; ACDP_LWOO:Category = "PMS Probe" ; ACDP_LWOO:SerialNumber = "CDP016" ; ACDP_LWOO:SampledRate = 10 ; ACDP_LWOO:DataQuality = "Preliminary" ; float CCDP_LWOO(Time, sps1, Vector31) ; CCDP_LWOO:_FillValue = -32767.f ; CCDP_LWOO:units = "#/cm3" ; CCDP_LWOO:long_name = "CDP Concentration (per cell)" ; CCDP_LWOO:actual_range = 0.f, 273.5868f ; CCDP_LWOO:Category = "PMS Probe" ; CCDP_LWOO:SerialNumber = "CDP016" ; CCDP_LWOO:DataQuality = "Preliminary" ; CCDP_LWOO:Dependencies = "2 ACDP_LWOO TASX" ; CCDP_LWOO:FirstBin = 1 ; CCDP_LWOO:LastBin = 30 ; CCDP_LWOO:CellSizes = 2.12f, 3.14f, 4.16f, 5.18f, 6.2f, 7.22f, 8.24f, 9.26f, 10.29f, 11.31f, 12.33f, 13.35f, 14.37f, 16.41f, 18.46f, 20.5f, 22.54f, 24.58f, 26.63f, 28.67f, 30.71f, 32.75f, 34.8f, 36.84f, 38.88f, 40.92f, 42.97f, 45.01f, 47.05f, 49.1f, 51.14f ; CCDP_LWOO:CellSizeUnits = "micrometers" ; CCDP_LWOO:CellSizeNote = "CellSizes are upper bin limits as particle " ; CCDP_LWOO:HistogramNote = "Zeroth data bin is an unused legacy placeholder." ; CCDP_LWOO:DepthOfField = 1.2f ; CCDP_LWOO:BeamDiameter = 0.2f ; CCDP_LWOO:Density = 1.f ; CCDP_LWOO:PLWfactor = 1.e-06f ; CCDP_LWOO:DBZfactor = 1000000.f ;
Difference (plus is new, minus is old):
+ float ACDP_LWOO(Time, sps1, Vector30) ; - float ACDP_LWOO(Time, sps1, Vector31) ; + float CCDP_LWOO(Time, sps1, Vector30) ; - float CCDP_LWOO(Time, sps1, Vector31) ; + CCDP_LWOO:FirstBin = 0 ; + CCDP_LWOO:LastBin = 29 ; - CCDP_LWOO:FirstBin = 1 ; - CCDP_LWOO:LastBin = 30 ; + CCDP_LWOO:CellSizeNote = "CellSizes are lower bin limits as particle " ; - CCDP_LWOO:CellSizeNote = "CellSizes are upper bin limits as particle " ; - CCDP_LWOO:HistogramNote = "Zeroth data bin is an unused legacy placeholder." ;
bin# | 0 | 1 | 2 | 3 | .... | 29 | |
---|---|---|---|---|---|---|---|
FirstBin | LastBin |
Chris Webster
National Center for Atmospheric Research
Research Aviation Facility
(303) 497-1044