Editing Steps

Steps on cleaning up Airborne Doppler Radar data

Step 1: Back up Initial Fields

Soloii does not have an undo button. This is the most commonly requested feature and should be implemented in any future radar editing packages. For now however, the user must take care to backup fields at regular intervals during the editing process. An initial backup of the DBZ and VG fields is recommended, so that one can always start over from scratch if need be.

Sample editor commands:

  • copy DBZ to ZZ
  • copy VG to VV

Step 2: Apply Navigation Corrections

Navigation corrections are essential for a research quality analysis. The current Dorade file format supports a single set of corrections per file. The design decision was made to include the CFAC (Correction FACtors) as part of the header block. This has the advantage of allowing the user to view what corrections have been applied to the data using the Examine widget. The disadvantage to this is that the corrections are not linked to any individual field. Thus, just because the CFAC block has values in it, doesn't necessarily mean that the VG or other velocity field has been corrected! It is highly recommended that the user perform either the ETHL or BLW corrections on the leg of interest, and apply those corrections directly. The user must first set an environmental variable in a UNIX terminal to tell soloii to use the proper CFAC file. From a terminal prompt (assuming C shell):

  • setenv CFAC_FILES "TA > cfac_file.aft TF > cfac_file.fore"

Open soloii, and if the VG field has not been edited, the corrections can be easily applied using the following editor commands:

  • copy VR to VG
  • remove-aircraft-motion in VG

This removes the corrected aircraft motion vector from the Doppler velocity, yielding a ground-relative Doppler velocity. If edits have already been performed on the velocity field of interest, don't worry! You can still correct the data and preserve your edits using the following commands:

  • copy VR to VC
  • remove-aircraft-motion in VC
  • clear-bad-flags
  • copy-bad-flags from VG
  • assert-bad-flags in VC

Step 3: Remove the surface

For most meteorological applications, the ground or ocean echo is an unwanted artifact. To concentrate on the wind field, this echo needs to be removed. Since ELDORA has 4 frequencies operating simultaneously the ground echo affects multiple gates, resulting in a 'fat' echo. This echo can be fairly easily removed directly underneath the aircraft, but it becomes more difficult the farther away from nadir due to the changing incidence angle. If strong precipitation echoes extend all the way to the surface, determining the boundary between ground and air can be difficult. For a first step, make sure your rotation angle correction is good so that your ground is flat, and try a basic surface removal:

  • remove-only-surface in VG
  • remove-only-surface in DBZ

You will most likely see some residual ground at extended ranges, evident by a 'rainbow' pattern in reflectivity, and near zero velocities. If your concern is retrieval of boundary layer winds, stop here and manually edit the remaining ground. In fact, if you are editing clear air returns of boundary layer structure (ie. IHOP), the automatic ground removal tool may remove too much, and you may have to manually edit all the ground. If you can live with loss of data below 700 m or so, an alternate technique to remove nearly all the ground is to modify the optimal beamwidth used in the ground removal algorithm. A value of 3 - 4 will get almost everything:

  • One Time Only Command: optimal-beamwidth is 3. degrees
  • remove-only-surface in VG
  • remove-only-surface in DBZ

If there is still ground left, you can try the following combination in conjunction with a well drawn boundary. This sets a point bad if there is high reflectivity and low velocity. You may have to play with the values a bit to suit your particular sweeps. Note that this most likely will not work with a moving surface (ie. hurricane ocean) You may need to use a different velocity criteria in that case.

  • use-boundary
  • clear-bad-flags
  • set-bad-flags when VG between -2. and 2.
  • and-bad-flags when DBZ above 30.
  • assert-bad-flags in VG
  • assert-bad-flags in DBZ

Step 4: Basic Noise Removal

Noise is a ubiqituous feature of radar data. Fortunately, there are quantitative measures which can describe much of the noise. For ELDORA data, the best thing to use is Normalized Coherent Power (NCP). Low NCP values indicate that the returned signal was noisy. A simple threshold will remove most of the noise in the sweep. Start low (0.2) and bump it up higher (0.25 - 0.3) if there is still some extra noise.

  • threshold VG on NCP below 0.2

Thresholding on Spectrum Width (SW) can also remove some noise. If you have NOAA data, this is your only option as NCP is not recorded. Be careful with this threshold however, high SW not only means possible noise, but also can indicate turbulence. Thresholding on SW too low can remove valuable data with strong turbulent motions (e.g. convection). Using a value greater than 13. is usually fairly safe. Better yet is to use a two-condition removal similar to the above ground removal:

  • clear-bad-flags
  • set-bad-flags when SW above 13.
  • and-bad-flags when DBZ below 20.
  • assert-bad-flags in VG
  • assert-bad-flags in DBZ

The double-condition provides some extra confidence that you are removing noise and not signal. Another area that needs special attention is a 'ring of fire'. The first few gates of data near the aircraft are often noisy. These can easily be eliminated with a simple command:

  • remove-ring in VG from 0. to 0.5 km.

If the aircraft was flying in stratiform precipitation, the fraction of noise around the ring may change, and occasionally go beyond 0.5 km range. Adjust the above as necessary, or remove the partial ring manually. If you have clear air, then you will see another ring farther out resulting from antenna side-lobes. This is a more difficult fire-ring to deal with since it is not well-defined. If there is no interesting data in the region, just take it out with a large annulus remove-ring command. More often than not, part of the ring has some good data. Then you are forced to draw accurate boundaries and manually edit. The more of the ring you remove, the better your dealiasing and defreckling will work.


Step 5: Dealiasing

Numerous papers have been written on dealiasing of radar data. Soloii implements the Bargen and Brown dealiasing scheme. You can click on Examples under the Editor menu to get a sample set of parameters and commands. ELDORA data rarely has to be unfolded, but be careful if you are editing a severe bow echo or category five hurricane! NOAA tail data almost always has to be unfolded. Airborne Doppler unfolding with Bargen-Brown has the advantage that the initial guess for the wind field is provided by the aircraft data system (BB-use-ac-wind). If your data is noisy, try increasing the 'BB-gates-averaged is X' parameter. Otherwise, your data will dictate how well the algorithm works. The more noise you remove before dealiasing, the better your results will likely be. If you have 'spikes', rays obviously unfolded badly, then use the Examine widget to fix them.


Step 6: Advanced Noise Removal

After you run the above thresholding, you will undoubtably have some noise left, both in the form of 'speckles' which are isolated gates of noise outside your main echo, and 'freckles' which are noisy gates embedded within your main echo, usually at longer range (edge of the cloud). These can both be removed automatically with the Editor. For the speckles, the command is self-explanatory:

  • despeckle VG

Freckles are a little trickier. The actual parameter values depend heavily on the type of weather phenomena you are studying. If you are too aggressive or the shear is too large, you will remove big streaks of data in your main echo. If you are too cautious, nothing will be removed. My experience is to start on the cautious side and gradully get more aggressive with multiple passes through the data. And always back up your fields before attempting a defreckle operation! You will regret it when you take out big slices because you underestimated the radial shear.Also, make sure you dealias your data and remove the 'ring of fire' before attempting a defreckling (see above). A cautious set of defreckling parameters is:

  • One Time Only Command: freckle-threshold is 40.
  • One Time Only Command: freckle-average is 5 gates

A more aggressive but still relatively safe set is:

  • One Time Only Command: freckle-threshold is 20.
  • One Time Only Command: freckle-average is 5 gates

The commands to execute the defreckling (with a backup) are:

  • clear-bad-flags
  • copy VG to VBK
  • flag-freckles in VG
  • assert-bad-flags in VG

Step 7: Manual Editing

If you have a strong weather echo, a threshold-despeckle-defreckle combo will have removed almost all of your bad data. If you have clear air, you will most likely still have some side-lobes and/or second-trip ground echo to deal with. These are especially difficult to remove with automatic editing commands. If you don't care about the clear air returns and want to focus on the stronger echo, you can threshold on DBZ below a certain threshold (-10. is a conservative place to start) and this will remove some of the side-lobes. Second trip is best removed with a boundary (use-boundary) and either a conditional removal (DBZ and/or VG above/below some threshold) or a straight up 'unconditional-delete'. Be very careful with that command, however, and always back it up before you try it.

In the end, one of the primary factors separating a good analysis from a great analysis is data quality. If you put junk into a dual-Doppler synthesis, you will get junk out. Conversely, if you hack-and-slash you may end up removing good data that could change your scientific conclusions. Manual editing is the only way at this point to finish the final product, but a middle-of-the-road approach is recommended -- don't get too bogged down in the details, but be cautious. Balance the time spent editing with the desire for the 'perfect' dataset. Refine your analysis step by step, and eventually you will feel confident that you have done the best you can without spending too much time doing so. Experience helps, so if there is an experienced radar meteorologist at your institution, utilize their knowledge.