MTP Data Analysis Checklist

MJ Mahoney
Version Date: 2013-03-13

Below I provide a superficial listing of the steps involved in the MTP data processing, which is highly slanted towards the new approach of doing the temperature calibration independently of using the MTP data itself. This new approach was developed for processing the NGV MTP data, and hence is most applicable for the new canister instruments at NCAR, DLR and JPL, which have a different data format from all our prior instruments. The plan is to update the analysis code for these older instruments to use the same techniques discussed here. To keep this checklist under control, only brief statements are used on this web [age, and no screenshots are provided -- with this one exception:

When referring to controls on a form, we will navigate from the form level to the control with each step separated by a pair of vertical lines. For example on the image above:
Form 1 form | Tab 0 tab | Button 1 button |, or
Form 1 form | Tab 0 tab | Frame 1 frame | Option 1 option |

Most of the links provide a deeper discussion of the steps, as well as many screenshots from the data processing software.

MTP Pre-Campaign and Campaign Activities

1. Measuring and Modelling the IF Bandpass

    Details can be found here.
    The IF Bandpass Model is located on the AllNorm tab of the MISSION_SU_IF_Bandpass.xls file
    Like all cells copied from a spreadsheet, it must be stripped of tabs. To do this,
    Copy cells M56:S50 on the AllNorm tab to the clipboard
    Open the | Tools form | in either RAOBman or MTPbin, and select the | RAF tab | and paste the clipboard into the | Code Text Box text box |
    Depress the | IF Bandpass button |. This writes the IF Bandpass Model in to the | Code Text Box text box |
    Copy this text devoid of tabs to the clipboard and open the /MISSION/Setup/MISSION_RCS.txt file
    Paste the clipboard to the [IF_BANDPASS] segment of the RCS file, and save the file in the /MISSION/Setup/ folder

2. Calibrate Platinum RTDs (for non-canister instruments)

Details can be found here

3. Determine Instrument Pointing Offsets

Details can be found here

4. Setup Analysis Software for New Campaign

    Select a prior mission most similar to the new mission and make it the default (Default_Mission)
    On FLTINFO | Flight tab | click the | Add New Mission button | to add the new mission (New_Mission). (Go here for more details)
    (In addition to creating various default folders and copying the RCS, WCT, HDR and CAL setup files, it will also copy the RAW, MTPH, and CAL
     files from the currently selected flight date folder (eg HIPPO-3/20100416/) to New_Mission folder. Never copy the REF, ERF, BIN or other files! It will
     also create a NUM file in the New_Mission Setup folder to show the currently selected default flight date (which can be deleted after some New_Mission flights.)
    Edit the MISSION_HDR.txt file to reflect the current mission

5. Add Data for Each New Flight

    Copy the flight data files (e.g., N2010032617.54, N2010032617.mtph, and IWG1_20100326_175447.txt) to the /MISSION/RAW/ folder
    (If there are multiple files, copy all of them. Step 3) below will ask if you want to concatenate them.)
    Before new flight data can be processed, relevant information must be added to the FLTINFO form
    To process the new flight, go to the MTPbin form

6. Check Flight Performance

Use the Excel spreadsheet MJMacros to plot diagnostic sheets to evaluate performance (This needs to be expanded on after the impact on MS Office 2007 is fixed!)

MTP Post-Campaign Data Analysis

0. Calibrate Platinum RTDs

This does not apply to canister instruments, which are automatically calibrated (Go here for more details)

1. Create Avionics Files for Temperature Calibration

For the NCAR GV create NG files from the RAF netCDF files (Be sure that the format is identical to previous campaign NG files. Remember to run the CRLF utility to remove line feeds)

2. Download a radiosonde data base

    Create a MISSION.RSL file in the /MISSION/RAOB/ folder

    Use RAOBget, go here for more information (This step usually downloads TXT files from the Wyoming radiosondes server.)

    Create a binary RAOB2 file containing all the radiosondes launched during the campaign (e.g., PREDICT_2010.RAOB2)
     (Note that radiosondes from prior years may be needed to augment those from the campaign year in order to calculate RCs; however, a file is
      needed with just those radiosondes launched during the campaign in order to create template radiosonde files.)

    Use RAOBman to open the RAOB2 file for the campaign to create a time-sorted RAOB2 file for the mission (Mission_RAOBs.RAOB2) in the following steps
    Optionally, on the RAOBman | Filters tab | Valid RAOB Data Needed frame | set the 'To' altitude to 20 km (to remove RAOBs that don't get high enough)
    On the RAOBman | Select tab | click the | Search button | to load all the soundings into the select list
Sort the soundings in time by clicking the | Sort button |
    On the RAOBman | I/O tab | Random RAOB Output frame | select the | New File option | and make sure the Output File is named: /Mission/RAOB/Mission_RAOBs.RAOB2
    Go back to the RAOBman | Select tab | and click the | Write button |
     (This writes the time-sorted Mission_RAOBs.RAOB2 file which is needed in many of the calibration steps below.)

3. Find radiosondes launched near the aircraft flight track

4. Fill in LR1, LR2 and Zb in MISSION_RAOBrangeAll.csv

    The steps shown below are discussed in much more detail here.

    Import MISSION_RAOBrangeAll.csv file (created in the previous step) into Excel

    Use RAOBman to search MISSION_RAOBs.RAOB2 file for each unique sounding using the | Filter tab | to specify the RAOB site before searching

    Estimate LR11, LR12, and Zb1 for each entry in the Excel spreadsheet

    Save the completed spreadsheet as a csv file with the new lapse rate information for extending soundings beyond their burst altitude

    Navigate to the MTPbin | Open Tools button | Run Pgm tab | or the RAOBman | Blend tab | Open Tools button | Run Pgm tab |

    Press the | RAOBrangeNR button | to generate the following files:
    MISSION_RAOBrangeAll.txt         Same as MISSION_RAOBrange.csv file, but tab delimited instead
    MISSION_RAOBrangeNR.txt          A sub-set of the MISSION_RAOBrangeAll.txt file with non-redundant soundings only
    MISSION_RAOBrangeUsed.txt   Initially the same as MISSION_RAOBrangeNR.txt, but editted to remove soundings that show before and after variability

    Press the | RAOBrangeUnique button | to generate the following file:
    MISSION_RAOBrangeUnique.txt     A sub-set of MISSION_RAOBrangeNR.txt with all the unique sounding for which comparisons will be done
    This file is used when the RAOBman | Search tab | Templates button | is used to generate template RAOB2 files to calculate retrieval coefficients (RCs)

5. Do the flight level RAOB comparisons

    On the RAOBman | Blend tab | there are a number of parameters in the lower right hand corner that need to be specified.
    See the Details for how to set the values up, there is a LOT of useful information here!

    Set these parameters to appropriate values for the MISSION under consideration. Be sure the | Save Images check box | is selected
    (The images are needed so that sounding variability can be assessed on the next iteration.)
    The following step differs depending on whether you are processing NGV data or other data.
    The reason is that the NGV uses NG-files to read research temperatures, whereas Other instruments now use the IWG1-files for this information.

    For the NGV instrument:
    Click the | NGV FL Tcal button | to start the flight level (FL) comparisons.
    For Other instruments:
    Click the | IWG FL Tcal button | to start the flight level (FL) comparisons.

    This program uses the following files that are all located in the /MISSION/RAOB/ folder
    RAOB2 File with time-sorted RAOBs:     MISSION_RAOBs.RAOB2        (from RAOBman, see Step 2)
    Output RAOB2 file with those used:         MISSION_RAOBused.RAOB2      (subset of RAOBs from MISSION_RAOBs.RAOB2)
    File with list of closest approaches:         MISSION_RAOBrangeUsed.txt
    Output file with RAOB comparisons:        RAOBcomparison.txt                   (comparison file to be imported into MISSION_RAOBcomparison_FL_NR.xls)

    Open the MISSION_RAOBcomparison_FL_NR.xls file from a prior mission and save it with the new mission name in the new mission's RAOB folder.

    Import the RAOBcomparison.csv file and paste it into the MISSION_RAOBcomparison_FL_NR.xls file All tab
    Make sure that the overwritten data does not extend beyond the bottom of the pasted data. If it does, delete the old data.

    Now edit the MISSION_RAOBrangeNR.txt file to remove comparisons with temporal variability and save the subset in the MISSION_RAOBrangeUsed.txt file
    To do this, use an image viewing program to navigate to the /MISSION/PNG/ folder.
    Open RAOBrangeNR.txt file in a text editor and save it as the RAOBrangeUsed.txt file.
    Now examine the PNG images one by one.
    If the before (yellow) and after (blue) radiosondes show differences, remove these comparisons from the RAOBrangeUsed.txt file.
    When all the PNG images have been examined, save the editted RAOBrangeUsed.txt file.

    On the RAOBman | Blend tab | repeat the earlier setup steps.
    Click the | NGV FL Tcal button | to start the flight level (FL) comparisons with the new RAOBrangeUsed.txt file (without redundant and variable comparisons).

    Back in MISSION_RAOBcomparison_FL_NR.xls, import the new RAOBcomparison.txt file into the Edit1 tab.
    Make sure that the overwritten data does not extend beyond the bottom of the pasted data. If it does, delete the old data.

    Finally, we are ready to calibrate the aircraft in situ temperature against radiosondes.
    This is done by plotting all the available avionics temperatures versus Mach Number squared and then deciding which one to use.
    The fit must be entered into the Calibration Module's fTc function. See here for details.

    Now redo the RAOB comparison with the RAOBman | Blend tab | TCal Corr checkbox | checked so that the new temperature calibration is applied during the comparison.
    Import RAOBcomparison.txt into MISSION_RAOBcomparison_FL_NR.xls
    On the Verify tab, verify that the slope and bias correction results in zero bias of the MTP compared to radiosondes
    Save this file as MISSION_RAOBcomparison_FL_NR.xls
    This completes the flight level temperature calibration.

6. Substitute the Calibrated Pressure and Temperature into the RAW Files

    With the temperature calibration performed and verified, we are finally ready to process the data!

    [ NB: As the NASA aircraft transition to using IWG1 files there are some changes. In the past for example, MMS data would be substituted into the RAW file
   to create the RMM file. Now we would substitute the preliminary MMS data into the IWG1 file to create a binary RIF file. On the first pass, since the MMS
   data has not be calibrated, we set fTc = T in the Calibration module. The after calibrating the FL temperature against radiosondes, we redo the substitution
   using fTc = T + slope*Mach^2 +bias, to create new RIF and RIW files.]

    The first step is to substitute the calibrated pressure and temperature in to the RAW files
    Navigate to the MTPbin | Open Tools button | Run Pgm tab | or the RAOBman | Blend tab | Open Tools button | Run Pgm tab |
    Verify that all the flight dates have been loaded into the | Flight Dates to Process frame |
    Switch to the | RAW tab |
    In the | RAW File Options frame |, select the pressure and temperature source in the | Source combo box | (e.g., NG for the NGV)
    Verify that the | Rename New File checkbox | and | Substitute in All Files checkbox | are checked
    The | Rename New File checkbox | does not change the RAW, but causes a new file extension to be generated for the corrected file
    (e.g., NGYYYYMMDD.RNG for the NGV)
    Click the | Substitute P and T Data into button |
    This substitutes the calibrated pressure and temperature data into A-line or IWG-line of the RAW data files, which are given a new file extension.

7. Determine Flight Levels that RCs should be calculated at

    This step is documented in detail elsewhere. Basically a histogram is generated of all level flight legs longer than some threshold.
    On the Tools form | Run Pgm tab | Level Flight Legs frame | set the desired options and depress the | Get button |
    Results are written to the MISSION_LevelFlight.txt file in the MISSION folder and should be imported into the MISSION_LevelFlight.xls file to complete the analysis
    Enter the flight altitudes into the MISSION_RCS.txt in the /MISSION/Setup/ folder

8. Create Template RAOB2 files using the MISSION_RAOBrangeUnique.txt File

    This code is now completely automated for Mission template RAOBs; go here to understand how this used to be done manually or see the next step

    Use RAOBman to open the MISSION_RAOBs.RAOB2 file
    Verify that the MISSION_RAOBrangeUnique.txt file exists (from step 4 above) in the /MISSION/RAOB/ folder. It is used as input for this process.

    On the RAOBman | Select tab | click the | Templates button |
    The template RAOB files will be written into the /MISSION/RAOB/Templates/ folder

9. Calculate Template Retrieval Coefficient Sets

    Navigate to the RCcalc | Setup tab | and verify the contents of the | Platform text box |, | Sensor Unit text box |, and | Mission text box |
    Enter a two letter RC set name in the | Region text box |
    Set the | Monthly Stratification combo boxes | to the month(s) of the campaign
    Set the | Mode combo box | to the proper mode
    (Normally Process Templates, to process all the files in the /MISSION/RAOB/Templates/ folder, or Process Single File to process a single file)
    Verify the settings on the | Frequencies tab |, | Angles tab | and | Altitudes tab |
    Press the | RCcalc | Start button | to start the RC calculations.

    Normally on the first pass RCs are calculated for all the template soundings in the /MISSION/RAOB/Templates/ folder.
    After the first pass of retrievals (next step), it is often necessary to add additional RC sets.
    This might happen if the MISSION_RAOBrangeAll.txt file did not pick up a potential template sounding because
    the aircraft did not get close enought to it as defined by the range threshold. It might also happen if there is an obvious temperature bias between the
    AA Observables and the measured Observables. In this case do the following:

    Use the RAOBman | Open RAOB button | to open the template sounding that has a bias
    Enter the temperature bias that you want added to the existing template soundings into the | Bias text box |
    Check the | Add Bias check box |
    Use the RAOBman | Select tab | Search button | to load the template soundings into the | Select tab | list box with the bias added
    On the RAOBman | I/O tab | Random RAOB Output frame | Output File text box | enter the name of the output RAOB2 file
    (e.g. use the conventional name with a different UT, the conventional filename being ICAO__YYYYMMDDUT.RABO2)
    On the RAOBman | Select tab | depress the | Write button |

10. Do Retrievals with Null OATnavCOR and Disabled WCT

    Start MTPbin. On the FLTINFO form | Flight tab | use the | Flt Date combo box to select the flight that you want to process and click the | Set Default button |
    Make the following check on these tabs:
     View - make sure the altitudes in the | pALT [km] frame | are correct
     RCs   - make sure that the proper retrieval coefficients have been loaded. If not, on the | FLTINFO form | RCs tab | depress the | Load All button |
     WCT - verify that the | WCT frame | Enable WCT check box | is NOT checked
     Gain - verify that the | OATnavCor text box | is set at 0.0 K and that the | Thresholds frame | and | Moving Average Info frame | have appropriate values
    Also select the gain scale in the | Gain Scale combo box | (usually EQN for the gain equation and ND for the noise diode,
    And the OAT source in the | OAT Source combo box | (usually A/C since the corrected IWG temperature has been substituted into the RXX file
    The | Ref Source combo box | is normally the OAT.

    On the MTPbin | Pgm Setup tab | Retrieval Program Flags frame | make sure that the | Use Substituted RAW check box | is checked

    Starting with the first flight for the mission, perform the following steps for each flight:
     -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Depress the | Open RXX button |, where 'XX' is the two letter aircraft designation (e..g. 'NG' for the NCAR NGV)
    This reads in the RAW data file from the flight date folder and edits it for interference according to the setup on the FLTINFO form | Threshold frame |
    and | Moving Average Info frame |

    Switch to the | TB Fit tab | and click the | Update button | to display the OAT (magenta) and flight altitude (blue) traces
    The TB Fit will be done in two ways: 1) to estimate the radiometric noise, and 2) to calculate the gainand TTO.
    The first uses a short level flight segment, and the second a longer light segment with some ascent and/or descent data to get the Time Tag Offset (TTO) and instrument gain

    To estimate the radiometric noise,
    Select a short level flight segment, and depress the | Auto Fit button |
    Switch to the MTPbin | Cts/Track tab | and verify that the | Display Kelvin check box | is checked
    Depress the | Scan-to-Scan AVG button |. The averages should all be close to zero.
    Depress the | Scan-to-Scan RMS button |. The first ten numbers in each column should be greater than 0.20 K (the smallest allowed)
    Depress the | Write Flt Date RMS button |. This writes a RMSYYYYMMDD.txt file to the /MISSION/RAOB/RMSerror/ folder

    To set the TTO and instrument gain, switch back to the MTPbin | TB Fit tab |:
    Use the mouse select the portion of the flight that you want to calibrate over, preferably with some ascent or descent data (could even be the entire flight!)
    Depress the | Auto Fit button | to do the fit. (If the | dTavg column | to the right are not all zeros repeat this step.)
    Depress the | TTO button | to determine the time tag offset (TTO)
    (The TTO is how far in time the MTP temperature measurement is in front of the plane compared to an in situ temperature measurement)
    Verify that the | TTO text box | in the FLTINFO form | Gain tab | T Offsets | is reasonable.

    Switch to the | MTPbin form | TBs tab | and then press the | Retrieve button | to start the retrievals for this flight date
    Watch the | TBs picture box | display and note times when the RCs may not be a good match to the measurements.
    Also watch the | ATP form | and note times when the AA temperature profiles (white trace(s)) and retrieval (yellow trace) are not a good match to each other.
    If there are a lot of RCs, use the | FLTINFO form | RCs tab | Pack button | to remove little used RC sets, and redo the retrievals.
    (This is done to avoid "having the tail shake the dog.")
    Repeat these steps for the remaining flights.
     -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

11. Determine the Window Correction Table and A Priori Observable Errors

    Copy the MISSION_RAOBrangeUsed.txt file to the MISSION_RAOBrangeWCT.txt file, which will be needed for this step. This is most easily done by loading the MISSION_RAOBrangeUsed.txt file into a text editor, and then saving it as the MISSION_RAOBrange.WCT file. Before saveing the latter file, several things should be considered. First, any RAOB comparisons editted out during the flight level temperature calibration should be editted out of this file. Second, and comparisons below a flight altitude of ~6 km should be editted out as well. While useful for the temperature calibration, downward-looking observables at flight levels below 6 km will be contaminated by ground or ocean emissivity and temperature.
    Run MTPsim and go to the | TAs tab |
    Verify that the | REF File Path text box | is for the correct MISSION
    Set the | Cycles Averaged text box | to an appropriate value (1 if tropospheric profiling is being done, 3 or 5 otherwise)

    Depress the | Calculate WCT button |
    For each entry in the MISSION_RAOBrangeWCT.txt file, the program retrieves the Measured Observables from the REF files generated in the previous step,
    and then finds the RAOB in the MISSION_RAOBs.RAOB2 file and calculates the Predicted Observables.
    The difference (Predicted Observables - Measured Observables) is written to the WCT.txt file in the /MISSION/RAOB/ folder.
    Import the WCT.txt file into the MISSION_WCT.xls file and carry out the steps described here to obtain the Window Correction Table

    On the OB Noise tab of the MISSION_WCT.xls file copy the cells Q5:T14 (yellow highlighted cells) to the clipboard
    Open the MISSION_WCT.txt file in the /MISSION/Setup/ folder and paste the clipboard into this file
    The clipboard is tab delimited so all the tabs have to be removed as this is what the code that reads the WCT expects. To do this:
    Open the | Tools form | in either RAOBman or MTPbin, and select the | RAF tab | and paste the clipboard into the | Code Text Box text box |
    Depress the | WCT button |. This writes out the WCT in the /MISSION/Setup/MISSION_WCT.txt file

    Now we will calculate the a priori observable errors, which are the quadrature sum of the standard error on the WCT and the radiometric noise
    Go to the | Tools form | Run Pgm tab | and verify that all the flight dates are loaded into the | Flight Dates to Process frame | Flight Dates list box |
    Depress the | Radiometer Noise button |.
    This reads all the RMSYYYYMMDD.txt files in the /RAOB/RMSerror/ folder and writes the RMS error into the RadiometerNoise.csv file in the same folder.
    Double click the RadiometerNoise.csv file to import it into Excel with the MISSION_WCT.xls open
    Copy its contents to the clipboard and on the | OB Noise tab | paste the clipboard at cell G28
    The quadrature sum of the WCT SE and the radiometer noise end up at cells Q41:Z43
    Copy cells Q41:Z43 to the clip board. They represent the a priori observable errors and must have tabs removed. To do this:
    Open the | Tools form | in either RAOBman or MTPbin, and select the | RAF tab | and paste the clipboard into the | Code Text Box text box |
    Depress the | A Priori Errors button |. This writes the a priori observable errors in to the | Code Text Box text box |
    Copy this text devoid of tabs to the clipboard and open the /MISSION/Setup/MISSION_RCS.txt file
    Paste the clipboard to the [OBSERVABLE_ERRORS] segment of the RCS file, and save the file.

12. Apply the Window Correction to All Flights

13. Calculate New Retrieval Coefficients with the New A Priori Observable Errors

14. Manually Reprocess All the Flights with the New Retrieval Coefficients

    After new RCs have been calculated, all flights must be reprocessed manually (i.e., not in batch mode) as in step 10
    The reason for this is that the new RCs may have new names, so the | Load All button | on | FLTINFO form | RC tab | must be used to load the new coefficients.
    After this is done for all the flights, use the | Pack button | and the | RC tab | to avoid "tail wagging the dog" issues as described in step 10.
    Do not do new retrievals after packing the RCs, as this will be taken care of in the next step as a batch process.

15. Reprocess All the Campaign Data in Batch Mode

16. Perform Full Profile Comparisons to RAOBs

    On the RAOBman | Blend tab | in the lower right hand corner select the appropriate comparison parameters
    Depress the | ATP Tcal button |
    This will write the comparison results into the /MISSION/RAOB/RAOBcomparison.txt file

17. Determine Whether a RAF Correction is Needed

     A RAF file is a REF After Fix file. To determine whether one would improve the ATP retrievals:

    Import the /MISSION/RAOB/RAOBcomparison.txt file into the MISSION_RAOBcomparison.xls file and copy it on to the | All tab |
    Switch to the | cAccuracy tab |
    If there appear to be systematic biases in the Zp vs Tmtp-Traob plot, then switch to the | cFit tab | to get the polynomial fit
    Determine the altitude range over which you want to do the fit (e.g. +/-8 km)
    Copy the fit equation to the clip board
    Open the | Tools form | in either MTPbin or RAOBman
    Switch to the | RAF tab | and paste the clip board into the | Paste Eqn Here text box |
    Depress the | Generate Code button |
    Copy the code in | Code Text Box text box | to the Calibration Module function fATPc( )
    Copy the line of seven comma separated numbers at the bottom of the | Code Text Box text box | to the clipboard
    Paste the clipboard contents into cell Q2 of the | E1 tab | of the MISSION_RAOBcomparison.xls spreadsheet and verify that the fit improves the | cAccuracy tab |

18. Read the REF Files and Apply the RAF Correction (Optional)

If you decided to make a RAF correction, navigate to MTPtools | Run Pgm tab | and click the | Create RAF Files button |

19. Perform Edit REF Individually for Each Flights

    The following three steps will be repeated for each flight.
     -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    On the FLTINFO form | Flight tab | select the flight that you want to edit on the | Flt Date combo box | and depress the | Set Default button |
    Depress the MTPbin | Edit REF button |
    (If a RAF file exists, this edits the data in the RAF file and puts it into an ERF file. Otherwise, it edits the REF file and puts the data into an ERF file.)
    Depressing the MTPbin | Edit REF button | switches the FLTINFO form to the | Words tab | where the editting results are displayed
    Select the appropriate limits (minimum and maximum) for editting the data and repeat the editting until satisfied with the results
     -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

20. Plot Curtain Plots

21. Write Final Archive Files

22. Inspect the Final Archive Files

    In addition to being able to plot ERF and REF files, the CTC form can also plot MP files (the NASA Ames or Gaines-Hipskind archive format)
    If in the MTPbin | MP Write Options frame | the | Edit out scans with MRI > check box | was checked, this should certainly be done
    Depress the MTPbin | Plot CTC button |
    On the MTPoptions form | depress the | Plot MP button |
    On the CTCoptions form | File menu | select the | Save PNG menu item | to the image in the /MISSION/PNG/ folder
    Instead of using the default image name, add a letter (e.g. m for MP or a for Archive) to the filename so that the plot from step 18 is not over-written
    Use an image viewing program to compare the uneditted and editted images, and decide whether the editting thresholds should be changed and a new MP generated.