MTPsetup Overview

MJ Mahoney
Last Revision: May 29, 2008

The MTPsetup program was created to simplify changing the location of data source and destination files used in the MTP data processing -- either in realtime or offline. This had been handled in the main data processing program MTPbin, but switching between offline data processing and realtime data processing was awkward when done in a single program.

The original MTPsetup program is now on the Setup tab of the current version of this program. In addtion, the functions of two other stand-alone programs (MTPrtsim and MTPftp) have been added as new tabs for the MTPsetup program because they also deal with the location of data source files for realtime data processing. So there are now three tabs: Setup, File and FTP, which are described separately below, and are shown in Figures 1, 2 and 3. The File tab allows you to play back any previously recorded RAW file at a specified rate to create MTPcurrent.RAW files used in the realtime data processing. The FTP tab allows files written to a FTP server to be retrieved and written as the MTPcurrent.RAW file. These two tabs work only duringrealtime data processing.

Figure 1. MTPsetup Setup tab.

MTPsetup Tab

This tab reads information located in the C:\MTP\Setup\MTPsetup.INI file, allows it to be modified, and then re-written. The MTPsetup tab defines the following parameters which are located in the [SYSTEM] section of the MTPsetup.INI file:

ProgramDrive$ - the hard drive on which the MTP data processing programs reside (e.g., C:), DataDrive$ - the hard drive on which the MTP data resides. Normally this is the same as the ProgramDrive$. It was created years ago when limited drive storage capacity of forced the programs and data to be on different hard drives.

UID$ - a three letter user identification which is needed in a few places

DefaultRTmode - TRUE for realtime data processing and FALSE otherwise

Data Source Mode	Realtime	Comment
0	No	Normal data processing
1	No	Data processing of a special Test File
2	Yes	Data processing of serial data
3	Yes	Data processing of MTPcurrent.RAW file

DataSourceMode - There are four sources for MTP data to be processed: two are for offline data processing and two are for real time data processing. 0 for normal offline data processing, 1 for offline data from a test file, 2 for realtime serial data from a Data Unit, and 3 for realtime data from a file named C:\MTP\RTA\RAW\MTPcurrent.RAW. DataSourceMode =3 doesn't care where the MTPcurrent.RAW file comes from. At the present time it can come from three sources: the MTP real time control program, from an existing RAW file (to simulate real time data acquistion), or from an FTP site (for example, MTPcurrent_YYYYMMDD_HHMMSS.txt files download in real time by satellite from the NCAR GV).

DataDestinationMode - this is an index (starting at 0) that specifies in real time mode the destination of files used to create the MTP realtime web page (index.html and MTPcurrent.PNG). New destination locations can be created by editting the [cboDestination] section of the C:\MTP\Setup\MTPsetup.INI file. See below for more information on this entry.

DataDestinationMode2 - it is possible when running in realtime mode that it may be necessary to send files to different locations. By convention, the DataDestinationMode is used to specify where the realtime MTP web page files are sent. DataDestinationMode2 is used to specify where RAW data files are sent. For example, on the NCAR GV DataDestinationMode2 specifies where on the ads server RAW data files (e.g., MTPcurrent_20080515_172112.txt) are located ("/mnt/r2/mtp/") so that they can be sent to the ground via satellite in realtime.

AlternateRoot - Used if the primary MTP folder is not located in the root of the ProgramDrive. In the code below taken from the GlobalParms.bas module of MTPbin, the AlternateRoot is represented by the parameter Rdir1$.

Public Rdir$          'Root Directory = "\" + Platform$ + "\"
Public Rdir1$         'MTP root if not null, ie MTP folder not in root directory
Public Rdir2$         'New Rdir = Rdir1$ + '\MTP\Data\" + Platform$ + "\"
Public Root$          'Root$ = Drive$ + Rdir$
Public MNpath$        'MNpath$ = Root$ + Mission$ + "\", Mission Path

In addition to the [SYSTEM] section of the MTPsetup.INI file there is the [cboDestination] section which can be editted to change the local or remote location of the realtime MTP web page.

[cboDestination]
C:\www\missions\realtime\ USERNAME= PASSWORD=
ftp://137.79.98.116/home/mahoney/ USERNAME=mahoney PASSWORD=!@#$%^&
ftp://192.168.84.1/var/www/html/mtp/ USERNAME=ads PASSWORD=santa;claus
Y:\jpl\group\mtp\www\missions\realtime\ USERNAME= PASSWORD=

The lines above are taken from a MTPsetup.INI file. If a remote server is mapped to the computer running the realtime software (MTPbin) then the realtime files are simply written into the specified folder. If it is necessary to FTP to the remote serve, that can be done as shown above on the two lines beginning "ftp." Note that a USERNAME= and PASSWORD= must be specified as shown above. Even if there a username and password are not needed, they must appear (as shown on lines 1 and 4) without values given.

Figure 2. MTPsetup File tab.

File Tab

As mentioned above, the File tab allows you to play back any previously recorded RAW file at a specified rate to create MTPcurrent.RAW files used in the realtime data processing. Its primary function is to simulate realtime data processing for software trouble-shooting and development purposes.

When MTPsetup starts up, the Stop button shown in Figure 2 is labelled Open File. When it is depressed, a dialog box opens to allow you to navigate to whatever RAW file that you want to process. The Write Interval text box allows your to change the rate at which the MTPcurrent.RAW file is changed. The unnamed list box shows the last record read and written to MTPcurrent.RAW. Depressing the Stop button will stop the updating of MTPcurrent.RAW, and it will be relabelled Open File, which can be depressed to restart sending files.

Figure 3. MTPsetup FTP tab.

FTP Tab

The FTP tab allows files written to an FTP server to be retrieved and written as the MTPcurrent.RAW file during realtime data processing. It is currently implemented to support data downlinked from the NCAR GV in realtime via satellite, so that retrievals can be performed and displayed on the MTP web site (http://mtp.jpl.nasa.gov/missions/realtime/index.html). Eventually it will also be used to serve the same purpose using REVEAL data on NASA research aircraft.

As implemented on the NCAR GV, the files written to the ads server by MTPbin in realtime must have a unique time stamp in the filename. So the files are named MTPcurrent_YYYYMMDD_HHMMSS.txt. (The filename extension is ".txt" instead of ".RAW" to simplify filehandling.) These file are downlinked by the ads server at whatever data rate can be accomodated, currently every two minutes. A cron job on the secure receiving server then relays the files to the anonymous FTP site shown in Figure 2, where they are accessed by the MTPsetup program. This program simply FTPs to this server to get a current directory listing. It then checks this listing to see if a new file has arrived, and if so, writes it to MTPcurrent.RAW.

Before starting to FTP files, first use the MonthView control on the right of Figure 3 to select the date of the flight. (By default, the current date will be selected, but clicking it is needed to update the Local Folder and Filename_Date text boxes so that the current date is correct. This is needed to save time if the anonymous server has data from more than one flight. This step could easily be automated, but for now it is not.) Having selected the current flight date, be sure that the File to FTP frame has the All .txt Files option selected, and that the FTP Operation frame has the Receive Files option selected. These are the default values. The Use Latest File check box if selected will start at the last file transferred. If it is not selected, files will be transferred starting with the first file generated on the current flight date. To begin FTPing files, depress the Start button.

Notes

RAW Files
RAW files contain the raw unprocessed MTP data as it is written by the computer controlling the instrument.

MTPcurrent.RAW
When MTP data is processed offline, the entire RAW file for a flight is read into arrays in memory so that data can be more quickly manipulated. This also facilitates the use of lead and lag times on critical component temperatures for more accurate gain calibration. In realtime this is not possible. Instead the realtime data processing software simply looks to see if the time stamp on a file name C:/MTP/RTA/RAW/MTPcurrent.RAW has increased. If it has, the data are processed and a retrieval done. If it has not, nothing is done. The realtime software doesn't care where MTPcurrent.RAW came from; it only cares whether the time stamp has changed. Currently the MTPcurrent.RAW file can come from three sources: the realtime program running the MTP, an existing RAW file used to simulate realtime data processing, and raw data downlinked from an aircraft via satellite to an FTP site.

MTPsetup.INI
An example of the MTPsetup.INI file

' C:\MTP\Setup\MTPsetup.INI was last written on 05-28-2008 at 23:27:39
' This INI-file follows a few simple but mandatory format requirements.
' Settings are entered into categories which occupy a single line and
' must be bracketed by square brackets (e.g. [PATH]). Variables for which
' settings are assigned must have exactly the same name in the program in
' which they are used and be followed by an equal sign (=). Spaces are
' not allowed. Anything beyond the first space in a line is ignored.
' Blank lines are also ignored, but are required betweeen categories.

[SYSTEM]
ProgramDrive$=c:              'Default Program hard drive
DataDrive$=c:                 'Default Data hard drive
UID$=mjm                      'Default user ID (initials)
DefaultRTmode=0               'Default Real Time Mode (0=FALSE, 1=TRUE)
DataSourceMode=0              'Data Source (0=Normal, 1=Test File, 2=Data Unit, 3=Sim File
DataDestinationMode=2         'Data Destination 1
DataDestinationMode2=0        'Data Destination 2
AlternateRoot=""              '

[cboDestination]
C:\www\missions\realtime\ USERNAME= PASSWORD=
ftp://137.79.98.116/home/mahoney/ USERNAME=mahoney PASSWORD=!@#$%^&
ftp://192.168.84.1/var/www/html/mtp/ USERNAME=ads PASSWORD=santa;claus
Y:\jpl\group\mtp\www\missions\realtime\ USERNAME= PASSWORD=

[cboDestination2]
ftp://192.168.84.1/mnt/r2/mtp/ USERNAME=ads PASSWORD=blue;spruce

[EOF]