A significant part of the radar system, or any distributed system for that matter, is the interface and transport mechanism between components, the interface through which radar data, control messages, and status information need to flow. There are many aspects to the requirements for the data transport interface, including communication and protocols between clients, notification, reliability and recoverability, data formats, and not least of all bandwidth and performance. The data transport layer needs to perform well for high-bandwidth radar data, it needs to be reliable so that no data are lost, and it needs to scale to several possible consumers and even producers of radar data. The data producers and consumers may be located on a single host (such is the case with MAPR) or they may be distributed among several hosts, as for ELDORA and SPOL.
Below is the basic block diagram being used as a starting reference point for the data system architecture.
The blocks in the diagram do not imply processing nodes, only components. At this point it's feasible that multiple components may run on a single computer or network node. (That is exactly the current MAPR architecture.) In fact, the expectation is that many other components will be attached to the system, especially components for generating derived products, but also including multiple instances of the display and archiving components. There may be multiple physical displays, such as at science stations on the P-3, or there may be multiple archivers, one for each of the desired archive media. There are also different kinds of displays, such as the engineering display versus the science display. The ELDORA development is concentrating on the science display, but there's as yet no reason an engineering display could not be part of this framework. Likewise display does not have to imply graphics. A component which displays status tables or log messages acts like a radar data consumer similarly to a graphical display.
The less coupling we need between different components of the system, the easier it will be to adapt the components to different radars, and each component will be more resistant to changes in other parts of the system. This should imply more reliability for critical parts like archiving. Likewise, individual components can be easier to test, and as a single component is tested and shared among multiple radars, all of the radars can benefit from the effort applied to making the component robust and complete. Fundamental requirements common to all radar archiving systems need only be implemented once.
The data bus is the universal decoupler. It represents the data interface and transport mechanism shared by all of the components in the radar system. In the current ELDORA it is analagous to the MCPL bus and a DORADE field format, while for SPOL it is high-speed ethernet broadcasting, PIRAQ headers, and sweepfiles.
None of the radar data system requirements are unique to radars. There are several existing software libraries, often called frameworks or layers or middleware, specifically designed to implement the general needs of networked and distributed systems, even where realtime or high-bandwidth performance is a concern. A few packages have been considered:
Joe has chosen ACE (see the section called “ACE”) as a base on which to build the network broadcasting/multicasting components.
[Notably, the block diagram in Figure 7 and its radar data flow are analogous to a distributed multimedia framework, so someday we might also look there for ideas for radar data systems. TAO contains just such an A/V streaming framework.]
One part of the data transport framework will be the DataSpace library, a C++ library for modelling data streams, where the model information can be used to generate the source code for marshalling and unmarshalling data structures. Current DataSpace documentation is available separately at the dataspace web directory, or just browse to Gary's home page.
Besides the transfer mechanism, there is also the issue of the on-the-wire format: how the data and the data interface will look between the signal processing side and the archive and display (consumer) sides. If generic and flexible enough, then the same design can be shared between SPOL and ELDORA and other radars. Note there is no point in distinguishing between interface and format in this design. If the actual on-the-wire format is not encapsulated within a software API, then that format is the interface, and clients of that interface have to do more work to interpret the format and access the data. If instead a software layer handles the marshalling and unmarshalling of data to and from the network, then that software layer is the de facto data interface. Naturally encapuslating data formats behind a software abstraction helps to insulate client components from changes and particularities in the wire format.
DORADE was meant to be a generic radar format, so presumably it would be a good place to start in studying fundamental (or at least historical) requirements for the data interface. Likewise, descriptions of several additional existing formats have been collected here for reference:
Regarding the existing formats based on large, sequential blocks of data, Dick Oye added a few more points to consider. One point relates to the proposed design for future SPOL radar product generation in which the receiver broadcasts linearized returns over a high-speed ethernet to multiple CPU nodes computing derived parameters in parallel. The parameter computations will be asynchronous, which differs from current convention where large blocks of per-beam data contain all of the parameters. So either something would have to collate the asynchronous derived data, or the consumer data interface would have to support reception of individual radar parameters asynchronously.
Another of Dick's points relates the advantages of the netCDF API. Since variables are always looked up by name in a netcdf file, consumers of the netcdf files are isolated from many changes to the netcdf layout. They will not notice and need not care about the additions of new variables nor the removal of variables they do not need. This isolation is possible because of the standard netcdf API and the level of indirection added by variable names.
The idea behind a radar data broker is to have some kind of agent to act as intermediary between a radar system component (ie, producer and/or consumer) and the rest of the distributed radar components. Rather than each client on a host opening its own network connections to other clients, one broker component interfaces with all the clients on a host and manages the set of data which have been requested among all the components on the host. Typically radar clients will want similar sets of data, so it makes sense to receive and store a single set of those data and share them among the host components. For example, an archiver and display might both run on the same host. Through the data interface middleware API they request the bulk of the radar data stream. On their behalf, the radar data broker makes a request to some remote radar data producer and begins listening for the requested data. As they arrive, the broker puts the data in shared memory where both the archiver and display can read them.
Since the radar data are mostly read-only, a shared memory implementation would work well since there will be little need for concurrency control. In fact, since data will usually have only a single producer, the broker can deploy multiple shared memory buffers, one for each producer on the host, where the broker itself is considered a "producer" since it is writing data from the network into a buffer.
One advantage to a data broker is scalability, or at least flexibility in the deployment topology. When deploying a radar system framework on a single host, such as for testing or for single-host radars like MAPR, there is no need for data to pass through the network interface just to be shared between the components on the host. Instead the data broker will transparently provide the same "data bus" capability as the network of a multi-host, distributed radar system. For that matter, the data broker may not even need to use shared memory buffers if the radar system components are linked into the same process and run in their own threads. That architecture is exactly analogous to the one used by MAPR currently.
The per-host data could be shared across shared memory, or pipes, or even just local file caches, but the sharing mechanism would be transparent to the applications and scalable and tunable. Obviously this is similar to SPOL circular buffers in shared memory. The difference is merely in adding more "intelligence" to the circular buffer component, as in allowing it to be a catalog, event service, and data subscription liaison for all the clients on a single host node. It represents a scalability layer: like an API is an interface abstraction layer, the broker is a performance abstraction, allowing the component topology to be distributed among nodes as needed for scalability and performance without modifying the components. ELDORA also already uses a similar scheme on the vxworks nodes. One task is responsible for pulling data off the bus so that other tasks can perform work with the data.
For the near-term implementation, the radar data broker component may be simplified to be very radar (ie, DORADE) specific. Since the archiver will receive and archive the raw DORADE network stream, the only consumer which needs any decoding of the DORADE structure is the display. Thus the benefits of a data broker per se would not be realized. At a minimum, there must be a component which marshalls DORADE packets off the network, byte swaps the DORADE structures, decodes them for available parameters and such, and provides them to the display with as succinct an API as possible. Figure 8 shows a block diagram containing this component. Eventually, the DORADE processing component can be generalized to perform as a radar data broker for multiple producers and consumers.
