Data Management Group (DMG)

Left to right: Amanda Orin, Bob Rilling, Linda Cully, Jose' Meitin, Linda Echo-Hawk, Scot Loehrer, Don Stott, Steve Williams, Janine Aquino, Janet Scannell, Mike Daniels.

What We Do

The CDS Data Management Group provides all facets of scientific data management including data collection, processing, quality assurance/control, archival, long-term data stewardship, and dissemination of information from the activities of National and International field and Research Projects.

Who We Are

In Fall 2005, the Field Project Support and Data Management team from the UCAR/Joint Office for Science Support (JOSS) merged with the former Atmospheric Technology Division (ATD) to form the new Earth Observing Laboratory (EOL). The former UCAR/JOSS Data Management team is now the Data Management Group within the EOL Computing, Data, and Software (EOL/CDS) Facility.  The EOL/CDS organization chart can be found here. The UCAR staff directory containing contact information can be found here.

The EOL/CDS Data Management Group is composed of associate scientists, software engineers, programmer techs, and student assistants dedicated to providing comprehensive data management services throughout the complete life cycle of a project. 

Collectively, we have over 150 years of data management experience supporting a variety of field and non-field project research on climate, weather, arctic studies, atmospheric chemistry, human health, etc.  In any given year, the CDS Data Management Group may be simultaneously supporting a multitude of projects at various stages in their life cycles from pre-field campaign to final dataset archive and analysis.

What Happens to the Data Collected during Field Campaigns?

Multi-disciplinary data (in a variety of formats) collected as part of field campaigns come from a variety of sources such as in-situ, remote sensing, and model output providers.  Such measurement platforms (or regional analyses) could be aircraft, ships, land, balloon, radars, or satellite based. Depending on the size of a field project, there could be hundreds of datasets collected from both routine operational sources to special project specific research instrumentation and models.  The DMG provides support in the stewardship of all these data and metadata from the initial collection to the long-term archival.  When data/metadata are received by the DMG, they are inventoried, reviewed, documented, and status recorded using the DMG Data Configuration Management System (as part of EMDAC).  The DMG performs a series of automatic and manual data/metadata consistency checks (quality assurance) which include format verification, gross limit checks, exact/inexact duplicate records, dataset completeness, visual inspection, followed by the production of statistical summaries and documentation of any problems encountered.  The DMG then coordinates closely with the respective data sources to resolve any problem issues and review any re-submitted data/metadata.  Once datasets are finalized, the data/metadata are merged and loaded into EMDAC for long-term archival and access by the scientific community.  Figure 1 shows the DMG data/metadata flow (from ingest to final archive).

Figure 1 – DMG Data/Metadata flow from various sources and types supporting a field project

In some cases, the DMG will provide additional data processing or form "composite" datasets from various networks [such as surface mesonets and upper air profile (balloon) data]. The "composite" data set involves the collection of all operational/research surface network data from all available sources (i.e., special research observations and existing mesonets in the field project domain), extraction of common standard meteorological parameters, conversion of all data to be a common format, provision of uniform quality control, and generation of final "composite" data sets at various time or spatial resolutions. The major advantage of creating these "composite" data sets is cost efficiency by eliminating the requirements of each investigator to individually re-process and re-format separate network data sets. In some cases data are collected from dozens of different networks. Other value-added integrated datasets an media products (such CDs, DVDs) have been reproduced to support project requirements.

As part of the overall project data archives, the DMG includes other project supporting information such as (1) links to related project web pages and other long-term Data Centers; (2) publication listings/citations; (3) meeting summaries and presentations; (4) participants and mailing lists; (4) complete project documentation and reports; (5) photography; and (6) media and outreach products.

EMDAC

To improve our data management services, CDS DMG is supporting the development of the EOL Metadata Database and Cyberinfrastructure (EMDAC). EMDAC is a comprehensive metadata database and integrated cyberinfrastructure that will be the hub of all EOL data services. Through EMDAC, CDS will create bridges to multi-agency data portals, creating compatible metadata and data access infrastructure which connects EOL to the common services of today while allowing us to meet future needs through a modular and extensible architecture. For more information about EMDAC, Web Access to archived data, DMG internal tools, Metadata Catalog Export, Browsing and Visualization, satellite and other personalized data services click here.

Long Term Data Management Collaborators

To coordinate data management activities, the DMG works routinely with many International Partners, National Agencies, Research Institutions, Universities, UCAR, and other long term data management collaborators.  Also, DMG personnel participate on various international and project data management committees to ensure that data management practices are consistent and standardized to the extent possible.

Meetings, Public Presentations, Posters

Data Management Overview

Arctic Project Support and Data Management Activities

Other Documents and Related Links

 

Contact or Visit Us

We are located on the east and south sides of the second floor at UCAR Foothills Laboratory I in Boulder, Colorado USA.

For more information, please contact:

Steve Williams

NCAR/Earth Observing Laboratory (EOL)

Telephone: (303)497-8164
Fax: (303)497-2044
Email: sfw at ucar dot edu

Mailing address:

P.O. Box 3000
Boulder, CO USA 80307

Shipping address:

3090 Center Green Drive
Boulder, CO USA 80301

To locate contact information for other members of the CDS Data Management Team, click here.