NASA's EO-1 Retired; Project Matsu


After 17 years in orbit, NASA's Earth Observing-1 (EO-1) satellite was powered off on March 30, 2017. Expected to only last a year, the EO-1 mission developed into a NASA testbed for sensors like Advanced Land Imager (ALI), hyperspectral sensors like Hyperion, and novel self-piloting AI. The OCC has collected and redistributed ALI and Hyperion sensor data from EO-1 in a data commons since 2009 as part of a commitment to Project Matsu.


On a shoestring budget contributed by NASA, the U.S. Geological Survey, the National Oceanic and Atmospheric Administration, National Reconnaissance Office and Naval Research Laboratory, the satellite continued to operate for sixteen more years, resulting in more than 1,500 papers published on EO-1 research.

E0-1 Mission director and Project Matsu lead Dan Mandl is quoted in Wired:

"In normal operations, you never get to try new things. It’s so expensive, you don’t want to mess anything up," Mandl says. "But when you have a platform to test things, a whole area of space technology evolves faster.”

Project Matsu is a collaboration between the NASA Goddard Space Flight Center and the OCC to develop open source technology for cloud-based processing of satellite imagery to support the earth science research community as well as human assisted disaster relief.

“The data from EO-1 has been essential for prototyping a cloud-based reanalysis framework for large volumes of streaming data, called the 'Matsu Wheel’, which has been running over daily batches of EO-1 satellite data for the last few years, " says Dr. Maria Patterson, a research scientist in the LSST data management group at University of Washington and former lead for Project Matsu.

A copy of the paper describing the framework and the analytics applied to EO-1 data, including spectral anomaly detection and land coverage classification, is available here: The Matsu Wheel: a reanalysis framework for Earth satellite imagery in data commons.

Project Matsu was maintained on hardware operated by the team at the Center for Data Intensive Science at University of Chicago.

RCMRD Workshop on Flood Forecasting in Nairobi

RCMRD Laura Holtzman, Dr. Zachary Flamig, Dr. Race Clark

OSDC Partnerships for International Research and Education (PIRE) Fellow and now postdoc at the Center for Data Intensive Science at the University of Chicago, Zachary Flamig, traveled to Nairobi, Kenya this past week to work with the Regional Centre for Mapping of Resources For Development (RCMRD). RCMRD operates underneath the African Union umbrella while contracting with 20 states in Eastern & Southern Africa with a goal of promoting sustainable development in member states. RCMRD focuses on hosting training workshops teaching people GIS, mapping and related skills. Included in this focus is a strong linkage to mapping disasters for mitigation and management to help save lives and property across the region.

NASA and the U.S. Agency for International Development (USAID) conduct a joint program called SERVIR aimed at helping people solve challenges in the areas of natural disasters, food security, and water resources. As part of this program NASA funds Applied Science Teams (AST) to do research and travel to the SERVIR hubs, of which RCMRD is one, for capacity building workshops. Dr. Flamig is working closely with a AST run by the University of Oklahoma focused on developing capacity for flash flood and flood prediction. The Oklahoma team has a distributed hydrologic modeling framework called the Ensemble Framework For Flash Flood Forecasting, previously developed by Dr. Flamig during his Masters and Ph.D. studies, which they are transferring to RCMRD for use in informing decision makers about impending flooding problems.

Last week was the first of six workshops that the AST will be hosting in the region over the next three years. 20 trainees were present at the workshop representing universities, Kenyan government agencies, and RCMRD itself. The workshop was considered a success by both the AST and RCMRD as the EF5 system was able to be used by the trainees at the end of the workshop. In addition to the workshop the AST focused on planning for the future with RCMRD. The next workshop this year is likely to feature two components with a starter workshop for EF5 in Uganda followed by more advanced training at RCMRD.

The OCC is very pleased that Dr. Flamig, the scientific lead for the Environmental Data Commons and the NOAA Big Data Project (BDP) was able to participate in this workshop promoting increased data use and access. The RCMRD workshop aligns with the OCC's mission to support and enhance access to community based science initiatives.

Flood Stream Flow in Southern Africa

OCC at the 97th AMS Annual Meeting


We are pleased to announce participation in a number of events at the 97th Annual Meeting of the American Meteorological Society (AMS). Dr. Zac Flamig, a postdoc at the Center for Data Intensive Science at the University of Chicago and scientific lead for the OCC NOAA Big Data Project, will be on-hand.

Highlights include:

OCC at the AGU Fall Meeting


As part of the scientific program at the American Geophysical Union (AGU) Fall Meeting, Zac Flamig, a postdoc at the Center for Data Intensive Science at the University of Chicago and scientific lead for the OCC NOAA Big Data Project, will present on "The OCC NOAA Data Commons: First Year Experiences."

This invited talk will take place Tuesday afternoon in the "Enabling Cloud Applications for Earth Science Data II session.” This session features 8 total speakers discussing how cloud computing is being used to enable science. AGU’s Fall Meeting boasts being the largest Earth and space science meeting in the world with more than 24,000 attendees in 2015. This talk provides a vital opportunity for OCC to share what it is working on within the NOAA Big Data Project as well as receive feedback on future directions.

OCC and CDIS @ SC16


We're pleased to announce our presence at the annual Super Computing conference. This year's conference will be in Salt Lake City, Utah and the OCC and the Center for Data Intensive Science will showcase:

  • innovations in data science applications in biology, medicine, health care, and the environment;
  • new releases of data commons and data peering technology that support research communities, including specialized commons for cancer genomic data, weather data, and satellite imagery;
  • data intensive computing systems;
  • high performance analytics;
  • and a Tuesday Birds-of-a-Feather session on Data Commons led by Dr. Robert Grossman, Dr. Allison Heath, and Dr. Zachary Flamig.

If you're in Salt Lake City and attending the conference please stop by booth #2611, ask for a demonstration, and learn more about how we've been making data intensive research easier and more accessible. We'll be giving away free gourmet coffee, CDIS and OCC tool kits, and OSDC embossed Belgian chocolates. Get yours while they last!

While you're in the booth be sure to check out some of the innovative networking demonstrations of our booth partners: Naval Research Lab and the NASA Sciences and Exploration Directorate. These are organized by OCC Board Member Joe Mambretti and his organization, ICAIR, who is leading 42(!!) high speed networking demonstrations this year. The OCC and CDIS will participate in 100Gbps demonstrations showcasing transfer and use of NOAA environmental data and components of Software Defined Networking for Bioinformatics applications.

Scheduled highlights of the CDIS/OCC presence can be downloaded here. Unless otherwise noted, scheduled presentations will take place in exhibition booth #2611.