Earth Observing System Data Information System (EOSDIS)
The Earth Observing System Data and Information System (EOSDIS) was started as part of the Earth Observing System (EOS). This system was meant to collect, process, distribute, and archive the large amount of data that was to be generated by the EOS program and to archive and distribute NASA Earth science data. The early concepts for the EOSDIS were shaped by NASA experiences with previous data systems. A large centralized data system was necessary to ensure on-time data processing, as well as distribution to those involved in the interdisciplinary research of Earth science. Data could not be hoarded by the Principal Investigators, as had been previously done. A large aerospace contractor was seen as the ideal developer, because these contractors had experience delivering large systems to the Department of Defense. In general, there was little experience within NASA or other civilian agencies with data systems of this magnitude and complexity.
The Earth Observing System Data Information System stores data from NASA Earth-observing satellites, such as the data employed to generate this image showing the progression of the 1997-98 El Nio as derived from the TOPEX/Poseidon satellite. Image Credit: NASA
During the start-up phase, contractor phase B studies were done and requirements were generated. However, these were done in the context of traditional procurement. A contract was signed for the development of a large, centralized EOSDIS Core System (ECS) to meet a fixed-end item specification. The issue of a continuing, long-term (10-year) development plan that evolved with technology was recognized, but this issue was never fully resolved with the work in the contract. Some involved saw it necessary to put blinders on and build the EOSDIS to the specifications. Others saw it as an R&D system that must — and could — evolve with time.
The split between the developers and the users came early in the implementation phase, and it was driven by these different views of the system. In the early to mid 1990s, attempts were made to reconcile these views and change the system to accommodate everyone. Such modifications, coupled with problems of poor performance, caused delays and budget overruns.
Simultaneously, external pressures were also having an effect. EOSDIS budgets were significantly reduced to save the Agency and the Office of Earth Science money for other purposes. This external reduction in funding was dramatic, reducing EOS funding by approximately 10 Billion dollars by mid-decade. The impact on the EOSDIS was significant. Most of the time, these reductions were not accompanied by corresponding reductions in requirements. Project visibility and poor performance combined to encourage reviews by many prestigious external review teams.
Under changing leadership in the mid to late 1990s, the necessity for basic changes was recognized to avoid total collapse of the EOSDIS. The focus was changed from a centralized system being developed all at once, to a smaller more doable system that would be developed incrementally. The main contract requirements were cut back to a core to be developed by the contractor, the science teams developed innovative ways of doing the science data processing. Thus, the system became more distributed.
Under these revisions, the EOSDIS was completed and is now delivering data to users. It does not have all the capabilities originally requested, but it also did not cost as much as originally budgeted. It handles over a terabyte of data per day, and includes the management of several petabytes of data, over 1.75 million lines of code, and support of over 900 data sets. Nevertheless, it is an impressive achievement that serves more than 2 million users per year.
The major challenge for the EOSDIS was to achieve a balance between the mission-critical operational activities and the necessary progress in science and technology. A number of lessons can be drawn from the difficult path followed by the EOSDIS during development.