Sustainable Digital Data Preservation and Access Network Partners (DataNet)

Sustainable Digital Data Preservation and Access Network Partners  (DataNet)

Science and engineering research and education are increasingly digital and increasingly data-intensive. Digital data are not only the output of research but provide input to new hypotheses, enabling new scientific insights and driving innovation. Therein lies one of the major challenges of this scientific generation: how to develop the new methods, management structures and technologies to manage the diversity, size, and complexity of current and future data sets and data streams. This solicitation addresses that challenge by creating a set of exemplar national and global data research infrastructure organizations (dubbed DataNet Partners) that provide unique opportunities to communities of researchers to advance science and/or engineering research and learning.

The new types of organizations envisioned in this solicitation will integrate library and archival sciences, cyberinfrastructure, computer and information sciences, and domain science expertise to:

• provide reliable digital preservation, access, integration, and analysis capabilities for science and/or engineering data over a decades-long timeline;
• continuously anticipate and adapt to changes in technologies and in user needs and expectations;
• engage at the frontiers of computer and information science and cyberinfrastructure with research and development to drive the leading edge forward; and
• serve as component elements of an interoperable data preservation and access network.

By demonstrating feasibility, identifying best practices, establishing viable models for long term technical and economic sustainability, and incorporating frontier research, these exemplar organizations can serve as the basis for rational investment in digital preservation and access by diverse sectors of society at the local, regional, national, and international levels, paving the way for a robust and resilient national and global digital data framework.

These organizations will provide:

• a vision and rationale that meet critical data needs, create important new opportunities and capabilities for discovery, innovation, and learning, improve the way science and engineering research and education are conducted, and guide the organization in achieving long-term sustainability;
• an organizational structure that provides for a comprehensive range of expertise and cyberinfrastructure capabilities, ensures active participation and effective use by a wide diversity of individuals, organizations, and sectors, serves as a capable partner in an interoperable network of digital preservation and access organizations, and ensures effective management and leadership; and
• activities to provide for the full data management life cycle, facilitate research as resource and object, engage in computer science and information science research critical to DataNet functions, develop new tools and capabilities for learning that integrate research and education at all levels, provide for active community input and participation in all phases and all aspects of Partner activities, and include a vigorous and comprehensive assessment and evaluation program.

Potential applicants should note that this program is not intended to support narrowly-defined, discipline-specific repositories.

Chapter 3 (Data, Data Analysis, and Visualization) of NSF’s Cyberinfrastructure Vision for 21^st Century Discovery (www.nsf.gov/pubs/2007/nsf0728/index.jsp) presents a vision in which “science and engineering digital data are routinely deposited in well-documented form, are regularly and easily consulted and analyzed by specialists and non-specialists alike, are openly accessible while suitably protected, and are reliably preserved.”  The goal of this solicitation is to catalyze the development of a system of science and engineering data collections that is open, extensible and evolvable. 

For purposes of this solicitation, data are defined as any information that can be stored in digital form and accessed electronically, including, but not limited to, numeric data, text, publications, sensor streams, video, audio, algorithms, software, models and simulations, images, etc. Digital data are not only the products of research but provide input to new hypotheses, enabling new scientific insights and driving innovation. This requires developing  new methods, management structures and technologies to manage the diversity, size, and complexity of current and future data sets and data streams.  This solicitation addresses that challenge by creating a set of exemplar national and global data research infrastructure organizations that provide unique opportunities to communities of researchers to advance science and/or engineering research and learning.  

 The new types of organizations envisioned in this solicitation will integrate library and archival sciences, cyberinfrastructure, computer and information sciences, and domain science expertise to:

• provide reliable digital data preservation, access, integration, and analysis capabilities over a decades-long timeline;
• continuously anticipate and adapt to changes in technologies and in user needs and expectations;
• engage at the frontiers of computer and information science and cyberinfrastructure with research and development to drive the leading edge forward; and
• serve as component elements of an interoperable data preservation and access network.

Characteristics of DataNet Partners

The goal of this program solicitation is to support creation of a small set of full-scale exemplars of these new types of organizations (dubbed ‘DataNet Partners’ or ‘Partners’ for purposes of this solicitation).  By demonstrating feasibility, identifying best practices, establishing viable models for long term technical and economic sustainability, and incorporating frontier research, these exemplar organizations can serve as the basis for rational investment in digital preservation and access by diverse sectors of society at the local, regional, national, and international levels.  If broadly made, the result of such investments would be a robust and resilient national and global digital data framework for preservation and access to the resources and products of the digital age.    

One defining feature of the new types of organizations envisioned in this solicitation is the capacity to be at once both risk-averse and risk capable. The goal of preserving digital information for periods of decades to centuries requires that an organization not risk loss of its holdings.  At the same time, the organization must operate in a swift, disruptive current of technological change.  Both the hardware and software components of the organization and the expectations of its users will change rapidly and continuously.  Thus, the organization must be adept at scanning the horizon, constantly planning for and then embracing new technologies. Further, the organization must be fluent and engaged in the deep research issues that support all aspects of the data management life cycle (including data creation, access, use, and preservation).  Successfully providing stability for long-term preservation and agility both to embrace constant technological change and to engage evolving research challenges requires a novel combination of expertise in library and archival sciences, computer, computational, and information sciences, cyberinfrastructure, and the other domain sciences and engineering.  A goal of this solicitation is to support the creation of new types of organizations that fully integrate all of these capabilities. 

A second essential feature is that of sustainability.  The organization must adopt economic models and technology strategies that allow it to meet its decades-long mission for preservation and access.  A key goal of this program solicitation is to develop implementation models and a body of underlying information and experience to facilitate other efforts in developing long-term digital preservation and access organizations. Thus, the solicitation envisions an initial five-year period of building capability followed by the possibility of a second, five-year period focused on achieving sustainability as NSF support is ramped down. 

A third critical feature is the ability to engage at the frontiers of science and engineering research and education. This includes functioning as an information resource for students and researchers across a wide variety of disciplines, serving as an object of research, and possessing its own active and engaged research capability. Serving as an effective information resource requires close interaction with the relevant domain science communities, along with appropriate domain expertise within the organization itself to understand the needs, practices, and expectations of the communities it serves and to provide the technical and intellectual context to inform all aspects of the data preservation and access lifecycle.  Serving as an object of external research includes providing access to large and complex data sets as test beds for research in data mining, information extraction, fusion, integration, visualization, and analysis; facilitating research in concepts and technologies that support long-term preservation and access for complex digital objects and assets; and enabling research in digital security, trust, authentication, and authorization.  Maintaining an engaged research capability requires that computer and information scientists and cyberinfrastructure researchers be an integral part of the organization at all levels.  

Finally, a key capacity to be created by these new types of organizations is the ability to support the re-purposing of digital data in innovative ways and in novel combinations not envisioned by the data authors. To provide for revolutionary new information integration capabilities and to promote broadly interdisciplinary research and education, the organizations created under this solicitation must be both individually heterogeneous in content and collectively interoperating. The former – content heterogeneity – requires that each awardee create a resource that serves a broad disciplinary and subject matter range, manages a diverse array of data types and formats, and provides the capability to support collections at the research, resource, and reference levels (see the report of the National Science Board on Long-lived Digital Data Collections; NSB 05-40; www.nsf.gov/pubs/2005/nsb0540/). The latter characteristic – operating as a data network – requires cooperation, coordination, and close interaction both among the members of this program portfolio and with other preservation and access organizations, national and international.  Participation in data networking activities, including activities focused on the development of plans for shared governance structures, will be a requirement for awards. Potential applicants should note that this program is not intended to support narrowly-defined, discipline-specific repositories.

Program Goals

The specific goals of this program are to support the development of a small set of full-scale exemplars of new types of digital data preservation and access organizations that: (1) combine expertise in library and archival sciences, computer, computational, and information sciences, cyberinfrastructure, and domain sciences and engineering; (2) develop models for economic and technological sustainability over multiple decades; (3) engage at the frontiers of science and engineering research and education as an information resource, an object of research, and a research entity; and (4) work cooperatively and in coordination to create a functional data network with revolutionary new capabilities for information access, use, and integration without regard to conventional barriers such as data type and format, discipline or subject area, and time and place.

Responsibilities of DataNet Partners

The responsibilities of a DataNet Partner include the following:

1. Vision and Rationale

• Opportunity and Impact - Identify and meet specific scientific needs and create new opportunities and capabilities for discovery, innovation,  and learning.
• Sustainable economic and technology models – Develop and continuously assess and revise a vision and rationale, with a corresponding organizational framework and implementation plan, that provides for economic and technological sustainability over a decades-long timeline.  Develop metrics and an implementation timeline with milestones that will be used during the period of the proposed award in assessing progress towards achieving economic sustainability.  Conduct ongoing planning for regular and reliable migration to new systems and execute migrations without disruption of service or risk to preservation.

2. Activities

• Support the full data preservation and access lifecycle, including acquisition; documentation; protection; access, analysis and dissemination; migration; and disposition as follows:
  • Data deposition/acquisition/ingest – Provide systems, tools, procedures, and capacity for efficient data and metadata deposition by authors and others; acquisition from appropriate sources; and ingest in accordance with well-developed and transparent policies and procedures that are responsive to community needs, maximize the potential for re-use, and ensure preservation and access over a decades timeline.
  • Data curation and metadata management – Provide for appropriate data curation and indexing, including metadata deposition, acquisition and/or entry and continuing metadata management for use in search, discovery, analysis, provenance and attribution, and integration.  Develop and maintain transparent policies and procedures for ongoing collection management, including deaccessioning of data as appropriate.
  • Data protection – Provide systems, tools, policies, and procedures for protecting legitimate privacy, confidentiality, intellectual property, or other security needs as appropriate to the data type and use.
  • Data discovery, access, use, and dissemination – Provide systems, tools, procedures, and capacity for discovery of data by specialist and non-specialist users, access to data through both graphical and machine interfaces, and dissemination of data in response to users needs.
  • Data interoperability, standards, and integration – Promote the efficient use and continuing evolution of existing standards (e.g. ontologies, semantic frameworks, and knowledge representation strategies). Support community-based efforts to develop new standards and merge or adapt existing standards.  Provide systems, tools, procedures, and capacity to enhance data interoperability and integration.
  • Data evaluation, analysis, and visualization – Provide systems, tools, procedures, and capacity to enable data driven visual understanding and integration and to enhance the ability of diverse users to evaluate, analyze, and visualize data.
• Engage at the frontiers of science and engineering research and education - Engage in research central to DataNet responsibilities, addressing the evolving technical challenges, and fully integrate computer and information scientists and cyberinfrastructure researchers at all levels of the organization. Facilitate research both as an information resource and as an object of research. Establish and maintain close interactions with the relevant domain science and education communities, fully integrate relevant domain expertise within the organization, and facilitate access by and interaction with researchers using the resources of the organization and studying its functions.
• Education and training – Provide systems, tools, and resources that enhance: (1) the use of digital data for educational and training purposes at all levels (K/12, undergraduate, graduate, professional, and public); and (2) capabilities for integration of research and education at all levels, including mechanisms that allow students and the lay public to participate in and contribute to science data activities, that facilitate the use of research data and information in science-based learning, and that enhance the ability of others to use scientific data as a means to forge a closer link between the classroom and the research environment.
• Community and user input and assessment – Provide policies, procedures, systems, tools, and forums (including workshops and conferences) for continuously obtaining input and responding to users and to the community on the mission, strategies, and policies of the Partner, including its effectiveness in meeting its responsibilities, and in planning and implementing the tools and services it provides. Develop and implement a vigorous and comprehensive assessment program including policies, metrics, procedures and tools for continuously assessing the effectiveness of the Partner in meeting the responsibilities listed in this section and in responding to the needs of users and the community. 
• International Participants - Collaborate and coordinate closely with other preservation and access organizations, both domestic and foreign, with the long-term goal of catalyzing the formation of a global data network.  In keeping with this goal, NSF strongly encourages active international collaboration by the DataNet Partners.  DataNet funds may be used to support U.S. investigators and students to work in international settings and foreign investigators and students to work in the U.S.  However, foreign collaborators are expected to secure support for their activities at their home or other non-U.S. institutions from their own national sources.

3. Organizational Structure

• Comprehensive expertise and infrastructure capacity/capabilities - Possess expertise in library and archival sciences; computer, computational, and information sciences; cyberinfrastructure; and domain sciences.  Integrate this expertise into a single, functional unit through an organizational structure that enables shared responsibility, close coordination and cooperation, and catalyzes the exchange of ideas.  Possess the computational and storage resources, network fabric (resources for internal and external access, dissemination, interaction, and communication), and expert support and administrative staff for meeting the goals for digital data preservation, access, and integration.  This capacity must support the ability to visualize and explore massive and/or heterogeneous data sets and to support analyses of data sets whose size or protection needs prevent their being moved to another site for analysis. 
• Diverse, multisector participation – Include as active project participants and/or key project partners organizations from a variety of sectors such as academic organizations of diverse geographical, programmatic, and institutional types; governmental organizations at the federal, state, and local levels; non-profit organizations; commercial entities; and international organizations. Envision the user base to include individuals and organizations from a diverse spectrum of sectors and institutional, geographical, and cultural settings with a wide range of needs, expectations, practices, and capabilities in pursuing their research and learning goals. Maintain a resource that serves a broad range of disciplinary and topic areas; manage a wide range of data types and formats; and provide advanced capabilities for searching and integrating information from various elements of the collection. Develop policies and practices that increase the participation of women and individuals from groups underrepresented in science and engineering at all levels and in all Partner activities. 
• Data Network - Collaborate and coordinate closely with other DataNet Partners and digital preservation/access organizations nationally and internationally with the goal of achieving a level of interoperability that allows seamless, single entry point discovery, access, and use of data from across the network. Work with other DataNet Partners and digital preservation/access organizations to develop and disseminate best practices and principles. It is anticipated that cooperative agreements under this program will encourage the establishment of a shared governance or coordination framework among the DataNet Partners and including representation among users and the community. Participation in this process is a requirement of an award.

DataNet Partner Leadership and Management

Anticipated Type of Award: Cooperative Agreement

Estimated Number of Awards: Two to three awards are anticipated in each of two review cycles (one review cycle for fiscal year FY2008 awards and one for FY2009) for a total of five awards, contingent on the quality of proposals received and pending the availability of funds. Each award is limited to a total of up to $20,000,000 (direct plus indirect costs) for up to 5 years. The initial term of each award is expected to be 5 years with the potential at NSF's sole discretion for one terminal renewal for another 5 years, subject to performance and the availability of funds. Such performance is to include serving the needs of the relevant science and engineering research and education communities and catalyzing new opportunities for progress. If a second five-year award is made, NSF funding is expected to decrease in each successive year of the award as the Partner transitions to a sustainable economic model with other sources of support. The actual amount of the annual decrease in NSF support will be established through the cooperative agreement. Note that the maximum period NSF will support a DataNet Partner is 10 years.

Anticipated Funding Amount: Up to $100,000,000 over a five year period is expected to be available contingent on the quality of proposals received and pending the availability of funds.

Estimated program budget, number of awards and average award size/duration are subject to the availability of funds.