IARPA has just announced how they will be tackling this pursuit.
FOR IMMEDIATE RELEASE ODNI News Release No. 28-11
September 27, 2011
IARPA LAUNCHES NEW PROGRAM TO ENABLE THE RAPID DISCOVERY OF EMERGING TECHNICAL CAPABILITIES
WASHINGTON – The Intelligence Advanced Research Projects Activity (IARPA), within the Office of the Director of National Intelligence (ODNI), announced today that it has embarked on a multi-year research effort to produce a new capability to accelerate the process of identifying and prioritizing emerging technologies across the globe. The Foresight and Understanding from Scientific Exposition (FUSE) Program seeks to develop automated methods that aid in the systematic, continuous, and comprehensive assessment of technical emergence using publicly available information found in published scientific, technical and patent literature.“Identifying what the technical trends and connections are, before the capability is fully developed, is critical to our mission of maintaining an intelligence advantage,” said Dewey Murdick, FUSE program manager at IARPA.
Today, the identification and assessment of emerging technical capabilities is a manual, time-consuming, domain-specific, and expert-intensive process. Analysts are in need of a reliable, automated, evidence-based capability that allows them to rapidly identify specific technical areas for in-depth review. IARPA plans to create a capability that can nominate both known and novel technical areas based on quantified indications and evidence of technical emergence.
“The globalization of science and technology means that capabilities can emerge from diverse technical areas anywhere in the world,” added Murdick. “We want to provide an evidence-based technical capability that enhances our competitiveness in a rapidly changing digital and technical age. In FUSE we are partnering with the scientific community to develop validated indicators and theories of technical emergence detection.” The FUSE Program is designed to provide the analytic workforce with a cutting edge capability to sustain technical vigilance across multiple disciplines and languages. Through a competitive Broad Agency Announcement (BAA) process, IARPA has awarded research contracts in support of FUSE to teams led by BAE Systems, Columbia University, Raytheon BBN Technologies Corporation, and SRI International.
IARPA invests in high-risk, high-payoff programs that have the potential to provide our nation with an overwhelming intelligence advantage over our future adversaries. For media inquiries, please contact dni-iarpapublicaffairs@ugov.gov.
We offer our congratulations to BAE Systems, Columbia University, Raytheon BBN Technologies Corporation and SRI International. We know many leaders in those four great organizations and are proud to say some of the greatest thinkers in the world are associated with those firms. We all look forward to progress. We can also make a few guesses here about what they are up to. I imagine every one has concluded they cannot automate this approach with old legacy technologies. Each is very likely working on ways to apply the right stack of new “Big Data” approaches like Apache Hadoop and the many capabilities that surround that core framework. If these firms have not already called on the experts at Cloudera I expect them to be doing that right now. By leveraging an integrated stack of ready to go capabilities in the Cloudera Distribution including Apache Hadoop (CDH) they will be able to field capabilities rapidly. In fact, if they really want to move fast they will buy optimized hardware and services from Dell, for reasons we point out here in our post on: The Quickest Way To Deploy A Well Engineered Apache Hadoop Solution To A Production Environment.
Looking for more on what will be accomplished? Here is info from the first synopsis:
SYNOPSIS
Technical emergence refers to the process whereby innovative ideas, capabilities, applications, and even entirely new fields of study arise, are tested, mature, and, if conditions are favorable, make a significant impact. Those able to “scan the horizon” for the early signs of technical emergence, and take advantage of the resulting capabilities and applications, can gain a significant competitive edge.
Today, the identification and assessment of emerging technical capabilities is a time-consuming, domain-specific, and expert-intensive process. This demanding process is often carried out under severe time constraints on either too much or too little data, with limited reproducible auditing and bias controls, and with limited systematic validation against real world activities. Furthermore, the increasing globalization of science and technology raises the potential for high-impact technical capabilities to emerge in increasingly diverse technical, socio-economic, and geographic areas.
Analysts and Subject-Matter Experts (SMEs) need a reliable, evidence-based capability that allows them to dramatically accelerate the horizon-scanning process and reduce the labor involved to identify specific technical areas for in-depth review. It is essential that this automated capability can nominate both known and novel technical areas based on quantified indications of technical emergence with sufficient supporting evidence and arguments for that nomination.
In FUSE, the real-world concept of a scientific or technical area or domain of inquiry will be represented by a Related Document Group (RDG). This set of documents serves as a proxy for one or more related technical concepts, capabilities, applications, fields of study, etc., that have developed over the span of time defined by the documents’ publication dates. An RDG can represent a narrowly circumscribed topic or domain of technical activity (e.g., support vector machines applied to face recognition), or something more general (e.g., artificial intelligence or computer science) or more exotic, such as a body of multi-disciplinary scientific work sharing both methods and applications. The sizes of RDGs will range from a few tens of documents to many thousands of documents.
The FUSE Program seeks to develop automated methods that aid in the systematic, continuous, and comprehensive assessment of technical emergence using information found in the published scientific, technical, and patent literature. FUSE envisions a system that could (1) process the massive, multi-discipline, growing, noisy, and multilingual body of full-text scientific, technical, and patent literature from around the world; (2) automatically generate and prioritize RDGs, nominate those that exhibit technical emergence, and provide compelling evidence for that emergence; and (3) provide this capability for literatures in English and at least two non-English languages. The FUSE Program will also address the vital challenge of validating such a system, using real world data.
ADDITIONAL INFORMATION
Contracting Office Address:
Office of the Director of National Intelligence
Intelligence Advanced Research Projects Activity
Washington, District of Columbia 20511
United States
Primary Point of Contact:
Dewey Murdick
Program Manager
dni-iarpa-baa-10-06@ugov.gov
