Solicitations - Office of Incisive Analysis
Babel Program - BAA Questions
IARPA-BAA-11-02
Proposers' Day Date: January 20, 2011
BAA Release Date: April 07, 2011
BAA Question Period: April 07, 2011 - May 24, 2011
Proposal Due Date: June 07, 2011
| # | Question | Answer | Date Posted |
|---|---|---|---|
| 001 | Where can I find answers to some of my general questions about IARPA BAAs? | For answers to generic IARPA BAA questions, please see the Frequently Asked Questions (FAQs) page on the IARPA website www.iarpa.gov/faq.html. | 04/14/11 |
| 002 | Would it be possible to discuss a project with you that we are working on that might be responsive to this BAA? | The Program will not respond to requests for feedback regarding a proposal idea; the BAA has sufficient information for interested proposers to understand the range of problems and solutions that are of interest. See also FAQ P5. | 04/14/11 |
| 003 | Are there any restrictions that preclude foreign national participation in the Program? | See BAA Section 3.A. (Eligible Applicants). | 04/14/11 |
| 004 | Can funds for the Program be used to pay for postdoctoral or graduate students who are proposed as team members for the program? | Yes. | 04/14/11 |
| 005 | May a subcontractor be a member of more than one offering team? | Yes. See FAQ T1. | 04/22/11 |
| 006 | Are academic institutions that host UARC’s such as University of Maryland and Johns Hopkins University eligible to participate in the Babel program (assuming that such participation involves only non-UARC personnel)? | Academic institutions that host UARCs may be eligible to participate. See BAA Section 3: Eligibility Information and FAQs P1 and P3. | 04/22/11 |
| 007 | Are researchers who receive partial funding from an FFRDC, UARC, or other similar type of organization that has a special relationship with the Government eligible to submit a proposal? | There may be an Organizational Conflict of Interest given this scenario, and it is important to resolve this issue as early as possible before the proposal due date. See BAA Section 3.A.1. (Procurement Integrity, Standards of Conduct, Ethical Considerations and Organizational Conflicts of Interest (OCI)) and FAQs P1 and P3. | 04/22/11 |
| 008 | What type of procurement contract will be allowed under this BAA? | The BAA does not specify the type of procurement contract an offeror must propose. | 04/22/11 |
| 009 | Must offerors propose approaches that use speech recognition methods? | The technology proposed must address the elements in Section 1.A for the key technical challenges of the Program, Section 1.B.1 for Program milestones, and Section 1.B.2, for Program waypoints, but the precise methods are at the discretion of the offeror. | 04/28/11 |
| 010 | Will there be "time-stamps" on transcriptions? | Yes. | 04/28/11 |
| 011 | On page 29, Section 4.D., the BAA states, "Travel should be limited to at most two conferences per year, in addition to mandatory site visits and PI meetings." Can you clarify whether there is a restriction on how many attendees per team may be funded to go to the two research conferences? | The number of attendees per team will depend on the composition of the team. There is no explicit number; however, excessive travel will not be approved. | 04/28/11 |
| 012 | How do I submit a question for the Babel BAA? | See the bullet titled “Questions” on page 3 of the Babel BAA, as well as the e-mail address under Points of Contact in Section 7. | 04/28/11 |
| 013 | We plan to submit two proposals from the same prime organization; these proposals take a different approach and a different teaming strategy. Will the fact that the proposals are from the same prime organization be perceived as a deficiency? | Each proposal will be judged based on its own merits. Please note that BAA section 4.B.1 specifies “Interested parties are encouraged to leverage personnel that are dedicated to BAA requirements no less than 20% of their time. If any participant is scheduled for less than 20% of his/her time, the proposer will provide a clear and compelling justification as to how benefit can be gained from that person’s participation at the specified level of effort.” Note that an individual or organization associated with multiple teams must take care not to commit more resources than can be effectively applied to the program under the BAA to which they are proposing. | 05/11/11 |
| 014 | In the case of a proposal with a prime and several subcontractors, will the site visits involve only the prime, or will site visits also include subcontractor representatives? | Generally, a site visit will be to the prime contractor location. The prime may, with agreement from IARPA, hold the site visit at an alternative location. Participation of key personnel is expected, and representative(s) of each subcontractor is encouraged but is at the discretion of the prime. | 05/11/11 |
| 015 | Will there be multiple languages mixed in test data? Will there be code switching in the training and test data? | As stated in BAA Section 1.A.3, “Evaluations will be conducted on a language-by-language basis,” thus there will be no mixture of languages in test data. There may be code-switching and other common features of spontaneous conversational speech, as indicated in Section 1.A.1. | 05/11/11 |
| 016 | What functionality is required of the software at the end of each period? | See the performance goals in BAA Table 1 (Overview of the Program Targets by Program Period) and section 1.A.2 (Program Phases) constraints (see Table 1). | 05/11/11 |
| 017 | Is there a preference for large or small teams? | As stated in IARPA FAQ T2, “There is no preference for large or small teams, or teams led by large or small organizations.” | 05/11/11 |
| 018 | How much funding can an organization or institution receive from IARPA? | See FAQ B1 under Engagement Opportunities, www.iarpa.gov. There is no predetermined award size. | 05/11/11 |
| 019 | Is cost sharing required? | As stated in BAA Section 3.C, “Cost sharing is not required and is not an evaluation criterion. Cost sharing is encouraged where there is a reasonable probability of a potential commercial application related to the proposed research and development effort.” It should be noted that IARPA will not cover costs associated with the commercialization of technology. | 05/11/11 |
| 020 | It is clear that metadata will be provided for the training data but not for the evaluation data. But will metadata also be provided for the development data? | The evaluation data will not contain metadata, as stated in section 1.A.1 of the BAA, but all build packs, which includes training and development data, will contain metadata. | 05/16/11 |
| 021 | Can we collect data from additional languages to support our work? | Section 1.A.5 of the BAA states: "Methods of gathering additional data can also be explored within the scope of this Program with some limitations. The Babel Program will not fund the acquisition of new audio collections beyond those provided. However, additional data can be utilized so long as the offeror complies with Section 4.B.1 under "Data sources", 6.B.2, and 6.B.5, and provides a compelling rationale for using that data to meet Program goals". | 05/16/11 |
| 022 | Is it legitimate to use the web for the surprise language task? | Use of the web or other electronic resources is acceptable; however, please note, as stated Section 1.A.5., "all human intervention, including crowd-sourcing, is prohibited during all evaluations". | 05/16/11 |
| 023 | Will sites run the evaluation, or do we have to provide a system that someone else (e.g. at NIST or IARPA) runs? | Section 1.A.3 of the BAA states "Performers will be required to complete all evaluations at their site without any human intervention and within the time frame specified by NIST, and deliver evaluation output to NIST for scoring in the required format." | 05/16/11 |
| 024 | As to the proposal structure, is it best to just follow the exact format in the BAA Section 4.B.1., Section 3: Detailed Proposal Information? | Yes. Please follow the format and provide the information specified in BAA Section 4.B., Proposal Content Specifics. | 05/16/11 |
| 025 | Will “term match” in the keyword search be defined as in the NIST 2006 STD evaluation, that is as an exact orthographic match at the word level, or will it be modified to account for the morphological structure of the target language? | As stated in section 1.A.3. of the BAA, “During the evaluation period, NIST will provide performers with an evaluation pack for the keyword search evaluation for each development language, together with a set of orthographic terms to search on.” Further details regarding the evaluation will be provided after program start, as stated in section 1.A.3, “NIST will provide the evaluation schedule, the evaluation procedure guidelines (e.g., www.itl.nist.gov/iad/mig/tests/std/2006/docs/std06-evalplan-v10.pdf), submission requirements (e.g., rules of participation, formats for system output to enable scoring, and scoring methods), and evaluation packs to performers at the appropriate times in each year of the program.” Note however that participants are required to provide additional metrics and analyses, as stated in Section 1.A.4 “To support analysis, offerors shall propose and develop additional metrics for keyword search, as well as other offeror-defined metrics, including transcription error. Offerors are responsible for developing and justifying their own scoring methods for new metrics.” Hence, those researching an approach that is best evaluated using a method that accounts for the morphological structure of a language should propose additional scoring methods. | 05/16/11 |
| 026 | Our organization is part of a University that does not have an analytical accounting system for its actual indirect costs (overhead). Is it therefore possible to specify the overhead via a flat rate? | No. Under no circumstances can an unsubstantiated rate be applied to a Government contract. The Government must be able to determine that the proposed costs are fair and reasonable. | 05/16/11 |
| 027 | Partial academic year salary for faculty members is not an acceptable budget item for NSF grants. Is a percentage of academic year salary for faculty members involved in Babel a legitimate budget request? | Funds are provided for work done on the research program, both during the academic year and in the summer. As specified in BAA Section 4.B.1, “Participation by key personnel and significant contributors is expected to exceed 20% of their time.” | 05/16/11 |
| 028 | For keyword search, will performers be given citation forms and expected to also detect inflected forms, or will we only have to detect the exact word specified? | See Babel Q&A 025. | 05/23/11 |
| 029 | For languages whose native scripts do not habitually use spaces, will these be introduced in IARPA data? For languages that do not mark short vowels, will appropriate diacritics be introduced? | Please read BAA Section 1.A.1 Government Furnished Information (GFI). Transcriptions will be provided in a normalized orthography. | 05/23/11 |
| 030 | Will the language packs be ITAR (international Traffic in Arms Regulations) restricted? | Our intention is to utilize language packs that have no ITAR restrictions. | 05/23/11 |
| 031 | Our university has an off-campus research facility that is a UARC. Are we still eligible to submit to Babel? | See Babel Q&A 006. Please note that Organizational Conflicts of Interest (OCIs) are based on the affiliations of your project personnel. | 05/23/11 |
| 032 | For the surprise language, will the identity of the language be provided? | As specified in Section 1.A.1, language build packs will “contain a description of the language, including limited information on the dialectal variation in the collection, and the phoneme set for the language.” The language build pack for each surprise language will be provided at the start of the surprise language evaluation, as shown in Figure 1. | 05/23/11 |
| 033 | In Section 4.B.2 of the BAA, item 2 in "Section 2: Detailed Estimated Cost" requests "Major Program tasks by fiscal year". Does this refer to US Government Fiscal Year? | Yes. | 05/31/11 |
