At the ARDG meeting on Wednesday, EFF presented a position on the protection of peer review and peer reviewers. We proposed that one sentence be added to the ARDG's proposal solicitation document, but the ARDG refused to add it. Our position and proposal are included below.
Proposal"Submitters acknowledge that responses to this RFI may be subject to technical review and subjected to testing by independent third parties for the purpose of evaluating submitter claims about such technologies, and that the detailed results of such third party evaluations will be made public."
PositionIndependent, third-party evaluation and testing -- that is, peer review -- is essential to evaluating the accuracy of claims made by vendors about technologies. In response to this RFI, many technical claims will be made. Some of those claims will be factually incorrect, either because vendors desire to misrepresent the properties of technologies for commercial advantage or because they lack sufficient information to recognize why their claims are wrong. As our colleague Scott Craver mentioned in a presentation to the ARDG, peer review is the only way we know to arrive at the truth about empirical claims made for technologies.
Unfortunately, some researchers, such as Mr. Craver, have in the past been threatened with legal liability when they sought to research or to disclose the results of their research about the properties of technologies such as those sought by this RFI. This is unacceptible both from the point of view of free expression and scientific inquiry and from the point of view of determining scientifically, and in a public and transparent way, whether or how well technologies actually work for particular purposes.
Researchers need assurances that they will not be punished or harmed for carrying out their research. Without such assurances, a substantial amount of important independent research simply will not occur, to the detriment of public evaluation of vendor claims.
EFF would like the ARDG to consider how the RFI might be amended to provide such security for independent researchers.
Posted by Seth Schoen at July 31, 2003 03:57 PM