Saturday, December 23, 2006
USGS internal review and censorship
Appointees of the Bush Administration have been accused of censoring and distorting data that was incompatible with their ideological views. George Deutsch a press officer at NASA who wanted the word 'theory' inserted after The Big Bang (insisting it was a religious view) and tried to keep reporters away from NASA scientist Jim Hansen is an example.
I'm a little late with this but earlier this week I read a post at Red State Rabble about the internal review policy of the U.S. Geological Survey (see here and here). The articles I linked to suggest that the USGS is instituting a policy to censor data that aren't compatible with Bush Administration policies. While I don't think the present administration has behaved very responsibly when it comes to handling data I don't think the USGS internal review policy is being installed to stifle scientists.
There has been a very interesting discussion of this policy on the listserv of the Affiliation of Christian Geologists (you'll need to set up an account in order to access the archive). Several current USGS employees have contributed to that discussion. I've reproduced one of the responses, by Peter Haeussler, who works for the USGS in Anchorage, Alaska below:
I spent a couple years working with the Earthquake Hazards Team at the USGS (the team I was part of was in the GD). I actually thought that the internal review process was USGS-wide and not just for GD (although commenter mark on the RSR thread says the same procedure is used in WRD). I've been through the internal review process both as an author and as a reviewer, and I've found it very helpful. Its main purpose is to make sure that publications by USGS authors are of a high quality, not to hinder or censor data. The process is pretty straightforward. The author picks two reviewers (Haeussler says that they don't have to be USGS employees, which is news to me – all of the internal reviewers in the team I worked on were either USGS employees or volunteers, although we did hit up visiting scientists too). The interval review is just like standard peer review for a journal. The reviewer goes over the document and makes sure the data support the conclusions, that the format of the paper is decent, etc. The author then revises the manuscript and submits it to the Team Chief Scientist, who signs off on it. The TCS looks at the reviewers comments and the revised manuscript and if the reviewers' comments have been addressed the manuscript is approved (although the review by the TCS can be cursory).
This process applies not only to papers but also to abstracts and newspaper articles. The review is pretty quick for abstracts, it usually doesn't take more than a day. As you can probably imagine, the days leading up to the AGU Fall Meeting abstract submission deadline are usually pretty busy. The review process for a paper is longer. It usually took a week or two, but it did take me a couple of months to get a paper back from a reviewer in one case (I expected that going in since he was really busy).
The USGS internal review policy is available online here and here (I should note that the date on these links is May 2006, so they might not be the final version). I've reproduced the section that deals with sensitive information below. From the second link:
I don't see anything particularly ominous in this policy. I can understand why people would be concerned since scientists have been censored in the past, but I don't feel threatened by this policy. I hope I'm right.
I'm going to end this post by looking in a case that was mentioned by both Peter Haeussler by CANOE. Here's the relevant quote from the CANOE article:
I think it's hard to characterize that paragraph as anything other than misleading. Here's a really nice article from the Washington Post from 2002.
Here is a link to the original report (section 3 is the relevant section) and here is a link to the follow up memo.
In 2002 a group headed by a USGS scientist published a report that dealt in part with the potential impact of the development of ANWR. This group had studied the caribou herds in that part of Alaska for 12 years and one of their findings was that female caribou who were either close to giving birth or had young calves tended to avoid roads and pipelines (they apparently preferred to be at least 4 km away from those features). That study reported that full development of 1002 area of ANWR (the area where the proposed drilling would occur) was likely to cause mortality rates of calves in the local caribou herd to decrease by 8.2%, and that a reduction of 4.6% was sufficient to cause growth of the herd to cease (Figure 3.28 in their report shows their data). The reason this caused a problem is that the scenario they used to arrive at their 8.2% calculation wasn't necessarily realistic since it was based on all of the 1002 area being developed and that wasn't necessarily realistic. Some development scenarios had plans with a much smaller footprint (you can see a map of all the plans in the follow up memo). The authors of the original study were taken to task for not including these plans in their report. The follow up memo included the predicted mortalities based on 2 more realistic drilling plans (the mortalities were estimated using exactly the same procedures as in the original report). The predicted mortalities for those two models were 0.7 and 1.2 %. The authors of the original report had in fact included models that didn't required full development (again you can see a map of their scenarios in the follow up memo), but they just focused on the 8.2 % figure (the worst case scenario). The USGS is not supposed to set or advocate policy although USGS authors can evaluate proposed scenarios, but they can't pick a favorite. The 2002 report crossed that line.
In addition both the original 2002 report and the follow up memo are available on line so I think it's difficult to claim that the authors were censored.
I'm a little late with this but earlier this week I read a post at Red State Rabble about the internal review policy of the U.S. Geological Survey (see here and here). The articles I linked to suggest that the USGS is instituting a policy to censor data that aren't compatible with Bush Administration policies. While I don't think the present administration has behaved very responsibly when it comes to handling data I don't think the USGS internal review policy is being installed to stifle scientists.
There has been a very interesting discussion of this policy on the listserv of the Affiliation of Christian Geologists (you'll need to set up an account in order to access the archive). Several current USGS employees have contributed to that discussion. I've reproduced one of the responses, by Peter Haeussler, who works for the USGS in Anchorage, Alaska below:
First, a history lesson on the different divisions of the USGS. There's the NMD - national mapping division (makes maps), the WRD - Water Resources Division (does water stuff), the GD - Geologic division (geologic mapping, energy, minerals, earthquakes, volcanos, climate change, etc), and now the BRD - Biological Resources Division (the biology folks pulled out of Fish and Wildlife Service, NPS, and perhaps BLM in the early days of the Clinton administration under Bruce Babbit as Secretary of the Interior. Babbit wanted to start an organization equivalent to the USGS for biology folks, but Newt Gingrich and the contract on America didn't give the new organization the organic act, so it never started. In the meantime, these scientists had no home, so, once it was clear the USGS was not to be eliminated under Newt's scheme, the director of the USGS offered to make a new division of the USGS for the biology folks. and thus was born the BRD part of the USGS.)
Here's how GD scientists publish in non USGS publications. Perhaps unbeknownst to most non-USGS folks, the GD geologists have always had to get two "internal" reviews of their publications before publishing in any outside journal. (these days, this includes abstracts. Also, neither reviewer has to be from the USGS) After you respond to those reviews, you make your corrections, and give the whole package to your immediate supervisor. (If you use geologic names in the paper, then there is an additional review that makes sure that geologic names are used properly) The supervisor should look it over, see that appropriate responses are made, then 'sign off' on the manuscript. Then it gets sent to a publications group chief, who then says it gets 'directors approval'. Then, you are officially given permission to submit the manuscript wherever. It does slow down the publication process, but invariably these interval reviews improve the manuscript and the content. The goal is to make sure the USGS is not publishing crap. You might be surprised to learn that there are a manuscripts that die after these reviews. It's a big no no to publish without internal review. You'd get spanked. The internal reviewers are chosen by the author of the paper and not by a supervisor or some political appointee. In a smaller office like the one I'm in, you can end up using your same friend to review manuscripts too many times. And avoiding certain people who have terminal indigestion. And you can end up having rubber stamp reviews. But it's OK.
I think the 'new' policy (I've not read it, but have been told) basically implements the GD internal review policy across all divisions of the USGS. I've heard the BRD scientists in my office (actually they are all located across town and we don't interact with them much) complain about this new policy. They see this as interfering. GD folks don't think this is an unnecessary burden. However, there have been instances of BRD people not having appropriate peer review when publishing things. There was one BRD pub that clearly had a biased slant toward caribou-pipeline interactions. The USGS really gets spanked in Washington when stuff like this happens. It's bad for everyone.
It's funny Charles mentioned this. I saw Al Gore speak at lunch today at AGU. Gore actually mentioned this press report and the quote from James Estes, and Gore said he was horrified of the censorship. He mentioned the climate guy at NASA (Ames??) who had felt he had been censored as well. Gore's main point of the talk was that scientists need to tell the truth on what they know, even if it may be "an inconvenient truth." He pointed out that Galielo's conclusions about our solar system were also an inconvenient truth and that he published on it. It was important to get the truth out.
I think partly in response to the NASA guy (I saw him on 60 minutes), we at the USGS were recently given more guidance about how to express personal opinions in the course of conveying our scientific results to the public. I think this is excellent, and I think it makes it much easier to give a personal opinion (often with policy implications). I do a fair amount of earthquake hazards work, and I get these questions all the time. I appreciate more freedom to speak on policy. I just need to be clear to the audience what is my USGS science versus what is my opinion as a citizen.
I tried to find which division of the USGS that Estes is in from USGS internal web sites, but I can't find it easily. His research is all about charismatic megafauna, so I think it's likely he's in BRD and not GD. Nonetheless, Estes has published in Science and received meritorious service awards, so I doubt he would be slowed down much by the new rules.
Regarding the last couple of things mentioned: is this high visibility science? or is this policy sensitive? I've seen those checkboxes on our manuscript routing sheets. The high visibility question is merely for the press people in the USGS to play up the science. NASA does so well in part because they have such a good press machine. The USGS tries to emulate that. I do find the question about 'policy sensitive' more troubling. But I'm not aware of that impacting anything.
Regards, Peter
I spent a couple years working with the Earthquake Hazards Team at the USGS (the team I was part of was in the GD). I actually thought that the internal review process was USGS-wide and not just for GD (although commenter mark on the RSR thread says the same procedure is used in WRD). I've been through the internal review process both as an author and as a reviewer, and I've found it very helpful. Its main purpose is to make sure that publications by USGS authors are of a high quality, not to hinder or censor data. The process is pretty straightforward. The author picks two reviewers (Haeussler says that they don't have to be USGS employees, which is news to me – all of the internal reviewers in the team I worked on were either USGS employees or volunteers, although we did hit up visiting scientists too). The interval review is just like standard peer review for a journal. The reviewer goes over the document and makes sure the data support the conclusions, that the format of the paper is decent, etc. The author then revises the manuscript and submits it to the Team Chief Scientist, who signs off on it. The TCS looks at the reviewers comments and the revised manuscript and if the reviewers' comments have been addressed the manuscript is approved (although the review by the TCS can be cursory).
This process applies not only to papers but also to abstracts and newspaper articles. The review is pretty quick for abstracts, it usually doesn't take more than a day. As you can probably imagine, the days leading up to the AGU Fall Meeting abstract submission deadline are usually pretty busy. The review process for a paper is longer. It usually took a week or two, but it did take me a couple of months to get a paper back from a reviewer in one case (I expected that going in since he was really busy).
The USGS internal review policy is available online here and here (I should note that the date on these links is May 2006, so they might not be the final version). I've reproduced the section that deals with sensitive information below. From the second link:
4. Policy. All information products (see SM 1100.1) must be reviewed and approved for official release and dissemination, whether they are published by the USGS or an outside entity, if the work has been funded, whole or in part, by the USGS or if USGS affiliation is identified with the authorship. Processes for review and approval may differ for different kinds of information products (print, digital, or audiovisual) but must be consistently applied throughout the USGS.
If during preparation, review, or approval, an information product is determined to be of a particularly sensitive nature, the appropriate Bureau and Departmental officials should be consulted. Issues or concerns of a sensitive nature include those that have current or future policy implications (such as land and resource management decisions) or that involve matters of national interest, security, or potential commercial gain. Additionally, coordination and review should include the appropriate Office of Communications representative in instances of policy-sensitive products or those that may be of such a level of visibility that a communication strategy is warranted to handle their release.
Information products should not recommend or appear to advocate or prescribe a particular public policy or course of action. Information products carry no disclaimer that could suggest that the product does not meet USGS standards of scientific excellence, integrity, and objectivity.
I don't see anything particularly ominous in this policy. I can understand why people would be concerned since scientists have been censored in the past, but I don't feel threatened by this policy. I hope I'm right.
I'm going to end this post by looking in a case that was mentioned by both Peter Haeussler by CANOE. Here's the relevant quote from the CANOE article:
President George W. Bush's administration has been criticized for scientific integrity issues. In 2002, the USGS was forced to reverse course after warning oil and gas drilling in Alaska's Arctic National Wildlife Refuge would harm the Porcupine caribou herd. One week later a new report followed, this time saying the caribou would not be affected.
I think it's hard to characterize that paragraph as anything other than misleading. Here's a really nice article from the Washington Post from 2002.
Here is a link to the original report (section 3 is the relevant section) and here is a link to the follow up memo.
In 2002 a group headed by a USGS scientist published a report that dealt in part with the potential impact of the development of ANWR. This group had studied the caribou herds in that part of Alaska for 12 years and one of their findings was that female caribou who were either close to giving birth or had young calves tended to avoid roads and pipelines (they apparently preferred to be at least 4 km away from those features). That study reported that full development of 1002 area of ANWR (the area where the proposed drilling would occur) was likely to cause mortality rates of calves in the local caribou herd to decrease by 8.2%, and that a reduction of 4.6% was sufficient to cause growth of the herd to cease (Figure 3.28 in their report shows their data). The reason this caused a problem is that the scenario they used to arrive at their 8.2% calculation wasn't necessarily realistic since it was based on all of the 1002 area being developed and that wasn't necessarily realistic. Some development scenarios had plans with a much smaller footprint (you can see a map of all the plans in the follow up memo). The authors of the original study were taken to task for not including these plans in their report. The follow up memo included the predicted mortalities based on 2 more realistic drilling plans (the mortalities were estimated using exactly the same procedures as in the original report). The predicted mortalities for those two models were 0.7 and 1.2 %. The authors of the original report had in fact included models that didn't required full development (again you can see a map of their scenarios in the follow up memo), but they just focused on the 8.2 % figure (the worst case scenario). The USGS is not supposed to set or advocate policy although USGS authors can evaluate proposed scenarios, but they can't pick a favorite. The 2002 report crossed that line.
In addition both the original 2002 report and the follow up memo are available on line so I think it's difficult to claim that the authors were censored.
Comments:
<< Home
There is a new need for clarification. The new review process in USGS can slow up the process of publishing research and submitting reports to funding agnecies.
From what I have observed, the author submits the manuscript to his/her supervisor along with up to 5 suggested reviewers. The supervisor then selects three from this list or others; when comments are returned, the original author must address them all. This constitutes anonymous peer review.
However, once that step is completed, the manuscript is then submitted to a journal to undergo the same process (2-3 different anonymous reviewers). It is now a two-step process, not including any manuscripts flagged as "sensitive" (deal with policy), which may undergo separate review at higher levels.
Maybe this seems smooth and easy, yet if an ambitious author submits 6 papers in one year, then that is 3 + 2-3 reviewers per manuscript = 30-36 reviews (for 6 papers).
This process is also required of reports. For example, if that ambitious scientist submits 5 reports to other agences, add in another 25-30 reviews for the review.
Delivery of "timely information" is a stated objective of the agency, and highly desired. A more streamlined system appears in order.
Scientists in USGS do not oppose the anonymous peer review system; they were weaned on it (grad school onward) and many serve as editors or reviewers for journals. The pace of conducting research and publishing results is increasing at a frantic pace.
Those scientists who have been submitted lots of manuscripts and reports are being ham-strung by the new process. One response may be to reduce the number of papers and reports per year. That does not seem like a way to utilize the time of your most productive scientists or serve other agencies and science in general.
This is offered for information on the process.
From what I have observed, the author submits the manuscript to his/her supervisor along with up to 5 suggested reviewers. The supervisor then selects three from this list or others; when comments are returned, the original author must address them all. This constitutes anonymous peer review.
However, once that step is completed, the manuscript is then submitted to a journal to undergo the same process (2-3 different anonymous reviewers). It is now a two-step process, not including any manuscripts flagged as "sensitive" (deal with policy), which may undergo separate review at higher levels.
Maybe this seems smooth and easy, yet if an ambitious author submits 6 papers in one year, then that is 3 + 2-3 reviewers per manuscript = 30-36 reviews (for 6 papers).
This process is also required of reports. For example, if that ambitious scientist submits 5 reports to other agences, add in another 25-30 reviews for the review.
Delivery of "timely information" is a stated objective of the agency, and highly desired. A more streamlined system appears in order.
Scientists in USGS do not oppose the anonymous peer review system; they were weaned on it (grad school onward) and many serve as editors or reviewers for journals. The pace of conducting research and publishing results is increasing at a frantic pace.
Those scientists who have been submitted lots of manuscripts and reports are being ham-strung by the new process. One response may be to reduce the number of papers and reports per year. That does not seem like a way to utilize the time of your most productive scientists or serve other agencies and science in general.
This is offered for information on the process.
Thanks for the comment. The new review process doesn't sound terribly different from the old one, the only difference seems to be that the supervisor selects the reviewers from a list provided by the author.
Internal review certainly does place a burden on folks at the USGS, but I also don't think the internal review procedure really slows down publication all that much. The internal review process in my experience has been much faster than journal peer review. Most of the papers I internally reviewed (and had internally reviewed) took less than a week. The turn around on abstracts was less than a day. I'm curious about how abstracts were reviewed in your experience. Did a supervisor have to pick the reviewers for them too? In that case I can certainly see how that would hinder publication since in the team I worked with most abstracts were internally reviewed and submitted the day before, or the day of, the abstract submission deadline.
Of course none of the papers I submitted or reviewed were on really sensitive topics, so I don't have any personal experience there.
The internal review process didn't greatly hinder publication of time sensitive material either. For example following a notable earthquake like the 2004 Parkfield or the 2004 Sumatra earthquake quite a few publications came out within a year (i.e., they went through both internal review and journal review). A turn around time of a year is not bad at all.
I'm sure that there have been (and will be) times when the internal review process unreasonably delays publication, but I don't think that's going to be a regular event.
Internal review certainly does place a burden on folks at the USGS, but I also don't think the internal review procedure really slows down publication all that much. The internal review process in my experience has been much faster than journal peer review. Most of the papers I internally reviewed (and had internally reviewed) took less than a week. The turn around on abstracts was less than a day. I'm curious about how abstracts were reviewed in your experience. Did a supervisor have to pick the reviewers for them too? In that case I can certainly see how that would hinder publication since in the team I worked with most abstracts were internally reviewed and submitted the day before, or the day of, the abstract submission deadline.
Of course none of the papers I submitted or reviewed were on really sensitive topics, so I don't have any personal experience there.
The internal review process didn't greatly hinder publication of time sensitive material either. For example following a notable earthquake like the 2004 Parkfield or the 2004 Sumatra earthquake quite a few publications came out within a year (i.e., they went through both internal review and journal review). A turn around time of a year is not bad at all.
I'm sure that there have been (and will be) times when the internal review process unreasonably delays publication, but I don't think that's going to be a regular event.
I've worked with a USGS-WRD cooperating agency for a number of years, and everything that was authored by a USGSer or was published by USGS (including Open-File Reports) required review. Overall, that has been a good thing, although there were occasional problems and delays. I currently have a report that sat on a USGS desk gathering dust for about 8 months before coming back to me. The question of who would be the reviewers always seemed to depend on who (with expertise in the area) was available, and has usually been open to suggestion. After colleague reviews, the report would always go to Headquarters for final review (checking things like appropriate report series, geologic names, consistency with USGS format of illustrations and tables, no obscene graffiti visible in photos, etc. We were always admonished to avoid advocating policy decisions (saying things like "must" or "ought") and to just present the science. Most of the requirements can be found in the USGS publication "Suggestions to Authors."
Hi All:
In my earlier post to the obscure ACG listserve, which was reproduced in this more public forum, I wrote, "...However, there have been instances of BRD people not having appropriate peer review when publishing things. There was one BRD pub that clearly had a biased slant toward caribou-pipeline interactions."
I was recently made aware that the said report (A) DID have full internal review, and (B) was NOT biased. I based my comments on the media representation of the report, and I was not fully informed. I apologize for misleading readers in this way and for mischaracterizing the report.
Regards, Peter
Post a Comment
In my earlier post to the obscure ACG listserve, which was reproduced in this more public forum, I wrote, "...However, there have been instances of BRD people not having appropriate peer review when publishing things. There was one BRD pub that clearly had a biased slant toward caribou-pipeline interactions."
I was recently made aware that the said report (A) DID have full internal review, and (B) was NOT biased. I based my comments on the media representation of the report, and I was not fully informed. I apologize for misleading readers in this way and for mischaracterizing the report.
Regards, Peter
<< Home