Jason Chaffetz

An update of US energy subsidies being prepared by the US Energy Information Administration, slated to be released on Tuesday, has been delayed.  Stephen Lacey, in a blog post at Climate Progress, notes that one source attributed the delay to  "quality assurance" issues. 

I noted back in April that the research mandate for the new study was problematic (my critique of EIA's last report is here).  The mandate issue pointed to a more important underlying principle for statistical agencies than simply how to report on energy subsidies.  At its core is the question of who gets to define the research agenda, and along what parameters. 

While it is important that organizations such as EIA help Congress evaluate options, they should not be in a position of having to meekly respond to a Congressional request verbatim.  The statistical agencies are funded by taxpayers not to provide fodder for a particular Congressional policy agenda (the lobbyists and trade associations already do a fine job there), but to provide unbiased analysis that helps answer broad questions on resource allocation and important societal patterns.  If a Congressional request skews how the statistical agencies are allowed to evaluate these key patterns, we risk seeing results that are more reflective of the scoping of the research task than of the actual problem being studied.

Thus, I view the delay for this particular report as a very positive signal.  If there are issues with scope of coverage, it is better they address them now than to come out with skewed coverage again.  I hope that in the future the holds won't be at the the time of release (when analytic modifications are both more difficult and more expensive).  Rather, they should be at the very beginning of the process, with the analytical staff working with Congressional requestors to refine their requests to ensure the statistical agencies have appropriate professional discretion to implement a robust and unbiased review.

My original information suggesting that Senator Lamar Alexander had put in a new request for EIA to update its work on energy subsidies turns out to have been incorrect.  The actual requestors this time were from the House side:  Congressmen Jason Chaffetz, Marsha Blackburn, and Roscoe Bartlett.  I'm grateful to the office of Congressman Chaffetz for providing a copy of the request they submitted to EIA in this matter.  That request is included below.

While not submitted by Alexander, the request is nonetheless almost identical to the letter Senator Alexander put in last time around.  Congressman Chaffetz' office said this was done purposefully so the data could be compared to earlier studies. In my view, replicating the old scoping is quite unfortunate.

While the comparability is certaintly a benefit, it is not inherently so.  First, as I documented last year there have already been substantial changes in EIA's methodology over time so the time series is already problematic.  Second, identical scoping means that the more troublesome of the assumptions and policy omissions in the 2007 study will be replicated in the 2011 release.  This is a far more important deficit than a time series between 2007 and 2011 is a benefit.  The Congressmen could just as easily have empowered a more robust research mandate for EIA, and asked the authors to highlight in a table or chapter how those changes affected the time series data.  A summary of how changes in assumptions, data, or other factors influence comparability from study-to-study would be a good thing for EIA to be including anyway.

Aside from the specific problems that using the Alexander research mandate will generate in understating the overall magnitude of energy subsidies and distorting their distribution across fuels, there are a number of broader policy issues that warrant consideration:

• Does EIA or any other statistical or oversight agency (e.g., GAO) of the federal government have the right to enter into a dialog with Congressional requestors over the scope of work they are being asked to do, or are they mere order-takers?  This would be important both in terms of resources required to complete the requested task, and with respect to the relevance and accuracy of the results that this expenditure of governmental time and taxpayer resources will generate.
• If EIA did have this right, but didn't exercise it, why not?  Clearly the subsidy study is widely cited and of interest to a much broader audience -- even within Congress -- than the three members who put in this particular request.
• When such a request for a non-routine analysis of broad interest to the Congress is placed, should there be a process by which EIA or other statistical agency can vet the scope to a somewhat wider audience -- at least within Congress -- before embarking on the research?  Such input at the early stages could result in a much more robust and useful product, but becomes increasingly difficult to integrate once research is well underway and a delivery time has been commited to. 
• Finally, if EIA did in fact work through the terms of reference with the requestors and determined the existing structure was appropriate, this too would be a problem.