{"id":2429,"date":"2021-08-11T13:59:34","date_gmt":"2021-08-11T13:59:34","guid":{"rendered":"https:\/\/blogs.sussex.ac.uk\/sussexenergygroup\/?p=2429"},"modified":"2024-01-31T14:32:42","modified_gmt":"2024-01-31T14:32:42","slug":"a-dialogue-on-examining-datasets-in-the-nuclear-vs-renewable-energy-debate","status":"publish","type":"post","link":"https:\/\/blogs.sussex.ac.uk\/sussexenergygroup\/2021\/08\/11\/a-dialogue-on-examining-datasets-in-the-nuclear-vs-renewable-energy-debate\/","title":{"rendered":"A dialogue on examining datasets in the nuclear vs renewable energy debate"},"content":{"rendered":"\n

Following the publication of their paper \u201c<\/em>Differences in carbon emissions reduction between countries pursuing renewable electricity versus nuclear power<\/em><\/a>\u201d <\/em>in Nature Energy<\/em> last October, Prof. Benjamin K Sovacool, Prof. Andy Stirling and their co-authors received a number of responses and challenges to the paper\u2019s findings.<\/p>\n\n\n\n

To advance scientific debate around independent research, they engaged in a series of dialogues with researchers offering critiques of our work. Below, they share an exchange with Daniel Perez<\/a>, PhD student at \u00c9cole Normale Sup\u00e9rieure in Paris.<\/p>\n\n\n\n

Mr Perez\u2019s paper, “<\/a>On Sovacool’s et al. study on the differences in carbon emissions reduction between countries pursuing renewable electricity versus nuclear power<\/a><\/em>“<\/a>, offers a critical perspective of Sovacool et al.\u2019s paper\u2019s models and statistical analysis.<\/p>\n\n\n\n

The exchange below begins with their response to Mr Perez\u2019s paper, followed by Mr Perez\u2019s response to theirs.<\/p>\n\n\n\n

By sharing the exchange here, Profs Sovacool and Stirling hope to encourage collegiate debate and support the critical importance of independent research, an issue considered in their earlier blog, Nuclear vs renewable energy and the critical importance of independent research<\/em><\/a>.<\/p>\n\n\n\n

Thanks to Mr Perez for his original response and for participating in this exchange.<\/p>\n\n\n\n

\n\n\n\n

Response to Daniel Perez\u2019s Matters Arising<\/strong><\/h2>\n\n\n\n

Benjamin K. Sovacool, Patrick Schmid, Andy Stirling, Goetz Walter & Gordon MacKerron<\/strong><\/p>\n\n\n\n

We thank Mr. Perez for engaging with our article. But we do not believe any of the concerns he raises are novel, nor do they hit the point on many aspects.<\/p>\n\n\n\n

First of all, if he had read thoroughly, Mr. Perez might have noticed that we never talk about greenhouse gases in their full generality. To say we use GDP to \u201cconfound something\u201d is a serious misrepresentation. We actually use GDP as a \u201ccontrol\u201d.<\/p>\n\n\n\n

Mr. Perez also seems to misunderstand us when he says: \u201cdespite the fact that decarbonated energy sources are not good predictors of GHG emissions\u201d and \u201cFossil fuels as the real predictor and the \u2018crowding out\u2019 hypothesis\u201d. Just as we never address GHG in their entirety, so we never claim that clean energy sources are a \u201cpredictor\u201d of CO2 emissions. Ours is not a predictive but a correlative study. The reader might wonder why Mr. Perez puts so much emphasis on such obvious red herrings.<\/p>\n\n\n\n

With respect to Mr. Perez\u2019s point that the crowding out hypothesis is not surprising at all (since \u201crenewables and nuclear power are structurally incompatible, so there is an anti-correlation between them\u201d), we would note that he is directly endorsing (without duly emphasized acknowledgement) one of the most crucial findings of our paper.<\/p>\n\n\n\n

And to be clear, we do not emphasize mere theoretical properties of random variables<\/em>, which need opaque assumptions and are devoid of empirical data. That Mr. Perez states on such an ostensibly precise theoretical basis, \u201clittle to no surprise\u201d, detracts from his idiom of precision. It raises the question: is it really no surprise<\/em> or is there something<\/em> to be investigated? It is the empirical findings we obtain \u2013 together with our qualifications \u2013 that strike us without doubt as being something to be investigated.<\/p>\n\n\n\n

With respect to the important role played by hydroelectricity in the earlier period we examine, Perez again deploys a misleading polemic. Why should this unavoidable empirical reality be treated as if it were somehow a deficiency of our study? The relative importance of hydroelectricity in the early stages of renewables uptake is simply a reflection of the established historical trajectory in renewable development. In later stages, the effects we document in this regard become much more influenced by wind and solar. With all these sources anyhow counting as \u2018renewable\u2019, why would this count as a \u2018flaw\u2019.<\/p>\n\n\n\n

With respect to timeframes, the question raised is (as we acknowledge) about nuanced differences of approach, not about \u201cmistakes\u201d. We are ourselves clear that there are multiple things to consider on this issue. This is exactly why we have chosen a robust data-averaging approach with several triangulation procedures. Together with our openness to the many conditionalities, this is the way to properly address uncertainties and ambiguities that are unavoidable in this kind of research. If Mr. Perez really wants to claim that there exists just one single definitive approach to this complexity, then he is arguably reproducing the kind of technocratic authoritarianism that has led for so long to the neglect of the kinds of questions we are raising.<\/p>\n\n\n\n

As to Mr. Perez\u2019s argument that \u201cyou do not have stationarity\u201d and \u201cyou need stationarity for time series analysis\u201d, we agree. But this is again a strangely misleading point. It is this need for stationarity in time series approaches that constitutes a key reason why we do not adopt such an approach.<\/p>\n\n\n\n

Here, the argument that we should have used panel data and that our analysis is unduly time-averaged actually go together. While panel data analysis may be an alternative, we intentionally chose time averaging since this procedure enables more robust statements to be made in the context of random variables (the underlying modelling for statistics). Such approaches are often used as a mean to average nuisance contributions in an environment with a presence of many influencing factors which clearly is our case at hand.<\/p>\n\n\n\n

We choose the stated time lag without involvement of a second category of assumptions that would not compellingly fit the purpose of an initial pioneering study. As we explain, the indicated time lag was chosen to optimally use the data set. Otherwise, we might have disregarded precious data points which would then in turn have raised the objection that we intentionally and deliberately used only some parts of the data, but not all of it, thereby wasting parts of the available data set. Crucial here, is that we still have to consider a directional effect since power plants typically involve a lot of down-stream processes (such as maintenance) that stretch over time but need to be clearly attributed.<\/p>\n\n\n\n

To Mr. Perez\u2019s statement that \u201cdistribution of the residuals is not exactly normal\u201d, we respond that any expert should be aware that any test of assumptions only gives hints for acceptance within defined error intervals. When invoking statistical pretests, all issues surrounding statistical tests, like \u201cfalse positives\u201d, power and efficiency of tests, have to be mentioned.<\/p>\n\n\n\n

On a further technical point, Mr. Perez refers to \u201cconfounding variable with a power law and not just a linear model\u201d. But there is no part of our work that relies on identifying a \u201cbest fit\u201d curve. This would be difficult to motivate from a theoretical perspective \u2013 for example: why squared or the root. We are not aiming to build a \u201ccausal model\u201d. We never claim to do so. Why does Perez imply otherwise?<\/p>\n\n\n\n

In similar vein, Mr. Perez makes statements about the \u201cpredictive power\u201d of our model that compound a further diversion with a misquote. What we actually said was \u201cCrucially, renewable energy strategies are, to an evidently noteworthy degree, associated with lower levels of national carbon emissions\u201d. This is not an attribution of causality. Whether one might have chosen a different analytical approach is a moot point that we acknowledge. But all methods hold pros and cons. Oddly for someone so focused on precision, Mr. Perez does not demonstrate that alternatives do not display their own more serious specific disadvantages.<\/p>\n\n\n\n

With regard to Mr. Perez\u2019s statement that the original data would yield a \u201cbias\u201d: we are not adjusting\/distorting the original data, we analyze it simply it as it is. His slurs about \u201cthe poor study of the data set\u201d and \u201csuboptimal modeling\u201d can be qualified in light of our response to his other misleading language addressed above.<\/p>\n\n\n\n

Multivariate linear regression is actually quite robust with respect to its assumptions. What is most crucial here is that it was not our aim in this pioneering study to test any particular model versus another as a candidate for an \u201coptimal fit\u201d. What we are instead doing, is investigating prevailing understandings of the form \u201cthe more \u2026 energy, the less emissions\u201d. So our methodology stands in this regard.  Given that the associated issues are so prominent and so high stakes, it is remarkable that our research question has not been posed before.<\/p>\n\n\n\n

In conclusion, we would urge that the reader cut through the many technicalities to see the underlying picture. Our study asks a very basic empirical question. We do not claim to have answered this definitively, but merely pointed to the significant implications and the grounds for further research. Our findings remain valid and salient.<\/p>\n\n\n\n

\n\n\n\n

Response to Sovacool et al.\u2019s response<\/strong><\/h2>\n\n\n\n

Daniel Perez<\/strong><\/p>\n\n\n\n

Benjamin K Sovacool et al.:<\/em><\/strong> We thank Mr Perez for engaging with our article. But we do not believe any of the concerns he raises are novel, nor do they hit the point on many aspects.<\/em><\/p>\n\n\n\n

First of all, if he had read thoroughly, Mr Perez might have noticed that we never talk about greenhouse gases in their full generality. To say we use GDP to \u201cconfound something\u201d is a serious misrepresentation. We actually use GDP as a \u201ccontrol\u201d.<\/em><\/p>\n\n\n\n

Mr Perez also seems to misunderstand us when he says: \u201cdespite the fact that decarbonated energy sources are not good predictors of GHG emissions\u201d and \u201cFossil fuels as the real predictor and the \u2018crowding out\u2019 hypothesis\u201d. Just as we never address GHG in their entirety, so we never claim that clean energy sources are a \u201cpredictor\u201d of CO2 emissions. Ours is not a predictive but a correlative study. The reader might wonder why Mr Perez puts so much emphasis on such obvious red herrings.<\/em><\/p>\n\n\n\n

Daniel Perez<\/strong>: The terms \u201cconfounding variable\u201d and \u201cpredictors\u201d are widespread and well-known concepts in statistics. Both of these terms are standard terminology in the context of regression analysis, as can be corroborated by looking at any statistics textbook. It\u2019s in the statistical sense that the correlative study made in Sovacool et al.\u2019s paper explicitly uses both nuclear and renewables as predictors of GHG emissions.<\/p>\n\n\n\n

Sovacool et al.:<\/em><\/strong> With respect to Mr Perez\u2019s point that the crowding out hypothesis is not surprising at all (since \u201crenewables and nuclear power are structurally incompatible, so there is an anti-correlation between them\u201d), we would note that he is directly endorsing (without duly emphasized acknowledgement) one of the most crucial findings of our paper.<\/em><\/p>\n\n\n\n

Perez:<\/strong> This is a misquote, we were simply explaining Sovacool et al.\u2019s reasoning. The full statement reads as follows: \u201cMoreover, the reasoning behind the \u201ccrowding out\u201d hypothesis is flawed. Indeed, the authors of [16] motivate the proposal of the \u201ccrowding out\u201d hypothesis as follows: Intermittent renewables require a decentralized electrical infrastructure as soon as they occupy a significant fraction of the electricity produced. By contrast, the optimal electrical infrastructure of non-intermittent power sources, such as fossil fuels, hydroelectricity and nuclear power is centralized [2]. The authors then suggest that, for these reasons, there should be an anticorrelation between R and N, which is the statement of the so-called \u201ccrowding out\u201d hypothesis. They back this statement by verifying that R and N are indeed anticorrelated and use this to justify their statements.\u201d It is clear that nowhere are we agreeing with their conclusions, but rather just explaining the reasoning proposed by Sovacool et al. as to their proposal of the \u201ccrowding out\u201d hypothesis.<\/p>\n\n\n\n

Sovacool et al.:<\/em><\/strong> And to be clear, we do not emphasize mere theoretical properties of random variables, which need opaque assumptions and are devoid of empirical data. That Mr Perez states on such an ostensibly precise theoretical basis, \u201clittle to no surprise\u201d, detracts from his idiom of precision. It raises the question: is it really no surprise or is there something to be investigated? It is the empirical findings we obtain \u2013 together with our qualifications \u2013 that strike us without doubt as being something to be investigated.<\/em><\/p>\n\n\n\n

Perez:<\/strong> Whether Sovacool et al. were aware of their emphasis on a phenomenon arising when studying fractions of a same whole in a regression analysis is irrelevant in the demonstration that their \u201cfindings\u201d are mere artefacts of this fact, as clearly demonstrated in our work.<\/p>\n\n\n\n

Sovacool et al.:<\/em><\/strong> With respect to the important role played by hydroelectricity in the earlier period we examine, Perez again deploys a misleading polemic. Why should this unavoidable empirical reality be treated as if it were somehow a deficiency of our study? The relative importance of hydroelectricity in the early stages of renewables uptake is simply a reflection of the established historical trajectory in renewable development. In later stages, the effects we document in this regard become much more influenced by wind and solar. With all these sources anyhow counting as \u2018renewable\u2019, why would this count as a \u2018flaw\u2019.<\/em><\/p>\n\n\n\n

Perez:<\/strong> That \u201cthe effects in this regard become much more influenced by wind and solar\u201d remains to be shown, as hydroelectricity accounts for a much higher percentage of energy produced world-wide than both of these sources of energy combined, particularly so in both the timeframes considered by Sovacool et al. To extrapolate their findings to a regime where solar and wind power were to become dominant deserves at the very least a justification, which is not present in their paper. Let us stress that, although hydro, wind and solar share the renewable characteristics, the large uncontrolled variability of wind and solar production make them very different to hydroelectricity in that respect.<\/p>\n\n\n\n

Sovacool et al.:<\/em><\/strong> With respect to timeframes, the question raised is (as we acknowledge) about nuanced differences of approach, not about \u201cmistakes\u201d. We are ourselves clear that there are multiple things to consider on this issue. This is exactly why we have chosen a robust data-averaging approach with several triangulation procedures. Together with our openness to the many conditionalities, this is the way to properly address uncertainties and ambiguities that are unavoidable in this kind of research. If Mr Perez really wants to claim that there exists just one single definitive approach to this complexity, then he is arguably reproducing the kind of technocratic authoritarianism that has led for so long to the neglect of the kinds of questions we are raising.<\/em><\/p>\n\n\n\n

As to Mr Perez\u2019s argument that \u201cyou do not have stationarity\u201d and \u201cyou need stationarity for time series analysis\u201d, we agree. But this is again a strangely misleading point. It is this need for stationarity in time series approaches that constitutes a key reason why we do not adopt such an approach.<\/em><\/p>\n\n\n\n

Here, the argument that we should have used panel data and that our analysis is unduly time-averaged actually go together. While panel data analysis may be an alternative, we intentionally chose time averaging since this procedure enables more robust statements to be made in the context of random variables (the underlying modelling for statistics). Such approaches are often used as a mean to average nuisance contributions in an environment with a presence of many influencing factors which clearly is our case at hand. We choose the stated time lag without involvement of a second category of assumptions that would not compellingly fit the purpose of an initial pioneering study. As we explain, the indicated time lag was chosen to optimally use the data set. Otherwise, we might have disregarded precious data points which would then in turn have raised the objection that we intentionally and deliberately used only some parts of the data, but not all of it, thereby wasting parts of the available data set. Crucial here, is that we still have to consider a directional effect since power plants typically involve a lot of down-stream processes (such as maintenance) that stretch over time but need to be clearly attributed.<\/em><\/p>\n\n\n\n

Perez:<\/strong> Non-stationarity is an important phenomenon in this particular timeframe, as many countries underwent rapid development in the 90s and the 00s. It was never claimed in our paper that \u201ctime series require stationarity\u201d, which is a false statement. Time series analysis can be performed even in a non-stationary setting, for instance by using a Moving Average (MA) or Moving Average Exogenous (MAX) model, which was not the case in Sovacool et al.\u2019s work. Other standard tools in this context are Autoregressive processes (ARPs) or Autoregressive exogenous processes (ARXs). All of these tools are well-adapted to indeed study whether the claims of Sovacool et al. regarding the nature of the time lag are justified or not. However, non-stationarity in particular implies that considering only two-time steps with an a priori arbitrary lag is an incorrect approach from a statistical point of view. The averaging chosen by the authors is not justified from a time-series analysis perspective and does not exploit the data in any sense of optimality (from a statistical standpoint). That this procedure is \u201crobust\u201d remains to be shown by, for instance, demonstrating its stability, i.e. whether a change in the time step and number of timeframes considered changes the conclusions of the regression analysis or not. This was never made explicit by the authors in their paper. Furthermore, whether their justification for the lag is correct or not also would require a finer time series analysis. Regardless, this was not the main argument provided in our paper, although we point out that more adequate tools exist for treating the data. As we stated in our paper, even taking the averaged-out data from Sovacool et al. there are many other problems regarding their analysis, which are not related to these time series considerations.<\/p>\n\n\n\n

Sovacool et al.:<\/em><\/strong> To Mr Perez\u2019s statement that \u201cdistribution of the residuals is not exactly normal\u201d, we respond that any expert should be aware that any test of assumptions only gives hints for acceptance within defined error intervals. When invoking statistical pretests, all issues surrounding statistical tests, like \u201cfalse positives\u201d, power and efficiency of tests, have to be mentioned.<\/em><\/p>\n\n\n\n

Perez:<\/strong> The only time we make this remark is when we are reporting the t-statistic, standard error of our regressions and their p-value. We remark that looking at p-values is irrelevant here, and that the standard error and t-statistics are thus the relevant metrics to look at.<\/p>\n\n\n\n

Sovacool et al.:<\/em><\/strong> On a further technical point, Mr Perez refers to \u201cconfounding variable with a power law and not just a linear model\u201d. But there is no part of our work that relies on identifying a \u201cbest fit\u201d curve. This would be difficult to motivate from a theoretical perspective \u2013 for example: why squared or the root.<\/em><\/p>\n\n\n\n

Perez:<\/strong> The full quote is \u201cdespite the fact that going forwards we should consider accounting for the confounding variable with a power law and not just a linear model.\u201d As is clear from inspection of the data in a log-log chart, the GDP vs CO2eq emissions are better described by a power law rather than just a linear model, as was shown in our analysis. That this should be the case is not necessarily a surprise, as the data is clearly heteroskedastic and spans many orders of magnitude. This is often a sign that the underlying distribution should be Pareto, hence our inspection of whether this hypothesis holds or not.<\/p>\n\n\n\n

Sovacool et al.:<\/em><\/strong> We are not aiming to build a \u201ccausal model\u201d. We never claim to do so. Why does Perez imply otherwise?<\/em><\/p>\n\n\n\n

Perez:<\/strong> On this point, let us quote the authors on the conclusions of their paper: \u201cWhen taken together with the finding that renewables seem significantly more positive for carbon abatement, important adverse implications arise for nuclear power. As the evidently less generally favourable of the two broad carbon emissions abatement strategies, a tendency of nuclear not to coexist well with its renewable alternative, does (all else being equal) raise doubts about the opportunity costs of investments in nuclear power rather than renewable energy. The direction of cost and learning trends discussed here, intensify this point. Given the current state of climate debates internationally and in many countries, it is troubling that nuclear and renewable energy pathways appear (both historically and, here, empirically) to display such mutual tension. It appears that countries planning large-scale investments in new nuclear power are risking suppression of greater climate benefits from alternative renewable energy investments. That the converse may also be true (with renewables tending to suppress nuclear investments) is evidently less important, because it is renewable strategies that are on balance evidently more effective at carbon emissions mitigation.\u201d If the authors did not seek to exploit a causal model in which the link between the variables studied was properly understood, drawing such conclusions from a simple correlation study exhibiting the several statistical caveats mentioned in our paper is at best unjustified. Alternatively, if the objective of the paper was to make policy recommendations, then the study of a causal model becomes necessary (albeit, not necessarily sufficient).<\/p>\n\n\n\n

Sovacool et al.:<\/em><\/strong> In similar vein, Mr Perez makes statements about the \u201cpredictive power\u201d of our model that compound a further diversion with a misquote. What we actually said was \u201cCrucially, renewable energy strategies are, to an evidently noteworthy degree, associated <\/strong>with lower levels of national carbon emissions\u201d. This is not an attribution of causality.<\/em><\/p>\n\n\n\n

Perez:<\/strong> Once again, \u201cpredictive power\u201d is a common expression in the statistical jargon typically used in regression analysis.<\/p>\n\n\n\n

Sovacool et al.:<\/em><\/strong> Whether one might have chosen a different analytical approach is a moot point that we acknowledge. But all methods hold pros and cons. Oddly for someone so focused on precision, Mr Perez does not demonstrate that alternatives do not display their own more serious specific disadvantages.<\/em><\/p>\n\n\n\n

Perez:<\/strong> The matter is not whether a particular method holds pros or cons, but rather to point out that there are many methodological mistakes in the analysis in Sovacool et al.\u2019s paper. For example, performing correlations over fractions of the same whole, disregarding that data concerning nuclear power necessarily has a considerably smaller variance than that of renewables, by circumstance (lots of countries have little to no nuclear power, whereas there are very few countries with a large portion of nuclear in their electrical mix). It is not a matter of a pro or con but simply a methodological mistake. Finally, we are explicit in our paper in stating that our goal was to reproduce the study of Sovacool et al. not to do our own on the same subject.<\/p>\n\n\n\n

Sovacool et al.:<\/em><\/strong> With regard to Mr Perez\u2019s statement that the original data would yield a \u201cbias\u201d: we are not adjusting\/distorting the original data, we analyse it simply it as it is. His slurs about \u201cthe poor study of the data set\u201d and \u201csuboptimal modelling\u201d can be qualified in light of our response to his other misleading language addressed above.<\/em><\/p>\n\n\n\n

Perez:<\/strong> cf. our previous discussion on time series analysis considerations.<\/p>\n\n\n\n

Sovacool et al.:<\/em><\/strong> Multivariate linear regression is actually quite robust with respect to its assumptions. What is most crucial here is that it was not our aim in this pioneering study to test any particular model versus another as a candidate for an \u201coptimal fit\u201d. What we are instead doing, is investigating prevailing understandings of the form \u201cthe more <\/strong>\u2026 energy, the less <\/strong>emissions\u201d.<\/em><\/p>\n\n\n\n

Perez:<\/strong> Precisely, and what we show in our paper is that it is not possible to conclude, using the methodology of Sovacool et al., anything other than \u201cfossil fuels emit CO2\u201d.<\/p>\n\n\n\n

Sovacool et al.:<\/em><\/strong>  So our methodology stands in this regard. Given that the associated issues are so prominent and so high stakes, it is remarkable that our research question has not been posed before. In conclusion, we would urge that the reader cut through the many technicalities to see the underlying picture. Our study asks a very basic empirical question. We do not claim to have answered this definitively, but merely pointed to the significant implications and the grounds for further research. Our findings remain valid and salient.<\/em><\/p>\n\n\n\n

\n\n\n\n

Having considered the points Mr Perez raises, Prof. Sovacool, Prof. Stirling and their co-authors feel they have been adequately covered in their initial response.<\/p>\n\n\n\n

Profs Sovacool and Stirling will share further reflections on the response to the paper and the challenge of maintaining open debate in energy debates in another blog post, to follow shortly on this site.<\/p>\nFollow Sussex Energy Group \"Facebook\"<\/a>\"twitter\"<\/a>\"linkedin\"<\/a><\/span>","protected":false},"excerpt":{"rendered":"

Following the publication of their paper \u201cDifferences in carbon emissions reduction between countries pursuing renewable electricity versus nuclear power\u201d in Nature Energy last October, Prof. Benjamin K Sovacool, Prof. Andy Stirling and their co-authors received a number of responses and…<\/span><\/p>\n

Read more ›<\/a><\/div>\n

<\/p>\n","protected":false},"author":322,"featured_media":2430,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","jetpack_publicize_message":"","jetpack_is_tweetstorm":false},"categories":[96027,274,39117],"tags":[151054,161324,161322],"yoast_head":"\n\n\n\n\n\n\n\n\n\n\n\n\t\n\t\n\n\n\t\n