Chapter 12: Epilogue — The Responsible Use of Statistics and Econometrics
🎯 Learning Objectives
Upon completing this Epilogue, you will be able to:
Recognize the ethical responsibilities of econometricians
Identify common statistical deceptions (Huff's classic lessons)
Understand Hendry's methodology as a defence against malpractice
Apply critical thinking to statistical claims
Appreciate the human dimension of econometric practice
"All Knowledge is, in final analysis, History.
All Sciences are, in the abstract, Mathematics.
All Judgments are, in their rationale, Statistics."
C.R. Rao (1920–2023)
12.1 Introduction: The Power and Peril of Numbers
Throughout this book, you have learned the technical tools of econometrics: OLS regression, hypothesis testing, diagnostic tests, panel data, time series, and even machine learning. These are powerful tools. But like any powerful tool, they can be used for good or for ill.
This Epilogue is about responsibility. It draws on three traditions:
Darrell Huff's How to Lie with Statistics — a classic exposé of statistical manipulation that remains as relevant today as in 1954.
David Hendry's methodology — a rigorous, automated approach to empirical model discovery that actively guards against malpractice.
Philip Hans Franses' Ethics in Econometrics — a guide to ethical research practice.
Key Insight: The goal of this Epilogue is not to make you cynical about statistics. It is to make you vigilant — to empower you to ask the right questions and to demand integrity in empirical research.
12.2 Huff's Warning: How to Lie with Statistics
Darrell Huff's 1954 classic remains the most accessible introduction to statistical deception. Its lessons are timeless.
12.2.1 The Sample with Built-in Bias
Huff warns: “A sample is a sample, not the whole population.” Yet many statistical claims are based on biased or non-random samples.
Example: In the 1936 US presidential election, the Literary Digest polled 10 million people and predicted a landslide victory for Alf Landon over Franklin D. Roosevelt. The poll was wrong because the sample was drawn from telephone directories and automobile registrations — which in 1936 overwhelmingly favoured wealthier (and Republican) households.
Key Question to Ask:How was the sample selected? Is it truly random?
12.2.2 The Well-Chosen Average
Huff famously points out that there are three kinds of averages: mean, median, and mode. The choice of which one to report can dramatically alter the impression conveyed.
Example: A real estate agent tells you that the "average" income in a neighbourhood is $100,000. What she doesn't tell you is that this is the mean, pulled up by a few wealthy executives. The median — the income of the typical resident — might be only $60,000.
Key Question to Ask:Which average is being used? Is it the mean, median, or mode?
12.2.3 The Little Figures That Are Not There
Many statistical claims omit crucial information: sample size, standard error, confidence intervals, or p-values.
Key Question to Ask:What's missing? Is the sample size large enough? Are the standard errors reported?
12.2.4 The Gee-Whiz Graph
Graphs can be manipulated by changing the scale of the axes, starting the Y-axis above zero, or using 3D effects that exaggerate differences.
Example: A graph showing a small increase in profits can look dramatic if the Y-axis starts at 90% instead of 0%. The visual impression is one of explosive growth, even if the actual increase is only a few percentage points.
Key Question to Ask:Does the graph distort the data? Look at the axes carefully.
12.2.5 The Semiattached Figure
Huff warns about comparing apples and oranges — using one result to imply something else that doesn't logically follow.
Key Question to Ask:Is the comparison legitimate? Or is the subject being changed?
12.2.6 Post Hoc Rides Again
Huff's most famous lesson: Correlation does not imply causation.
Example: A study found that cancer was more common in milk-drinking regions. But milk-drinking regions also had longer life expectancies — and cancer is a disease of older age. The correlation was spurious.
Key Question to Ask:Does X really cause Y, or could there be a third factor (or reverse causation)?
12.2.7 Huff's Final Advice: How to Talk Back to a Statistic
Huff ends with five questions every critical reader should ask:
Who says so? — What is the source? Is there a conflict of interest?
How does he know? — What is the methodology? Is the sample adequate?
What's missing? — Are standard errors, sample size, or other key details omitted?
Did somebody change the subject? — Is the comparison legitimate?
Does it make sense? — Does the conclusion pass the "common sense" test?
12.3 Espasa on Hendry's Methodology: A Defence Against Malpractice
Antoni Espasa's paper Avoiding Malpractice in Econometrics presents David Hendry's methodology as a systematic framework for rigorous empirical research.
12.3.1 The Problem: Traditional Modelling Can Be Misleading
Espasa argues that traditional econometric modelling — where researchers "specify" a model based on theory alone — can lead to malpractice: omitted variables, incorrect functional forms, and biased estimates.
12.3.2 Hendry's Solution: A Discovery Process
Hendry's methodology treats econometric modelling as a discovery process, not a one-step specification. The key principles are:
The Local Data Generation Process (LDGP): The true data-generating process is unknown. The researcher must discover it.
The General Unrestricted Model (GUM): The LDGP must be nested in a very general model that includes all potentially relevant variables (from theory, data, and indicators).
General-to-Specific (Gets) Reduction: The GUM is systematically simplified through a series of tests and diagnostic checks until a congruent model is found.
Automation (Autometrics): The entire process is automated using machine learning algorithms that explore many possible paths and select the best-fitting, congruent model.
Indicator Saturation Estimation (ISE): Hendry's innovation to detect and account for outliers, location shifts, and other "contaminating influences".
"The specification of econometric models are not known at the beginning and a modelling discovery process is required."
Automated, systematic search with rigorous testing
Overfitting
Diagnostic checks and parsimonious encompassing
Ignoring non-linearities
Non-linear selection capabilities
Ignoring dynamic structure
Lag selection and equilibrium correction mechanisms
Key Insight: Hendry's methodology does not eliminate the need for economic theory. Rather, it nests theory within a rigorous empirical framework — allowing theory to inform the GUM while letting the data speak.
12.4 Franses on Ethics in Econometrics
Philip Hans Franses' Ethics in Econometrics (2024) provides a comprehensive guide to ethical research practice.
12.4.1 The Ethical Landscape
Franses argues that econometricians face unique ethical challenges because:
They work with complex data and models that are often poorly understood by the public and policymakers.
They have considerable discretion in choosing model specifications, variables, and tests.
Their results can have significant policy implications.
12.4.2 Specific Ethical Pitfalls
P-Hacking: Trying different specifications until a significant p-value is obtained.
HARKing: Hypothesising after the results are known (presenting post-hoc hypotheses as if they were pre-specified).
Data Dredging: Searching for patterns in data without a clear theoretical foundation.
Cherry-Picking: Reporting only the results that support a preferred conclusion.
Overstating Results: Claiming causality when only correlation has been established.
12.4.3 Franses' Ethical Guidelines
Transparency: Report all decisions made in the modelling process.
Reproducibility: Provide data and code so others can replicate your results.
Honesty: Acknowledge limitations, uncertainties, and alternative interpretations.
Humility: Remember that all models are simplifications of reality.
12.5 Synthesising the Lessons: A Code of Responsible Practice
Drawing on Huff, Hendry, and Franses, here is a Code of Responsible Practice for econometricians:
1. Be Skeptical — Especially of Your Own Results
Ask Huff's five questions of your own work.
Test alternative specifications, not just the ones that support your preferred conclusion.
Remember: correlation is not causation.
2. Be Transparent
Report all variables considered, not just those retained.
Report all diagnostic tests and their outcomes.
Make your data and code available for replication.
3. Use a Systematic Methodology
Follow a principled approach like Hendry's General-to-Specific methodology.
Nest your model in a General Unrestricted Model (GUM).
Use automated selection methods (like Autometrics) to reduce subjectivity.
Use indicator saturation to detect outliers and breaks.
4. Embrace Uncertainty
Report standard errors, confidence intervals, and p-values honestly.
Acknowledge the limitations of your data and model.
Remember that economic reality is more complex than any model can capture.
5. Focus on Substance, Not Just Significance
Statistical significance ≠ economic or practical significance.
Ask: Does the magnitude of the effect matter?
Ask: Does the result make sense in the context of economic theory?
12.6 Practical Exercises
Exercise 12.1: Spot the Lie
Consider the following claim: "A study of 1,000 people found that 90% of those who drank red wine daily lived to be over 90 years old."
What questions would you ask about the sample?
What questions would you ask about the methodology?
Is there a possible confounding factor (a third variable)?
Does correlation imply causation here?
Solution:
Sample questions: Who was sampled? Was it random? What was the sample size? Was there a control group?
Methodology questions: How was consumption measured? Was it self-reported? Were other factors (e.g., diet, exercise, wealth) controlled for?
Confounding factors: People who drink red wine daily may also be wealthier, have better healthcare, or have other healthy habits.
Causation: No — correlation does not imply causation. A controlled experiment would be needed.
Exercise 12.2: Average or Averages?
You are presented with the following data on salaries in a company: $25,000, $28,000, $30,000, $32,000, $35,000, $40,000, $45,000, $50,000, $1,000,000.
Calculate the mean, median, and mode.
Which average would a manager use to attract recruits? Why?
Which average would a union representative use to argue for higher pay? Why?
Which average best represents the "typical" salary?
Manager: Would use the mean ($142,778) to make salaries look generous.
Union: Would use the median ($35,000) to show most workers earn much less.
Best representation: The median ($35,000) best represents the typical salary, as the mean is distorted by the outlier.
Exercise 12.3: The Misleading Graph
You are shown a bar chart where the Y-axis starts at 80% instead of 0%, making a small increase from 82% to 86% look like a dramatic rise.
Why is this misleading?
What should the graph look like to be honest?
How would you redraw the graph to show the true change?
Solution:
Why misleading: Starting the Y-axis above zero exaggerates visual differences. A 4% increase looks like a 33% increase (from 82 to 86 is a 4.9% increase, but visually it's distorted).
Honest graph: The Y-axis should start at 0% to show the true proportion.
Redraw: Use a Y-axis from 0% to 100%. The bars will be much smaller and the change will be shown in proper perspective.
Exercise 12.4: Post Hoc Reasoning
A researcher finds that ice cream sales and shark attacks are positively correlated. She concludes that eating ice cream causes shark attacks.
Why is this conclusion flawed?
What is the likely confounding factor?
How would you explain this to a non-statistician?
Solution:
Flaw: Correlation does not imply causation. The researcher has committed the post-hoc fallacy.
Confounding factor: Hot weather. Both ice cream sales and shark attacks increase in summer.
Explanation: "Ice cream sales and shark attacks both go up in summer because more people are swimming in the ocean. The ice cream doesn't cause the shark attacks — the common cause is the season."
Exercise 12.5: Applying Hendry's Methodology
You are modelling the effect of education on wages. You have data on 1,000 individuals with variables: education, experience, age, gender, and region. You suspect there may be outliers, structural breaks, or omitted variables.
How would you apply Hendry's General-to-Specific approach?
What would be the GUM (General Unrestricted Model)?
How would you use Indicator Saturation Estimation (ISE) to detect outliers?
What diagnostic tests would you run to ensure congruence?
Solution:
Gets approach: Start with a very general model that includes all plausible variables (education, experience, age, gender, region, interactions, polynomials). Then systematically simplify using hypothesis tests and diagnostic checks.
GUM: Wage = β₀ + β₁Education + β₂Experience + β₃Age + β₄Gender + β₅Region + β₆Education² + β₇Experience² + β₈Education×Experience + ... + u (including all possible variables and transformations).
ISE: Include impulse indicators for each observation to detect outliers. The algorithm will retain those that are statistically significant, revealing potential outliers or breaks.
Diagnostic tests: Test for heteroskedasticity (BP, White), serial correlation (DW), normality (JB), functional form (RESET), and multicollinearity (VIF).
Huff, D. (1954). How to Lie with Statistics. W.W. Norton & Co.
Espasa, A. (2025). Avoiding Malpractice in Econometrics: David Hendry's Automated Methodology Nesting Theory-Driven and Data-Driven Approaches. Preprints.
Hendry, D.F. & Doornik, J.A. (2014). Empirical Model Discovery and Theory Evaluation. MIT Press.
Franses, P.H. (2024). Ethics in Econometrics: A Guide to Research Practice. Cambridge University Press.