What Meta-Analysis Is For
A single study, however well-designed, can be misleading for reasons that have nothing to do with whether its conclusion is true. Small sample sizes produce noisy results. A study that finds nothing might simply lack the statistical power to detect a real but small effect. A study that finds something might be one of the unlucky (or lucky) draws that occur by chance even when nothing is really there.
Meta-analysis was developed to address exactly this problem: pool the results of many studies, weight them appropriately, and look at what emerges from the combined data — an effect too small to see reliably in any single study might become clearly visible when many studies are stacked together, or an effect that seemed to appear in a few individual studies might wash out entirely once the full body of evidence is considered.
For a question like “does astrology have predictive validity,” meta-analysis seems like exactly the right tool. Individual studies have used different methodologies, different sample sizes, different astrological systems, and different outcome measures — exactly the kind of heterogeneous, scattered evidence base that meta-analysis exists to make sense of. The hope, reasonable on its face, was that combining everything would produce a clearer answer than any individual study could.
The Dean and Kelly Synthesis
The most comprehensive attempt at this kind of synthesis came from Geoffrey Dean and Ivan Kelly, two researchers who had spent decades — Dean had himself trained and worked as an astrologer earlier in his career — compiling and evaluating astrological research. Their 2003 paper, published in the Journal of Consciousness Studies, became what one review later called a standard reference: a wide-ranging assessment covering sun-sign studies, full natal chart studies, the “time twins” research discussed elsewhere in this series, and the Gauquelin planetary-position findings, among others.
Their overall conclusion was that astrology, across this body of evidence, showed no effects that couldn’t be more economically explained by the cognitive and social mechanisms covered throughout this series — what they characterized as the operation of “hidden persuaders” rather than anything astrological. For sun-sign personality research specifically, the combined evidence was unambiguous: correlations between sun sign and personality measures, across very large samples, clustered around zero.
This is, by most measures, a strong and well-supported conclusion for the specific claim it addresses. But “astrology” is not one claim — it’s a family of related but distinct claims, made by different traditions with different mechanisms, and the Dean and Kelly synthesis’s treatment of the most empirically interesting part of the astrological research literature — the Gauquelin findings — has itself become a site of ongoing dispute.
The Gauquelin Problem
Michel Gauquelin, a French researcher working from the 1950s onward, conducted what remains the most extensively replicated body of empirical astrology research: studies of the relationship between planetary positions at birth and professional eminence. The most famous finding, the so-called “Mars effect,” found that eminent athletes were born, at rates exceeding chance, with Mars in certain positions relative to the horizon — specifically, just past rising or just past culmination.
What makes the Gauquelin findings unusual, and genuinely difficult to dismiss with the mechanisms that explain sun-sign astrology’s apparent accuracy, is that they don’t rely on subjective reading or interpretation at all. There’s no practitioner involved, no Barnum statement, no cold reading. It’s a direct statistical question: do birth records show planetary positions clustering in certain ways for certain professions, at rates exceeding chance? And by several independent replications — including, notably, a replication by a Belgian skeptics’ organization (the Comité Para) that set out expecting to debunk the finding and instead reported results consistent with Gauquelin’s — the answer for the Mars-and-athletes finding specifically has been yes, at a statistically significant level.
This puts the Mars effect in a genuinely unusual position in this literature: a finding that is statistically significant, independently replicated by researchers including skeptics, and not obviously explainable by the cognitive biases that account for most of astrology’s apparent validity elsewhere. It’s also, by most assessments including skeptical ones, a weak effect — and one that creates problems for astrological theory as much as for skeptics: the effect appears for only some planets, not the full set astrology assigns significance to, and the “favored” positions in the data are described as weak ones rather than the positions traditional astrology would predict as strongest.
Two Meta-Analyses, Two Conclusions
This is where the story gets genuinely contentious, in a way that’s instructive about what meta-analysis can and can’t resolve.
Dean and Kelly’s 2003 synthesis addressed the Gauquelin findings as part of their broader null conclusion — proposing explanations including the “parental tampering” hypothesis (the idea that parents who were aware of astrological traditions might have, consciously or not, adjusted birth registration times to produce astrologically favorable charts for their children, which could produce a statistical artifact resembling a real effect without any planetary influence being involved).
A 2023 rebuttal, published in the same general literature, pushed back specifically on this treatment — arguing that Dean and Kelly’s meta-analysis combined studies with substantially different data types in ways that obscured rather than clarified the picture, that the parental tampering hypothesis lacks direct supportive evidence for the specific datasets in question, and that a careful reanalysis of even the famous Carlson 1985 study (discussed in detail elsewhere in this series, and often cited as astrology’s most decisive empirical refutation) shows results closer to marginal significance than the study’s “devastating verdict” framing suggested — specifically, a three-way forced-choice result at p = .054, just outside the conventional p < .05 threshold, with a difference assessment method producing p = .04, just inside it.
Neither side in this exchange is arguing that astrology, broadly, has been vindicated. The 2023 rebuttal explicitly states that its reanalysis results are “insufficient to deem astrology as empirically verified” — its argument is narrower: that the negative verdict was overstated relative to what the data, carefully analyzed, actually shows.
What “Combining All the Studies” Actually Requires
The deeper issue this dispute illustrates is that meta-analysis isn’t a mechanical, assumption-free procedure. It requires decisions: which studies to include, how to weight them, how to handle studies that measured different things in different ways, how to define the outcome being tested, and how to handle borderline results relative to significance thresholds.
For a research literature as heterogeneous as astrology’s — sun-sign studies, full natal chart studies, Gauquelin-style planetary position studies, time-twin studies, each using different methodologies developed by researchers with very different starting assumptions about what astrology claims and how to test it — these decisions matter enormously, and reasonable researchers can make them differently while believing they’re doing the same thing: “summarizing what the evidence shows.”
This doesn’t mean meta-analysis is useless here, or that “everything is contested so nothing is known.” Some conclusions from this literature are robust across virtually every reasonable approach to combining the data: sun-sign personality correlations are not detectably different from zero in large samples, regardless of how you slice the meta-analysis. Other conclusions — particularly around the Gauquelin findings — remain genuinely contested in ways that depend on exactly these methodological choices, and where reasonable researchers, looking at substantially the same underlying data, reach different characterizations of what it shows.
What This Means for “What Does the Research Say”
The honest summary of astrology’s meta-analytic research base is more textured than either “science has debunked astrology” or “studies show astrology works” — both of which oversimplify a literature where different categories of claim have fared very differently.
Sun-sign personality astrology — the most widely consumed and most culturally prominent form — has been tested extensively, combined across large samples, and shows no effect. This conclusion is about as solid as null results in social science get.
The Gauquelin planetary-position findings occupy genuinely contested territory: independently replicated by multiple research groups including skeptical ones, statistically significant by conventional thresholds, but weak in magnitude, inconsistent with astrological theory in important ways (working for some planets and not others, in directions astrology wouldn’t have predicted), and subject to ongoing methodological dispute about how to interpret them within the broader research literature.
What “combining all the studies” has actually produced, in this specific case, isn’t a single answer. It’s a clearer picture of where the genuine uncertainty lies — which is itself a meaningful result, even if it’s not the kind of result either side of the broader astrology debate finds satisfying.