A Familiar Problem, in an Unfamiliar Domain
Algorithmic bias is a well-established area of research, mostly developed in contexts with obvious, serious stakes: hiring algorithms that systematically disadvantage certain demographic groups, facial recognition systems that perform less accurately on darker skin tones, risk-assessment tools used in criminal justice that replicate historical patterns of discrimination. This research has identified several distinct mechanisms by which bias enters algorithmic systems — not usually through any single line of deliberately biased code, but through more diffuse processes: biased training data, unexamined design defaults, and incentive structures that optimize for something other than the outcome users assume they’re getting.
Divination apps are a much lower-stakes domain than hiring or criminal justice, and it would be a category error to treat the concerns as equivalent in severity. But the same three mechanisms — training data, design defaults, and incentive structures — operate in this domain too, and understanding how is useful both for evaluating these apps and for understanding algorithmic bias research more generally, since this is a domain where the mechanisms are visible without the same ethical weight that makes the higher-stakes cases harder to examine dispassionately.
Training Data: Whose Astrology Gets Learned
A language model generating astrological content, as discussed in the companion piece on how LLMs produce horoscopes, learns the statistical patterns of the text it was trained on. This creates an immediate and underexamined bias question: whose astrological writing, from which traditions, in which languages, dominates that training data?
Western tropical astrology has an overwhelming presence in English-language internet text relative to its share of global astrological practice — there is vastly more English-language content about sun signs, Mercury retrograde, and natal chart synastry than there is comparably accessible English-language content about Vedic dashas, BaZi Day Masters, or Nine Star Ki annual cycles, even though these traditions have comparably long histories and large practitioner bases in their countries of origin. A model trained predominantly on English-language internet text will, as a direct consequence, be more fluent, more detailed, and more reliable when generating Western astrological content than when generating content in less-represented traditions — not because Western astrology is more valid, but because it’s more textually documented in the specific corpus the model learned from.
This produces a specific, measurable failure mode: AI systems asked to generate content in underrepresented traditions may produce output that sounds confident and fluent while actually being less accurate to that tradition’s actual conventions — blending in elements from better-represented traditions, flattening important distinctions, or defaulting to generic “mystical” language when the specific tradition’s actual technical vocabulary is sparse in the training data. This is structurally identical to documented bias patterns in other domains, where models perform worse on underrepresented categories while often expressing equal or even greater confidence in their (lower-quality) output for those categories — confidence and accuracy are not the same thing, and nothing in standard training procedures guarantees they’ll track each other.
Design Defaults: What Gets Chosen Without Being Decided
A second source of bias operates at the level of product design decisions that get made, often for practical or historical reasons, without anyone explicitly deciding “this is the bias we want.” Which house system does a birth chart calculator use by default — Placidus, Whole Sign, Koch — when these systems can produce meaningfully different house placements from identical input data? Which hemisphere’s seasonal associations does a system default to, when many astrological and seasonal frameworks were developed in the Northern Hemisphere and don’t automatically invert for Southern Hemisphere users? Which calendar system serves as the default for date input, when Western (Gregorian), Chinese lunar, and other calendar systems can shift which “year” or “month” a birth falls into for systems that key off those boundaries?
None of these are bias in the sense of intentional discrimination. They’re closer to what algorithmic fairness researchers call “default effects” — a system has to make some choice, the choice that’s easiest to implement or most familiar to the system’s original developers tends to become the default, and the default then silently shapes output for every user who doesn’t know to question it or doesn’t have the option to change it. A Southern Hemisphere user receiving Northern-Hemisphere-calibrated seasonal commentary isn’t being deliberately disadvantaged — but they are receiving output calibrated for a context that isn’t theirs, through a default nobody specifically chose to apply to them.
Incentive Structures: What Gets Optimized
The third and, in some ways, most consequential mechanism involves what the overall system — not just the AI model, but the product surrounding it — is actually optimized to produce. Most consumer apps, divination apps included, are commercial products with engagement and retention metrics that influence design decisions, sometimes explicitly and sometimes through less direct organizational pressure.
This creates a specific risk, discussed in the companion piece on dopamine and anticipation: a system optimized for engagement may, whether by deliberate choice or by the kind of unexamined A/B-testing-driven evolution that shapes most consumer software, drift toward producing content that maximizes checking behavior and emotional engagement rather than content that’s most useful, most accurate to its stated tradition, or most genuinely reflective. Content that’s slightly more dramatic, slightly more emotionally charged, or slightly more anxiety-inducing may perform better on engagement metrics than calmer, more measured content — a pattern well-documented in research on social media and news algorithm design, where engagement-optimization has been repeatedly shown to favor emotionally activating content over accurate or measured content, without any explicit decision to prioritize drama over accuracy.
If a divination app’s content generation is tuned (explicitly or through iterative optimization against engagement data) in this direction, the resulting bias isn’t about which tradition or which demographic group is favored — it’s a bias toward content that performs well on a specific business metric, which may systematically diverge from content that best serves the stated purpose of helping someone reflect on their day. This is a bias that’s particularly hard to detect from the outside, because engagement-optimized content and genuinely useful content can look similar on the surface, and the divergence only becomes visible through systematic analysis of what kinds of messages perform best on engagement metrics versus what kinds of messages users, asked directly and reflectively, would say were actually helpful.
What Mitigates Each Mechanism
Each of these three bias sources has a different, partially independent mitigation. Training data bias is addressed by deliberately incorporating well-sourced material from underrepresented traditions, ideally reviewed by practitioners with genuine expertise in those traditions, rather than relying on whatever happens to dominate a general-purpose training corpus — and by being transparent about which traditions a given system has been more carefully developed for versus which are more thinly supported. Design-default bias is addressed by making defaults visible and adjustable rather than silent and fixed — letting users see and change house system, hemisphere, and calendar assumptions rather than burying them in an unstated default. Incentive-structure bias is addressed by deliberately decoupling content-generation tuning from pure engagement metrics, and by building in some accountability to a stated standard (accuracy to a tradition’s actual conventions, usefulness as self-reported by users after reflection rather than measured purely by immediate engagement) that isn’t reducible to “did this make people check the app more.”
None of these mitigations require abandoning AI-assisted divination tools — they require treating the design choices involved with the same scrutiny that algorithmic fairness research has developed for higher-stakes domains, scaled appropriately to the actual stakes involved here, which are real (people’s daily reflective practice, their sense of which traditions are taken seriously, their time and attention) even if they’re not life-altering in the way hiring or criminal justice algorithms can be.
The Quiet Version of a Familiar Problem
What divination apps illustrate, in miniature, is that algorithmic bias isn’t a phenomenon limited to high-stakes domains with obvious victims. It’s a structural feature of how algorithmic systems get built — shaped by what data was available, what defaults were convenient, and what the system was actually being optimized to do — that shows up wherever an algorithm makes choices on someone’s behalf, including in a domain as low-stakes as a daily reflective message. The mechanisms are the same. The consequences here are smaller. The need to actually look at the mechanisms, rather than assuming a fluent and pleasant-sounding output is therefore a neutral one, is not smaller at all.