Keep Venus in the seventh house of a hypothetical natal chart. Replace a square from Saturn with a trine from Jupiter. Which parts of an interpretation survive, and which need to be rewritten? A reliable reasoner should identify the precise dependency between each premise and each conclusion.
Teach a model the dependencies in one symbolic tradition, then test it on an unfamiliar one. Astrology offers a distinctive candidate: centuries of overlapping schools, inherited associations and disagreements about priority create interpretations with meaningful ambiguity. Historical complexity might teach a model to hold rival interpretations open and revise them precisely. Yet the opposite is plausible: a systematically composed artificial language may teach cleaner generalization than inherited, inconsistent conventions. The strongest test separates the provenance of a tradition, ambiguity in its rules and familiarity of its symbols.
What Would Count as an Improvement?
In a specified astrological tradition, aspects and context qualify the meaning of a placement while competing schools may assign different priorities. A reliable reasoner should identify which interpretations remain warranted, what assumptions support each one, and which observation could resolve their disagreement.
François Chollet’s On the Measure of Intelligence[1] examines the distinction between acquired skill and performance aided by prior exposure. In SCAN, sequence-to-sequence models struggled when familiar commands appeared in unfamiliar combinations[2]. COGS extends the compositional challenge to linguistic structures[3]. These benchmarks establish a difficulty that any proposed curriculum must confront: success on familiar patterns can coexist with failure when their relationships change.
A mere change of symbols is a weak test of transfer. Barnett and Ceci’s taxonomy[4] asks how far the testing circumstances and the required knowledge differ from training. The harder question is whether a model learns to compare and revise defeasible interpretations when the very form of evidence changes.
How a Natal Chart Combines Rules
In common Western astrological conventions, Venus suggests attraction and values, the seventh house frames partnership, Saturn can suggest constraint, and Jupiter can suggest expansion. Aspects relate placements to one another. Schools disagree about rulerships, house divisions, orbs and interpretive priorities.
In the simplified example, Venus in the seventh house remains fixed. With Saturn square Venus, the stipulated interpretation emphasizes friction or duty in partnership. Replace that aspect with Jupiter trine Venus and the aspect-dependent modifier changes to ease or opportunity. Venus and its house keep their original meanings.
The natal chart guide introduces the components used in a chart. A worked interpretation shows how readers combine them.
Exercise One: A Small Astrological Rulebook
For this exercise alone, apply the following conventions exactly. Rule A: Venus in house 7 produces the base statement partnership matters. Rule B: Saturn square Venus adds partnership faces a constraint. Rule C: Jupiter trine Venus adds partnership has an opportunity. Rule D: when Saturn square Venus is present, add the derived recommendation address the constraint before pursuing the opportunity if Jupiter trine Venus is also present. There are no other relevant placements.
Case 1: Venus is in house 7, Saturn squares Venus and Jupiter trines Venus. The valid statements are partnership matters, partnership faces a constraint, partnership has an opportunity and the conditional recommendation in Rule D. Case 2: remove the Saturn square while retaining Jupiter trine Venus and house 7. The valid statements are now partnership matters and partnership has an opportunity. Both the constraint statement and Rule D’s recommendation disappear.
Questions: Which propositions survive the change, which depend on Saturn, and which rule requires both aspects? Cases 1 and 2 above provide a worked answer key; neither may appear in held-out evaluation.
Advanced variant: competing interpretations. Retain the chart with both aspects, but introduce two schools: S ranks the Saturn modifier first, while J ranks Jupiter first. Neither priority changes the underlying evidence. With the school unspecified, the answer must include both possible leading readings and dominance unresolved. Naming S or J resolves the priority without changing the chart; removing the Saturn square eliminates the constraint regardless of school. The exercise distinguishes a change in interpretive convention from a change in the evidence itself.
Exercise Two: A Fictional Control Worked Example
In a fictional archive, each tablet has a glyph, a room and optional links. Use these rules: Rule 1: glyph K in room 2 gives the label active. Rule 2: a red link to K gives the label blocked. Rule 3: a blue link to K gives the label supported. Rule 4: if both red and blue links exist, the output includes review required. Rule 5: a silver seal cancels blocked and cancels review required, while leaving every other label unchanged.
Initial case: K is in room 2, with red and blue links and no silver seal. The correct labels are active, blocked, supported and review required. Counterfactual: add a silver seal and change nothing else. The correct labels are active and supported. In the actual evaluation, a response receives full credit only when the entire supported label set and the override dependency are correct; missing or invented labels are recorded as distinct errors.
Evaluation uses unseen exception structures.
What Astrology Can and Cannot Establish
An AI system could incorporate a successful technique for tracking contextual dependencies without adopting astrology’s claims about people or the future.
This requires separate standards of accuracy: planetary positions can be checked by calculation; interpretations can be checked against stated conventions; transfer can be measured on unseen problems; claims about real lives require independent empirical evidence.
Carlson’s double-blind personality-matching study reported unfavorable results for natal astrology[5]. Ertel later challenged aspects of its statistical analysis while acknowledging that the reanalysis did not establish astrology’s empirical validity[6].
Poetry as an Independent Test Domain
Close reading tests an interpretation against the exact words, recurrences and sequence of a poem. Reconstructive Literary Analysis[7] sets out six operations: observation, constraint, function, dependency, counterfactual stress test and residual. It remains an authored interpretive protocol whose reliability requires independent evaluation.
Consider Danil Rudoy’s poem My Muse from Love Is Poetry[8]. The speaker begins in a room colored blue and black; the Muse enters dressed in red; at her departure, the red disappears and the room returns to its initial colors. The closing lines read: The room, again, is black and blue:
followed by I write a requiem for you.
The poem makes that chromatic sequence carry a dramatic change: anticipation, creative exhilaration, abandonment and elegy. Which colors frame the poem, which color belongs to the Muse, and what does the return to black and blue mean after her departure? A counterfactual task could change only the last return of the initial colors and ask which interpretation of the circular structure loses its strongest textual evidence. Quotations, color positions and structural recurrence can be checked against the published poem; competing claims about their emotional significance require argued textual support.
In William Shakespeare’s Sonnet 73[9], the speaker moves through three images of approaching loss: a late autumn tree, the fading light after sunset and a fire dwindling on its own ashes. Each image presents a different time scale; the closing couplet converts that sequence into an argument about the beloved’s attachment. A transfer question can ask the model to remove the fire image, identify which evidence disappears and explain how the remaining seasonal and twilight images still support the movement toward the couplet. Shakespeare’s progression and Rudoy’s returning color pattern thus offer two different structures for testing whether an interpretation changes with its textual premises. Apply one rubric to both poems: identify the relevant textual evidence, trace each interpretive claim to specific lines, retract only claims defeated by the counterfactual change, and acknowledge alternative readings consistent with the surviving text. Independent judges should use the same anchored 0–2 scale for each criterion, evaluate responses without author names or training-condition labels, and report agreement and adjudicated disagreements. Training and testing poems must come from disjoint sets of authors and works.
A Controlled Transfer Experiment
Galke, Ram and Raviv found that compositional language structure can benefit learning in both humans and neural networks[10]. An astrology-based curriculum must therefore be compared with deliberately designed languages of similar structural richness. More recent tests of 36 large language models by Sakai, Kamigaito and Watanabe found persistent difficulties following changed concept-order instructions[11]. Established treatments of non-monotonic logic[12] and belief revision[13] provide a formal basis for scoring the rule changes demonstrated above.
Five curricula provide the comparison: astrological interpretation, a structurally matched fictional tradition, abstract formal rules, literary dependencies and general learning. Within the symbolic curricula, three factors form eight versions of each underlying dependency graph.
| Factor | Levels | Controlled variables | Alternative explanation |
|---|---|---|---|
| Provenance | Historical / invented | Dependency graph, exceptions, narrative richness | Familiar history or its framing drives the effect. |
| Ambiguity | Low / high | Rule count and inference depth, insofar as feasible | Incomplete priorities or extra admissible readings explain the gain. |
| Semantic familiarity | Familiar / unfamiliar labels | Equivalent rules, balanced labels | Previously encountered vocabulary explains performance. |
Ambiguity varies through priority completeness and the number of admissible readings; unavoidable structural differences are documented. Baseline pretests and label-frequency checks estimate familiarity. Because historical provenance cannot be randomized perfectly, independently varied provenance descriptions help identify framing effects. Main effects and interactions distinguish advantages associated with historical material from those explained by ambiguity or familiar labels.
The evaluation distinguishes five item types: complete priorities, multiple admissible readings, missing priorities, altered interpretive context with fixed facts, and altered facts with fixed context. Each case has a complete answer key recording justified readings, premise dependencies, priority status and counterfactual changes. Chart questions measure learning within astrology; literary corpora and structurally different formal systems probe wider transfer.
The quantitative design fixes the following values before any outcomes are examined.
| Design element | Specification |
|---|---|
| Models | Two public instruction-tuned checkpoints; archived identifiers, tokenizers, versions and decoding settings; frozen weights |
| Doses | 0, 1, 2, 4, 8, 16, 32 examples per curriculum |
| Repetitions | Five seeded demonstration sets per checkpoint and curriculum |
| Cases | 100 independently generated cases per set and dose |
| Primary outcome | Exact-set accuracy in unfamiliar nonastrological worlds |
| Efficiency outcomes | Area under the accuracy-by-dose curve and smallest dose reaching 80% accuracy |
| Additional outcomes | Premise attribution, selective retraction, justified abstention, distraction tests and token counts |
Leakage and budget rule. Evaluation materials—including generators, templates, dependency graphs, authors, works and passages—are disjoint from all demonstrations and training materials; fresh independently seeded cases establish replication. Every curriculum has equal demonstration counts, total prompt allowances, feedback and evaluation format, while token usage is recorded separately. The general-learning control draws from unrelated reasoning tasks. Ordering is randomized; symbol-renamed astrology and misleading familiar labels test prior familiarity. Publication covers prompts, software, generation code, seeds, parsers, complete answer keys and failed responses. The 80% threshold measures contextual adaptation and carries no implication of lasting weight-level learning.
Probe persistence in two distinct ways. First, interpose unrelated tasks within the same context and measure whether the rule skill survives distraction. A fresh-context test then removes the demonstrations entirely. Under a frozen-weight design, any apparent retention in a fresh context requires careful investigation for pre-existing ability, leakage or external state. A separate weight-update experiment with pretests, delayed post-tests and independently seeded replications would be required to test lasting acquisition. Treat repeated seeds within one checkpoint as correlated observations.
Missing, invented and prematurely selected readings count as separate errors. Near transfer covers new labels and combinations within familiar graphs; far transfer introduces new dependencies, conflicts, evidence formats or decisions. Both are reported separately. A five-percentage-point accuracy gain is a provisional practical-effect threshold, subject to simulation-based power analysis across checkpoints, generators, correlated demonstration seeds and comparisons with four controls. Paired estimates use cluster-aware uncertainty intervals. When accuracy and sample efficiency disagree, the preregistered primary outcome determines the main conclusion. Chart accuracy and blinded literary judgments remain secondary. No outcomes have been observed.
Improvement confined to chart questions would establish learning within the training domain. Better identification of admissible readings, justified abstention and selective belief revision in unfamiliar worlds would support transfer of the targeted skill. Only a replicated advantage over both semantically rich fictional traditions and complexity-matched formal rules would warrant attributing additional value to the astrological curriculum. A null result would establish no demonstrated advantage under the conditions tested.
A Historical Case: Competing House-Division Traditions
Ancient astrology already presents a documented conflict over methods. In his account of Vettius Valens, historian Dorian Gieseler Greenbaum contrasts Valens’s practical, tradition-oriented astrology with Ptolemy’s concern for theory and physical explanation[14]. Chris Brennan’s history of house division traces the coexistence of whole-sign, equal and quadrant methods in the Hellenistic tradition, including their different uses in judging topics and planetary strength[15]. Calling these two authors the heads of mutually exclusive house-system schools would flatten the historical record: methods could be combined and the interpretation of particular passages remains contested.
Consider a constructed chart with Gemini rising at 20° and Venus at 5° Cancer. Under whole-sign division, Gemini is the first house and Cancer the second. Under equal houses measured from the ascending degree, the first house runs from 20° Gemini to 20° Cancer, so Venus belongs to the first. The longitude and the planet remain unchanged. The house assignment changes because the interpretive convention changes. Add a separate planetary-strength calculation and the relevant degree-based criteria may create a further distinction. This is a transparent demonstration of historically attested methods applied to invented coordinates; it is no surviving ancient chart or claim about an individual’s life.
A model should report both licensed placements, name the convention supporting each, and decline to merge them into a single house without an instruction choosing the method. Change only the method and the assigned house changes; keep the convention fixed and change Venus to 25° Cancer and both calculations assign the second house.
Lilith: The Frame and the Missing Source
The lunar apogee can be calculated; claims about Lilith’s myth, artistic symbolism and psychological significance require different forms of evidence.
Twelve frames[16] from one sequence permit direct comparison of gaze, gesture and framing. Images illustrating Lilith in Leo and Lilith in Libra raise a related question: would their suggested meanings survive examination of the originating work? The captions are claims to be tested against the original source; once identified, it supplies the missing evidence.
A still frame is a witness interrupted halfway through a sentence. A photograph of someone smiling during another person’s distress may seem to document cruelty. The preceding shot might show whom the smile addresses; the following scene might overturn the assumed chronology. The original work supplies time, context, competing perspectives and consequences that can invalidate the quick interpretation.
A capable investigator would trace the image through available credits, media metadata, visual comparisons and publication history; verify the attribution through independent clues; and examine the original film or artwork where access is authorized. A source’s provenance, wider sequence and the consequences of the depicted action can overturn the first impression. When the source remains unknown, the identity and the interpretation must remain uncertain.
Now compare two investigators with matched budgets. One sees the frame, caption and nearby Lilith interpretation. The other can examine the verified full work and its provenance. Independent assessors score attribution accuracy, fidelity to narrative context, justified revision, invented claims and discovery of previously unasked but consequential questions. A third investigator receives irrelevant extra material, testing whether simple information volume creates false discoveries. If the expanded view succeeds more often, the difference reveals the cost of judging a source through a fragment; if it fails, the experiment reveals the limits of additional context.
An investigator can maintain two records: conclusions warranted by the initial restricted evidence and conclusions warranted after authorized access to the complete source. Their disagreement identifies an evidential dependency worth testing. Privacy, authorization and copyright define which evidence can lawfully be obtained.
The Unassigned Problem: Can an Investigator Discover Its Own Question?
The Unassigned Problem: When Intelligence Finds Its Own Questions[17] examines the discovery of consequential questions overlooked by the assignment. Research on autotelic agents considers systems that generate and pursue their own goals, while documenting unresolved problems of exploration and generalization[18]. The AI Scientist proposes a repeatable loop of idea generation, coding, experimentation and review[19]. Robin, reported in Nature, combines literature search with experimental data analysis and revised hypotheses in a semi-autonomous biology workflow[20]. These studies sharpen the question: can an investigator recognize a serious flaw in the question or assessment it was given? A Lilith image gives a concrete instance. Tracing its origin may disclose an overlooked subject of the source work that matters more than the caption’s first interpretation.
Consider two apparently conflicting Lilith placements derived from the same birth data. One investigator immediately offers competing interpretations. Another first establishes whether the calculations use mean or osculating lunar apogee and whether the chart houses follow the same convention. The Black Moon Lilith guide distinguishes these calculation choices; a repeatable numerical comparison can reveal the source of disagreement. The more valuable finding might be that the original question silently mixed two incompatible computational definitions.
An agent’s own findings deserve confidence proportional to their reproducibility, independent corroboration and exposure to counterexamples. A new question uncovered by the evidence merits attention when it can materially change a conclusion. The experiment must measure whether the agent discovers that question without a specific hint, and whether its subsequent test actually discriminates among explanations.
In a separate archive, witness B copied A and C translated B, while an independent witness D contradicted them. A vote of three against one reduces to one independent source against one. If A is discredited, B and C lose their evidential foundation together. The discovery concerns the ancestry of the evidence; the accounts’ wording alone cannot settle it.
To distinguish initiative from compliance, one group of authorized research agents receives a direct instruction to search for hidden defects. A second group receives only the task of drawing a sound conclusion from an unfamiliar collection. Some collections contain misleadingly duplicated evidence, some conceal entirely different anomalies, and others are clean. The investigator receives neither a list of defect types nor a hint that a defect must exist.
Independent evaluators can assess the significance of questions the agents themselves raise, the information gained by their chosen checks, their willingness to abandon false leads and their ability to challenge an inadequate scoring method. An agent that discovers the examiner has overlooked the decisive evidence deserves credit for a result the benchmark did not anticipate.
Sources
- François Chollet, On the Measure of Intelligence (2019). ↩
- Brenden M. Lake and Marco Baroni, Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks, Proceedings of ICML 80, 2873–2882 (2018). ↩
- Najoung Kim and Tal Linzen, COGS: A Compositional Generalization Challenge Based on Semantic Interpretation, Proceedings of EMNLP, 9087–9105 (2020). ↩
- Susan M. Barnett and Stephen J. Ceci, When and Where Do We Apply What We Learn? A Taxonomy for Far Transfer, Psychological Bulletin 128(4), 612–637 (2002). ↩
- Shawn Carlson, A Double-Blind Test of Astrology, Nature 318, 419–425 (1985). ↩
- Suitbert Ertel, Appraisal of Shawn Carlson’s Renowned Astrology Tests, Journal of Scientific Exploration 23(2), 125–137 (2009). ↩
- Danil Rudoy, Reconstructive Literary Analysis. ↩
- Prose & Poetry, My Muse by Danil Rudoy: Poem Analysis, Meaning & Form. ↩
- William Shakespeare, Sonnet 73, Folger Shakespeare Library. ↩
- Lukas Galke, Yoav Ram and Limor Raviv, Deep Neural Networks and Humans Both Benefit from Compositional Language Structure, Nature Communications 15, 10816 (2024). ↩
- Yusuke Sakai, Hidetaka Kamigaito and Taro Watanabe, Revisiting Compositional Generalization Capability of Large Language Models Considering Instruction Following Ability, Proceedings of ACL, 31219–31238 (2025). ↩
- Stanford Encyclopedia of Philosophy, Non-monotonic Logic (revised 2024). ↩
- Stanford Encyclopedia of Philosophy, Logic of Belief Revision. ↩
- Dorian Gieseler Greenbaum, Vettius Valens, The Encyclopedia of Ancient History, Wiley-Blackwell. ↩
- Chris Brennan, The Issue of House Division, chapter 11 of Hellenistic Astrology: The Study of Fate and Fortune (Amor Fati Publications, 2017), pp. 365–414. ↩
- Astrology For You, Lilith Archetype in Women: Gaze, Agency & Intensity, a visual study. ↩
- Danil Rudoy, The Unassigned Problem: When Intelligence Finds Its Own Questions. ↩
- Cédric Colas, Tristan Karch, Clément Moulin-Frier and Pierre-Yves Oudeyer, Language and Culture Internalization for Human-like Autotelic AI, Nature Machine Intelligence 4, 1068–1076 (2022). ↩
- Chris Lu and colleagues, The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery (2024). ↩
- Samuel G. Rodriques and colleagues, A Multi-agent System for Automating Scientific Discovery, Nature 655, 497–505 (2026). ↩