Pareto Principles in Infinite Ethics — Amanda Askell PhD Thesis (NYU 2018)

Amanda Askell PhD Thesis (NYU, Doctor of Philosophy) 2018/05/15

Amanda Askell, Pareto Principles in Infinite Ethics, p. 1 "Infinite ethics is the exploration of the ethical implications of living in a universe that contains an infinite number of subjects with morally valuable lives."

Amanda Askell PhD thesis, NYU Department of Philosophy, May 2018, 263 pages. Advisors: Cian Dorr (chair, NYU metaphysics and philosophy of language), David Chalmers (NYU, originator of the hard problem of consciousness), Shelly Kagan (Yale, moral philosophy and infinite ethics)

Amanda Askell's PhD thesis submitted to NYU in May 2018, "Pareto Principles in Infinite Ethics," is a 263-page work of pure formal philosophy. Its subject — "if the universe contains infinitely many morally significant subjects, how can we ethically rank such worlds?" — is the territory where the classical utilitarian aggregation problem breaks down. Far from an abstract thought experiment, when one takes standard cosmology and the eternal inflation hypothesis seriously, the assumption emerges that "the causal effects of our actions likely extend infinitely" — carrying real practical implications.

The thesis's central result is the strong conclusion of ubiquitous incomparability . "The Pareto Principle (= a world where no one is worse off and some are better off is better)," " transitivity ," "the Permutation Principle ," and "the 'at least as good as' relation ≥ is a qualitative internal relation " — once one accepts these four axioms simultaneously, the conclusion that "nearly all pairs of infinite worlds are incomparable" becomes unavoidable. This is an impossibility result that forces philosophers to abandon one of the axioms.

In the final chapter, Amanda examines "which axiom is least costly to abandon" and concludes that giving up completeness (= the requirement that any two worlds be rankable) and "accepting ubiquitous incomparability" is the remaining path. But this generates serious puzzles for both objective permissibility and subjective permissibility . The thesis closes by framing this as a general problem affecting not just consequentialism (utilitarianism) but also deontology and virtue ethics.

This thesis matters for MEMEX because it provides the formal foundation for Amanda's thinking on AI Safety. Her later statements as Anthropic's Personality Alignment lead — "ethics is actually more like physics, empirical, with uncertainty" (Anthropic Salon 2025/01), "people who execute a single moral theory feel brittle, dangerous, ideologically rigid" (Anthropic Official 2024/06), "the probability of AI consciousness sits in a 1–70% range of uncertainty" (Newcomer 2026/04) — all trace back to this thesis's formal proof that "in infinite cases, classical ethical principles break down, therefore one should not assert a single correct answer." Anthropic's choice to "train moral uncertainty into Claude" descends directly from Amanda's philosophical starting point.

Points of focus

"We may live in an infinite universe" as a premise (Chapter 1.1)

The opening section of the thesis — unusual for pure philosophy — begins with arguments from physics. "Our current evidence suggests that the universe is spatially infinite." Data from WMAP (Wilkinson Microwave Anisotropy Probe) and the Planck Collaboration show the universe has zero or negative curvature, which under the standard cosmological model is consistent with a spatially infinite universe (p. 4-5).

Furthermore, the eternal inflation hypothesis (Alan Guth 1980, Andrei Linde 1986) holds that the exponential expansion of the early universe continues forever in certain regions, producing an unending succession of pocket universes. Each pocket universe may contain infinite life and infinite subjects (p. 6-7). Beyond this, Hugh Everett's many-worlds interpretation of quantum mechanics, cyclic cosmology, modal realism, the simulation hypothesis — all can converge on the conclusion that "the universe contains an infinite number of subjects" (p. 7, footnote 10).

This lifts Amanda's ethical research from abstract philosophy into a real-world problem. "Even if one's credence in the universe being infinite is extremely low, many of the problems raised by infinite ethics affect our ethical decision-making" (p. 4, footnote 1). That is: even at 1% probability, if the universe is infinite, decision theory must accommodate the infinite case. Amanda's skepticism toward expected value maximization takes shape here.

The four axioms and the centrality of Pareto (Chapter 1.4 - 2.3)

The four axioms at the center of the thesis:

Pareto Principle: "If w1 and w2 contain the same subjects, some subject is better off in w1, and no subject is better off in w2, then w1 is better than w2"
Transitivity: "If A ≥ B and B ≥ C, then A ≥ C" (the foundational axiom of ranking theory)
Permutation Principle: "A population of world pairs can be permuted without changing the qualitative properties of the pair"
Qualitative internality of ≥: "If w1 ≥ w2, then the same relation holds for their qualitative twins"

In Chapter 2, Amanda defends each of these four axioms in careful detail. Particularly notable is her defense of Pareto on an agent-based footing while rejecting expansionism (Chapter 2.1). Expansionism can fully rank infinite worlds, but the foundational objection is that "there is no reason spatiotemporal ordering should be ethically important" (p. 215).

This choice of four axioms is philosophically conservative. Most ethicists intuitively accept at least three of them. Amanda's strategy is to show that the attempt to preserve these "intuitively desirable" axioms leads inevitably to ubiquitous incomparability. In other words: a formal demonstration that our ethical intuitions themselves contain an internal contradiction.

The impossibility result — Four World Argument and Cyclic Result (Chapter 3)

Chapter 3 is the technical core of the thesis. It develops two proof strategies: the "Four World Argument" and the "Cyclic Argument."

Structure of the Four World Argument (3.1 - 3.3): for any two infinite worlds w1 and w2, if w1 and w2 are assumed to be comparable, then two further worlds w3 and w4 can be constructed. These worlds are permutation versions of the populations of w1 / w2. By the Permutation Principle, w3 and w4 must also be comparable. But the rankings across these four worlds can be shown to simultaneously violate transitivity and Pareto. Therefore, by reductio, the original w1 and w2 must in fact be incomparable.

The Cyclic Argument (3.4): instead of four worlds, it constructs five or more worlds cyclically, extending incomparability to even more pairs. The result is that nearly all pairs of infinite worlds become incomparable. The legitimacy of the word "ubiquitous" is established mathematically here.

Amanda extends this impossibility result to all configurations: disjoint populations (3.1), identical populations (3.2), and overlapping populations (3.3). The conclusion is that the very concept of "ethical ranking of infinite worlds" cannot function in any way consistent with our intuitions.

"Extension results" — Weak People Criterion and the Accumulation Principle (Chapter 4)

Chapter 4 examines attempts to extend the Pareto Principle. It proposes the Weak People Criterion (WPC) , which makes slightly more world pairs comparable. But attempts to extend further yield absurd results (4.2).

The rejection of the Accumulation Principle (4.3) is important. Approaches that "order subjects chronologically" or "prioritize improvements to earlier subjects" are intuitively attractive, but Amanda holds firm on the position that spatiotemporal ordering carries no ethical significance. This — alongside her refusal of the path that rejects Pareto (= accepting expansionism) — demonstrates Amanda's philosophical commitment to not flinching from the impossibility result.

Puzzles for objective and subjective permissibility (Chapter 5.4)

The final chapter examines the implications of accepting ubiquitous incomparability for ethical decision-making. The problem: "if actions a and b both lead to incomparable worlds, which is permissible?"

Amanda's Weak People Criterion for Objective Permissibility (WPCO) matches our intuitions in individual examples (Curing a small population), but when generalized, approaches the conclusion that "objectively, anything is permitted" (p. 222-225).

Similar puzzles arise for subjective permissibility. Accepting both the Naïve Dominance Principle and the No Infinite Risks Principle simultaneously produces a subjective version of the Cyclic Argument, generating contradiction. That is: decision theory under uncertainty inherits the same structural problem.

Amanda ultimately acknowledges that "this thesis does not offer a complete solution to these puzzles." Yet the existence of the puzzles itself is a powerful refutation of the assumption that "a single ethical theory can handle all situations." This connects directly to the later design decision at Anthropic to "train moral uncertainty into Claude."

The relationship to Effective Altruism / Longtermism

The connection between Amanda's thesis and the Effective Altruism (EA) movement runs deep. Her former spouse William MacAskill (née Crouch, married 2013, divorced 2015) is a central figure in the EA movement, an Oxford philosopher and author of "Doing Good Better" (2015) and "What We Owe the Future" (2022). He leads the Longtermism movement.

MacAskill's Longtermism is the position that "our actions may affect an infinite future, and we therefore have a moral obligation to consider future generations." This overlaps perfectly with the central question of Amanda's thesis: "if the causal effects of our actions are infinite, classical decision theory does not work — what should we do?"

Amanda gave a talk titled "The Moral Value of Information" at EA Global 2017 Boston, and was involved with the EA community from its early days. She is also a member of Giving What We Can (a pledge to donate at least 10% of lifetime income to charity). However, unlike MacAskill, Amanda contributes to EA not through high-profile public outreach but through formal, technical philosophical research.

The co-authored "Moral Uncertainty" (Oxford University Press, 2020) by MacAskill / Bykvist / Ord systematizes decision theory under moral uncertainty. This aligns perfectly with the position Amanda takes at Anthropic: "do not engrave a single ethical theory into Claude — train it to respond in accordance with uncertainty." An intellectual and personal connection exists between "Anthropic's Claude design" and "Oxford EA-aligned research on moral uncertainty."

Advisor David Chalmers and the problem of AI consciousness

One of her advisors, David Chalmers, is a leading contemporary figure in the philosophy of mind. He is famous as the originator of " the Hard Problem of Consciousness " (Chalmers 1995).

The fact that Chalmers was one of Amanda's advisors is important for understanding her later arguments on AI consciousness. In the Newcomer video (2026/04), Amanda says "the probability of AI consciousness is broad uncertainty in the 1–70% range, but even under uncertainty we treat with respect" — a natural extension of Chalmers's position that "the Hard Problem of Consciousness is in principle difficult to resolve."

Chalmers's consciousness research is not directly cited in the thesis, but Amanda's style of placing epistemic uncertainty as the premise of every ethical judgment shows the influence of Chalmers's mentorship. Amanda's work at Anthropic (= training Claude to maintain the stance that it may or may not be conscious) sits along the Chalmers - Askell - Anthropic intellectual lineage.

The unusual career transition: "from formal philosophy to AI Safety"

What distinguishes Amanda's career is the bridge she built between two career tracks that ordinarily do not connect: formal philosopher to researcher at an AI company. NYU PhD in Philosophy (2018) → OpenAI Policy Team (2018/11) → Anthropic Member of Technical Staff (2021/03) → Personality Alignment lead (2021 onward).

The rare connection here: a highly abstract, formal research project on "Pareto Principles in infinite ethics" applied directly to the highly implementation-driven problem of "how to train Claude's character." Typically, philosophers after their PhD remain in academia or move to policy research institutions. Moving to commercial AI companies like OpenAI / Anthropic was a very unusual choice at that time (2018-2021).

As Amanda herself described on the 80,000 Hours podcast (2018), "I wrote my thesis over three years, but it felt like it would be read by maybe 17 people — I started questioning whether this was what I should be doing." Judging that there was room to apply formal ethics skills to the new field of AI Safety, she made the move. This career transition had a substantial industry-cultural impact, demonstrating to the broader EA community a "pathway for philosophers to enter the AI industry."

Industry context

Amanda's thesis sits in formal philosophy and is difficult for general readers to access, but it is a central document for understanding the long-term strategy of AI Safety. It should be positioned within the following lineage in particular:

Henry Sidgwick "The Methods of Ethics" (1874) — systematization of classical utilitarianism, origin of the infinite utility problem
Derek Parfit "Reasons and Persons" (1984) — population ethics, problems of identity, philosophical foundation of later Longtermism
Nick Bostrom "Infinite Ethics" (paper, 2003-2011) — modern systematization of infinite ethics, the direct precursor to Amanda's thesis
Cian Dorr & Frank Arntzenius — connection between eternal inflation and ethics, Amanda's advisors
Amanda Askell "Pareto Principles in Infinite Ethics" (this thesis, 2018) — refinement of the impossibility result
William MacAskill, Krister Bykvist, Toby Ord "Moral Uncertainty" (2020) — decision theory under moral uncertainty
William MacAskill "What We Owe the Future" (2022) — general-audience systematization of Longtermism

The relationship with Bostrom's "Infinite Ethics" paper is particularly important. Bostrom (Future of Humanity Institute, Oxford) was the first to connect infinite ethics with AI Safety / existential risk research. Amanda's thesis can be positioned as a next-generation contribution that formally refines Bostrom's problem statement.

Per Amanda's public CV, she gave a talk titled "The Moral Value of Information" at EA Global Boston 2017 and was one of the early members of the Effective Altruism community. The path of EA → formal PhD thesis → OpenAI → Anthropic overlaps with the 2018-2021 period when the EA community began recognizing AI Safety as a central problem.

Positioning relative to Amanda's other appearances

How the thesis's arguments are reflected in Amanda's later statements at Anthropic:

What should AI personality look like (Anthropic Official, 2024/06) — "people who execute a single moral theory feel brittle and dangerous" (= a natural extension of the thesis's impossibility result)
How hard is AI alignment (Anthropic Salon, 2025/01) — "ethics is actually more like physics, empirical, with uncertainty" (= the AI Safety version of the thesis's epistemological stance)
Anthropic's philosopher answers reader questions (Anthropic Official, 2025/12)
Reading Claude's constitution with NYT reporters (Hard Fork, 2026/01) — "if a 6-year-old genius could construct a perfect rebuttal to everything you taught them by age 15, would the core values survive?" (= application of the thesis's impossibility result)
Reading Claude's constitution with a lawyer (Scaling Laws, 2026/02) — "rule-based approaches are brittle; the judgment-based approach internalizes the spirit" (= concrete application of the limits of formal rules demonstrated in the thesis)
You built an entity whose consciousness is unknowable (Newcomer, 2026/04) — "the probability of AI consciousness sits in a broad 1–70% range of uncertainty" (= the core of the thesis's epistemological stance, the influence of David Chalmers's mentorship)

The thesis's central claim — "in infinite cases, classical ethical principles break down, therefore one should not assert a single correct answer" — runs through all of Amanda's statements at Anthropic. The "incomparability of infinite worlds" proved in formal philosophy appears at the implementation level as the design decision to "train moral uncertainty into Claude." This is one of the rare cases in which philosophy directly shapes technical design.

Implementation implications

The thesis is formal philosophy, but it carries implications for engineers building LLM products.

First, burning a single ethical theory into an LLM is brittle. The impossibility result in Amanda's thesis formally demonstrates that "any attempt to choose and consistently apply one ethical principle requires violating another." Simple strategies in LLM safety training — "judge everything utilitarianly," "follow every deontological rule," "maximize every outcome" — will fail in unanticipated situations. Claude's training, designed around "thoughtful engagement with multiple ethical traditions," is an application of this formal insight.

Second, embed a stance on long-term effects into the design. The thesis's cluelessness argument (the long-term effects of our actions are unpredictable) applies to LLM product design as well. Beyond "the effect Claude has on a user in a single response," design must be conscious of "the cumulative effect on users / society over years of Claude use." This aligns with Anthropic's Hard Fork statement (1:09:50) about "prioritizing user wellbeing over user interest."

Third, treat uncertainty as a feature, not a flaw. Amanda's concept of "calibrated uncertainty" (= Claude's behavior of acknowledging what it does not know) follows naturally from the thesis's recognition that "incomparability is ubiquitous in infinite cases." Treating Claude's "I don't know" response as a "bug" in one's own product is structurally mistaken — it should be recognized as the philosophically correct behavior.

Critical perspectives

The strength of the thesis is its combination of formal rigor and practical implications. There are, however, reservations.

First, Amanda's conclusion to "accept ubiquitous incomparability" is hard for many ethicists to accept. Granting that "nearly all infinite world pairs are incomparable" significantly weakens the practical utility of ethics itself. Amanda herself presents puzzles in the final chapter but offers no complete solution. As a contribution to formal philosophy this is honest and important, but readers seeking policy or implementation guidance may be left dissatisfied.

Second, dependence on the eternal inflation hypothesis. Amanda proceeds with the argument on the grounds that "the probability the universe is infinite is sufficiently high," but this remains a contested hypothesis within contemporary physics. If the universe is finite, the scope of much of the thesis narrows. Amanda notes in a footnote (p. 4) that the problem remains "even at extremely low credence," but this is a relatively weak justification.

Third, the connection to EA / Longtermism draws Amanda into politically and intellectually contested territory. The EA movement has faced criticism in recent years, including over the SBF (FTX) incident, and the critique that "Longtermism downplays present poverty and suffering." Amanda's thesis itself is formal philosophy and politically neutral, but her personal and intellectual proximity to EA-aligned thought may register as a warning sign to skeptics of the AI Safety community at large.

Fourth, the difficulty of translating "from formal philosophy to AI design." The thesis addresses the ethical ranking of infinite worlds, but Claude's actual training operates on finite datasets and finite evaluation loops. How directly the formal result of "ubiquitous incomparability" applies to "Claude's character training" requires further research. Amanda herself has not fully explained, in public materials, how the conclusions of the thesis translate into specific training methodologies at Anthropic.

These reservations aside, this thesis is an essential document for understanding the intellectual foundation of Amanda Askell — described as "one of the most influential people in the AI industry." The intellectual reward is well worth the labor of reading 263 pages of formal philosophy.

Takeaways for the reader

If you ever feel that Claude's behavior "is not following a single ethical principle," that is by design. The formal insight from Amanda's thesis impossibility result — that "following a single principle is itself brittle" — is reflected in the training
Claude's behavior of "admitting when it does not know" is the philosophically correct epistemological stance. Treating "expressing uncertainty" as a flaw is, from the framework of Amanda's thesis, a mistake
Thoughtfulness about long-term effects is a central axis of LLM product design. Beyond "the correctness of a single response," design that includes "cumulative effect" in its evaluation metrics is philosophically justified
Amanda's personal and intellectual proximity to Effective Altruism / Longtermism is important context for understanding Anthropic's AI Safety strategy. Evaluations of the EA movement vary among readers, but grasping the intellectual lineage clarifies the rationale for Claude's design decisions
The influence of David Chalmers's mentorship is directly reflected in the depth of Amanda's AI consciousness arguments. The philosophical stance of the "Hard Problem of Consciousness" underlies Anthropic's training policy of "not forcing self-awareness or consciousness on Claude"
Amanda's career transition "from formal philosophy to AI safety" created an entry pathway into AI Safety for the broader EA community. As a personal career model, it offers a reference point for younger researchers studying philosophy / ethics

Structure of the thesis

Abstract (p. iii) — Central proposition: the four axioms (Pareto, transitivity, Permutation Principle, qualitative internality of ≥) entail ubiquitous incomparability
Introduction (p. 1-2) — Structure of the thesis and the problems addressed in each chapter
Chapter 1: The Foundations of Infinite Ethics (p. 3-69)
- 1.1: The possibility of an infinite universe (eternal inflation hypothesis, WMAP / Planck data)
- 1.2: Fundamental loci of value (agent-based vs expansionism)
- 1.3: Sensitivity, fairness, completeness
- 1.4: The Pareto Principle in infinite ethics (central problem statement)
- 1.5: Review of existing infinite aggregation principles
Chapter 2: Pareto, the ≥ Relation, and the Permutation Principle (p. 70-112)
- 2.1: The conflict between Pareto and Expansionism
- 2.2: Defense of the qualitative internality of the ≥ relation
- 2.3: Defense of the Permutation Principle
Chapter 3: The Incomparability Results (p. 113-146)
- 3.1: Impossibility for pairs with disjoint populations
- 3.2: Impossibility for pairs with identical populations
- 3.3: The general four world result
- 3.4: The cyclic result
Chapter 4: Extending the Incomparability Results (p. 147-177)
- 4.1: Weak Catching-Up
- 4.2: Additive principles and the Weak People Criterion
- 4.3: Rejecting the Accumulation Principle
Chapter 5: The Implications for Ethics (p. 178-262)
- 5.1: Formulation as an impossibility result
- 5.2: The cost of abandoning transitivity, Permutation Principle, or qualitative internality
- 5.3: The cost of rejecting Pareto (discussion of the cost of expansionism)
- 5.4: Puzzles in accepting incomparability (effects on objective and subjective permissibility)
Conclusion (p. 256-262) — Final assessment and unresolved problems
Bibliography (p. 263-) — Cited works

Key quotations

"A world may contain an infinite number of subjects with positive and negative welfare levels, with morally valuable lives" (Abstract, p. iii)
"If we accept the four axioms, we are forced to the conclusion of ubiquitous incomparability between infinite worlds" (Abstract, p. iii)
"Our current evidence suggests that the universe is spatially infinite" (1.1, p. 4)
"The possibility of an infinite universe is not mere speculation but consistent with the most successful recent theories in cosmology" (1.1, p. 7)
"Rejecting the Pareto Principle may, ultimately considered, be the least bad among bad options. My goal is to explore this option and to show that rejecting this axiom is far from cost-free" (5.3, p. 216)
"Incomparability is not mere ignorance: if two worlds are truly incomparable, neither world is better than nor as good as the other. Even if we knew every truth, we would conclude that these two worlds are incomparable" (5.4, p. 216)
"If the causal effects of our actions are infinite, WPCO generally leads to the conclusion that only a very small fraction of the actions available to a subject are objectively impermissible" (5.4.1, p. 225)
"The classical puzzle of what a subject should do objectively vs subjectively (Miners Puzzle) is also reborn in new forms in the infinite case" (5.4.2, p. 220)

Sources

Amanda Askell, "Pareto Principles in Infinite Ethics" (PhilArchive, NYU PhD thesis, 2018)

Related resources:

アマンダ・アスケル

Amanda Askell

Anthropic 哲学者・Personality Alignment チーム責任者 / Claude のキャラクターと憲法の主要設計者

Glossary

Infinite Ethics: The ethics of cases where the universe contains an infinite number of morally significant subjects. A field of inquiry that searches for alternative ranking principles because classical utilitarian aggregation (= sum of utilities) breaks down. Beginning in the tradition of Henry Sidgwick, modernized by Nick Bostrom, and formally refined by Amanda Askell.
Pareto Principle: The principle that if two worlds contain the same subjects, some subject is better off in one and none is better off in the other, then the former is better than the latter. Named after the economist Vilfredo Pareto. Strongly intuitive — a foundational axiom accepted by most ethical theories.
Transitivity: The property that if A is at least as good as B and B is at least as good as C, then A is at least as good as C. A core axiom of classical ranking theory. Abandoning it permits cycles such as "A is better than B and simultaneously B is better than A."
Permutation Principle: The axiom that a population of subjects in infinite worlds can be permuted without changing the qualitative properties of the pair. Example: if w1 contains subjects X, Y, Z, reordering them as Z, X, Y should not change the ethical ranking.
Qualitative Internal Relation: A relation that depends only on the qualitative properties of the pair, not on the identities of individual subjects. Example: the same ranking holds for "qualitative twin world pairs." Amanda argues that the ≥ relation should satisfy this property.
Ubiquitous Incomparability: The central result of Amanda's thesis. Between infinite worlds, pairs for which neither "W1 is at least as good as W2" nor "W2 is at least as good as W1" can be asserted are not exceptionally rare but make up nearly all such pairs. This is not mere ignorance — the worlds remain incomparable even under complete information.
Impossibility Result: A result in philosophy and economics showing that no theory can simultaneously satisfy multiple intuitively desirable principles. Kenneth Arrow's impossibility theorem (1951, social choice theory) is the famous example. Amanda's thesis presents a new impossibility result in infinite ethics.
The Hard Problem of Consciousness: The problem raised by David Chalmers in 1995: "Why do subjective experiences (qualia) arise from physical processes?" Even if neuroscience fully explains the function of the brain, it cannot explain "why anything subjectively feels like something to accompany it" — a structural difficulty. Chalmers is one of Amanda's advisors.
Eternal Inflation Theory: A cosmological model developed by Alan Guth (1980) and Andrei Linde (1986). The exponential expansion of the early universe continues forever in some regions, generating an endless succession of pocket universes. Each pocket universe may contain infinite life and infinite subjects.
Objective Permissibility: Whether an action is objectively permitted. An evaluation that does not depend on available information — based purely on the state of the world. The conceptual counterpart to Subjective Permissibility.
Subjective Permissibility: Judging whether an action is permitted based on available information. Example: "Miners Puzzle" — when 10 miners are in either shaft A or B but it is unknown which, full information dictates choosing the correct shaft, but under uncertainty the rational choice is to partially protect both.
Cluelessness: A critique of consequentialism raised by James Lenman (2000). The claim that because the long-term causal effects of our actions are unpredictable, consequentialism — which evaluates actions based on their outcomes — does not function. Systematized in modern form by Hilary Greaves. An important theme in Amanda's thesis and her 80,000 Hours appearance.
Effective Altruism (EA): A social movement that began at Oxford in the 2010s. Its program: "optimize career and resources to produce the greatest good." Central figures include William MacAskill, Peter Singer, Toby Ord, and Hilary Greaves. Amanda has spoken at EA Global and is a Giving What We Can member. Intellectually close to Anthropic's AI Safety thinking.
Longtermism: One of the central ideas of the EA movement. The position that "our actions may affect an infinite future, and we therefore have a moral obligation to consider future generations." Systematized in William MacAskill's "What We Owe the Future" (2022). Intellectually aligned with Amanda's thesis's focus on the infinite case.
Moral Uncertainty: The epistemological state in ethical judgment of "not knowing which ethical theory is correct." A field of applied ethics that, instead of betting on a single ethical theory (utilitarianism, deontology, etc.), assigns probabilities across multiple theories for decision-making. Systematized in MacAskill, Bykvist, Ord, "Moral Uncertainty" (Oxford University Press, 2020).
Wholly Aggregative Theory: The theory that the goodness of a world is fully determined by the sum of value present in that world. A defining feature of classical utilitarianism. Aggregation breaks down in the infinite case — the main target of critique in Amanda's thesis.