Response Synthesis

Strategic Direction

Research focus and pivotal questions

Responses cluster around two broad areas, with a clear tension between fit and impact.

The AI × economics/labor cluster draws the strongest convergence. Pacchiardi, Garavaglia, Du, Habermacher, Kao, and Tagat all point here, with framings ranging from macro-trend tracking à la Ord"AI-economics interface (eg impacts on labour market [Anthropic's work] or macro-trends à la Ord) seems the most relevant area"— Lorenzo Pacchiardi to workplace retraining, privacy, and AI literacy/inequality. Habermacher pushes for politically grounded regulatory frameworks for labor impacts"politically grounded (I mean 'realpolitik' type not 'purely ivory tower abstract') Regulatory Frameworks for dealing with labor impacts"— Florian Habermacher — though DR notes this needs concrete examples: what methods, what producers, what decision-relevance? Tagat observes there is already substantial labor-impacts work in the pipeline, which from UJ's curation standpoint is a feature not a bug. DR: "Why is it a 'warning'? That would seem to be a good thing. We're not the ones doing the research; we're the ones prioritizing it, curating it, commissioning its evaluation." Kao additionally suggests evaluating high-effort informal pieces like Trammell/Patel and Citrini"Phil Trammel and Dwarkesh Patel's post on Capital in the 22nd Century ... Citrini research's 2028 global intelligence crisis"— Andrew Kao. The open question DR flags: does this cluster reach the highest-GCR-impact issues, or stay comfortably mid-impact?

The governance/risk-parameters cluster (Schwab, Kalkar) is more GCR-adjacent but stretches UJ's comfort zone. Kalkar proposes evaluable questions on what governance mechanisms actually change frontier lab behavior"What governance mechanisms actually change frontier lab behavior, and under what conditions?"— Uma Kalkar and quantitative risk-parameter estimation (DR: "This seems quantitative, which is more comfortable to us"). Schwab pushes toward middle-power regulatory strategies and lessons from arms control with China"effective middle power strategies for AI regulation ... how has China cooperated in other risk domains? What treaties has it ratified?"— Josephine Schwab — DR flags this as a possible move away from UJ's quantitative/empirical comfort zone toward IR-style analysis, and wants concrete examples before committing.

Staying timely

Convergent process proposals:

Fast-track with one evaluator. Manheim, Garavaglia, and Kao agree that securing >1 evaluator should not block publication. Manheim proposes inviting multiple reviewers and moving forward as soon as one is submitted"Fast track, 1+ evaluators, invites to more than 1 reviewer and move forward as soon as 1 is submitted"— David Manheim.

Pre-booking evaluator time. Pacchiardi suggests reserving reviewer time before paper selection, paying more for flexibility, and targeting specific cruxes requiring only ~1 hour of work"targeting very specific cruxes in a paper ... could be more efficient, for instance requiring reviewers to be useful with 1 hour of work only"— Lorenzo Pacchiardi. DR: "This seems like a great idea to me. Talked about it in the past, as well as giving evaluators some limited choice over which ones they would like to evaluate."

Commentary roundups. Kao proposes ACX/Zvi-style clustered comment digests as a complement to formal evaluation. Manheim adds fast peer review of normally non-peer-reviewable documents like model cards and technical reports"peer review of non-peer reviewable documents (Model cards, technical reports from think tanks, etc)"— David Manheim. DR notes this proposal sits awkwardly with the consensus against technical-safety expansion below.

AI assistance with human-in-loop. Garavaglia endorses AI-assisted briefings; Kalkar cautions against deeper automation for quality reasons. Tagat suggests an NBER-style pipeline with quicker turnaround.

Who to bring in

Specific names: Jonathan Prunty (CFI Cambridge) and Marko Tesic (DSIT) for labor markets (Pacchiardi). Networks: RAND TASP Fellows and GovAI Fellows (Kalkar, Manheim), BlueDot (Manheim), the AI Slack channel for crowdsourcing (Tagat), the ILO and AI developers themselves (Du). Tagat also recommends engaging AI/x-risk funders (Schmidt) to evaluate work they commission.

The Ord RL Scaling Evaluation

Should we still pursue it?

The picture has changed materially. Before Ord's April 29 reply, views ranged from "moment passed" (Schwab) and "not top priority" (Habermacher) to genuine on-the-fence (Pacchiardi, Manheim) to enthusiastic first-mover framing (Kalkar: a window of opportunity given Ord's reframe to 2027+"it may actually make sense to be a 'first-mover' and evaluate it now ... there's a window of opportunity to assess in light of the other RL papers"— Uma Kalkar).

Ord's April 29 reply substantially reopens the question. Three updates matter:

1. Ord endorses formal analysis: "I'd be very happy for there to be more formal analysis of these questions (mine were very quick analyses)." He won't participate but doesn't object.
2. There is an unwritten wrap-up essay Ord intended for the EA Forum Scaling Series but won't write — a gap a UJ synthesis could directly fill.
3. He clarified his quantitative position more precisely than anywhere public: speed was ~60% compute / 40% other in the GPT-2/3/4 era; ~half the compute boost is going away, leaving ~30% + 40% = 70% of prior speed absent new compensating factors. This is not "RL has hit a fundamental limit" — it is a claim about declining relative contribution, with explicit room for algorithms, data, RL environments, or recursive self-improvement to compensate.

This gap between Ord's actual modest position and the "near effective limit" headline reading is itself worth documenting. The "moment passed" view should be updated accordingly.

If yes — what form?

Kalkar's proposal is the most concrete: a long-form EA Forum/PubPub post separating empirically established, contested, and open"long-form EA Forum/ PubPub post. It should cover the full evidence base ... with clear separation between what's empirically established, what's contested"— Uma Kalkar claims, with a draft circulated to GovAI (and Ord). Given Ord's endorsement, this now looks tractable. A crux-targeted, ~1-hour Pacchiardi-style review of the specific 60/40 → 30/40 claim could be a lightweight complement.

Points of apparent consensus

• AI × economics/labor is squarely in UJ's wheelhouse and well-supplied with evaluable work.
• Process should accept single-evaluator publication when needed; pre-booking and crux-targeting are promising.
• Broad reluctance to expand into technical AI safety — though DR flags that rapid, credible evaluation of non-alignment technical safety work may genuinely be uncovered.
• GovAI, RAND TASP, and the existing AI Slack are the right first-call networks.

Key tensions worth discussing

• Wheelhouse vs. GCR-relevance: labor-market work fits comfortably but may not reach the highest-impact questions; governance and risk-parameter work is more GCR-relevant but stretches expertise.
• Model cards/technical reports vs. anti-expansion consensus: Manheim's proposal arguably is technical-safety territory.
• Speed vs. rigor: single-evaluator fast-track and AI assistance vs. Kalkar's quality concerns.
• The Ord evaluation specifically: respondents leaned skeptical, but Ord's own endorsement and newly precise quantitative framing materially update the calculus — worth revisiting now.