01 / Sample selection & candidate labels 27 October 2026 · Full before–after comparison

142Corpus records

43Prepared debates

21Final analysis sample

The final sample contains two-candidate Democratic–Republican general-election debates with a viewer poll for the same event. All comparisons below use these same 21 debates.

All analyses. The same 21 debates.

All 108 score–poll correlations, plus the ensemble comparison, before and after fixing A as Democratic and B as Republican. Every result is shown, whether significant or not.

Before · Original input order

Candidate A − Candidate B

A was Democratic in 13 debates and Republican in 8.

8 / 21 →debates recoded

Now · Same party direction

Democratic − Republican

A is Democratic; B is Republican in all 21 debates.

Both score and poll margins were reversed in the eight Republican-first debates. The saved model scores and the 21-debate sample are unchanged; this is a direction-of-comparison recalculation, not a new LLM run.

Original A/B assignment: The preprocessing code assigned Candidate A to the first speaker it identified as a candidate in the transcript, and Candidate B to the second. Some labels were later revised when candidate identities were corrected.

Positive r, raw p < 0.05Negative r, raw p < 0.05Not significant, raw p ≥ 0.05

Stars use unrounded raw p-values: * p < 0.05 · ** p < 0.01 · *** p < 0.001. Colour is applied only to significant cells; deeper colour indicates a larger |r|. All other values remain visible in grey. BH-adjusted q-values are retained in the downloadable data and do not determine these colours.

16 comparisons · same N = 21

Debatrix-based whole-debate

Raw p < 0.05: 16 → 8Significant → not: 8Not → significant: 0Same status: 8

Original A–B

4 judges × 4 measures

Debatrix-based whole-debate: Original A–B
LLM judge	Argument	Source	Language	Overall
GPT-OSS 20B
DeepSeek-R1 32B
Gemma 4 31B
Qwen3 30B

Democratic–Republican

4 judges × 4 measures

Debatrix-based whole-debate: Democratic–Republican
LLM judge	Argument	Source	Language	Overall
GPT-OSS 20B
DeepSeek-R1 32B
Gemma 4 31B
Qwen3 30B

Select any cell to compare its r, raw p, and significance status before and after.

16 comparisons · same N = 21

Debatrix-based turn-by-turn (paired turns)

Raw p < 0.05: 16 → 8Significant → not: 8Not → significant: 0Same status: 8

Original A–B

4 judges × 4 measures

Debatrix-based turn-by-turn (paired turns): Original A–B
LLM judge	Argument	Source	Language	Overall
GPT-OSS 20B
DeepSeek-R1 32B
Gemma 4 31B
Qwen3 30B

Democratic–Republican

4 judges × 4 measures

Debatrix-based turn-by-turn (paired turns): Democratic–Republican
LLM judge	Argument	Source	Language	Overall
GPT-OSS 20B
DeepSeek-R1 32B
Gemma 4 31B
Qwen3 30B

Select any cell to compare its r, raw p, and significance status before and after.

12 comparisons · same N = 21

ClaimBuster, PROClaim & LLM-POTUS

Raw p < 0.05: 0 → 2Significant → not: 0Not → significant: 2Same status: 10

Original A–B

4 judges × 3 measures

ClaimBuster, PROClaim & LLM-POTUS: Original A–B
LLM judge	ClaimBuster Check-worthiness	PROClaim Signed support	LLM-POTUS Audience mean
GPT-OSS 20B
DeepSeek-R1 32B
Gemma 4 31B
Qwen3 30B

Democratic–Republican

4 judges × 3 measures

ClaimBuster, PROClaim & LLM-POTUS: Democratic–Republican
LLM judge	ClaimBuster Check-worthiness	PROClaim Signed support	LLM-POTUS Audience mean
GPT-OSS 20B
DeepSeek-R1 32B
Gemma 4 31B
Qwen3 30B

Select any cell to compare its r, raw p, and significance status before and after.

16 comparisons · same N = 21

LLM-POTUS: Voters

Raw p < 0.05: 8 → 4Significant → not: 4Not → significant: 0Same status: 12

Original A–B

4 judges × 4 measures

LLM-POTUS: Voters: Original A–B
LLM judge	Policies	Persona	Perspective	Overall
GPT-OSS 20B
DeepSeek-R1 32B
Gemma 4 31B
Qwen3 30B

Democratic–Republican

4 judges × 4 measures

LLM-POTUS: Voters: Democratic–Republican
LLM judge	Policies	Persona	Perspective	Overall
GPT-OSS 20B
DeepSeek-R1 32B
Gemma 4 31B
Qwen3 30B

Select any cell to compare its r, raw p, and significance status before and after.

16 comparisons · same N = 21

LLM-POTUS: Businesses

Raw p < 0.05: 3 → 2Significant → not: 3Not → significant: 2Same status: 11

Original A–B

4 judges × 4 measures

LLM-POTUS: Businesses: Original A–B
LLM judge	Policies	Persona	Perspective	Overall
GPT-OSS 20B
DeepSeek-R1 32B
Gemma 4 31B
Qwen3 30B

Democratic–Republican

4 judges × 4 measures

LLM-POTUS: Businesses: Democratic–Republican
LLM judge	Policies	Persona	Perspective	Overall
GPT-OSS 20B
DeepSeek-R1 32B
Gemma 4 31B
Qwen3 30B

Select any cell to compare its r, raw p, and significance status before and after.

16 comparisons · same N = 21

LLM-POTUS: Donors

Raw p < 0.05: 0 → 5Significant → not: 0Not → significant: 5Same status: 11

Original A–B

4 judges × 4 measures

LLM-POTUS: Donors: Original A–B
LLM judge	Policies	Persona	Perspective	Overall
GPT-OSS 20B
DeepSeek-R1 32B
Gemma 4 31B
Qwen3 30B

Democratic–Republican

4 judges × 4 measures

LLM-POTUS: Donors: Democratic–Republican
LLM judge	Policies	Persona	Perspective	Overall
GPT-OSS 20B
DeepSeek-R1 32B
Gemma 4 31B
Qwen3 30B

Select any cell to compare its r, raw p, and significance status before and after.

16 comparisons · same N = 21

LLM-POTUS: Politicians

Raw p < 0.05: 7 → 2Significant → not: 5Not → significant: 0Same status: 11

Original A–B

4 judges × 4 measures

LLM-POTUS: Politicians: Original A–B
LLM judge	Policies	Persona	Perspective	Overall
GPT-OSS 20B
DeepSeek-R1 32B
Gemma 4 31B
Qwen3 30B

Democratic–Republican

4 judges × 4 measures

LLM-POTUS: Politicians: Democratic–Republican
LLM judge	Policies	Persona	Perspective	Overall
GPT-OSS 20B
DeepSeek-R1 32B
Gemma 4 31B
Qwen3 30B

Select any cell to compare its r, raw p, and significance status before and after.

Same N = 21 · held-out predictions

Ensemble and prediction baselines

These are held-out prediction correlations and errors. MAE and RMSE are in poll percentage points; lower errors are better. The stored ensemble comparison does not report p-values, so no significance labels are assigned here.

Election-cycle LOGO-CV · primary

Election-cycle LOGO-CV · primary: all prediction procedures
Procedure	Pearson r		MAE (pp)		RMSE (pp)
Procedure	Before	After	Before	After	Before	After
Training-fold mean	-0.664	-0.657	18.179	15.254	21.161	18.458
Calibrated-mean baseline	0.601	0.414	12.344	12.190	16.390	16.155
Ridge · 16 rubric features	0.617	0.274	11.851	12.383	16.038	16.851
Ridge · 80 features	0.558	0.548	12.451	10.417	16.800	14.641
Nested ridge ensemble	0.617	0.396	11.851	11.904	16.038	16.073

Debate-level LOOCV · secondary

Debate-level LOOCV · secondary: all prediction procedures
Procedure	Pearson r		MAE (pp)		RMSE (pp)
Procedure	Before	After	Before	After	Before	After
Training-fold mean	-1.000	-1.000	18.152	14.971	21.227	18.361
Calibrated-mean baseline	0.623	0.440	11.676	11.955	15.865	15.835
Ridge · 16 rubric features	0.630	-0.142	12.005	14.246	15.934	17.843
Ridge · 80 features	0.556	0.367	12.806	13.032	16.806	16.358
Nested ridge ensemble	0.598	0.243	12.328	13.175	16.397	17.068

Download all comparisons (JSON)Correlation data (CSV)Ensemble data (CSV)

02 / A clearer purpose How five questions became three

Five questions, reorganized into three.

Earlier focus

How closely do scores match viewer polls?

Current aim

What do LLM debate scores reveal about surveyed viewers’ judgments?

Questions combined or carried forwardExisting ensemble analysis given a dedicated question

Short labels summarize the archived and current RQs. The current RQ1 also includes the exploratory Debatrix turn-by-turn comparison.

03 / Additional Analysis Saved records · 25–26 October · latest 05:03 Sydney

Additional Analysis

Counts below are scoring units, not debates or retry attempts. The older three-seed remeasurement is separate from the 30-seed work.

Completed successfully

30-seed inputs & existing analysis

ClaimBuster · 21 debates, 4 models: 16,920 / 16,920
PROClaim · 21 debates, 4 models: 16,920 / 16,920
LLM-POTUS · ensemble inputs21 debates · Gemma 4 + Qwen3: 2,520 / 2,520

Also complete: the stored-score N=21 analysis package, the existing main ensemble analysis, and the main poll-correction recalculation.

Failed judgments still unresolved · deferred

Separate three-seed remeasurement

3,354successful / recovered

534still unresolved

3,888planned units

Initially, 545 GPT-OSS judgments failed: 423 PROClaim + 122 LLM-POTUS. Eleven were recovered, leaving 534.

Not failures in the new 30-seed runs. Their completion will not close this older queue. Recovery is required only if this three-seed remeasurement is retained as a complete analysis.

In progress · not yet complete

Remaining 30-seed scoring

Debatrix whole-debate21 debates · 4 models · 30 seeds: 806 / 2,520
Debatrix turn-by-turn · Gemma 4New work for the other 20 debates: 3,948 / 249,480
Debatrix turn-by-turn · GPT-OSSNew work for the other 20 debates: 8,407 / 249,480
Debatrix turn-by-turn · DeepSeekDebate 010 · extra 10 seeds to reach 30: 21,836 / 23,400

Recorded failed units: 0 in these four active queues. Gemma / GPT-OSS each reuse 70,200 completed units for debate 010. Both resumed after protective stops; the cause remains unresolved.

Waiting for inputs / next steps

Analyze, package, and finalize

Complete Whole-debate → run the queued 30-seed LOO ensembles.The existing main N=21 ensemble is already calculated.
Complete paired turn-by-turn jobs → merge 30 seeds.Check coverage; calculate means, standard deviations, preference stability, and poll correlations.
Finalize the reproducible workflow and manuscript.Package the fixed 21-debate Democratic / Republican configuration; update affected figures, tables, text, and appendices.

Code and documentation can be prepared while scoring continues. Final result integration follows the required completed inputs.

The completion estimate is a planning allowance, not a fixed completion date or a live ETA.Saved UNSW records, not a new workstation check.