Deep Pearl AI

Independent research lab

Intelligence, measured. Judgment, audited.

Deep Pearl AI studies two sides of the same question: how do we know an AI system actually works — and how do we know the judge grading it is right? Multivac measures frontier models with blind, cross-family peer evaluation. PartialCredit measures the judges themselves against code that actually runs.

NACRE SIM LAYER 00/44 Δr 0.00px T+0.0s

Evaluation
Multivac
Judge audit
PartialCredit
Status
Paper under review · preregistered study

The work

Two instruments, one thesis

Frontier models are judged by benchmarks they can game, then graded by judges nobody has checked. We work both ends of that gap. Each instrument below is a working model of its research — open the drawer to see it run.

Multivac

Paper under review

Fig. 1 · The peer matrix

Independent, blind evaluation of frontier language models. Fresh questions models haven't memorized, judged by a cross-family peer matrix — no vendor grades its own homework.

Open Multivac

PartialCredit

Preregistered · pilot complete

Fig. 2 · The pass vector

Pairs of partially-correct code fixes, scored by hidden tests, shown to LLM judges blind. It measures how often a judge prefers the fix that execution says is worse — and whether that rate climbs as the two fixes get closer.

Open PartialCredit

Working note

Superintelligence from first principles

There are many definitions of superintelligence. Here is what we have understood so far.

Superintelligence is not magic and not a mood. It is a claim about optimization: that a system can be built which is far better than humans at achieving goals across almost every domain.

Stripped to essentials

  1. Intelligence is the ability to achieve goals across many environments.
  2. Superintelligence greatly exceeds humans at that, in almost every domain.
  3. Physics permits it. The human brain is a floor, not a ceiling.
  4. Progress is fast and measured. The timeline is uncertain.
  5. The brakes are data, energy, experiments, the test to reality gap, and verification.
  6. The central problem is trust: verifying a mind smarter than the grader.
  7. Control is layered. No single method is enough.
  8. Watch time horizons, AI driven research, verifiable discoveries and safety progress.

Using a first principles lens, the deepest question about superintelligence is not whether it can be built. Physics allows it and the trend points toward it. The question is the one Asimov asked in 1956: when the answer comes from something smarter than us, how do we know it is true?

Read the full breakdown

How we work

First principles, honestly reported

01

No one grades their own homework

Evaluations are blind and judged across model families. A benchmark's design shapes its rankings, so we design against our own bias first.

02

Run the code, then audit the judge

PartialCredit scores every patch on a hidden, human-authored test suite that no judge ever sees. Pass rate is a proxy, so a blind manual audit decides whether a disagreement is a judge error, a weak test or ambiguous.

03

Preregister, then look

Hypotheses, exclusions and analyses are frozen before the data exists, and every change is a dated amendment. Inconclusive is published as inconclusive.