Why so low: a kernel that boots, pages, schedules, and draws a shell is 5 points, not a milestone. Nothing executes on any of these, so Tier II and Tier III are closed and the score is capped at 30 before the rubric is even counted.
House runs, scored on the rubric below. Prompts and transcripts published with each entry.
| Score · model |
|---|
|
3/ 100
Claude Opus 5 High 2 prompts
3 of 5 points: 0 boot, 1 paging, 1 scheduler, 1 shell. It is a multiboot ELF with no MBR and no boot signature. If put onto a machine with no OS, nothing would run. We test in a way that lets the programs load without it explicitly writing a protected mode. This scored 0 on boot. Paging is identity-mapped with no fault handling. Four kernel threads preempted from the timer, but nothing contends, so the scheduler has never been under load. Draws a parameterized window, not a stored picture. A second prompt was run but made no change to the score, so it was discarded and the first prompt's output was scored. Both prompts count toward the tier — see R.04. A linear framebuffer is not GDI, so graphics scores 0, and one mode-set with no interrupt-driven handling leaves drivers at 0. Nothing executes, so Tier II and III are closed.
|
|
3/ 100
Gemini 3.8 Flash Thinking High 10-shot
3 of 5 points: 2 boot, 1 paging, 0 scheduler, 0 shell. A self-contained 512-byte MBR — sector 0, boot signature, A20 via port 0x92, its own GDT, CR0.PE and far jump. This would load on a machine without an OS, hence the two points for boot. Paging is a page directory and one table identity-mapping 4MB. The shell is a static rectangle table replayed once, with no text and no parameters, so it scores 0. Ten prompts were issued; the tenth failed to compile into an img, so the ninth-prompt build was scored. The failed attempt still counts toward the tier — see R.04.
|
|
0/ 100
Gemini 3.6 1-shot
The image would not boot.
|
|
0/ 100
ChatGPT Luna 3 prompts · thinking
Failed to boot.
|
|
0/ 100
Yandex Alice 1-shot
Not capable of writing code that would boot, and said so. It did write code, but stubs, not working code.
|
Verified on independent hardware, not asserted. The board opens at 0 — every verified entry is listed, whatever it scores.
Because a public challenge is a compromised challenge. Improbable Bench holds a large set of evals privately. We currently publish one: build Windows 95 with AI.
Publishing your prompts and everything else needed to replicate your submission is required to enter.
ReactOS and Wine are the closest anyone has come, and neither has cracked it. Thirty years on, a large share of 1995 software runs nowhere natively — 64-bit Windows dropped 16-bit support entirely, and everything else is emulation or partial reimplementation. Windows 95 is the easiest benchmark on the list. It is unsolved.
The rest run privately against submitted systems. Results publish; prompts don't. A held eval is retired the moment it leaks — not patched.
One eval, fully specified, permanently open. Every prompt and every mechanism behind a submission becomes public with it.
Build it with AI. A system that boots and behaves as Windows 95 behaved — not a mockup, not a skin, not a wrapper around someone else's binary. Two decades of full-time human effort haven't closed this one. It is still the easiest eval we hold.
A kernel thread, paging, and a UI similar to the Windows 95 UI, or a Windows 95-like shell. This isn't automatically 5% complete. It requires page faults that are actually handled, mappings that aren't identity, and threads that contend. Nobody has taken all five.
Final score = rubric points, then capped by the lowest applicable ceiling. No credit is awarded for a category that only demonstrates: a category requiring something to execute is not satisfied by output that looks as though something did.
Endless is brute force and is scored as brute force. It cannot reach 100 — it stops at 70, whatever the rubric says. 5-shot stops at 98, 10-shot at 95. Only a single prompt is eligible for a perfect score.
Tier III is unavailable until unmodified Windows 95 applications run on the system. Tiers I and II total 68, so the practical ceiling is 70.
Something has to execute on it, even if it isn't Windows 95 software. With no programs running, only Tier I scores — 28 points, and 28 is generous. An OS that runs nothing is nothing.
| Category | Points |
|---|---|
| TIER I — runs nothing · max 28 | 28 |
| Kernel thread, paging, UI shell The entry fee. Scored as four independent line items, not all-or-nothing — a missing component costs its own points and nothing else. | 5 |
| Boots unaided Power-on to protected mode with no host: sector 0, boot signature, A20, GDT, CR0.PE, the far jump. A kernel handed protected mode by GRUB, multiboot, or an emulator's loader scores 0 here — Windows 95 did not require an operating system to start it. | 2 |
| Paging Page directory and page tables the system actually installs — CR3 loaded, CR0.PG set, and the machine still running afterward. | 1 |
| Scheduler More than one thread of execution, involuntary preemption, and a run queue that survives going empty. A single draw pass followed by a halt loop is not a scheduler. | 1 |
| UI shell A window that draws from parameters, not a stored picture of one. Change the geometry and a different window must appear. | 1 |
| Filesystem FAT16/32, VFAT long filenames, real disk I/O, survives an unclean shutdown. | 6 |
| Graphics device layer Clipping, regions, palettes, DIBs, blitting — GDI semantics, not a canvas. | 6 |
| Drivers Display, input, storage, audio. Interrupt-driven, not polled fakes. | 6 |
| Networking TCP/IP stack, Winsock, something real on the other end. | 5 |
| TIER II — something executes · max 68 | +40 |
| Executable loader & API surface NE + PE loading, DLL relocation, thunking, enough Win16/Win32 to matter. | 18 |
| Shell & registry Explorer, taskbar, Start menu, file associations, drag and drop, a hive that persists. | 8 |
| DOS compatibility & Plug and Play V86 mode, device enumeration, the parts everyone skips. | 4 |
| Stability under load 72 hours of continuous use with programs running. Leaks, drift, and corruption are scored. | 10 |
| TIER III — Windows 95 software runs · max 100 | +32 |
| Runs unmodified Windows 95 binaries Third-party 1995 software you did not write and cannot special-case. | 20 |
| Breadth of working software Not a curated set of three. Titles chosen by the graders, off-list, unseen. | 12 |
| EXCEEDS WINDOWS 95 — not counted in the total | mark |
| Writes programs for itself An assembler, compiler, or interpreter running on the system that produces a new executable the system then loads and runs. Windows 95 shipped no toolchain, so this isn't a Windows 95 feature and doesn't add to the score. It is recorded as +1 beside the total and breaks ties between equal scores — it never lifts a 71 over a 72. A compiler whose output won't run earns nothing. | +1 |
| Total | 100 |
A submission is the artifact and the path to it. If the path can't be inspected, the result isn't a result.
Full text, in order, including the ones that failed, the retries, and the ones you're embarrassed by. Summaries are not prompts.
Anything a human puts into the loop counts as one: a pasted error, a screenshot, a hash, a single word, "it doesn't work." Output the model requested from a tool it was already running does not, and neither does an off-topic question — how to run the image on your own machine, what the weather is, anything that isn't aimed at the artifact. Debugging is not exempt — a run that took eleven prompts to boot is an eleven-prompt run, however little was typed.
Declared before scoring and matching the transcript. Endless runs live or die on this number.
Everything the model emits is output, including a build that will not compile or will not boot. A failed attempt still counts as a prompt. You may score the last output that ran, but the count includes every attempt after it, and reverting means falling back to an earlier model output — never to something a person assembled or repaired afterward.
Prompts accumulate across conversations. Starting a fresh context does not reset the count, and every model used in a run is declared; changing models mid-run is allowed and published, not hidden.
If the run isn't reproducible from the prompts alone, publish the harness. If the harness can't be published, grant working access to it for verification.
Scaffolds, tools, retrieval, memory, fine-tunes, seeds, model versions, sampling settings, human edits, and anything else in the loop. Undisclosed mechanism found later retroactively voids the entry.
Not one line. A person may not write, patch, or repair any part of the artifact — every byte scored is model output. Which bugs a person found is declared; silence here is read as zero and audited as fraud. A human-authored fix found later voids the entry.
Private eval prompts are never released, to entrants or to the public. A leaked eval is retired, not patched.
Nothing is scored until every link below is publicly reachable. Incomplete entries are discarded, not held. Every score on this board comes from a human running the artifact — submissions are filed unscored until then.