Skip to content

Commit 11f0470

Browse files
committed
Add DistDebug-Bench project page
1 parent 086209b commit 11f0470

2 files changed

Lines changed: 59 additions & 0 deletions

File tree

DSD-Bench/index.html

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
---
2+
layout: default
3+
title: DistDebug-Bench
4+
permalink: /DSD-Bench/
5+
---
6+
<section class="section-margin">
7+
<div class="container">
8+
<div class="section-intro pb-85px text-center">
9+
<h2 class="section-intro__title">DistDebug-Bench</h2>
10+
<p class="section-intro__subtitle">A benchmark for evaluating code-level diagnosis of distributed-system failures.</p>
11+
</div>
12+
<p>
13+
DistDebug-Bench evaluates whether language-model agents can diagnose real
14+
distributed-system failures from issue reports, repository snapshots, and
15+
project context. The benchmark focuses on finding the source-code blocks
16+
that explain a reported failure before the fixing patch is applied.
17+
</p>
18+
<p>
19+
The benchmark contains 120 bug reports from 11 distributed systems and 5
20+
programming languages, with gold root-cause labels for code-level
21+
evaluation.
22+
</p>
23+
<p>
24+
Benchmark repository:
25+
<a href="https://github.com/OrderLab/DSD-Bench">github.com/OrderLab/DSD-Bench</a>
26+
</p>
27+
</div>
28+
</section>
29+
30+
<section class="section-padding bg-magnolia">
31+
<div class="container">
32+
<div class="section-intro pb-85px text-center">
33+
<h3 class="section-intro__title">Benchmark Artifacts</h3>
34+
<p class="section-intro__subtitle">Issue evidence, repository snapshots, gold root-cause blocks, and evaluation scripts.</p>
35+
</div>
36+
<div class="row">
37+
<div class="col-md-4">
38+
<h4>Reports</h4>
39+
<p>Original issue descriptions and metadata from real distributed-system bug reports.</p>
40+
</div>
41+
<div class="col-md-4">
42+
<h4>Labels</h4>
43+
<p>Gold source-code blocks that capture the pre-fix root cause for each failure.</p>
44+
</div>
45+
<div class="col-md-4">
46+
<h4>Evaluation</h4>
47+
<p>Scoring logic for comparing predicted diagnosis blocks against human labels.</p>
48+
</div>
49+
</div>
50+
</div>
51+
</section>

software.html

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,14 @@ <h3 class="section-intro__title">Group GitHub Repository</h3>
1010
</div>
1111
</div>
1212
</section>
13+
<section class="section-padding bg-magnolia">
14+
<div class="container">
15+
<div class="section-intro pb-85px text-center">
16+
<h3 class="section-intro__title">DistDebug-Bench</h3>
17+
<p class="section-intro__subtitle">DistDebug-Bench evaluates code-level diagnosis of distributed-system failures. Visit the <a href="/DSD-Bench/">project page</a> or the <a href="https://github.com/OrderLab/DSD-Bench">benchmark repository</a>.</p>
18+
</div>
19+
</div>
20+
</section>
1321
<section class="section-padding bg-magnolia">
1422
<div class="container">
1523
<div class="section-intro pb-85px text-center">

0 commit comments

Comments
 (0)