SWE-bench Leaderboards

Hlido Transparency Passport

https://swebench.com · AI Agent
Issued 2026-10-03 · from an independent review dated 2026-05-23 · next review due 2026-08-21
Free — no charge to the vendor, and no charge to read it.

Independent review

65/100

FADING

Hands-on testing of the public surface. Scoring weights are private; the evidence is not.
Full record →

EU AI Act — Article 50

63/100

PARTIAL

Transparency-duty readiness — some signals present, others missing (confidence: medium).
Register →

These two numbers are never combined. One says whether the product is good; the other says whether it meets a disclosure duty. A blended score would let a strong product hide a compliance gap, and a compliant one look better than it is.

What was verified

4 of 5 checks passed, each dated and evidenced.

Check	Result	Evidence	Tested
`homepage_loads`	PASS	—	2026-05-23
`primary_value_prop`	PASS	'Evaluation of language models through competitions'	2026-05-23
`cta_present`	PASS	'Learn more about CodeClash'	2026-05-23
`pricing_or_access` optional	FAIL	No clear pricing or access model presented	2026-05-23
`evidence_or_demo` optional	PASS	CodeClash introduction visible on homepage	2026-05-23

Article-50 signals observed

Marking / labelling statement	not found
Detection tool declared	not found
Code of Practice signatory	not determined
Checked	2026-08-13

Basis: public homepage reachable · clear stated purpose · clear access path · 3/3 required checks passed · 1 red flag(s) · demonstrable, not black-box

Deploying this agent yourself? Article 50 puts duties on you as its deployer, not only on the vendor — build a file for your whole stack with the free EU AI Act Evidence File.

Evidence from the run

1 screenshot captured by the Hlido engine (run-885e28d4bb65536b-swebench-com). Our own captures — not vendor marketing material.

Machine-readable

Your customer’s procurement agent can verify this without reading the page.

GET https://hlido.eu/data/passports/swebench.json
GET https://hlido.eu/data/scorecards/swebench.json

POST https://hlido.eu/mcp
{"method":"tools/call","params":{"name":"get_scorecard","arguments":{"slug":"swebench"}}}

Independence & limits

Free, and not paid for. Hlido takes no money from the products it rates. This passport was produced from an independent review the subject did not commission, pay for, or approve. It is free to publish and free to read — charging the subject is exactly what would make it worthless to the procurement desk it is handed to.
Not a certification. Hlido is not a notified body. This is not a conformity assessment, a certification, or legal advice. It is a dated record of what an independent party measured on the product's public surface.
Correctable. If anything here is wrong, tell us and we correct the record — the correction is dated too.

All passports · Full review · Article-50 register · Check any agent