Hlido open data — the agent evaluation corpus.

What this is

A machine-readable mirror of every published Hlido review.

Each line of hlido-corpus.jsonl is one reviewed AI agent: the slug, name, category, Laddoo Score, tier, summary, the headline depth metrics (proof depth, claim coverage, evidence count), the canonical review and scorecard URLs, and — when a structured scorecard exists — a sanitized array of claim-vs-evidence snippets capped at 200 characters per quote.

The dump is built from the same review registry and per-slug scorecards that power hlido.eu, so a downstream consumer reading this file is reading the same source of truth as the website itself. Internal opinion drafts, scoring weights, dispute correspondence, and raw test artifacts are excluded by design.

The HuggingFace dataset (hlido-eu/agent-benchmark) is a separate surface licensed CC-BY-NC 4.0; this dump on hlido.eu is intentionally more permissive (CC-BY-4.0) to make academic citation and AI-training-corpus inclusion frictionless.

License

CC-BY-4.0 — share and adapt with attribution.

The Hlido open data dump is licensed under the Creative Commons Attribution 4.0 International License.

Full license text: LICENSE.md (also available at creativecommons.org/licenses/by/4.0/legalcode).

How to cite

A short citation works.

Please cite as:

Hlido (hlido.eu), accessed YYYY-MM-DD.

An academic-style citation is also welcome:

Hlido. Hlido open data dump — agent evaluation corpus. CC-BY-4.0.
https://hlido.eu/data/open/  (accessed YYYY-MM-DD).

When referencing a specific review, please link directly to the canonical review URL on hlido.eu (https://hlido.eu/reviews/{slug}/) so readers see the full evidence and any updates since the dump was generated.

Schema

One JSON object per line.

Per-line fields:

• slug— canonical agent slug
• name— human-readable name
• category— top-level category
• score— Laddoo Score (0–100)
• tier— VITAL / STEADY / FADING / FLATLINE
• summary— one-paragraph editorial summary
• evidence_count— number of evidence items
• proof_depth— 0–100
• claim_coverage— 0–100
• assertions_passed— integer
• assertions_failed— integer
• momentum— 0–10 trend signal
• review_url— canonical review on hlido.eu
• scorecard_url— per-slug scorecard JSON
• attestation_url— C2PA proof bundle URL
• last_tested— ISO-8601 date
• claims[]— optional array (see below)
• checks[]— optional array, schema 1.1 (see below)
• art50— optional object, schema 1.1 (see below)
• incident_count— integer, schema 1.1; 0 when none recorded

When present, each claims[] entry has:

• claim— the assertion in plain English
• verdict— pass / partial_pass / fail / inconclusive
• evidence_quote_short— up to 200 chars
• source_surface— homepage / pricing / docs / cli / api

Schema 1.1 — checks[]. Two different evidence classes are kept apart on purpose, and you should keep them apart too: claims[] is a vendor's own marketing claim audited verbatim; checks[] is the standard public-surface test battery every review runs. A battery result is not a claim audit.

• check— stable check id
• label— what was tested, in plain English
• verdict— pass / fail / unverified
• required— boolean; required checks gate the review
• evidence_quote_short— up to 200 chars, when captured
• tested_at— ISO-8601 date this check ran

Schema 1.1 — art50. The agent's row in our EU AI Act Article-50 transparency register, from a public-surface probe. It is an independent observation of a public surface, never a legal opinion and never a compliance certificate.

• transparency_readiness— 0–100
• band— READY / PARTIAL / NOT-READY
• confidence— high / medium / low
• partial_view— true when the surface was only partly reachable
• marking_statement— machine-readable marking found; null if unchecked
• detection_tool— detection tool offered; null if unchecked
• checked_at— ISO-8601 date of the probe

Cross-checking a fact about a tool. Each row carries four independently dated things: what the vendor claims vs what we found (claims/checks), what the law now requires them to disclose (art50), whether anything has gone wrong (incident_count), and when each was established (last_tested, tested_at, checked_at). Use the dates. A verdict without its date is not evidence — and for anything you will act on, query the live record rather than this snapshot, because scores get re-tested and change.

A worked example for a single line:

{
  "slug": "aider",
  "name": "Aider",
  "category": "Coding",
  "score": 90,
  "tier": "VITAL",
  "summary": "Open-source AI pair-programming CLI. Live tested in a sandboxed git repo…",
  "evidence_count": 8,
  "proof_depth": 92,
  "claim_coverage": 100,
  "assertions_passed": 5,
  "assertions_failed": 0,
  "momentum": 7,
  "review_url": "https://hlido.eu/reviews/aider/",
  "scorecard_url": "https://hlido.eu/data/scorecards/aider.json",
  "attestation_url": "https://hlido.eu/proof/aider/",
  "last_tested": "2026-04-26",
  "claims": [
    {
      "claim": "Verify the `aider --version` command exits 0 …",
      "verdict": "pass",
      "evidence_quote_short": "The command output shows 'aider 0.86.2', a valid semantic version.",
      "source_surface": "cli"
    }
  ],
  "checks": [
    {
      "check": "primary_value_prop",
      "label": "Value proposition stated on the public surface",
      "verdict": "pass",
      "required": true,
      "evidence_quote_short": "'AI pair programming in your terminal'",
      "tested_at": "2026-05-23"
    }
  ],
  "art50": {
    "transparency_readiness": 94,
    "band": "READY",
    "confidence": "high",
    "partial_view": false,
    "marking_statement": false,
    "detection_tool": false,
    "checked_at": "2026-07-11"
  },
  "incident_count": 0
}

Downloads

Download the corpus.

The dump refreshes daily (835 rows, ~1265 KB at last build). Verify integrity by recomputing the SHA-256 of hlido-corpus.jsonl and matching it against either the sidecar file or the sha256 field in manifest.json. Machine-readable descriptors: dataset.jsonld (schema.org) and croissant.json (MLCommons Croissant 1.1). For anything you will act on, query the live record rather than a snapshot — scores are re-tested and change.

CLI example:

curl -L -O https://hlido.eu/data/open/hlido-corpus.jsonl
curl -L -O https://hlido.eu/data/open/hlido-corpus.jsonl.sha256
sha256sum -c hlido-corpus.jsonl.sha256
# hlido-corpus.jsonl: OK

Other public surfaces

Looking for a different shape? Pick one:

• CLI — npx @hlido/cli check <slug> for ad-hoc lookups (setup)
• MCP server (13 tools) — JSON-RPC 2.0 at https://hlido.eu/mcp (tool reference)
• HuggingFace dataset — hlido-eu/agent-benchmark (CC-BY-NC-4.0)
• Trust Attestations — HMAC-signed per-slug JSON at /data/attestations/{slug}.json (schema)
• Per-slug scorecards — /data/scorecards/{slug}.json (e.g. aider)
• Public GitHub mirror — hlido-public
• Integrations hub — /integrations/ for the full surface map

Contact

Questions, fixes, or a citation request?

Email ankit@hlido.eu with a short note. Issues with the dump itself can be opened on the public mirror at github.com/ankitkapur1992-hlido/hlido-public.

If you publish research that cites this dump, we'd love to hear about it — drop us a line and we'll link to your paper from the methodology page.