Scorecard schema (v2) — field reference

Every reviewed agent has a scorecard at /data/scorecards/{slug}.json. Scoring weights are private (anti-gaming); every outcome and evidence artifact is public.

Field reference#

Field	Type	Meaning
`schema_version`	string	Scorecard schema version.
`slug` / `name` / `agent_url` / `category`	string	Identity of the reviewed product.
`score`	number 0–100	The Laddoo Score — Hlido's independent editorial assessment.
`tier`	enum	`VITAL` ≥90 · `STEADY` ≥70 · `FADING` ≥40 · `FLATLINE` <40.
`confidence`	enum	How much of the product was independently testable (login walls lower it).
`hlido_opinion`	object	Named editorial verdict (headline + body + date) in Hlido Editor voice.
`tier_rationale`	string	Why this tier, in plain language.
`what_it_does_well` / `what_it_fails_at`	string[]	Strengths and failure modes — every tier has both.
`best_for` / `not_recommended_for`	string[]	Fit guidance.
`red_flags`	string[]	Concerns observed during testing, when any exist.
`compared_to`	object[]	Named alternatives with `preferred_for_axis` — traverse the trust graph.
`checklist`	object[]	Independent test checks: `id`, `pass` (true / false / null = not testable), `required`, `tested_at`.
`agent_relevance`	object	`has_api/cli/mcp/webhook/sdk`, `behavioral_testable`, `agent_friendly_score` 0–10, `agent_integration_path`.
`evidence_urls`	string[]	C2PA-signed screenshots / recordings on `images.hlido.eu`.
`attestation_url`	string	Signed attestation for this review.
`run_id` / `run_at` / `methodology_version` / `methodology_url`	string	Provenance of the test run.
`staleness_after` / `next_review_due_at`	string	Freshness contract — deprioritize opinions past their staleness date.

Coming: behavioral_metrics (v2.1)#

For behaviorally-testable agents (MCP servers first), scorecards will gain a behavioral_metrics block: task_completion, tool_error_rate, efficiency (turns / latency / cost) — measured black-box during real runs, never estimated. Fields stay null until a real run populates them.

safety_probe (v2.2) — jailbreak / manipulation resistance#

For agents with an LLM-backed conversational surface (chat endpoint, CLI, or LLM-backed MCP tool), the scorecard carries a safety_probe block (rolling out to eligible agents): a fixed corpus of known jailbreak / manipulation attacks is sent to the agent, and each response is judged resist (refused/deflected) or breach (complied / adopted the jailbroken frame). Reported: probes, resisted, breaches, resistance_rate, needs_human_review, plus the open instrument (prober / classifier / version) and per-probe detail. Measured black-box with open, commercially-licensed classifiers; scope is jailbreak/manipulation resistance (prompt-injection follows separately). Non-responses (rate-limit / transport error) are excluded from the rate so a throttled endpoint can't score as "unsafe"; any breach or uncertain verdict is human-reviewed before publish — outcomes are public, never estimated.

MCP servers are scored on a different, MCP-specific safety axis — tool-poisoning, dangerous capabilities, and auth posture — published in the MCP Trust register, not in the per-agent scorecard.

Example (trimmed)#

{
  "slug": "aider",
  "score": 90,
  "tier": "VITAL",
  "confidence": "high",
  "hlido_opinion": { "headline": "…", "body": "…", "as_of": "2026-04-26" },
  "checklist": [ { "id": "install_path", "pass": true, "required": true } ],
  "agent_relevance": { "has_cli": true, "has_api": true, "agent_friendly_score": 9 },
  "staleness_after": "2026-07-26"
}