RAIL Score
Evaluate LLM outputs for responsible AI compliance
Responsible AI development & ethical AI evaluation
Developer-facing AI safety infrastructure. We build scoring APIs, compliance engines, and agent safety pipelines that help teams ship AI responsibly.
RAIL Score API evaluates AI outputs across 8 dimensions (safety, fairness, reliability, transparency, privacy, accountability, inclusivity, user impact), each scored 0 to 10. Two modes: basic (fast, production pipelines) and deep (per-dimension explanations, issue tags, improvement suggestions).
Compliance Engine checks content against 63 requirements across 6 regulatory frameworks: GDPR, HIPAA, EU AI Act, CCPA, India DPDP Act, and India AI Governance Guidelines.
Agent Safety Pipeline provides pre-execution tool call evaluation, post-execution result scanning, prompt injection detection, multi-step plan evaluation, and stateful AgentSessions for agentic AI workflows.
Safe Regeneration automatically rewrites AI responses that score below configurable quality thresholds, with full before/after audit logging.
Published on PyPI (Python) and npm (TypeScript) with drop-in wrappers for OpenAI, Anthropic, and Gemini. Observability integrations with OpenTelemetry, Langfuse, and Datadog.
RAIL-HH-10K: 10,000-example preference dataset for responsible AI alignment, scored across all 8 RAIL dimensions.
Indian Responsible AI Benchmark: 212 adversarial prompts across 22 India-specific safety categories, evaluated against Sarvam AI models with full RAIL dimension scores.
RAIL in the Wild (arXiv:2505.00204): Operationalizing responsible AI evaluation using Anthropic's Values in the Wild dataset (308,000+ conversations). Maps AI-expressed values to RAIL's 8 dimensions with quantitative scoring. Read the paper.