> ## Documentation Index
> Fetch the complete documentation index at: https://docs.errorbar.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Score a rollout group

> Score up to 64 rollouts against a session's reward. Billing ids are namespaced `reward:<session>:<step>` so a re-scored step is idempotent. Refusals are explicit: 402 `budget_exhausted` (the session's kill switch), 409 `held` (the anchor held the run — resume deliberately or ship the pinned step), 404 (unknown/foreign session), 422 (the reward no longer validates). An unparseable judge returns `grade: null` for that item: mask it, never zero it.



## OpenAPI

````yaml /openapi.json post /v1/reward/score
openapi: 3.1.0
info:
  title: errorbar Management API
  description: >-
    The management API behind the improvement loop: capture and setup, request
    logs and datasets, grades (labels), judges (criteria), evals and deploy
    gates, fine-tuning and reinforcement learning, dedicated GPU endpoints, and
    model aliases and versions. Authenticated with a workspace API key
    (sk_sovereign_...). The inference API (chat, embeddings, rerank, responses)
    is OpenAI-compatible and documented separately.


    Responses are snake_case, list endpoints on the loop products use the
    {"object": "list", "data": [...]} envelope, and refusals use the same nested
    error shape the gateway emits: {"error": {"message", "type", "code"}}.
    Request bodies on the loop products (logs, labels, criteria, evals,
    datasets, aliases) are snake_case; the training and infrastructure products
    (fine-tuning, GRPO, environment tools, dedicated, model-version adoption)
    validate camelCase bodies, and each schema below says which it is. Endpoints
    that spend money require a key minted by a workspace owner or admin and
    return 403 otherwise.
  version: 1.0.0
servers:
  - url: https://gateway.errorbar.ai
    description: Production
  - url: https://www.errorbar.ai/api
    description: Control plane (also served at this base URL)
security:
  - bearerAuth: []
tags:
  - name: Certified reward
    description: >-
      The certified reward for your own trainer: sessions with judge
      certificates, scoring with a ledger-true kill switch, and the mid-run
      anchor that holds a run when the reward diverges from an independent
      judge.
paths:
  /v1/reward/score:
    post:
      tags:
        - Certified reward
      summary: Score a rollout group
      description: >-
        Score up to 64 rollouts against a session's reward. Billing ids are
        namespaced `reward:<session>:<step>` so a re-scored step is idempotent.
        Refusals are explicit: 402 `budget_exhausted` (the session's kill
        switch), 409 `held` (the anchor held the run — resume deliberately or
        ship the pinned step), 404 (unknown/foreign session), 422 (the reward no
        longer validates). An unparseable judge returns `grade: null` for that
        item: mask it, never zero it.
      operationId: scoreReward
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              required:
                - sessionId
                - items
              properties:
                sessionId:
                  type: string
                step:
                  type: string
                  pattern: ^[A-Za-z0-9_.-]{1,40}$
                  description: Optimizer step tag; defaults to "s".
                items:
                  type: array
                  minItems: 1
                  maxItems: 64
                  items:
                    $ref: '#/components/schemas/RewardItem'
      responses:
        '200':
          description: Scores in the order the server grouped them (match on request_id).
          content:
            application/json:
              schema:
                type: object
                properties:
                  session_id:
                    type: string
                  scores:
                    type: array
                    items:
                      $ref: '#/components/schemas/RewardScore'
                  spend_micros:
                    type: string
                  criteria:
                    type: array
                    items:
                      type: string
        '400':
          $ref: '#/components/responses/BadRequest'
        '401':
          $ref: '#/components/responses/Unauthorized'
        '402':
          $ref: '#/components/responses/InsufficientBalance'
        '403':
          $ref: '#/components/responses/Forbidden'
        '404':
          $ref: '#/components/responses/NotFound'
        '409':
          description: >-
            The session is held by its anchor (`code: held`); the body names the
            pinned step.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Error'
components:
  schemas:
    RewardItem:
      type: object
      required:
        - requestId
        - conversation
      properties:
        requestId:
          type: string
          maxLength: 200
          description: >-
            Your id for the rollout; re-sending the same (step, requestId)
            settles into the same ledger rows.
        conversation:
          type: string
          description: The prompt/conversation the policy answered.
        response:
          type: string
          description: Single-turn completion (omit when sending steps).
        steps:
          type: array
          maxItems: 200
          items:
            type: object
            required:
              - role
              - content
            properties:
              role:
                type: string
                enum:
                  - assistant
                  - tool
              content:
                type: string
              source:
                type: string
                enum:
                  - code
                  - recorded
                  - declared
                  - simulated
                  - flagged
              toolName:
                type: string
          description: >-
            Agentic sessions: the whole rollout as structured steps; the server
            renders and sanitises it through the same instrument trace
            calibration uses.
        sessionId:
          type: string
          description: >-
            The episode's errorbar environment session id (exec verifiers +
            attestation).
    RewardScore:
      type: object
      properties:
        request_id:
          type: string
        grade:
          type:
            - number
            - 'null'
          description: >-
            Composite grade in [0,1]: mean of judge P(pass) and 0/1 assertions.
            null = the judge could not parse a verdict — MASK the sample, never
            treat it as 0.
        verdict:
          type: string
          enum:
            - pass
            - fail
            - unparsed
          description: 'Strict read: every component passes.'
        sim_fraction:
          type: number
        raw_grade:
          type: number
        exec_unverified:
          type: boolean
        attested:
          type: boolean
        attestation_mismatch:
          type: boolean
    Error:
      type: object
      properties:
        error:
          type: object
          properties:
            message:
              type: string
            type:
              type: string
              description: >-
                invalid_request_error, insufficient_quota, rate_limit_error, or
                api_error.
            code:
              type: string
              description: >-
                Machine-stable cause, e.g. invalid_api_key, not_found,
                insufficient_permissions, precondition_failed.
          required:
            - message
            - type
            - code
      description: >-
        Every refusal — gateway and management API alike — uses this one
        envelope.
  responses:
    BadRequest:
      description: Malformed request or invalid field.
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Error'
    Unauthorized:
      description: Missing, malformed, or revoked API key.
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Error'
          example:
            error:
              message: Invalid API key
              type: invalid_request_error
              code: invalid_api_key
    InsufficientBalance:
      description: Your wallet can't cover the required prepaid runway.
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Error'
          example:
            error:
              message: >-
                Insufficient balance: deploying this endpoint requires at least
                1h of runway. Top up and try again.
              type: insufficient_quota
              code: insufficient_balance
    Forbidden:
      description: The key lacks the required owner/admin permission.
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Error'
    NotFound:
      description: The resource does not exist in your workspace.
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Error'
  securitySchemes:
    bearerAuth:
      type: http
      scheme: bearer
      description: >-
        Your workspace API key, e.g. `sk_sovereign_...`, sent as `Authorization:
        Bearer <key>`.

````