Training & Feedback
Flag trace nodes as good, bad, or needs_review to build feedback loops for agent improvement.
import { Invariance } from '@invariance/sdk';
Overview
The training system lets you flag individual trace nodes with quality labels (good, bad, needs_review) and optional notes. These flags create a feedback dataset that can inform agent improvements.
Use stats to track the distribution of flags across agents and identify which agents need the most attention.
Quick Example
const inv = Invariance.init({ apiKey: process.env.INVARIANCE_API_KEY! });
// Flag a trace node
await inv.training.flag({
trace_node_id: 'node-123',
session_id: 'session-456',
agent_id: 'my-agent',
flag: 'bad',
notes: 'Hallucinated a non-existent API endpoint',
});
// View stats
const stats = await inv.training.stats();
console.log('Total flags:', stats.total);
console.log('Bad:', stats.bad);
API Reference
training.listFlags
List trace flags with optional filters.
async listFlags(opts?: { flag?: string; limit?: number }): Promise<TraceFlag[]>
Parameters
flag'good' | 'bad' | 'needs_review'Filter by flag type
limitnumberMax results
ReturnsPromise<TraceFlag[]>
training.flag
Flag a trace node.
async flag(body: CreateTraceFlagBody): Promise<TraceFlag>
Parameters
trace_node_idstringTrace node to flag
session_idstringSession ID
agent_idstringAgent ID
flag'good' | 'bad' | 'needs_review'Quality label
notesstringOptional notes
ReturnsPromise<TraceFlag>
training.stats
Get flag statistics across all agents.
async stats(): Promise<TraceFlagStats>
ReturnsPromise<TraceFlagStats>
Use Cases
- Build feedback loops for agent improvement by flagging good and bad behaviors
- Identify agents with the highest rates of bad flags for prioritized fixing
- Create training datasets from flagged trace nodes