Streams
You can handle streams in Evalite by returning any async iterable (including a ReadableStream
) from your task. This means you can test functions like the AI SDK streamText
function easily:
import { evalite } from "evalite";import { streamText } from "ai";import { openai } from "@ai-sdk/openai";import { Factuality } from "autoevals";
evalite("My Eval", { data: async () => { return [{ input: "What is the capital of France?", expected: "Paris" }]; }, task: async (input) => { const result = await streamText({ model: openai("your-model"), system: `Answer the question concisely.`, prompt: input, });
return result.textStream; }, scorers: [Factuality],});