Package dev.braintrust.devserver

Class RemoteEval<INPUT,OUTPUT>

java.lang.Object

dev.braintrust.devserver.RemoteEval<INPUT,OUTPUT>

Type Parameters:

INPUT - The type of input data for the evaluation

OUTPUT - The type of output produced by the task

public class RemoteEval<INPUT,OUTPUT>
extends Object

Represents a remote evaluator that can be exposed via the dev server.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	RemoteEval.Builder<INPUT,OUTPUT>
static class	RemoteEval.Parameter	Represents a configurable parameter for the evaluator
static enum	RemoteEval.ParameterType	Parameter type enumeration

Method Summary

Modifier and Type	Method	Description
static <INPUT, OUTPUT> RemoteEval.Builder<INPUT,OUTPUT>	builder()
String	getName()	The name of this evaluator (used as identifier)
Map<String,RemoteEval.Parameter>	getParameters()	Optional parameters that can be configured from the UI
List<Scorer<INPUT,OUTPUT>>	getScorers()	List of scorers for this evaluator
Task<INPUT,OUTPUT>	getTask()	The task function that performs the evaluation

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

builder

public static <INPUT,
OUTPUT>
RemoteEval.Builder<INPUT,OUTPUT> builder()

getName

@Nonnull
public String getName()

The name of this evaluator (used as identifier)

getTask

@Nonnull
public Task<INPUT,OUTPUT> getTask()

The task function that performs the evaluation

The task function must be thread safe.

getScorers

@Nonnull
public List<Scorer<INPUT,OUTPUT>> getScorers()

List of scorers for this evaluator

The score function must be thread safe.

getParameters

@Nonnull
public Map<String,RemoteEval.Parameter> getParameters()

Optional parameters that can be configured from the UI