Package dev.braintrust.eval

Class ScorerBrainstoreImpl<INPUT,OUTPUT>

java.lang.Object

dev.braintrust.eval.ScorerBrainstoreImpl<INPUT,OUTPUT>

All Implemented Interfaces:

Scorer<INPUT,OUTPUT>

public class ScorerBrainstoreImpl<INPUT,OUTPUT>
extends Object
implements Scorer<INPUT,OUTPUT>

A scorer that invokes a remote Braintrust function to compute scores.

This implementation fetches a scorer function from Braintrust and invokes it via the API for each task result. The remote function receives the input, output, expected, and metadata as arguments.

Constructor Summary

Constructors

Constructor	Description
ScorerBrainstoreImpl(BraintrustApiClient apiClient, String functionId, String scorerName, String version)	Create a new remote scorer.

Method Summary

Modifier and Type	Method	Description
String	getName()
List<Score>	score(TaskResult<INPUT,OUTPUT> taskResult)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

ScorerBrainstoreImpl

public ScorerBrainstoreImpl(BraintrustApiClient apiClient,
 String functionId,
 String scorerName,
 @Nullable
 String version)

Create a new remote scorer.

Parameters:

apiClient - the API client to use for invoking the function

functionId - the ID of the function to invoke

scorerName - the name of the scorer (used as default score name)

version - optional version of the function to invoke. null always invokes latest version.

Method Details

getName

public String getName()

Specified by:

getName in interface Scorer<INPUT,OUTPUT>

score

public List<Score> score(TaskResult<INPUT,OUTPUT> taskResult)

Specified by:

score in interface Scorer<INPUT,OUTPUT>