objectiveai-laboratory allows you to benchmark agents.
add a system for splitting ObjectiveAI Function calls that occur within an objectiveai-laboratory sandbox.
if supported, when an agent running within an objectiveai-laboratory sandbox executes an ObjectiveAI Function, split the sandbox into multiple new sandboxes.
each sandbox gets a different result from the function call. if it's a vector function, for each input item, each different item gets ranked top 1.
by then running the benchmark on the final product of the sandbox, we can see which decisions were the best.
this creates data that can be used for computing function profiles.
this + #28 = GG
objectiveai-laboratoryallows you to benchmark agents.add a system for splitting ObjectiveAI Function calls that occur within an
objectiveai-laboratorysandbox.if supported, when an agent running within an
objectiveai-laboratorysandbox executes an ObjectiveAI Function, split the sandbox into multiple new sandboxes.each sandbox gets a different result from the function call. if it's a vector function, for each input item, each different item gets ranked top 1.
by then running the benchmark on the final product of the sandbox, we can see which decisions were the best.
this creates data that can be used for computing function profiles.
this + #28 = GG