最新消息:雨落星辰是一个专注网站SEO优化、网站SEO诊断、搜索引擎研究、网络营销推广、网站策划运营及站长类的自媒体原创博客

python - Langsmith TypeError: DynamicRunEvaluator.evaluate_run() got an unexpected keyword argument 'evaluator_run_id&am

programmeradmin1浏览0评论

Error on running evaluation on dataset with ground truth to check generated SQL queries.

Getting Error for missing evaluator_run_id, not able to pass in evaluate function.

from langsmith import wrappers, Client
from pydantic import BaseModel, Field
from openai import OpenAI

client = Client()
openai_client = wrappers.wrap_openai(OpenAI())

def target(inputs: dict) -> dict:  
    payload = {
        "flowId": "xxx",
        "input": {
            "input_query": inputs['question']
        },
        "operateCmd": "testFlow",
        "userName": 'xxx'
    }

    response = requests.post(url, headers=headers, json=payload)
    print(response.json()['data']['result'])
    return response.json()['data']['result']

experiment_results = client.evaluate(
  target,
  data="Sample dataset",
  evaluators=[
      accuracy,
  ],
  max_concurrency=2,
)
Error running evaluator <DynamicRunEvaluator accuracy> on run 2e488952-ce0f-47a7-a112-adc49f49dfc3: TypeError("DynamicRunEvaluator.evaluate_run() got an unexpected keyword argument evaluator_run_id)
Traceback (most recent call last):
  File c:\Python313\Lib\site-packages\langsmith\evaluation\_runner.py, line 1634, in _run_evaluators
    evaluator_response = evaluator.evaluate_run(  # type: ignore[call-arg]
        run=run,
        example=example,
        evaluator_run_id=evaluator_run_id,
    )
TypeError: DynamicRunEvaluator.evaluate_run() got an unexpected keyword argument evaluator_run_id

与本文相关的文章

发布评论

评论列表(0)

  1. 暂无评论