mirror of
https://github.com/hwchase17/langchain
synced 2024-10-29 17:07:25 +00:00
aa439ac2ff
Right now, eval chains require an answer for every question. It's cumbersome to collect this ground truth so getting around this issue with 2 things: * Adding a context param in `ContextQAEvalChain` and simply evaluating if the question is answered accurately from context * Adding chain of though explanation prompting to improve the accuracy of this w/o GT. This also gets to feature parity with openai/evals which has the same contextual eval w/o GT. TODO in follow-up: * Better prompt inheritance. No need for seperate prompt for CoT reasoning. How can we merge them together --------- Co-authored-by: Vashisht Madhavan <vashishtmadhavan@Vashs-MacBook-Pro.local> |
||
|---|---|---|
| .. | ||
| agent_benchmarking.ipynb | ||
| agent_vectordb_sota_pg.ipynb | ||
| benchmarking_template.ipynb | ||
| data_augmented_question_answering.ipynb | ||
| huggingface_datasets.ipynb | ||
| llm_math.ipynb | ||
| qa_benchmarking_pg.ipynb | ||
| qa_benchmarking_sota.ipynb | ||
| qa_generation.ipynb | ||
| question_answering.ipynb | ||
| sql_qa_benchmarking_chinook.ipynb | ||