5 Commits (d481d887bccea9ccbb1fecbfd053fe2acf078199)

Author	SHA1	Message	Date
SangamSwadiK	8cded3fdad	fix typo (#2532) ... 1) Any breaking changes ? None 2) What does this do ? Fix typo in QA eval cc @hwchase17	1 year ago
Vashisht Madhavan	aa439ac2ff	Adding an in-context QA evaluation chain + chain of thought reasoning chain for improved accuracy (#2444) ... Right now, eval chains require an answer for every question. It's cumbersome to collect this ground truth so getting around this issue with 2 things: * Adding a context param in `ContextQAEvalChain` and simply evaluating if the question is answered accurately from context * Adding chain of though explanation prompting to improve the accuracy of this w/o GT. This also gets to feature parity with openai/evals which has the same contextual eval w/o GT. TODO in follow-up: * Better prompt inheritance. No need for seperate prompt for CoT reasoning. How can we merge them together --------- Co-authored-by: Vashisht Madhavan <vashishtmadhavan@Vashs-MacBook-Pro.local>	1 year ago
Harrison Chase	2d098e8869	Harrison/agent eval (#1620) ... Co-authored-by: jerwelborn <jeremy.welborn@gmail.com>	2 years ago
Nicolas	91d7fd20ae	feat: add custom prompt for QAEvalChain chain (#610) ... I originally had only modified the `from_llm` to include the prompt but I realized that if the prompt keys used on the custom prompt didn't match the default prompt, it wouldn't work because of how `apply` works. So I made some changes to the evaluate method to check if the prompt is the default and if not, it will check if the input keys are the same as the prompt key and update the inputs appropriately. Let me know if there is a better way to do this. Also added the custom prompt to the QA eval notebook.	2 years ago
Harrison Chase	985496f4be	Docs refactor (#480) ... Big docs refactor! Motivation is to make it easier for people to find resources they are looking for. To accomplish this, there are now three main sections: - Getting Started: steps for getting started, walking through most core functionality - Modules: these are different modules of functionality that langchain provides. Each part here has a "getting started", "how to", "key concepts" and "reference" section (except in a few select cases where it didnt easily fit). - Use Cases: this is to separate use cases (like summarization, question answering, evaluation, etc) from the modules, and provide a different entry point to the code base. There is also a full reference section, as well as extra resources (glossary, gallery, etc) Co-authored-by: Shreya Rajpal <ShreyaR@users.noreply.github.com>	2 years ago