22 Commits (84a97d55e1f0061f7796387c6909017a06ca1511)

Author	SHA1	Message	Date
William FH	9eb7e6e27f	Delete Old Evals Examples (#8252) ... Still retain: - Comparison Examples - Data + QA walkthrough - QA (but really minimize it)	1 year ago
William FH	30c2d3cd06	Update references (#8243)	1 year ago
Bagatur	483f6c2fe3	mv eval docs (#8209)	1 year ago
William FH	c6f2d27789	Docs Nits (#7874) ... Add links to reference docs	1 year ago
William FH	3179ee3a56	Evals docs (#7460) ... Still don't have good "how to's", and the guides / examples section could be further pruned and improved, but this PR adds a couple examples for each of the common evaluator interfaces. - [x] Example docs for each implemented evaluator - [x] "how to make a custom evalutor" notebook for each low level APIs (comparison, string, agent) - [x] Move docs to modules area - [x] Link to reference docs for more information - [X] Still need to finish the evaluation index page - ~[ ] Don't have good data generation section~ - ~[ ] Don't have good how to section for other common scenarios / FAQs like regression testing, testing over similar inputs to measure sensitivity, etc.~	1 year ago
William FH	c58d35765d	Add examples to docstrings (#7796) ... and: - remove dataset name from autogenerated project name - print out project name to view	1 year ago
William FH	051fac1e66	Improve walkthrough links for sphinx (#7672) ... Co-authored-by: Ankush Gola <9536492+agola11@users.noreply.github.com>	1 year ago
William FH	a673a51efa	[Breaking] Update Evaluation Functionality (#7388) ... - Migrate from deprecated langchainplus_sdk to `langsmith` package - Update the `run_on_dataset()` API to use an eval config - Update a number of evaluators, as well as the loading logic - Update docstrings / reference docs - Update tracer to share single HTTP session	1 year ago
os1ma	2667ddc686	Fix `make docs_build` and related scripts (#7276) ... **Description: a description of the change** Fixed `make docs_build` and related scripts which caused errors. There are several changes. First, I made the build of the documentation and the API Reference into two separate commands. This is because it takes less time to build. The commands for documents are `make docs_build`, `make docs_clean`, and `make docs_linkcheck`. The commands for API Reference are `make api_docs_build`, `api_docs_clean`, and `api_docs_linkcheck`. It looked like `docs/.local_build.sh` could be used to build the documentation, so I used that. Since `.local_build.sh` was also building API Rerefence internally, I removed that process. `.local_build.sh` also added some Bash options to stop in error or so. Futher more added `cd "${SCRIPT_DIR}"` at the beginning so that the script will work no matter which directory it is executed in. `docs/api_reference/api_reference.rst` is removed, because which is generated by `docs/api_reference/create_api_rst.py`, and added it to .gitignore. Finally, the description of CONTRIBUTING.md was modified. **Issue: the issue # it fixes (if applicable)** https://github.com/hwchase17/langchain/issues/6413 **Dependencies: any dependencies required for this change** `nbdoc` was missing in group docs so it was added. I installed it with the `poetry add --group docs nbdoc` command. I am concerned if any modifications are needed to poetry.lock. I would greatly appreciate it if you could pay close attention to this file during the review. **Tag maintainer** - General / Misc / if you don't know who to tag: @baskaryan If this PR needs any additional changes, I'll be happy to make them! --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	1 year ago
Ruixi Fan	0b69a7e9ab	[Document fix] Fix an expired link qa_benchmarking_pg.ipynb (#7110) ... ## Change description - Description: Fix an expired link that points to the readthedocs site. - Dependencies: No	1 year ago
Johnny Lim	9dc77614e3	Polish reference docs (#7045) ... This PR fixes broken links in the reference docs.	1 year ago
William FH	8c73037dff	Simplify eval arg names (#6944) ... It'll be easier to switch between these if the names of predictions are consistent	1 year ago
Zander Chase	e1fdb67440	Update description in Evals notebook (#6808)	1 year ago
Zander Chase	ad028bbb80	Permit Constitutional Principles (#6807) ... In the criteria evaluator.	1 year ago
Zander Chase	d7dbf4aefe	Clean up agent trajectory interface (#6799) ... - Enable reference - Enable not specifying tools at the start - Add methods with keywords	1 year ago
Zander Chase	cc60fed3be	Add a Pairwise Comparison Chain (#6703) ... Notebook shows preference scoring between two chains and reports wilson score interval + p value I think I'll add the option to insert ground truth labels but doesn't have to be in this PR	1 year ago
Zander Chase	c460b04c64	Update String Evaluator (#6615) ... - Add protocol for `evaluate_strings` - Move the criteria evaluator out so it's not restricted to being applied on traced runs	1 year ago
Davis Chase	3298bf4f00	docs/fix links (#6498)	1 year ago
Davis Chase	d3c2eab0b3	Docs nit (#6350)	1 year ago
Davis Chase	6640293087	fix eval guide links (#6319)	1 year ago
Davis Chase	24b2af5218	nit (#6305)	1 year ago
Davis Chase	87e502c6bc	Doc refactor (#6300) ... Co-authored-by: jacoblee93 <jacoblee93@gmail.com> Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago