Archives/reflexion-human-eval
2
0
Fork 0
mirror of https://github.com/GammaTauAI/reflexion-human-eval synced 2024-11-16 00:12:59 +00:00
Code Issues Packages Projects Releases Wiki Activity

note about paper

Browse Source
This commit is contained in:
Noah Shinn 2023-04-09 02:18:52 -04:00 committed by elleven11
parent a0162a065d
commit 3148695707
1 changed files with 5 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
README.md
View File

@ -10,19 +10,17 @@ Check out the code for the original paper [here](https://github.com/noahshinn024
Check out a new superhuman programming agent gym [here](https://github.com/GammaTauAI/leetcode-hard-gym)
### Note
This repo contains scratch code that was used for testing. The real version of Reflexion for benchmark-agnostic, language-agnostic code generation will be released after the first version of the upcoming paper to respect the privacy of the work (and collaboration) in progress.
If you have any questions, please contact [noahshinn024@gmail.com](noahshinn024@gmail.com)
![architecture](./media/architecture.png)
![result](./media/performance.png)
### Cloning The Repository
**The repository contains git submodules**. To clone the repo with the submodules, run:
```bash
git clone --recurse-submodules
```
### Note
### Another Note
Due to the nature of these experiments, it may not be feasible for individual developers to rerun the results due to limited access to GPT-4 and significant API charges. Due to recent requests, both trials have been rerun once more and are dumped in `./root` with a script [here](https://github.com/noahshinn024/reflexion-human-eval/blob/main/validate_py_results.py) to validate the solutions with the unit tests provided by [HumanEval](https://github.com/openai/human-eval).