4.6 KiB
EVAL
EVAL(Elastic Versatile Agent with Langchain) will execute all your requests. Like the eval method!
You don't have to think about how. If you tell them the results you want, they'll search, code, run, and test the Internet themselves, and they'll return the final results.
EVAL Making a full-fledged web application with multiple files
EVAL Making a UI for itself
EVAL-BOT
EVAL's self-managed github account. EVAL does everything except for signup and bio setting.
Examples
Here is an example.
EVAL's FEATURE
- Multimodal Conversation
- It understands and generates data formats for text, image, dataframe, audio (TODO), video (TODO).
- Services
- It can serve services (blocking processes) such as web apps.
- Evolving
- It can create it's own tools by writing, modifying, executing and testing code.
BUILT-IN TOOLS
- Terminal
- SyscallTracer
- Code Editor
- READ: Read and understand file.
- WRITE: Write code to create a new tool.
- PATCH: Correct the error throught the code patch if an error occurs.
- DELETE: Delete code in file for a new start.
- Search
- Google, Bing, Wikipedia
- Custom DB Search (Currently using Corca's Wine Data in https://www.workershop.kr/en)
- Use GPT index to quickly find the information you need in a document and use that information to answer
- Requests.get (Get information from anywhere you want)
- Image Understanding, Generation, Editing
- Image Understanding
- Image Understanding: blip-image-captioning
- Visual Question&Answering: blip-vqa
- Image Generation: Stable Diffusion 1.5
- Image Editing
- Replace or remove an object: Stable Diffusion Inpainting
- Change Image's style: InstructPix2Pix
- Image Understanding
Thanks to LangChain, Visual ChatGPT, llama index.
CUSTOM TOOLS
We also don't know what tools EVAL will create. Every day, It will create the right tools to execute your request.
Usage
- Environment variables
- Run with docker-compose
- Send request to EVAL
1. Environment Variables
You need to write some environment variables in the .env
file. Refer .env.example if you don't know how to format it.
Mandatory
Manatory envs are required in order to serve EVAL.
OPENAI_API_KEY
- OpenAI api key
Optional
Each optional env has default value, so you don't need to set unless you want to change it.
EVAL_PORT
- port (default: 8000)SERVER
- server address (default: http://localhost:8000)LOG_LEVEL
- INFO | DEBUG (default: INFO)BOT_NAME
- give it a name! (default: Orca)MODEL_NAME
- model name for GPT (default: gpt-4)
For More Tools
Some tools requires environment variables. Set envs depend on which tools you want to use.
- Google search tool
SERPAPI_API_KEY
- Bing search tool
BING_SEARCH_URL
BING_SUBSCRIPTION_KEY
2. Run with docker-compose
- There are 2 services in docker-compose.yml
eval
- without GPU, much lighterdocker-compose up --build eval
eval.gpu
- with GPU, for multi-modal conversationdocker-compose up --build eval.gpu
- The one with GPU is much heavier and unstable for now because of the massive dependencies. We recommend you to use the one without GPU if you don't need multi-modal conversation.
3. Send request to EVAL
-
POST /command
key
- session idfiles
- urls of file inputsquery
- prompt
-
You can send request to EVAL with
curl
orhttpie
.curl -X POST -H "Content-Type: application/json" -d '{"key": "sessionid", "files": ["https://example.com/image.png"], "query": "Hi there!"}' http://localhost:8000/command
http POST http://localhost:8000/command key=sessionid files:='["https://example.com/image.png"]' query="Hi there!"
-
We are planning to make a GUI for EVAL so you can use it without terminal.
TODO
- GUI
- memory saving
- session manage
- convert to alpaca
- prompt upgrade
- give a tool to create tools
- etc.
Reference
Thanks to the following repositories.