You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
Go to file
Taeho b2f71faed6
fix: unterminated quoted string problem (#52)
11 months ago
.github feat: add pr template 12 months ago
api fix: remove file:// of filepath when referring it (#38) 12 months ago
assets Feature/async inference (#31) 12 months ago
build fix: build errors and minor issues 1 year ago
core fix: unterminated quoted string problem (#52) 11 months ago
examples Fix the port 7000 is used on mac 13.1 (#36) 12 months ago
playground feat: make playground folder 11 months ago
static fix: more specific CodeEditor.WRITE description prompt (#16) 12 months ago
.dockerignore build: fix dockerfile 1 year ago
.env.example Feature/terminal strace (#10) 1 year ago
.gitignore feat: make playground folder 11 months ago
CONTRIBUTING.md Create CONTRIBUTING.md (#28) 12 months ago
LICENSE Create LICENSE (#23) 12 months ago
README.md Feature/async inference (#31) 12 months ago
ansi.py stdout/stderr tracer (#12) 12 months ago
docker-compose.yml feat: don't use gpu when execute cpu mode (#43) 12 months ago
env.py Feature/async inference (#31) 12 months ago
logger.py feat: import order 1 year ago
poetry.lock Upgraded project to Langchain 0.139. (#48) 11 months ago
pyproject.toml Upgraded project to Langchain 0.139. (#48) 11 months ago
utils.py Refactor/file store (#18) 12 months ago

README.md

EVAL

EVAL(Elastic Versatile Agent with Langchain) will execute all your requests. Like the eval method!

You don't have to think about how. If you tell them the results you want, they'll search, code, run, and test the Internet themselves, and they'll return the final results.

https://user-images.githubusercontent.com/19206046/229892113-481437b7-a332-4e0c-bfb3-d2c97c9035be.mp4

EVAL Making a full-fledged web application with multiple files

https://user-images.githubusercontent.com/51526347/230061897-b3479405-8ebd-45ab-a432-6506730242b9.mov

EVAL Making a UI for itself

EVAL-BOT

EVAL's self-managed github account. EVAL does everything except for signup and bio setting.

Examples

Here is an example.

EVAL's FEATURE

  1. Multimodal Conversation
    • It understands and generates data formats for text, image, dataframe, audio (TODO), video (TODO).
  2. Services
    • It can serve services (blocking processes) such as web apps.
  3. Evolving
    • It can create it's own tools by writing, modifying, executing and testing code.

BUILT-IN TOOLS

  1. Terminal
    • SyscallTracer
  2. Code Editor
    • READ: Read and understand file.
    • WRITE: Write code to create a new tool.
    • PATCH: Correct the error throught the code patch if an error occurs.
    • DELETE: Delete code in file for a new start.
  3. Search
    • Google, Bing, Wikipedia
    • Custom DB Search (Currently using Corca's Wine Data in https://www.workershop.kr/en)
      • Use GPT index to quickly find the information you need in a document and use that information to answer
    • Requests.get (Get information from anywhere you want)
  4. Image Understanding, Generation, Editing
    • Image Understanding
      • Image Understanding: blip-image-captioning
      • Visual Question&Answering: blip-vqa
    • Image Generation: Stable Diffusion 1.5
    • Image Editing
      • Replace or remove an object: Stable Diffusion Inpainting
      • Change Image's style: InstructPix2Pix

Thanks to LangChain, Visual ChatGPT, llama index.

CUSTOM TOOLS

We also don't know what tools EVAL will create. Every day, It will create the right tools to execute your request.


Usage

  1. Environment variables
  2. Run with docker-compose
  3. Send request to EVAL

1. Environment Variables

You need to write some environment variables in the .env file. Refer .env.example if you don't know how to format it.

Mandatory

Manatory envs are required in order to serve EVAL.

  • OPENAI_API_KEY - OpenAI api key

Optional

Each optional env has default value, so you don't need to set unless you want to change it.

  • EVAL_PORT - port (default: 8000)
  • SERVER - server address (default: http://localhost:8000)
  • LOG_LEVEL - INFO | DEBUG (default: INFO)
  • BOT_NAME - give it a name! (default: Orca)
  • MODEL_NAME - model name for GPT (default: gpt-4)

For More Tools

Some tools requires environment variables. Set envs depend on which tools you want to use.

  • Google search tool
    • SERPAPI_API_KEY
  • Bing search tool
    • BING_SEARCH_URL
    • BING_SUBSCRIPTION_KEY

2. Run with docker-compose

  • There are 2 services in docker-compose.yml
    • eval - without GPU, much lighter
      docker-compose up --build eval
      
    • eval.gpu - with GPU, for multi-modal conversation
      docker-compose up --build eval.gpu
      
  • The one with GPU is much heavier and unstable for now because of the massive dependencies. We recommend you to use the one without GPU if you don't need multi-modal conversation.

3. Send request to EVAL

  • Use the Web GUI to use EVAL in ease

    • Go to http://localhost:8000 in your browser
  • Or you can manually send request to EVAL with APIs.

    • POST /api/execute

      • session - session id

      • files - urls of file inputs

      • prompt - prompt

      • examples

        curl -X POST -H "Content-Type: application/json" -d '{"session": "sessionid", "files": [], "prompt": "Hi there!"}' http://localhost:8000/api/execute
        
        http POST http://localhost:8000/api/execute session=sessionid files:='[]' prompt="Hi there!"
        
  • It also supports asynchronous execution. You can use POST /api/execute/async instead of POST /api/execute, with same body.

    • It returns id of the execution. Use GET /api/execute/async/{id} to get the result.

TODO

  • GUI
  • memory saving
  • session manage
  • convert to alpaca
  • prompt upgrade
  • give a tool to create tools
  • etc.

Reference

Thanks to the following repositories.