You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
2.5 KiB
2.5 KiB
DataChad 🤖
This is an app that let's you ask questions about any data source by leveraging embeddings, vector databases, large language models and last but not least langchains
How does it work?
- Upload any
file
or enter anypath
orurl
- The data source is detected and loaded into text documents
- The text documents are embedded using openai embeddings
- The embeddings are stored as a vector dataset to activeloop's database hub
- A langchain is created consisting of a LLM model (
gpt-3.5-turbo
by default) and the vector store as retriever - When sending questions to the bot this chain is used as context to answer your questions
- Finally the chat history is cached locally to enable a ChatGPT like Q&A conversation
Good to know
- As default context this git repository is taken so you can directly start asking question about its functionality without chosing an own data source.
- To run locally or deploy somewhere, execute
cp .env.template .env
and set credentials in the newly created.env
file. Other options are manually setting of system environment variables, or storing them into.streamlit/secrets.toml
when hosted via streamlit. - If you have credentials set like explained above, you can just hit
submit
in the authentication without reentering your credentials in the app. - To enable
Local Mode
(disabled for the demo) setENABLE_LOCAL_MODE
toTrue
indatachad/constants.py
. You need to have the model binaries downloaded and stored inside./models/
- Currently supported
Local Mode
OSS model is GPT4all. To add more models updatedatachad/models.py
- If you are running
Local Mode
all your data stays locally on your machine. No API calls are made. Same with the embeddings database which stores its data to./data/
- Your data won't load? Feel free to open an Issue or PR and contribute!
- Yes, Chad in
DataChad
refers to the well-known meme
How does it look like?
TODO LIST
If you like to contribute, feel free to grab any task
- Refactor utils, especially the loaders
- Add option to choose model and embeddings
- Enable fully local / private mode
- Add Image caption and Audio transcription support