2023-04-04 19:17:13 +00:00
|
|
|
{
|
|
|
|
"cells": [
|
|
|
|
{
|
|
|
|
"cell_type": "markdown",
|
|
|
|
"id": "9fc6205b",
|
|
|
|
"metadata": {},
|
|
|
|
"source": [
|
|
|
|
"# Metal\n",
|
|
|
|
"\n",
|
2023-05-17 22:29:22 +00:00
|
|
|
">[Metal](https://github.com/getmetal/metal-python) is a managed service for ML Embeddings.\n",
|
|
|
|
"\n",
|
2023-04-04 19:17:13 +00:00
|
|
|
"This notebook shows how to use [Metal's](https://docs.getmetal.io/introduction) retriever.\n",
|
|
|
|
"\n",
|
|
|
|
"First, you will need to sign up for Metal and get an API key. You can do so [here](https://docs.getmetal.io/misc-create-app)"
|
|
|
|
]
|
|
|
|
},
|
|
|
|
{
|
|
|
|
"cell_type": "code",
|
|
|
|
"execution_count": 1,
|
|
|
|
"id": "1a737220",
|
|
|
|
"metadata": {},
|
|
|
|
"outputs": [],
|
|
|
|
"source": [
|
|
|
|
"# !pip install metal_sdk"
|
|
|
|
]
|
|
|
|
},
|
|
|
|
{
|
|
|
|
"cell_type": "code",
|
|
|
|
"execution_count": 3,
|
|
|
|
"id": "b1bb478f",
|
|
|
|
"metadata": {},
|
|
|
|
"outputs": [],
|
|
|
|
"source": [
|
|
|
|
"from metal_sdk.metal import Metal\n",
|
|
|
|
"API_KEY = \"\"\n",
|
|
|
|
"CLIENT_ID = \"\"\n",
|
2023-04-18 14:28:13 +00:00
|
|
|
"INDEX_ID = \"\"\n",
|
2023-04-04 19:17:13 +00:00
|
|
|
"\n",
|
2023-04-18 14:28:13 +00:00
|
|
|
"metal = Metal(API_KEY, CLIENT_ID, INDEX_ID);\n"
|
2023-04-04 19:17:13 +00:00
|
|
|
]
|
|
|
|
},
|
|
|
|
{
|
|
|
|
"cell_type": "markdown",
|
|
|
|
"id": "ae3c3d16",
|
|
|
|
"metadata": {},
|
|
|
|
"source": [
|
|
|
|
"## Ingest Documents\n",
|
|
|
|
"\n",
|
|
|
|
"You only need to do this if you haven't already set up an index"
|
|
|
|
]
|
|
|
|
},
|
|
|
|
{
|
|
|
|
"cell_type": "code",
|
|
|
|
"execution_count": 8,
|
|
|
|
"id": "f0425fa0",
|
|
|
|
"metadata": {},
|
|
|
|
"outputs": [
|
|
|
|
{
|
|
|
|
"data": {
|
|
|
|
"text/plain": [
|
|
|
|
"{'data': {'id': '642739aa7559b026b4430e42',\n",
|
|
|
|
" 'text': 'foo',\n",
|
|
|
|
" 'createdAt': '2023-03-31T19:51:06.748Z'}}"
|
|
|
|
]
|
|
|
|
},
|
|
|
|
"execution_count": 8,
|
|
|
|
"metadata": {},
|
|
|
|
"output_type": "execute_result"
|
|
|
|
}
|
|
|
|
],
|
|
|
|
"source": [
|
|
|
|
"metal.index( {\"text\": \"foo1\"})\n",
|
|
|
|
"metal.index( {\"text\": \"foo\"})"
|
|
|
|
]
|
|
|
|
},
|
|
|
|
{
|
|
|
|
"cell_type": "markdown",
|
|
|
|
"id": "944e172b",
|
|
|
|
"metadata": {},
|
|
|
|
"source": [
|
|
|
|
"## Query\n",
|
|
|
|
"\n",
|
|
|
|
"Now that our index is set up, we can set up a retriever and start querying it."
|
|
|
|
]
|
|
|
|
},
|
|
|
|
{
|
|
|
|
"cell_type": "code",
|
|
|
|
"execution_count": 9,
|
|
|
|
"id": "d0e6f506",
|
|
|
|
"metadata": {},
|
|
|
|
"outputs": [],
|
|
|
|
"source": [
|
|
|
|
"from langchain.retrievers import MetalRetriever"
|
|
|
|
]
|
|
|
|
},
|
|
|
|
{
|
|
|
|
"cell_type": "code",
|
|
|
|
"execution_count": 10,
|
|
|
|
"id": "f381f642",
|
|
|
|
"metadata": {},
|
|
|
|
"outputs": [],
|
|
|
|
"source": [
|
|
|
|
"retriever = MetalRetriever(metal, params={\"limit\": 2})"
|
|
|
|
]
|
|
|
|
},
|
|
|
|
{
|
|
|
|
"cell_type": "code",
|
|
|
|
"execution_count": 11,
|
|
|
|
"id": "20ae1a74",
|
|
|
|
"metadata": {},
|
|
|
|
"outputs": [
|
|
|
|
{
|
|
|
|
"data": {
|
|
|
|
"text/plain": [
|
|
|
|
"[Document(page_content='foo1', metadata={'dist': '1.19209289551e-07', 'id': '642739a17559b026b4430e40', 'createdAt': '2023-03-31T19:50:57.853Z'}),\n",
|
|
|
|
" Document(page_content='foo1', metadata={'dist': '4.05311584473e-06', 'id': '642738f67559b026b4430e3c', 'createdAt': '2023-03-31T19:48:06.769Z'})]"
|
|
|
|
]
|
|
|
|
},
|
|
|
|
"execution_count": 11,
|
|
|
|
"metadata": {},
|
|
|
|
"output_type": "execute_result"
|
|
|
|
}
|
|
|
|
],
|
|
|
|
"source": [
|
|
|
|
"retriever.get_relevant_documents(\"foo1\")"
|
|
|
|
]
|
|
|
|
},
|
|
|
|
{
|
|
|
|
"cell_type": "code",
|
|
|
|
"execution_count": null,
|
|
|
|
"id": "1d5a5088",
|
|
|
|
"metadata": {},
|
|
|
|
"outputs": [],
|
|
|
|
"source": []
|
|
|
|
}
|
|
|
|
],
|
|
|
|
"metadata": {
|
|
|
|
"kernelspec": {
|
|
|
|
"display_name": "Python 3 (ipykernel)",
|
|
|
|
"language": "python",
|
|
|
|
"name": "python3"
|
|
|
|
},
|
|
|
|
"language_info": {
|
|
|
|
"codemirror_mode": {
|
|
|
|
"name": "ipython",
|
|
|
|
"version": 3
|
|
|
|
},
|
|
|
|
"file_extension": ".py",
|
|
|
|
"mimetype": "text/x-python",
|
|
|
|
"name": "python",
|
|
|
|
"nbconvert_exporter": "python",
|
|
|
|
"pygments_lexer": "ipython3",
|
2023-05-17 22:29:22 +00:00
|
|
|
"version": "3.10.6"
|
2023-04-04 19:17:13 +00:00
|
|
|
}
|
|
|
|
},
|
|
|
|
"nbformat": 4,
|
|
|
|
"nbformat_minor": 5
|
|
|
|
}
|