2023-03-31 03:58:14 +00:00
|
|
|
# Apify
|
|
|
|
|
|
|
|
This page covers how to use [Apify](https://apify.com) within LangChain.
|
|
|
|
|
|
|
|
## Overview
|
|
|
|
|
|
|
|
Apify is a cloud platform for web scraping and data extraction,
|
|
|
|
which provides an [ecosystem](https://apify.com/store) of more than a thousand
|
|
|
|
ready-made apps called *Actors* for various scraping, crawling, and extraction use cases.
|
|
|
|
|
2023-06-16 18:52:56 +00:00
|
|
|
[![Apify Actors](/img/ApifyActors.png)](https://apify.com/store)
|
2023-03-31 03:58:14 +00:00
|
|
|
|
|
|
|
This integration enables you run Actors on the Apify platform and load their results into LangChain to feed your vector
|
|
|
|
indexes with documents and data from the web, e.g. to generate answers from websites with documentation,
|
|
|
|
blogs, or knowledge bases.
|
|
|
|
|
|
|
|
|
|
|
|
## Installation and Setup
|
|
|
|
|
|
|
|
- Install the Apify API client for Python with `pip install apify-client`
|
|
|
|
- Get your [Apify API token](https://console.apify.com/account/integrations) and either set it as
|
|
|
|
an environment variable (`APIFY_API_TOKEN`) or pass it to the `ApifyWrapper` as `apify_api_token` in the constructor.
|
|
|
|
|
|
|
|
|
|
|
|
## Wrappers
|
|
|
|
|
|
|
|
### Utility
|
|
|
|
|
|
|
|
You can use the `ApifyWrapper` to run Actors on the Apify platform.
|
|
|
|
|
|
|
|
```python
|
|
|
|
from langchain.utilities import ApifyWrapper
|
|
|
|
```
|
|
|
|
|
2023-07-25 04:20:32 +00:00
|
|
|
For a more detailed walkthrough of this wrapper, see [this notebook](/docs/integrations/tools/apify.html).
|
2023-03-31 03:58:14 +00:00
|
|
|
|
|
|
|
|
|
|
|
### Loader
|
|
|
|
|
|
|
|
You can also use our `ApifyDatasetLoader` to get data from Apify dataset.
|
|
|
|
|
|
|
|
```python
|
|
|
|
from langchain.document_loaders import ApifyDatasetLoader
|
|
|
|
```
|
|
|
|
|
2023-07-25 04:20:32 +00:00
|
|
|
For a more detailed walkthrough of this loader, see [this notebook](/docs/integrations/document_loaders/apify_dataset.html).
|