mirror of https://github.com/hwchase17/langchain
Web Loader: Add proxy support (#6792)
Proxies are helpful, especially when you start querying against more anti-bot websites. [Proxy services](https://developers.oxylabs.io/advanced-proxy-solutions/web-unblocker/making-requests) (of which there are many) and `requests` make it easy to rotate IPs to prevent banning by just passing along a simple dict to `requests`. CC @rlancemartin, @eyurtsevpull/6030/head
parent
f92ccf70fd
commit
3ca1a387c2
Loading…
Reference in New Issue