Harrison/text splitter docs (#1188)

searx-query-suffixy
Harrison Chase 1 year ago committed by GitHub
parent 91446a5e9b
commit 926c121b98
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

@ -39,7 +39,7 @@
},
{
"cell_type": "markdown",
"id": "1c8e504a",
"id": "072eee66",
"metadata": {},
"source": [
"## Generic Recursive Text Splitting\n",
@ -53,7 +53,7 @@
{
"cell_type": "code",
"execution_count": 8,
"id": "1fedab44",
"id": "14662639",
"metadata": {},
"outputs": [],
"source": [
@ -63,7 +63,7 @@
{
"cell_type": "code",
"execution_count": 9,
"id": "edd10895",
"id": "fc6e42c8",
"metadata": {},
"outputs": [],
"source": [
@ -78,7 +78,7 @@
{
"cell_type": "code",
"execution_count": 12,
"id": "7c7badcd",
"id": "bd1a0a15",
"metadata": {},
"outputs": [
{
@ -98,7 +98,7 @@
},
{
"cell_type": "markdown",
"id": "71e12a11",
"id": "80f6cd99",
"metadata": {},
"source": [
"## Markdown Text Splitter\n",
@ -112,7 +112,7 @@
{
"cell_type": "code",
"execution_count": 13,
"id": "b5a64592",
"id": "96d64839",
"metadata": {},
"outputs": [],
"source": [
@ -122,7 +122,7 @@
{
"cell_type": "code",
"execution_count": 14,
"id": "06beaf9b",
"id": "cfb0da17",
"metadata": {},
"outputs": [],
"source": [
@ -146,7 +146,7 @@
{
"cell_type": "code",
"execution_count": 15,
"id": "38bd78a2",
"id": "d59a4fe8",
"metadata": {},
"outputs": [],
"source": [
@ -156,7 +156,7 @@
{
"cell_type": "code",
"execution_count": 16,
"id": "681d5d19",
"id": "cbb2e100",
"metadata": {},
"outputs": [
{
@ -178,10 +178,10 @@
},
{
"cell_type": "markdown",
"id": "400d82d8",
"id": "c350765d",
"metadata": {},
"source": [
"# Python Code Text Splitter\n",
"## Python Code Text Splitter\n",
"\n",
"PythonCodeTextSplitter splits text along python class and method definitions. It's implemented as a simple subclass of RecursiveCharacterSplitter with Python-specific separators. See the source code to see the Python syntax expected by default.\n",
"\n",
@ -192,7 +192,7 @@
{
"cell_type": "code",
"execution_count": 17,
"id": "29fd2bcd",
"id": "1703463f",
"metadata": {},
"outputs": [],
"source": [
@ -202,7 +202,7 @@
{
"cell_type": "code",
"execution_count": 18,
"id": "c5197adc",
"id": "f17a1854",
"metadata": {},
"outputs": [],
"source": [
@ -224,7 +224,7 @@
{
"cell_type": "code",
"execution_count": 19,
"id": "6a11320d",
"id": "6cdc55f3",
"metadata": {},
"outputs": [],
"source": [
@ -234,7 +234,7 @@
{
"cell_type": "code",
"execution_count": 20,
"id": "8ccd4be3",
"id": "8cc33770",
"metadata": {},
"outputs": [
{
@ -582,7 +582,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "e974286e",
"id": "f4ec9b90",
"metadata": {},
"outputs": [],
"source": [

Loading…
Cancel
Save