Add ne comparator (#10006)

Description: Adds the not comparator and operator to pinecone, chroma
and deeplake.
Issue: Not a registered issue but when using a selfqueryretriever with
pinecone I got this error + stacktrace when I entered a query that asked
to not include specific data:
 
>  raised following `error:`
> Received unrecognized function ne. Valid functions are [<Operator.AND:
'and'>, <Operator.OR: 'or'>, <Operator.NOT: 'not'>, <Comparator.EQ:
'eq'>, <Comparator.GT: 'gt'>, <Comparator.GTE: 'gte'>, <Comparator.LT:
'lt'>, <Comparator.LTE: 'lte'>]

I noticed that chroma and deeplake also support not equals/not filtering
so I added it there as well



[pinecone](https://docs.pinecone.io/docs/metadata-filtering#metadata-query-language)
[chroma](https://docs.trychroma.com/usage-guide#filtering-by-metadata)

[deeplake](https://docs.activeloop.ai/enterprise-features/compute-engine/querying-datasets/query-syntax#and-or-not)
pull/9999/head
Eddie Cohen 1 year ago committed by GitHub
parent 2221194450
commit 565c021730
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

@ -81,6 +81,7 @@ class Comparator(str, Enum):
"""Enumerator of the comparison operators."""
EQ = "eq"
NE = "ne"
GT = "gt"
GTE = "gte"
LT = "lt"

@ -17,6 +17,7 @@ class ChromaTranslator(Visitor):
"""Subset of allowed logical operators."""
allowed_comparators = [
Comparator.EQ,
Comparator.NE,
Comparator.GT,
Comparator.GTE,
Comparator.LT,

@ -22,6 +22,7 @@ COMPARATOR_TO_TQL = {
OPERATOR_TO_TQL = {
Operator.AND: "and",
Operator.OR: "or",
Operator.NOT: "NOT",
}
@ -37,7 +38,7 @@ def can_cast_to_float(string: str) -> bool:
class DeepLakeTranslator(Visitor):
"""Translate `DeepLake` internal query language elements to valid filters."""
allowed_operators = [Operator.AND, Operator.OR]
allowed_operators = [Operator.AND, Operator.OR, Operator.NOT]
"""Subset of allowed logical operators."""
allowed_comparators = [
Comparator.EQ,

@ -15,6 +15,7 @@ class PineconeTranslator(Visitor):
allowed_comparators = (
Comparator.EQ,
Comparator.NE,
Comparator.LT,
Comparator.LTE,
Comparator.GT,

Loading…
Cancel
Save