Update the parser regex of map_rerank (#6419)

Sometimes the score responded by chatgpt would be like 'Respone
example\nScore: 90 (fully answers the question, but could provide more
detail on the specific error message)'
For the score contains not only numbers, it raise a ValueError like 


Update the RegexParser from `.*` to `\d*` would help us to ignore the
text after number.

Co-authored-by: Bagatur <baskaryan@gmail.com>
pull/7643/head
Ma Donghao 1 year ago committed by GitHub
parent b08f903755
commit 6f62e5461c
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

@ -3,7 +3,7 @@ from langchain.output_parsers.regex import RegexParser
from langchain.prompts import PromptTemplate
output_parser = RegexParser(
regex=r"(.*?)\nScore: (.*)",
regex=r"(.*?)\nScore: (\d*)",
output_keys=["answer", "score"],
)

@ -0,0 +1,17 @@
"""Test map_rerank parser"""
import pytest
from langchain.chains.question_answering.map_rerank_prompt import output_parser
GOOD_SCORE = "foo bar answer.\nScore: 80"
SCORE_WITH_EXPLANATION = "foo bar answer.\nScore: 80 (fully answers the question, but could provide more detail on the specific error message)" # noqa: E501
@pytest.mark.parametrize("answer", (GOOD_SCORE, SCORE_WITH_EXPLANATION))
def test_parse_scores(answer: str) -> None:
result = output_parser.parse(answer)
assert result["answer"] == "foo bar answer."
score = int(result["score"])
assert score == 80
Loading…
Cancel
Save