You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
34 lines
1.3 KiB
Markdown
34 lines
1.3 KiB
Markdown
# Factuality
|
|
|
|
LLMs have a tendency to generate responses that sounds coherent and convincing but can sometimes be made up. Improving prompts can help improve the model to generate more accurate/factual responses and reduce the likelihood to generate inconsistent and made up responses.
|
|
|
|
Some solutions might include:
|
|
- provide ground truth (e.g., related article paragraph or Wikipedia entry) as part of context to reduce the likelihood of the model producing made up text.
|
|
- configure the model to produce less diverse responses by decreasing the probability parameters and instructing it to admit (e.g., "I don't know") when it doesn't know the answer.
|
|
- provide in the prompt a combination of examples of questions and responses that it might know about and not know about
|
|
|
|
Let's look at a simple example:
|
|
|
|
*Prompt:*
|
|
```
|
|
Q: What is an atom?
|
|
A: An atom is a tiny particle that makes up everything.
|
|
|
|
Q: Who is Alvan Muntz?
|
|
A: ?
|
|
|
|
Q: What is Kozar-09?
|
|
A: ? Q:
|
|
|
|
How many moons does Mars have?
|
|
A: Two, Phobos and Deimos.
|
|
|
|
Q: Who is Neto Beto Roberto?
|
|
```
|
|
|
|
*Output:*
|
|
```
|
|
A: ?
|
|
```
|
|
|
|
I made up the name "Neto Beto Roberto" so the model is correct in this instance. Try to change the question a bit and see if you can get it to work. There are different ways you can improve this further based on all that you have learned so far. |