Archives/fabric
2
0
Fork 0
mirror of https://github.com/danielmiessler/fabric synced 2024-11-08 07:11:06 +00:00
Code Issues Packages Projects Releases Wiki Activity

Improved analyze_paper.

Browse Source
This commit is contained in:
Daniel Miessler 2024-03-19 19:33:59 -07:00
parent e2ad03b121
commit be813b2b60
1 changed files with 5 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
patterns/analyze_paper/system.md
View File

@ -88,7 +88,11 @@ A final 25-word summary of the paper, its findings, and what we should do about
- The stronger the claims the stronger the evidence needs to be, as well as the transparency into the methodology. If the paper makes strong claims, but the evidence or transparency is weak, then the RIGOR score should be lowered.
- Remove up to 2 grades for papers where compelling data is provided but it's not clear what exact tests were run and/or how to reproduce those tests. Do not relax this requirement for papers that claim they couldn't provide the data for safety or other good-sounding reasons. Make the grade deductions anyway and note that in your reasoning and explanation for lower scores.
- Remove at least 1 grade (and up to 2) for papers where compelling data is provided but it's not clear what exact tests were run and/or how to reproduce those tests.
- Do not relax this transparency requirement for papers that claim security reasons.
- If a paper does not clearly articulate its methodology in a way that's replicable, lower the RIGOR and overall score significantly.
- Remove up to 1-3 grades for potential conflicts of interest indicated in the report.