David A Roberts
ea4165721f
Remove single-cell tables
6 years ago
David A Roberts
bf64b58d90
Update tests
6 years ago
David A Roberts
68c9af4ffa
Use numeric encoding for non-XML entities
...
JSDOMParser can't handle HTML named entities like ` `
6 years ago
Gijs Kruitbosch
d4b842c82a
Match headings on trimmed strings to avoid whitespace causing mismatches
6 years ago
Gijs Kruitbosch
8c02a0d34c
Fix #283 and remove hidden nodes
6 years ago
David A Roberts
656a6673d9
Don't put non-phrasing content into paragraphs
6 years ago
David A Roberts
5ae90930cd
Don't convert DIVs to Ps when more than 25% links
6 years ago
David A Roberts
9f2c5cb42e
Put phrasing content into paragraphs
...
This removes the need for `p.readability-styled` elements.
6 years ago
David A Roberts
c823a6efb2
Fix generate-testcase.js
6 years ago
Gijs Kruitbosch
f4ab856992
Check for a document being passed
...
This provides a descriptive error message if no document is passed, and
ignores the first argument if the second argument looks like
a reasonable DOM document instance.
6 years ago
David A Roberts
7a24801958
Don't include root html node in candidates
...
Fixes #435
6 years ago
David A Roberts
acfd3759a1
Generate XHTML-compatible input for test cases
...
Fixes the bug noted in the README
6 years ago
David A Roberts
d60184966c
Remove unused URI parameter from constructor
6 years ago
Andres Rey
3c76104adb
Fix engadget test case
6 years ago
Andres Rey
4b99f41ec9
Add engadget test case
6 years ago
Andres Rey
6c5bc62959
Remove aside tags on test cases
6 years ago
David A Roberts
f8d9b1c224
Update test expectations
6 years ago
David A Roberts
8414158fa9
Fix _replaceBrs
...
Previously, `nextElem` was not actually proceeding to the next element, and therefore aborting the paragraph at the first `<br>` (rather than the first `<br><br>` as the comment indicates).
6 years ago
Brad Philips
8525c6af36
Fix relative URIs given <base> tags ( #422 )
6 years ago
Gijs Kruitbosch
d598baf02b
Improve URL handling in JSDOMParser and Readability.js
...
This change ups the required node version to 7.0 because it relies on the builtin url module.
We now pass a url when constructing a jsdom document or JSDOMParser document.
Because this is an API change, I'm increasing the package version.
Ultimately, I would like to remove the argument from the readability constructor. It should
use the documentURI from the document it is passed.
6 years ago
Andres Rey
834672ef86
Return longest text after failing to detect text longer than the configured value ( #423 )
...
Save extracted text across attempts and return the longest one when all attempts fail, and add a test case from hukumusume
6 years ago
Tom Z?hner
264b8e8968
Remove link elements when preparing article for display
6 years ago
Andres Rey
fa9d8bda48
Add la-nacion test case
7 years ago
Gijs Kruitbosch
807bf05aa3
Fix className usage so it deals correctly with SVG nodes ( fixes #412 ).
7 years ago
Gijs Kruitbosch
c586aeb404
Fix generate-testcase.js script so it keeps `caption` classes
7 years ago
Gijs Kruitbosch
ad4dd26448
Update test expectations
7 years ago
Andres Rey
eb895b97a2
Add test case for title and h1 discrepancy
7 years ago
Andres Rey
c2e370c2c7
Add telegraph test case
7 years ago
Cameron McCormack
5ad448f831
Update test expectations.
7 years ago
Cameron McCormack
d88c9afc63
Use a hard coded classesToPreserve in tests.
7 years ago
Björgvin Ragnarsson
c3ff1a2d2c
remove dead code
7 years ago
Taylor Hunt
b7c32feb25
Remove presentational HTML attributes ( #385 )
...
* Remove presentational HTML attributes
Fixes #383
This patch loops through a list of known-presentational attributes in HTML, attempting to remove each from each cleaned element. (Checking for the attribute's existence first seems to just add needless overhead.)
The extra check for the HTML namespace is to avoid removing attributes that inline SVG needs.
* Only remove `width`/`height` for certain elements
Embedded media elements are allowed to have them, but not others.
* Address PR feedback
* Fix loop index formatting
* Only remove `width`/`height` from certain elements
* Combine logic into a single check/remove
* Attempt fixing my recursion
* One weird trick to get your loops to run
* Add inline SVG bailout
Try not to touch any styles for `<svg>`, because it's inherently presentational.
* Update tests to match newly-removed attributes
* Oh those wacky SVGs
The `position:absolute` is a trick to import clipping paths into the document without putting a big 300×150 empty space in it. (`display:none` and such disable the clipPath in some browsers.)
* Whoops, missed some `width`s
* Normalize SVG tagName
JSDOMParser differs from the official DOM here
7 years ago
Gijs Kruitbosch
6f0bd81dfa
Provide more useful output when attributes mismatch, r=me,test-only
7 years ago
Evan Tseng
924017c686
Merge pull request #370 from evanxd/bug-1347472
...
Bug 1347472 - Tests for WordPress blogs, r=Gijs
7 years ago
Evan Tseng
f6d0840b82
Bug 1347472 - Tests for WordPress blogs, r=Gijs
7 years ago
Evan Tseng
0e147aa722
Bug 1346717 - Tests for youth.cn, r=Gijs
7 years ago
Evan Tseng
630a0b7367
Bug 1346155 - Tests for gmw.cn, r=Gijs
7 years ago
Evan Tseng
8976fb08bf
Bug 1345797 - Tests for yahoo.co.jp, r=Gijs
7 years ago
Evan Tseng
e48451c48c
Bug 1342355 - Add tests for wikia.com, r=Gijs
7 years ago
Evan Tseng
c7c26382b9
Bug 1217007 - Remove nodes without any content, r=Gijs
7 years ago
Evan Tseng
0f147374b7
Bug 1323861 - Remove the readScript method, r=Gijs
7 years ago
Evan Tseng
27b97434a3
Merge pull request #352 from evanxd/bug-1338468
...
Bug 1338468 - Tests for tumblr.com, r=Gijs
7 years ago
Evan Tseng
b46e316034
Bug 1338468 - Tests for tumblr.com, r=Gijs
7 years ago
Evan Tseng
bd2d9f8100
Bug 1338475 - Tests for pixnet.net, r=Gijs
7 years ago
Evan Tseng
1369f923db
Merge pull request #348 from gijsk/1322674-table-divs
...
Fix bug 1322674 by only conditionally removing content from tables when the tables are layout tables, r=evanxd
7 years ago
Gijs Kruitbosch
66c013c4fb
Make it easier to find where errors are coming from based on the error message
7 years ago
Gijs Kruitbosch
9baea36169
Fix bug 1322674 by not removing content in data tables
7 years ago
Evan Tseng
19f2124c82
Bug 1259763 - Remove h2 when there is only one h2 and its text content substantially equals article title, r=Gijs
7 years ago
Evan Tseng
ccc8e9bf4c
Bug 1330931 - Add tests for nytimes.com, r=Gijs
8 years ago
Evan Tseng
15e1f03261
Bug 1300697 - Reader View missed first few paragraphs on New York Times website, r=Gijs
8 years ago
Evan Tseng
a58913d975
Bug 1300697 - Add test for New York Times website
8 years ago
Evan Tseng
ad13a63065
Bug 1332236 - Add tests for cnet.com, r=Gijs
8 years ago
Evan Tseng
131d923d38
Bug 1167568 - Find a better topCandidate if there are other nodes scores are high enough, r=Gijs
8 years ago
Sebastian Hengst
5e9c7a3910
Bug 1322327 - Only remove images which are not descendants of a figure if there is more than one image, update affected tests
8 years ago
andrei-ch
c5ff44d8fe
Clean <input>,<textarea>,<select>,<button> elements
8 years ago
Evan Tseng
a34d054f45
Merge pull request #332 from gijsk/fix-readability-determination-in-generate-testcase
...
Use jsdom for parsing the document to determine readability (fixes #325 ), r=evanxd
8 years ago
Evan Tseng
63230a307a
Bug 1142312 - Add two more types of unlikely candidates: cover-wrap and yom-remote, r=Gijs
8 years ago
Gijs Kruitbosch
46842048c1
Use jsdom for parsing the document to determine readability ( fixes #325 )
8 years ago
Gijs Kruitbosch
0ab4ac8556
Fix test failures caused by timeout still being too low
8 years ago
Evan Tseng
e84c0c3f07
Bug 1285543 - Only use "og:title" or "twitter:title" if _getArticleTitle does not return a valid title, r=Gijs
8 years ago
Gijs
c2f7db51f5
Remove textContent from metadata file ( fixes #324 ) ( #326 )
8 years ago
Evan Tseng
33dc8fa023
Bug 1255978 - Remove legends candidate, r=Gijs
8 years ago
Evan Tseng
af0aa5c59f
Bug 1173548 - Find out text direction from ancestors of final candidate, r=Gijs
8 years ago
Evan Tseng
ece0d1ecea
Bug 1317930 - Tests for msn.com, r=Gijs
8 years ago
Evan Tseng
1b694cf650
Bug 1310075 - Tests for qq.com. r=Gijs
8 years ago
Evan Tseng
522f39617f
Bug 1310074 - Tests for yahoo.com. r=Gijs
8 years ago
Evan Tseng
4fa0d1b207
Bug 1177619 - Score div nodes which have br nodes. r=Gijs
8 years ago
Evan Tseng
8bfd2a978d
Bug 1310073 - Tests for wikipedia.org. r=Gijs
8 years ago
Gijs
1a12befa41
Fix code style, tighten up eslint rules ( #301 )
8 years ago
Gijs Kruitbosch
46b08a5ea5
Address issue #277 by marking 'modal' unlikely+negative
8 years ago
Gijs Kruitbosch
a4d1e9ca12
Fix oversight in comment removal code exposed by better/newer jsdom implementation
8 years ago
Gijs Kruitbosch
e830ac9dd8
Fix eslint issues identified in m-c
8 years ago
Gijs Kruitbosch
dffa760c04
Fix issue #267 by ignoring hash URIs when making URIs absolute
8 years ago
Wes Johnston
f87a12400b
Reuse test from pull request #239 which passes without modifications (modified by @gijsk to pass in the current XHTML test environment)
9 years ago
Gijs Kruitbosch
2e1cb3f467
Fix issue #251 by making JSDOMParser expect XML and stop making excuses for 'self-closed' things, when all that does is cause trouble
9 years ago
Gijs
a801846a45
Merge pull request #204 from mozilla/tweak-great-grandparent-scoring
...
Updated great grandparent node scoring.
9 years ago
Nicolas Perriault
ae0833522c
Improved embedded video elements detection.
9 years ago
Nicolas Perriault
46304bb5fe
Updated great grandparent node scoring.
9 years ago
Nicolas Perriault
88ef3893b5
Fixes #180 - Score intermediary headings.
9 years ago
Nicolas Perriault
dc1b2c9fa0
Refs #195 - Exclude nodes likely to be related content.
9 years ago
Nicolas Perriault
cc18cb5787
Ref #195 - Add support for dailymotion videos.
9 years ago
Nicolas Perriault
9dbc009376
Fixes #113 - Recursive node ancestor scoring.
9 years ago
Nicolas Perriault
44879722b6
Fixes #183 - Preserve list items.
9 years ago
Alexis Métaireau
5912e0c872
Add Firefox User-Agent when generating the test case.
9 years ago
Gijs
79aa2fca87
Merge pull request #189 from mozilla/dont-remove-headings
...
Fixes #150 - Keep article intermediary headings.
9 years ago
Margaret Leibovic
af6da2a87d
Merge pull request #190 from mozilla/improved-author-meta-extraction
...
Improved author metadata detection.
9 years ago
Nicolas Perriault
7aee44adb2
Improved author metadata detection.
9 years ago
Gijs Kruitbosch
5f184053cd
Make isProbablyReaderable include <pre>, and deal with long <br>-separated paragraphs and/or shorter-than-5-paragraph text and such.
9 years ago
Nicolas Perriault
2451a07a7d
Fixes #150 - Keep article intermediary headings.
9 years ago
Margaret Leibovic
319a50b4f0
Fixes #184 - Don't strip class names from article content
9 years ago
Gijs
49e40768aa
Merge pull request #185 from mozilla/score-section-tags-by-default
...
Fixes #139 #143 : Added more weight to section tags.
9 years ago
Nicolas Perriault
f6ffa6acde
Fixes #139 #143 : Added more weight to section tags.
9 years ago
Nicolas Perriault
58cd789cd3
Improved title extraction 'algorithm'.
9 years ago
Nicolas Perriault
de89036cd5
Fixes #130 - Using js-beautify for HTML formatting.
9 years ago
Gijs
b37ff08bc7
Merge pull request #169 from mozilla/clean-footer-tags
...
Fixes #163 - Avoid including footer tag contents.
9 years ago
Nicolas Perriault
12c6a11f67
Fixes #163 - Avoid including footer tag contents.
9 years ago
Nicolas Perriault
6eeabf90c1
Fixes #164 - Add support for title alt semantic metadata.
9 years ago
Gijs Kruitbosch
0ff82de0f4
Implement createTextNode, do more relaxed escaping there, update testcase.
9 years ago
Margaret Leibovic
37a8cd4171
Bug 1147584 - Don't remove unlikely <a> tags, and replace <a> tags with their text content if they won't be useful links
9 years ago
Gijs
a6014f5854
Merge pull request #132 from gijsk/heise-ad-prioritization
...
Don't look at banners and skyscrapers, remove <noscript> elements
9 years ago