Commit Graph

136 Commits (814f0a3884350b6f1adfdebb79ca3599e9806605)

Author SHA1 Message Date
PalmerAL 814f0a3884 Add support for detecting lazy-loaded images (#542)
Add support for detecting lazy-loaded images using `src` or `srcset` attributes.
5 years ago
Gijs Kruitbosch cb5771fd4a Add nested font tags to test _setNodeTag on those (see #59) 6 years ago
Radhi 9009f64f9c Fix table header missing (#530) 6 years ago
Radhi 6761a7e412 Fix embedded videos getting removed (#526)
Fix embedded videos getting removed
6 years ago
PalmerAL b9cece3e58 add test 6 years ago
PalmerAL a014e0c9c8 exclude graphs from nytimes articles 6 years ago
Radhi Fadlillah c942b32945 Revert source files and fix expected results 6 years ago
Radhi Fadlillah bd5087d2f1 fix error in testing "wikipedia" 6 years ago
Radhi Fadlillah 3e025d58e5 fix error in testing "lwn-01" 6 years ago
Radhi Fadlillah df95c9d717 fix error in testing "keep-tabulard-data" 6 years ago
Radhi Fadlillah 6a5066abe2 Fix tabular data got removed 6 years ago
PalmerAL f70d36852b check itemprop when determining whether a node is a byline 6 years ago
Andres Rey d41de78c26 Close img tag 6 years ago
Andres Rey 1187b2dae1 Update test expectations 6 years ago
Andres Rey 3ca8c12d87 Update test expectations 6 years ago
Andres Rey 4ffd482004 Add medicalnewstoday test case with incorrect results 6 years ago
Gijs Kruitbosch 60ef565b67 Don't choke on <meta> tags that do not have a content attribute 6 years ago
Gijs Kruitbosch e8bb7f722f Fix whitespace normalization in title metadata
When switching to a newer version of JSDOM, it is more literal
about listing whitespace as part of textContent, including
newlines and not normalizing multiple spaces.

It seems prudent to just always normalize whitespace for titles,
which are guaranteed to be pretty short anyway.
6 years ago
Gijs Kruitbosch 3610476663 Remove CSS that jsdom struggles to parse 6 years ago
Maria Luiza Soares 8c41d92560 Assert on siteName in all test cases 6 years ago
Maria Luiza Soares 1bac47c70d Add newly generated test case 6 years ago
Daniel Aleksandersen 5a69d4a8eb Improve metadata extraction (#478)
* Improve metadata extraction

* Recognize meta[property] as a space-separated list
* Recognize Dulin Core (dc|dcterm): metadata.
* Prefer Dublin Core, Open Graph, Twitter, and HTML in that order.
* _getArticleTitle() is now only used as fallback if document
 doesn't provide good metadata.
6 years ago
Gijs Kruitbosch f782bc5f06 Avoid global flag when looking for metadata using regexes 6 years ago
David A Roberts ea4165721f Remove single-cell tables 6 years ago
David A Roberts bf64b58d90 Update tests 6 years ago
Gijs Kruitbosch d4b842c82a Match headings on trimmed strings to avoid whitespace causing mismatches 6 years ago
Gijs Kruitbosch 8c02a0d34c Fix #283 and remove hidden nodes 6 years ago
David A Roberts 656a6673d9 Don't put non-phrasing content into paragraphs 6 years ago
David A Roberts 5ae90930cd Don't convert DIVs to Ps when more than 25% links 6 years ago
David A Roberts 9f2c5cb42e Put phrasing content into paragraphs
This removes the need for `p.readability-styled` elements.
6 years ago
David A Roberts 7a24801958 Don't include root html node in candidates
Fixes #435
6 years ago
Andres Rey 3c76104adb Fix engadget test case 7 years ago
Andres Rey 4b99f41ec9 Add engadget test case 7 years ago
Andres Rey 6c5bc62959 Remove aside tags on test cases 7 years ago
David A Roberts f8d9b1c224 Update test expectations 7 years ago
David A Roberts 8414158fa9 Fix _replaceBrs
Previously, `nextElem` was not actually proceeding to the next element, and therefore aborting the paragraph at the first `<br>` (rather than the first `<br><br>` as the comment indicates).
7 years ago
Brad Philips 8525c6af36 Fix relative URIs given <base> tags (#422) 7 years ago
Andres Rey 834672ef86 Return longest text after failing to detect text longer than the configured value (#423)
Save extracted text across attempts and return the longest one when all attempts fail, and add a test case from hukumusume
7 years ago
Tom Z?hner 264b8e8968 Remove link elements when preparing article for display 7 years ago
Andres Rey fa9d8bda48 Add la-nacion test case 7 years ago
Gijs Kruitbosch 807bf05aa3 Fix className usage so it deals correctly with SVG nodes (fixes #412). 7 years ago
Gijs Kruitbosch ad4dd26448 Update test expectations 7 years ago
Andres Rey eb895b97a2 Add test case for title and h1 discrepancy 7 years ago
Andres Rey c2e370c2c7 Add telegraph test case 7 years ago
Cameron McCormack 5ad448f831 Update test expectations. 7 years ago
Taylor Hunt b7c32feb25 Remove presentational HTML attributes (#385)
* Remove presentational HTML attributes

Fixes #383

This patch loops through a list of known-presentational attributes in HTML, attempting to remove each from each cleaned element. (Checking for the attribute's existence first seems to just add needless overhead.)

The extra check for the HTML namespace is to avoid removing attributes that inline SVG needs.

* Only remove `width`/`height` for certain elements

Embedded media elements are allowed to have them, but not others.

* Address PR feedback

* Fix loop index formatting
* Only remove `width`/`height` from certain elements
* Combine logic into a single check/remove

* Attempt fixing my recursion

* One weird trick to get your loops to run

* Add inline SVG bailout

Try not to touch any styles for `<svg>`, because it's inherently presentational.

* Update tests to match newly-removed attributes

* Oh those wacky SVGs

The `position:absolute` is a trick to import clipping paths into the document without putting a big 300×150 empty space in it. (`display:none` and such disable the clipPath in some browsers.)

* Whoops, missed some `width`s

* Normalize SVG tagName

JSDOMParser differs from the official DOM here
7 years ago
Evan Tseng 924017c686 Merge pull request #370 from evanxd/bug-1347472
Bug 1347472 - Tests for WordPress blogs, r=Gijs
8 years ago
Evan Tseng f6d0840b82 Bug 1347472 - Tests for WordPress blogs, r=Gijs 8 years ago
Evan Tseng 0e147aa722 Bug 1346717 - Tests for youth.cn, r=Gijs 8 years ago
Evan Tseng 630a0b7367 Bug 1346155 - Tests for gmw.cn, r=Gijs 8 years ago