Commit Graph

158 Commits

Author SHA1 Message Date
Adam Pash
17317823de fix: bug that stopped proper attr cleaning in certain cases 2016-10-26 14:17:52 -07:00
Adam Pash
40768fa188 feat: support lazy loading video on deadspin 2016-10-26 11:53:42 -07:00
Adam Pash
38c90d239e fix: removeEmpty shouldn't remove elements with images or iframes inside 2016-10-26 11:53:02 -07:00
Adam Pash
c63f500433 fix: narrowed selector to fix blogspot title selector 2016-10-26 11:16:31 -07:00
Adam Pash
d3b11be473 feat: keeping youtube and vimeo iframe embeds (#14)
* feat: keeping youtube and vimeo iframe embeds

* fix: removing class from article correctly
2016-10-26 11:14:37 -07:00
Adam Pash
5c7f2cd28e fix: better selector for nytimes authors 2016-10-17 18:55:58 -07:00
Adam Pash
3b87b557be feat: pulling score from whitelist 2016-10-17 16:26:16 -07:00
Adam Pash
94321111e9 Merge pull request #13 from postlight/feat-apartmenttherapy-parser
feat: Add custom extrator for Apartment Therapy
2016-10-17 08:40:09 -07:00
Drew Bell
76db95e884 feat: Add custom extrator for Apartment Therapy 2016-10-17 10:35:22 -05:00
Adam Pash
4ebfd37408 Merge pull request #12 from postlight/feat-broadwayworld-extractor
feat: Add custom parser for broadwayworld.com
2016-10-13 17:02:04 -07:00
Drew Bell
a708ad3b4f feat: Add custom parser for broadwayworld.com 2016-10-13 16:22:33 -05:00
Adam Pash
896021227d feat: added deadspin custom parser 2016-10-13 13:46:36 -07:00
Adam Pash
422deb4600 feat: generator generates potential selectors for all custom selectable fields 2016-10-10 15:57:47 -07:00
Adam Pash
c314e3befa feat: dek returns null if it's basically the same as the excerpt
Squashed commit of the following:

commit 0ee7d51ce609ad23d2deca1af41e7b4e56681bd7
Author: Adam Pash <adam.pash@gmail.com>
Date:   Mon Oct 10 15:44:28 2016 -0700

    feat: dek does not return if it's basically the same as the excerpt

commit 6ad27f994fff3652e04ffe7c81f1ae0b1647e941
Author: Adam Pash <adam.pash@gmail.com>
Date:   Mon Oct 10 14:35:54 2016 -0700

    feat: added excerpt util
2016-10-10 15:44:58 -07:00
Adam Pash
63c06c8a00 fix: babel-polyfill mess (I think) 2016-10-10 14:16:14 -07:00
Adam Pash
eb0aa0b1f6 feat: some small tweaks to toy's excellent parsers ☺️
Squashed commit of the following:

commit 9638220124a325322d6cda7d16c645185d5fe827
Author: Adam Pash <adam.pash@gmail.com>
Date:   Mon Oct 10 11:02:29 2016 -0700

    fix: removed eslint plugin that was adding unneded async parens

commit ce2268c0f7c1b093c06f156730a0f1bc2aaba39c
Author: Adam Pash <adam.pash@gmail.com>
Date:   Mon Oct 10 10:47:36 2016 -0700

    style: fix async in parens

commit 9591856915eddaf93170da1ce9225b8a378bdf55
Author: Adam Pash <adam.pash@gmail.com>
Date:   Mon Oct 10 10:37:11 2016 -0700

    fix: remove parens around async

commit 6c56054717acc1f7e5499691780f8273f6d07bac
Author: Adam Pash <adam.pash@gmail.com>
Date:   Mon Oct 10 10:35:50 2016 -0700

    fix msn fixture; adjusted yahoo test

commit 4fc117ad5fdc5528f29b0873d60a6a1709642f15
Author: Adam Pash <adam.pash@gmail.com>
Date:   Mon Oct 10 10:14:38 2016 -0700

    removed dek and date_publised tests; neither exist in littlethings

commit 401094b4abc52901255fd2461f5839624f11d8a3
Author: Adam Pash <adam.pash@gmail.com>
Date:   Mon Oct 10 10:08:44 2016 -0700

    feat: updated buzzfeed for content extraction

commit 19548a5485f70ff9b65e3e725d2364d07734ac9c
Author: Adam Pash <adam.pash@gmail.com>
Date:   Mon Oct 10 09:54:30 2016 -0700

    fix: generator should make transforms an object, not array

commit b92113f9f7c97aca9e6d3ce9243abac967d26b63
Author: Adam Pash <adam.pash@gmail.com>
Date:   Mon Oct 10 08:54:38 2016 -0700

    feat: updated politico

commit c026591040f7671cb2a6dd5177a995e21d015482
Author: Adam Pash <adam.pash@gmail.com>
Date:   Mon Oct 10 08:48:52 2016 -0700

    fix: typos

commit 14aa8fa4ce38ff1c2a212cd0225437ae3042c2c3
Author: Adam Pash <adam.pash@gmail.com>
Date:   Mon Oct 10 08:36:12 2016 -0700

    fix: incorrect command in readme

commit fe260e6122877e2cb0130a1ecde0e503017057a3
Author: Adam Pash <adam.pash@gmail.com>
Date:   Mon Oct 10 08:31:11 2016 -0700

    fix: removed dek test because there is no dek on wikia
2016-10-10 11:03:43 -07:00
Toy Vano
3c99404566 Merge pull request #11 from postlight/feat-politico-extractor
feat: added politico extractor
2016-10-05 13:52:57 -04:00
Toy Vano
e766494922 feat: added politico extractor 2016-10-05 13:51:11 -04:00
Toy Vano
dd20e56bc5 Merge pull request #10 from postlight/feat-littlethings-extractor
feat: added littlethings extractor
2016-10-04 15:04:28 -04:00
Toy Vano
fd1ac3f2b9 feat: added littlethings extractor 2016-10-04 15:02:23 -04:00
Toy Vano
84619641b0 Merge remote-tracking branch 'origin/master' 2016-10-04 13:01:36 -04:00
Toy Vano
6c18551ed0 Merge pull request #9 from postlight/feat-wikia-extractor
feat: added wikia extractor
2016-10-04 12:11:29 -04:00
Toy Vano
017b9dfcc2 Merge pull request #8 from postlight/feat-buzzfeed-extractor
feat: added incomplete buzzfeed extractor
2016-10-04 12:11:22 -04:00
Toy Vano
bdf66314ea Merge pull request #7 from postlight/feat-yahoo-extractor
feat: added incomplete yahoo extractor
2016-10-04 12:11:15 -04:00
Toy Vano
b0e1a873c0 Merge pull request #6 from postlight/feat-msn-extractor
feat: added incomplete msn extractor
2016-10-04 12:11:07 -04:00
Toy Vano
1519eed3e5 feat: added wikia extractor 2016-10-04 12:06:19 -04:00
Toy Vano
9416ec73a4 feat: added incomplete buzzfeed extractor 2016-10-04 11:28:01 -04:00
Toy Vano
46e26c9b85 Merge branch 'feat-msn-extractor' 2016-10-03 17:51:32 -04:00
Toy Vano
c6c35bd237 feat: added incomplete yahoo extractor 2016-10-03 17:48:11 -04:00
Toy Vano
320c740676 feat: added incomplete msn extractor 2016-10-03 13:27:51 -04:00
Adam Pash
e3ee5e93bf chore: small doc fixes 2016-09-30 15:01:48 -04:00
Adam Pash
d786a7ae0c Merge pull request #5 from postlight/feat-wired-extractor
feat: added wired custom extractor

AMAAAAAAZING!!!!
2016-09-30 14:49:45 -04:00
Toy Vano
7ecc696248 feat: added wired custom extractor 2016-09-30 14:32:28 -04:00
Adam Pash
20b7c5a8b6 chore: fix a few typos/links 2016-09-30 12:46:46 -04:00
Adam Pash
173f885674 feat: custom parser + generator + detailed readme instructions
Squashed commit of the following:

commit 02563daa67712c3679258ebebac60dfa9568dffb
Author: Adam Pash <adam.pash@gmail.com>
Date:   Fri Sep 30 12:25:44 2016 -0400

    updated readme, added newyorker parser for readme guide

commit 0ac613ef823efbffbf4cc9a89e5cb2489d1c4f6f
Author: Adam Pash <adam.pash@gmail.com>
Date:   Fri Sep 30 11:16:52 2016 -0400

    feat: updated parser so the saved fixture absolutizes urls

commit 85c7a2660b21f95c2205ca4a4378a7570687fed0
Author: Adam Pash <adam.pash@gmail.com>
Date:   Fri Sep 30 10:15:26 2016 -0400

    refactor: attribute selectors must be an array for custom extractors

commit f60f93d5d3d9b2f2d9ec6f28d27ae9dcf16ef01e
Author: Adam Pash <adam.pash@gmail.com>
Date:   Thu Sep 29 10:13:14 2016 -0400

    fix: whitelisting srcset and alt attributes

commit e31cb1f4e8a9fc9c3d9b20ef9f40ca6c8d6ad51a
Author: Adam Pash <adam.pash@gmail.com>
Date:   Thu Sep 29 09:44:21 2016 -0400

    some housekeeping for coverage tests

commit 39eafe420c776a1fe7f9fea634fb529a3ed75a71
Author: Adam Pash <adam.pash@gmail.com>
Date:   Wed Sep 28 17:52:08 2016 -0400

    fix: word count for multi-page articles

commit b04e0066b52f190481b1b604c64e3d0b1226ff02
Author: Adam Pash <adam.pash@gmail.com>
Date:   Thu Sep 22 10:40:23 2016 -0400

    major improvements to output

commit 3f3a880b63b47fe21953485da670b6e291ac60e5
Author: Adam Pash <adam.pash@gmail.com>
Date:   Wed Sep 21 17:27:53 2016 -0400

    updated test command

commit 14503426557a870755453572221d95c92cff4bd2
Author: Adam Pash <adam.pash@gmail.com>
Date:   Wed Sep 21 16:00:30 2016 -0400

    shortened generator command

commit 5ebd8343cd4b87b3f5787dab665bff0de96846e1
Author: Adam Pash <adam.pash@gmail.com>
Date:   Wed Sep 21 15:59:14 2016 -0400

    feat: can disable fallback to generic parser (this will be useful for testing custom parsers)
2016-09-30 12:26:25 -04:00
Adam Pash
39a3c0690d chore: readme improvement 2016-09-21 15:00:00 -04:00
Adam Pash
ef047107ea feat: content cleaner still runs, but can disable some cleaners 2016-09-21 14:38:03 -04:00
Adam Pash
75b1880f01 chore: cleaned up unused files, slight reorg 2016-09-20 11:08:02 -04:00
Adam Pash
ad42055f8f feat: switched test framework to jest 2016-09-20 10:52:16 -04:00
Adam Pash
8f42e119e8 feat: generator for custom parsers and some documentation
Squashed commit of the following:

commit deaf9e60d031d9ee06e74b8c0895495b187032a5
Author: Adam Pash <adam.pash@gmail.com>
Date:   Tue Sep 20 10:31:09 2016 -0400

    chore: README for custom parsers

commit a8e8ad633e0d1576a52dbc90ce31b98fb2ec21ee
Author: Adam Pash <adam.pash@gmail.com>
Date:   Mon Sep 19 23:36:09 2016 -0400

    draft of readme

commit 4f0f463f821465c282ce006378e5d55f8f41df5f
Author: Adam Pash <adam.pash@gmail.com>
Date:   Mon Sep 19 17:56:34 2016 -0400

    custom extractor used to build basic parser for theatlantic

commit c5562a3cede41f56c4e723dcfa1181b49dcaae4d
Author: Adam Pash <adam.pash@gmail.com>
Date:   Mon Sep 19 17:20:13 2016 -0400

    pre-commit to test custom parser generator

commit 7d50d5b7ab780b79fae38afcb87a7d1da5d139b2
Author: Adam Pash <adam.pash@gmail.com>
Date:   Mon Sep 19 17:19:55 2016 -0400

    feat: added nytimes parser

commit 58b8d83a56927177984ddfdf70830bc4f328f200
Author: Adam Pash <adam.pash@gmail.com>
Date:   Mon Sep 19 17:17:28 2016 -0400

    feat: can do fuzzy search or go straight to file

commit c99add753723a8e2ac64d51d7379ac8e23125526
Author: Adam Pash <adam.pash@gmail.com>
Date:   Mon Sep 19 10:52:26 2016 -0400

    refactored export for custom extractors for easier renames

commit 22563413669651bb497f1bb2a92085b71f2ae324
Author: Adam Pash <adam.pash@gmail.com>
Date:   Fri Sep 16 17:36:13 2016 -0400

    feat: custom extractor generation in place

commit 2285a29908a7f82a5de3c81f6b2b902ddec9bdaa
Author: Adam Pash <adam.pash@gmail.com>
Date:   Fri Sep 16 16:42:20 2016 -0400

    good progress
2016-09-20 10:37:03 -04:00
Adam Pash
c4f06c7ebc fix: .babelrc was still referencing iris 2016-09-19 13:56:19 -04:00
Adam Pash
f58ccec7aa fix: including babel-runtime as a bandaid for polyfill error 2016-09-19 11:24:43 -04:00
Adam Pash
59fb4c4974 fix: using transform-runtime to avoid babel-polyfill conflicts when used
in external code
2016-09-19 11:04:35 -04:00
Adam Pash
b4fbc5b581 chore: barebones readme 2016-09-16 15:11:46 -04:00
Adam Pash
f439f9d2cf refactor: slightly better preview 2016-09-16 15:10:14 -04:00
Adam Pash
7ade83692a feat: improve wikipedia parser 2016-09-16 13:59:05 -04:00
Adam Pash
1597fc79c2 feat: added preview script to test urls on-the-fly 2016-09-16 13:58:41 -04:00
Adam Pash
2ae2dba690 chore: renamed iris to mercury 2016-09-16 13:26:37 -04:00
Adam Pash
005ba47f6f fix: wikpedia transform only grabs one image from .infobox 2016-09-16 13:17:21 -04:00
Adam Pash
44d3a547b2 fix: added dist back to git 2016-09-16 12:56:08 -04:00