Commit Graph

36 Commits (fix-remove-moment-js)

Author SHA1 Message Date
touchRED 6a5f892c68 fix: update tests to remove dayjs where possible, update formats 1 year ago
touchRED 98b8f69d41 fix: replace moment with dayjs 1 year ago
Sarah Doire c0364ec52b
feat: update all fixtures and custom parsers to match (#713)
* feat: Refactor and update fixtures

This patch changes how fixtures are stored. Previously, a fixture's folder identified its domain and its filename identified when it was fetched. This has been changed so that the filename indicates the domain and the modified time of the file indicates how recently it was fetched. A fixture's filename can optionally include a modifier to distinguish between two different page types on the same domain, for example.

Also included here are changes to the update-fixture script, both to accomodate the new filename scheme as well as to actually update all fixtures. The functionality for running automatically and opening PRs has been removed but will likely be reintroduced.

Finally, all fixtures have been updated.

* Remove reference to deleted extractor

* feat: first batch of test and parser updates due to new fixtures

* feat: update more custom parsers and unit tests

* feat: update more custom parsers and unit tests and remove unnecessary parser

* feat: update more custom parsers and unit tests

* feat: update more parsers and add correct bloomberg html files

* fix: remove console statement

* feat: all parsers updated and tests passing

* fix: update date_published tests to account for test server time difference

* fix: cleanup remaining fixtures in folders

* feat: move fixtures for newest custom parsers

* feat: remove script changes

* fix: update dist files to account for reverting script changes

* adding .DS_Store to .gitignore

* adding .DS_Store to .gitignore -- 2

* adding .DS_Store to .gitignore -- 3 lol

* cleaning up some tests

* fix: ran build:generator command to update generate-custom-parser dist file

* fix: update rollup configs to generate source maps and update source maps

* fix: use underscore in place of unused error variable

* fix: remove unused fixture

Co-authored-by: Postlight Bot <adam.pash+postlight-bot@postlight.com>
Co-authored-by: flbn <overasc@gmail.com>
2 years ago
Austin 635fcf6356
fix: handle sec & ms timestamps properly (#702) 2 years ago
John Holdun 112846f74f
chore: Inline test fixtures (#683)
Not to be confused with extractor fixtures, which are snapshots of a webpage.

This change removes the pattern of separate JS files that provide "fixtures" for tests, which are used as provided or expected strings in tests. They were inconsistent and disorganized, and generally just served to add indirection to test files. So now all those strings are defined where they are used in their respective tests.
2 years ago
Toufic Mouallem 7844129fda feat: Add custom parser for Reddit (#307) 5 years ago
Adam Pash 663cc45bf4
fresh run of prettier; remove NOTES.md (#233) 5 years ago
Adam Pash 76d333f0be
deps: upgrade (#218) 5 years ago
George Haddad 56badb51f5
dx: remove unnec comments in source (#205)
* dx: remove commented code and obvious comments that can be looked up

* dx: remove commented out eslint options

* dx: remove commented out code

* dx: remove commented out code

* dx: remove commented out code

* dx: remove test block as all its code was commented out

* dx: remove commented out code

* dx: remove commented out code

* dx: remove commented out code

* dx: remove regex example comments

* dx: remove commented out code

* dx: remove commented out code

* dx: remove commented out import

* dx: remove commented out code

* dx: remove commented out code

* dx: remove commented out code

* dx: remove commented out code

* dx: remove commented out code

* dx: remove commented out code

* dx: remove commented out code

* dx: remove commented out code

* dx: remove commented out code

* dx: remove commented out code

* dx: remove commented out code

* chore: remove empty files

* chore: re-prettier code that may have missed it

* added back nec comments
5 years ago
Adam Pash e4b057f9ea
chore: update node and some deps (#209)
* chore: update .nvmrc

* added prettier and pre-commit hooks

* update docker image to new node

* add karma-cli to get web tests working

* explictly install karma... seems to fix problem

* remove pre-built phantomjs

* swap install order
5 years ago
Adam Pash 61f0f4e1af fix: kept elements being removed (#166)
Elements marked to keep were removeable under specific circumstances.
This PR fixes these edge cases.
7 years ago
Adam Pash 08b5bb7ff1 feat: allow parser to define custom date formats (#141)
* feat: allow parser to define custom date formats

* feat: updating macrumors to test/verify format working correctly
7 years ago
Adam Pash cfe7f34be4 fix: normalizing spaces for authors/dek/title (#31)
* fix: normalizing spaces for authors/dek/title
8 years ago
Adam Pash 6343946dd8 Feat: custom timezones (#29)
* using moment-timezone to allow custom timezones

* added tz to tmz, even though still so-so
8 years ago
Adam Pash a8face796a Fix extension bugs (#23)
* feat: cleaning supplemental elements in nytimes (visible in web only)

closes https://github.com/postlight/mercury-reader-chrome-extension/issues/102

* wip

* fix: more generous date published bits

* feat: added washington post extractor (including figure transforms)

closes https://github.com/postlight/mercury-reader-chrome-extension/issues/100

* feat: cleaning zoom lightbox from gizmodo/kinja

* lint fix
8 years ago
Adam Pash 60a6861e18 Feat: browser support (#19)
Big undertaking to support Mercury in the browser. Builds are working and all tests are passing both for web and node builds. Most code is closely shared.
8 years ago
Adam Pash 65c641a879 feat: enforcing line break rules in linter 8 years ago
Adam Pash 007ddec8ac feat: allowing iframes from src domain 8 years ago
Adam Pash 17317823de fix: bug that stopped proper attr cleaning in certain cases 8 years ago
Adam Pash d3b11be473 feat: keeping youtube and vimeo iframe embeds (#14)
* feat: keeping youtube and vimeo iframe embeds

* fix: removing class from article correctly
8 years ago
Adam Pash 422deb4600 feat: generator generates potential selectors for all custom selectable fields 8 years ago
Adam Pash c314e3befa feat: dek returns null if it's basically the same as the excerpt
Squashed commit of the following:

commit 0ee7d51ce609ad23d2deca1af41e7b4e56681bd7
Author: Adam Pash <adam.pash@gmail.com>
Date:   Mon Oct 10 15:44:28 2016 -0700

    feat: dek does not return if it's basically the same as the excerpt

commit 6ad27f994fff3652e04ffe7c81f1ae0b1647e941
Author: Adam Pash <adam.pash@gmail.com>
Date:   Mon Oct 10 14:35:54 2016 -0700

    feat: added excerpt util
8 years ago
Adam Pash 63c06c8a00 fix: babel-polyfill mess (I think) 8 years ago
Adam Pash ef047107ea feat: content cleaner still runs, but can disable some cleaners 8 years ago
Adam Pash cbd0636dcf chore: cleaned up python and other unneeded comments 8 years ago
Adam Pash bf13b38a9b feat: some basic error handling for bad urls 8 years ago
Adam Pash ffaf7db0f1 fix: some improvements to date parsing. punting on localization issues 8 years ago
Adam Pash 396313aeae feat: added twitter custom extractor
Squashed commit of the following:

commit 8116f14364869b72a8afabfcb44b2ac154caed96
Author: Adam Pash <adam.pash@gmail.com>
Date:   Thu Sep 15 16:27:27 2016 -0400

    feat: added twitter custom extractor

commit e478eb1b0bcdcb65fdd5fa64e37be92b6defd702
Author: Adam Pash <adam.pash@gmail.com>
Date:   Thu Sep 15 16:22:54 2016 -0400

    fix: made custom extractors and cleaners adhere to underscore keys
8 years ago
Adam Pash 7c375aded7 chore: cleanup 8 years ago
Adam Pash b325a4acdd chore: clean up junk tests 8 years ago
Adam Pash 62ae330db2 fix: bug in scoring and converting to paragraphs 8 years ago
Adam Pash 7e2a34945f chore: refactored and linted 8 years ago
Adam Pash c48e3485c0 chore: code reorganization
Squashed commit of the following:

commit 636296841d5cf5e685237fe70db7a15305d8e966
Author: Adam Pash <adam.pash@gmail.com>
Date:   Fri Sep 9 13:37:21 2016 -0400

    final cleanup

commit 51f712b3074d41a1f2da91519289d4dd09719ad0
Author: Adam Pash <adam.pash@gmail.com>
Date:   Fri Sep 9 13:25:28 2016 -0400

    Another big pass

commit 3860e6d872a9adb9290093fd9c8708dfcc773c28
Author: Adam Pash <adam.pash@gmail.com>
Date:   Fri Sep 9 12:49:52 2016 -0400

    chore: started reorganizing
8 years ago
Adam Pash edfb54c532 feat: links are rewritten to absolute in cleaner
Squashed commit of the following:

commit 9057d411a5458f80c316604559c469a239ef3a40
Author: Adam Pash <adam.pash@gmail.com>
Date:   Fri Sep 9 11:42:19 2016 -0400

    feat: links are rewritten to absolute in cleaner
8 years ago
Adam Pash 33c7e0d1c9 feat: Improved dateString parsing to handle more; first trying to parse without cleaning 8 years ago
Adam Pash 91881df523 refactor: cleaners now run on custom extractors
Squashed commit of the following:

commit e4c7d1d149d1846f0d589b3653655b81b477c682
Author: Adam Pash <adam.pash@gmail.com>
Date:   Thu Sep 8 19:29:26 2016 -0400

    refactor: cleaners now run on custom extractors

commit ca08d2482c54bf6a40f50758da9353f00987a4d7
Author: Adam Pash <adam.pash@gmail.com>
Date:   Thu Sep 8 14:42:19 2016 -0400

    moved cleaners, refactored as necessary

commit ec2c5d36410b255c6d8ee264deca990c46709c3c
Author: Adam Pash <adam.pash@gmail.com>
Date:   Thu Sep 8 14:07:01 2016 -0400

    moved datePublished cleaner

commit 5e55e397eecb3e88d64cd2aa2c6071c9cffed272
Author: Adam Pash <adam.pash@gmail.com>
Date:   Thu Sep 8 13:34:21 2016 -0400

    moved dek cleaner

commit 2dfb0c44d7882336992fdc864792df6eac094c21
Author: Adam Pash <adam.pash@gmail.com>
Date:   Thu Sep 8 13:29:37 2016 -0400

    moved lead-image-url

commit cef7a213b80ddd671249225622f1388f9e68896c
Author: Adam Pash <adam.pash@gmail.com>
Date:   Thu Sep 8 13:26:20 2016 -0400

    moved author
8 years ago