Commit Graph

37 Commits

Author SHA1 Message Date
Michael Ashley
e12c916499
feat: ability to add custom extractors via api (#484)
* feat: ability to add custom extractors via api

* docs: updating readme

* fix: example.com was being used in another test

* fix: timezone was messing up date_published test

* fix: using a unique site for testing

* fix: updated custom extractor api

* docs: updating readme

* fix: removing unused fixture

* fix: updating test description

* feat: ability to add custom extractors via cli
2019-09-04 07:32:28 -07:00
Gina Trapani
9f6f07508c
docs: Add links to README 2019-05-19 13:30:16 -04:00
Drew Bell
b6c82f2b16 doc: fix extend typo in README (#340) 2019-03-26 15:23:50 -07:00
Toufic Mouallem
144a797564
feat: Support passing custom headers in requests (#337) 2019-03-26 13:48:41 +02:00
Toufic Mouallem
da9606a4cb docs: Add parsing custom HTML to README.md (#326) 2019-03-25 15:40:51 -07:00
Drew Bell
b3e2a0ffd1 feat: extract custom types with extend option (#313)
* feat: extract custom types with extend option

Adds an `extend` option that lets you add custom types to be extracted
and returned alongside the defaults, either in a call to `parse()` or in
a custom extractor.

```
Mercury.parse(
  url,
  extend: {
    last_edited: { selectors: ['#last-edited'], defaultCleaner: false }
  }
)
```

* chore: use Reflect.ownKeys

* feat: add CLI options

* doc: add extend param to cli help

* refactor: extract selectExtendedTypes

* feat: only overwrite null extended results

* feat: add allowMultiple extraction option

* feat: accept extendList CLI args

* feat: allow attribute selectors in extends on CLI

* test: update extend tests

* fix: don't invoke cleaner for custom types

* feat: always return array if allowMultiple

* test: add test for array of single result

* refactor: extract extractHtml

* refactor: destructure allowMultiple

* fix: wrap multiple matches in $ for cheerio shim

* fix: find extended types before any other munging

* feat: absolutize all links

* fix: clean content more directly

* doc: Update CLI docs in README

* chore: update dist

* doc: Document extend in custom extractor README
2019-03-25 15:36:20 -07:00
Matthew Watkins
6e66887048 docs: add content formats to README.md (#318) 2019-03-12 08:37:38 -07:00
Adam Pash
ffb25f34d7
docs: add usage gif (#308) 2019-03-05 11:37:56 -08:00
Madison Kanna
b3fa18b6d9 docs: delete extra semicolon (#266) 2019-02-11 15:44:00 -08:00
Adam Pash
02476f4336
docs: add instructions for cli to README (#251) 2019-02-06 09:46:13 -08:00
Keith Mancuso
44edcda53f docs: added gitter badge (#249) 2019-02-06 08:18:55 -08:00
Paul Ford
cfd9b59345
docs: add custom parsers to README
Added paragraph about custom parsers to README with links to relevant code and documentation.
2019-02-06 10:10:56 -05:00
Adam Pash
d0726a2d32
chor: remove appveyor yml and badge (#247) 2019-02-05 15:32:42 -08:00
greenkeeper[bot]
7bdbbc8ed8 deps: update dependencies to enable Greenkeeper 🌴 (#243) 2019-02-05 11:39:25 -08:00
Adam Pash
e38aff9c17
docs: add npm install instructions (#240) 2019-02-04 09:03:19 -08:00
Gina Trapani
dc3dff6584 docs: add hero to README (#239) 2019-02-01 18:01:39 -08:00
Adam Pash
c6f42c1278
docs: cleanup and update docs (#238) 2019-02-01 14:10:59 -08:00
Adam Pash
92de5ce4ed
docs: remove contributors (github already has this covered) (#237) 2019-01-31 09:50:38 -08:00
George Haddad
2845a1bb7e docs: add gitter room text and link (#235) 2019-01-31 09:25:54 -08:00
George Haddad
380196b709 docs: change text to include AMP and Reader (#236) 2019-01-31 09:11:52 -08:00
George Haddad
33bf5882b9
docs: add mit license badge (#234) 2019-01-31 10:28:01 +02:00
Toufic Mouallem
0668f5d75b docs: add instructions for browser usage to parse current page (#231) 2019-01-30 10:36:26 -08:00
Ralph Jbeily
199fe70b03 Docs contributors (#227)
* Add @george-haddad as a contributor

* Add @toufic-m as a contributor

* Add @WajeehZantout as a contributor

* Add @e55o as a contributor

* Add @JadTermsani as a contributor

* Add @RalphJbeily as a contributor

* Add @alexiakl as a contributor

* docs: fix Alexi and George profiles

* docs: update README.md

* docs: update contributors table

* docs: fix image size and contributors per line

* docs: add license badge
2019-01-30 09:26:50 +02:00
Ralph Jbeily
9756e6ee67 docs: update mercury parser installation (#228)
* docs: update mercury parser installation

* docs: fix import for example source code
2019-01-30 09:17:23 +02:00
Marc Esso
5ad02b6f28
docs: add license files (#217)
* docs: add license files

* docs: license sentence in readme

* docs: change contributing section sentence in readme.md

* docs: small grammar mistake in README.md
2019-01-24 12:10:04 +02:00
Ralph Jbeily
ca44ce3dd1
docs: add install build and test guide (#215)
* docs: add install build and test guide

* docs: remove install build and test guides

* docs: add installation guide
2019-01-24 11:15:23 +02:00
Adam Pash
e325d860fd Feat: improving ci (#16)
This commit also swaps in yarn for npm and tweaks circle ci a bit.

* appveyor.yml first go

* changing node

* ps

* narrow it down

* trying this

* fix airbnb module

* trying with yarn

* logging

* hybrid?

* trying yarn w/circle

* bump workers?

* build off?

* updating script

* tweaking script for appveyor

* bumping maxworkers

* cleaning up

* build step?

* yarn it

* added appveyor badge
2016-10-28 09:16:21 -07:00
Adam Pash
41c3454590 fix: circle test passing badge 2016-10-27 16:35:52 -07:00
Adam Pash
4c9910384a Feat: adding circle ci (#15)
* added circle.yml config

* set maxworkers in circle

* trying diff node versions

* multiple node

* pre nvm install

* testing parallel

* added badge to readme

* clean up circle.yml
2016-10-27 16:20:30 -07:00
Adam Pash
90501ec3b5 fix: increased avatar size 2016-10-26 15:44:49 -07:00
Adam Pash
efd8ebbc69 Add @mutewinter as a contributor 2016-10-26 15:42:14 -07:00
Adam Pash
227bd6a36e Add @droob as a contributor 2016-10-26 15:41:48 -07:00
Adam Pash
9897d9bee8 Add @spiffytoy as a contributor 2016-10-26 15:41:37 -07:00
Adam Pash
615f142c5e Update @adampash as a contributor 2016-10-26 15:41:12 -07:00
Adam Pash
659301b4ab Add @adampash as a contributor 2016-10-26 15:40:16 -07:00
Adam Pash
8f42e119e8 feat: generator for custom parsers and some documentation
Squashed commit of the following:

commit deaf9e60d031d9ee06e74b8c0895495b187032a5
Author: Adam Pash <adam.pash@gmail.com>
Date:   Tue Sep 20 10:31:09 2016 -0400

    chore: README for custom parsers

commit a8e8ad633e0d1576a52dbc90ce31b98fb2ec21ee
Author: Adam Pash <adam.pash@gmail.com>
Date:   Mon Sep 19 23:36:09 2016 -0400

    draft of readme

commit 4f0f463f821465c282ce006378e5d55f8f41df5f
Author: Adam Pash <adam.pash@gmail.com>
Date:   Mon Sep 19 17:56:34 2016 -0400

    custom extractor used to build basic parser for theatlantic

commit c5562a3cede41f56c4e723dcfa1181b49dcaae4d
Author: Adam Pash <adam.pash@gmail.com>
Date:   Mon Sep 19 17:20:13 2016 -0400

    pre-commit to test custom parser generator

commit 7d50d5b7ab780b79fae38afcb87a7d1da5d139b2
Author: Adam Pash <adam.pash@gmail.com>
Date:   Mon Sep 19 17:19:55 2016 -0400

    feat: added nytimes parser

commit 58b8d83a56927177984ddfdf70830bc4f328f200
Author: Adam Pash <adam.pash@gmail.com>
Date:   Mon Sep 19 17:17:28 2016 -0400

    feat: can do fuzzy search or go straight to file

commit c99add753723a8e2ac64d51d7379ac8e23125526
Author: Adam Pash <adam.pash@gmail.com>
Date:   Mon Sep 19 10:52:26 2016 -0400

    refactored export for custom extractors for easier renames

commit 22563413669651bb497f1bb2a92085b71f2ae324
Author: Adam Pash <adam.pash@gmail.com>
Date:   Fri Sep 16 17:36:13 2016 -0400

    feat: custom extractor generation in place

commit 2285a29908a7f82a5de3c81f6b2b902ddec9bdaa
Author: Adam Pash <adam.pash@gmail.com>
Date:   Fri Sep 16 16:42:20 2016 -0400

    good progress
2016-09-20 10:37:03 -04:00
Adam Pash
b4fbc5b581 chore: barebones readme 2016-09-16 15:11:46 -04:00