Mercury Parser - Extracting content from chaos #parser #url #html #extractor
Go to file
2017-02-02 10:23:20 -07:00
dist release: 1.0.5 (#136) 2017-02-01 15:39:19 -08:00
fixtures feat: sciencefly extractor (#116) 2017-02-02 11:26:29 -05:00
scripts feat: changed user agent to latest chrome (#121) 2017-01-26 08:10:43 -08:00
src feat: adds www.polygon.com to list of www.theverge.com supportedDomains 2017-02-02 10:23:20 -07:00
.agignore chore: renamed iris to mercury 2016-09-16 13:26:37 -04:00
.all-contributorsrc fix: increased avatar size 2016-10-26 15:44:49 -07:00
.babelrc feat: making yarn-friendly for package manager (#17) 2016-10-28 11:10:42 -07:00
.eslintignore Feat: browser support (#19) 2016-11-21 14:17:06 -08:00
.eslintrc feat: recording/playing back network requests with nock (#18) 2016-10-28 14:54:12 -07:00
.gitattributes fix: i put a bad comment in .gitattributes (#125) 2017-01-27 10:26:03 -08:00
.gitignore feat: add money.cnn custom parser (#26) 2016-11-29 15:13:29 -08:00
appveyor.yml Feat: improving ci (#16) 2016-10-28 09:16:21 -07:00
CHANGELOG.md release: 1.0.5 (#136) 2017-02-01 15:39:19 -08:00
circle.yml feat: ci speedup (#37) 2016-11-30 16:11:35 -08:00
karma.conf.js Feat: browser support (#19) 2016-11-21 14:17:06 -08:00
NOTES.md feat: generator for custom parsers and some documentation 2016-09-20 10:37:03 -04:00
package.json release: 1.0.5 (#136) 2017-02-01 15:39:19 -08:00
preview feat: preview with optional rebuild (#36) 2016-11-30 16:37:42 -08:00
README.md Feat: improving ci (#16) 2016-10-28 09:16:21 -07:00
rollup.config.js feat: add money.cnn custom parser (#26) 2016-11-29 15:13:29 -08:00
rollup.config.web.js feat: add money.cnn custom parser (#26) 2016-11-29 15:13:29 -08:00
score-move chore: refactored and linted 2016-09-13 15:22:27 -04:00
yarn.lock Refactor: running tests more efficiently (#49) 2016-12-05 15:39:45 -05:00

Mercury Parser - Extracting content from chaos

CircleCI Build status

The Mercury Parser extracts the bits that humans care about from any URL you give it. That includes article content, titles, authors, published dates, excerpts, lead images, and more.

The Mercury Parser module powers the Mercury Parser API, a free API from Postlight that puts all of this information one API request away..

How? Like this.

import Mercury from 'mercury-parser';

Mercury.parse(url).then(result => console.log(result););

The result looks like this:

{
  "title": "Thunder (mascot)",
  "content": "<div><div><p>This is the content of the page!</div></div>",
  "author": "Wikipedia Contributors",
  "date_published": "2016-09-16T20:56:00.000Z",
  "lead_image_url": null,
  "dek": null,
  "next_page_url": null,
  "url": "https://en.wikipedia.org/wiki/Thunder_(mascot)",
  "domain": "en.wikipedia.org",
  "excerpt": "Thunder Thunder is the stage name for the horse who is the official live animal mascot for the Denver Broncos",
  "word_count": 4677,
  "direction": "ltr",
  "total_pages": 1,
  "rendered_pages": 1
}

If Mercury is unable to find a field, that field will return null.

Contributing

If you'd like to write a custom parser for a site, here's how.

Contributors

All Contributors


Adam Pash

📝 💻 📖 💡

Toy Vano

💻

Drew Bell

💻

Jeremy Mack

💻