Mercury Parser - Extracting content from chaos #parser #url #html #extractor
You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
Go to file
Adam Pash 39a3c0690d chore: readme improvement 8 years ago
dist feat: content cleaner still runs, but can disable some cleaners 8 years ago
fixtures feat: generator for custom parsers and some documentation 8 years ago
scripts chore: cleaned up unused files, slight reorg 8 years ago
src chore: readme improvement 8 years ago
.agignore chore: renamed iris to mercury 8 years ago
.babelrc fix: .babelrc was still referencing iris 8 years ago
.eslintignore chore: refactored and linted 8 years ago
.eslintrc chore: disable camelcase for linting 8 years ago
.gitignore feat: generator for custom parsers and some documentation 8 years ago
NOTES.md feat: generator for custom parsers and some documentation 8 years ago
README.md feat: generator for custom parsers and some documentation 8 years ago
package.json feat: switched test framework to jest 8 years ago
preview refactor: slightly better preview 8 years ago
rollup.config.js fix: using transform-runtime to avoid babel-polyfill conflicts when used 8 years ago
score-move chore: refactored and linted 8 years ago
test-runner feat: generator for custom parsers and some documentation 8 years ago

README.md

Mercury Parser - Extracting content from chaos

The Mercury Parser extracts the bits that humans care about from any URL you give it. That includes article content, titles, authors, published dates, excerpts, lead images, and more.

The Mercury Parser module powers the Mercury Parser API, a free API from Postlight that puts all of this information one API request away..

How? Like this.

import Mercury from 'mercury-parser';

Mercury.parse(url).then(result => console.log(result););

The result looks like this:

{
  "title": "Thunder (mascot)",
  "content": "<div><div><p>This is the content of the page!</div></div>",
  "author": "Wikipedia Contributors",
  "date_published": "2016-09-16T20:56:00.000Z",
  "lead_image_url": null,
  "dek": null,
  "next_page_url": null,
  "url": "https://en.wikipedia.org/wiki/Thunder_(mascot)",
  "domain": "en.wikipedia.org",
  "excerpt": "Thunder Thunder is the stage name for the horse who is the official live animal mascot for the Denver Broncos",
  "word_count": 4677,
  "direction": "ltr",
  "total_pages": 1,
  "rendered_pages": 1
}

If Mercury is unable to find a field, that field will return null.

Contributing

If you'd like to write a custom parser for a site, here's how.