Mercury Parser - Extracting content from chaos #parser #url #html #extractor
You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
Go to file
Janet f9ab9eb885 feat: howtogeek extractor (#108)
* feat: howtogeek extractor

This one is a bit tricky - the author and date info appear in a comment
section at the bottom. Was able to parse the author but not the date
info. Halp

* howtogeek update

Thanks to @fdsimms I was able to parse the date, but not sure what to
test it against, so I left it blank.

* fix: date_published assertion, it was comparing against empty string

* fix: timezone

* amend: generalize author selector
8 years ago
dist release: 1.0.5 (#136) 8 years ago
fixtures feat: howtogeek extractor (#108) 8 years ago
scripts feat: changed user agent to latest chrome (#121) 8 years ago
src feat: howtogeek extractor (#108) 8 years ago
.agignore
.all-contributorsrc fix: increased avatar size 8 years ago
.babelrc feat: making yarn-friendly for package manager (#17) 8 years ago
.eslintignore Feat: browser support (#19) 8 years ago
.eslintrc feat: recording/playing back network requests with nock (#18) 8 years ago
.gitattributes fix: i put a bad comment in .gitattributes (#125) 8 years ago
.gitignore feat: add money.cnn custom parser (#26) 8 years ago
CHANGELOG.md release: 1.0.5 (#136) 8 years ago
NOTES.md
README.md Feat: improving ci (#16) 8 years ago
appveyor.yml Feat: improving ci (#16) 8 years ago
circle.yml feat: ci speedup (#37) 8 years ago
karma.conf.js Feat: browser support (#19) 8 years ago
package.json release: 1.0.5 (#136) 8 years ago
preview feat: preview with optional rebuild (#36) 8 years ago
rollup.config.js feat: add money.cnn custom parser (#26) 8 years ago
rollup.config.web.js feat: add money.cnn custom parser (#26) 8 years ago
score-move
yarn.lock Refactor: running tests more efficiently (#49) 8 years ago

README.md

Mercury Parser - Extracting content from chaos

CircleCI Build status

The Mercury Parser extracts the bits that humans care about from any URL you give it. That includes article content, titles, authors, published dates, excerpts, lead images, and more.

The Mercury Parser module powers the Mercury Parser API, a free API from Postlight that puts all of this information one API request away..

How? Like this.

import Mercury from 'mercury-parser';

Mercury.parse(url).then(result => console.log(result););

The result looks like this:

{
  "title": "Thunder (mascot)",
  "content": "<div><div><p>This is the content of the page!</div></div>",
  "author": "Wikipedia Contributors",
  "date_published": "2016-09-16T20:56:00.000Z",
  "lead_image_url": null,
  "dek": null,
  "next_page_url": null,
  "url": "https://en.wikipedia.org/wiki/Thunder_(mascot)",
  "domain": "en.wikipedia.org",
  "excerpt": "Thunder Thunder is the stage name for the horse who is the official live animal mascot for the Denver Broncos",
  "word_count": 4677,
  "direction": "ltr",
  "total_pages": 1,
  "rendered_pages": 1
}

If Mercury is unable to find a field, that field will return null.

Contributing

If you'd like to write a custom parser for a site, here's how.

Contributors

All Contributors


Adam Pash

📝 💻 📖 💡

Toy Vano

💻

Drew Bell

💻

Jeremy Mack

💻