python-readability

Go to file

Yuri Baburov 75f2ea0d00 Version bump to 0.6.2		2016-04-09 14:34:00 +06:00
readability	Update readability.py	2016-04-03 21:38:17 -05:00
tests	Failure if best_elem is root (fix #58 )	2015-05-06 09:34:55 +02:00
.gitignore	Adds tox configuration.	2015-04-29 16:16:46 +02:00
Makefile	Fixed makefile testing.	2015-04-27 14:32:40 +06:00
README	Added dev branch features.	2015-07-27 00:22:45 +06:00
requirements.txt	Adds tox configuration.	2015-04-29 16:16:46 +02:00
setup.py	Version bump to 0.6.2	2016-04-09 14:34:00 +06:00
tox.ini	Adds compatibility `raise_with_traceback` method to support different `raise` syntax	2015-04-29 23:35:18 +02:00

README

This code is under the Apache License 2.0.  http://www.apache.org/licenses/LICENSE-2.0

This is a python port of a ruby port of arc90's readability project

http://lab.arc90.com/experiments/readability/

In few words,
Given a html document, it pulls out the main body text and cleans it up.
It also can clean up title based on latest readability.js code.

Based on:
 - Latest readability.js ( https://github.com/MHordecki/readability-redux/blob/master/readability/readability.js )
 - Ruby port by starrhorne and iterationlabs
 - Python port by gfxmonk ( https://github.com/gfxmonk/python-readability , based on BeautifulSoup )
 - Decruft effort to move to lxml ( http://www.minvolai.com/blog/decruft-arc90s-readability-in-python/ )
 - "BR to P" fix from readability.js which improves quality for smaller texts.
 - Github users contributions.

Installation::

    easy_install readability-lxml
    or
    pip install readability-lxml

Usage::

    from readability.readability import Document
    import urllib
    html = urllib.urlopen(url).read()
    readable_article = Document(html).summary()
    readable_title = Document(html).short_title()

Command-line usage::

    python -m readability.readability -u http://pypi.python.org/pypi/readability-lxml

To open resulting page in browser::

    python -m readability.readability -b -u http://pypi.python.org/pypi/readability-lxml

Using positive/negative keywords example::

    python -m readability.readability -p intro -n newsindex,homepage-box,news-section -u http://python.org


Document() kwarg options:

 - attributes:
 - debug: output debug messages
 - min_text_length:
 - retry_length:
 - url: will allow adjusting links to be absolute
 - positive_keywords: the list of positive search patterns in classes and ids, for example: ["news-item", "block"]
 - negative_keywords: the list of negative search patterns in classes and ids, for example: ["mysidebar", "related", "ads"]


Updates

 - 0.3 Added Document.encoding, positive_keywords and negative_keywords
 - 0.4 Added Videos loading and allowed more images per paragraph
 - 0.5 Preparing a release to support Python versions 2.6, 2.7, 3.3 and 3.4
 - 0.6 Finally a release which supports Python versions 2.6, 2.7, 3.3 and 3.4