blob42
/
hugobot
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
Content fetch and aggregation bot for hugo data-driven websites
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
10 Commits
1 Branch
2 Tags
966 KiB
 
 
 
 
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'master'
${ noResults }
Go to file
blob42 c4ed0c48ba
migrate module location 		4 weeks ago
bitcoin migrate module location 4 weeks ago
config initial 4 years ago
db initial 4 years ago
encoder initial 4 years ago
export migrate module location 4 weeks ago
feeds migrate module location 4 weeks ago
filters migrate module location 4 weeks ago
github migrate module location 4 weeks ago
handlers migrate module location 4 weeks ago
logging initial 4 years ago
posts migrate module location 4 weeks ago
static migrate module location 4 weeks ago
types initial 4 years ago
utils migrate module location 4 weeks ago
.gitignore initial 4 years ago
Dockerfile initial 4 years ago
Dockerfile-sqliteweb initial 4 years ago
LICENSE add license 4 years ago
Makefile initial 4 years ago
README.md update 1 month ago
api.go migrate module location 4 weeks ago
commands.go migrate module location 4 weeks ago
config.toml initial 4 years ago
docker-compose.yml initial 4 years ago
docker-entrypoint.sh initial 4 years ago
feed_commands.go migrate module location 4 weeks ago
go.mod migrate module location 4 weeks ago
go.sum migrate module location 4 weeks ago
hugobot.pdf update 1 month ago
jobs.go migrate module location 4 weeks ago
main.go migrate module location 4 weeks ago
parse_test.go migrate module location 4 weeks ago
posts_test.go migrate module location 4 weeks ago
scheduler.go migrate module location 4 weeks ago
server.go migrate module location 4 weeks ago

README.md

MIRRORED FROM: https://git.blob42.xyz/blob42/hugobot

HUGOBOT

hugobot is a bot that automates the fetching and aggregation of content for Hugo data-driven websites. It has the following features:

Data fetch

  • Use the feeds table to register feeds that will be fetched periodically.
  • Currently, it can handle these types of feeds: RSS, Github Releases, Newsletters
  • To define your own feed types, implement the JobHandler interface (see handlers/handlers.go).
  • Hugobot automatically fetches new posts from the registered feeds.
  • The database uses Sqlite for storage. It has feeds and posts tables.
  • The scheduler can handle an unlimited number of tasks and uses leveldb for caching and resuming jobs.

Hugo export

  • Data is automatically exported to the configured Hugo website path.
  • It can export data as markdown files or json/toml data files.
  • You can customize all fields in the exported files.
  • You can define custom output formats by using the FormatHandler interface.
  • You can register custom filters and post-processing for exported posts to prevent altering the raw data stored in the database.
  • You can force data export using the CLI.

API

  • It uses gin-gonic as the web framework.
  • hugobot also includes a webserver API that can be used with Hugo Data Driven Mode.
  • You can insert and query data from the database. This feature is still a work in progress, but you can easily add the missing code on the API side to automate inserting and querying data from the database.
  • For example, it can be used to automate the generation of Bitcoin addresses for new articles on bitcointechweekly.com.

Other

  • Some commands are available through the CLI (github.com/urfave/cli), you can add your own custom commands.

Sqliteweb interface

  • See the Docker files for more information.

First time usage

  • The first time you run the program, it will automatically generate the database. You can add your feeds to the Sqlite database using your preferred Sqlite GUI.

Contribution

  • We welcome pull requests. Our current priority is adding tests.
  • Check the TODO section.

TODO:

  • Add tests.
  • Handle more feed formats: tweets, mailing-list emails ...
  • TLS support in the API (not a priority, can be done with a reverse proxy).