diff --git a/README.md b/README.md index 0b53f93..ab53e7c 100644 --- a/README.md +++ b/README.md @@ -22,7 +22,7 @@ You can start the crawler in detached mode by passing --detach to it. Ensure you have at least 3GB of memory as the Elasticsearch stack docker will require 2GB. -# How to start the crawling process +# How to initiate crawling Since the API is exposed on localhost:15005, one can use it to start the crawling process: @@ -42,6 +42,17 @@ $ docker run creekorful/trandoshanctl --api-uri schedule https://www.faceb this will schedule given URL for crawling. +## How to speed up crawling + +If one want to speed up the crawling process, he can scale the instance of crawling process in order +to increase performances. This may be done by issuing the following command after the crawler is started: + +```sh +$ ./scripts/scale.sh crawler=5 +``` + +this will set the number of crawler instance to 5. + # How to view results ## Using trandoshanctl diff --git a/scripts/log.sh b/scripts/log.sh deleted file mode 100755 index b8f1761..0000000 --- a/scripts/log.sh +++ /dev/null @@ -1,3 +0,0 @@ -#!/bin/bash - -./scripts/exec.sh logs "$@" diff --git a/scripts/scale.sh b/scripts/scale.sh new file mode 100755 index 0000000..0f0f03d --- /dev/null +++ b/scripts/scale.sh @@ -0,0 +1,3 @@ +#!/bin/bash + +./scripts/exec.sh scale "$@"