diff --git a/prometheus_grafana/readme.md b/prometheus_grafana/readme.md
index 658a9a8..52563e9 100644
--- a/prometheus_grafana/readme.md
+++ b/prometheus_grafana/readme.md
@@ -4,12 +4,15 @@
![logo](https://i.imgur.com/e03aF8d.png)
+WORK IN PROGRESS
+Loki and caddy monitoring parts are not finished yet
+
# Purpose
Monitoring of the host and the running cointaners.
* [Official Prometheus site](https://prometheus.io/)
-* [Github](https://github.com/prometheus)
+* [Official Grafana site](https://grafana.com/)
Most of the stuff here is based off the magnificent
[stefanprodan/dockprom.](https://github.com/stefanprodan/dockprom)
@@ -17,13 +20,11 @@ So maybe just go play with that.
# Chapters
-Setup here starts off with the basics and then theres chapters how to add features
-
* **[Core prometheus+grafana](#Overview)** - to get nice dashboards with metrics from docker host and containers
* **[Pushgateway](#Pushgateway)** - how to use it to allow pushing metrics in to prometheus from anywhere
* **[Alertmanager](#Alertmanager)** - how to use it for notifications
-* **Loki** - how to do the above things but for logs, not just metrics
-* **Caddy** - adding dashboard for reverse proxy info
+* **[Loki](#Loki)** - how to do the above things but for logs, not just metrics
+* **[Caddy monitoring](#Caddy_monitoring)** - adding dashboard for reverse proxy info
# Overview
@@ -110,8 +111,8 @@ services:
image: prom/prometheus:v2.42.0
container_name: prometheus
hostname: prometheus
- restart: unless-stopped
user: root
+ restart: unless-stopped
depends_on:
- cadvisor
command:
@@ -134,9 +135,9 @@ services:
image: grafana/grafana:9.3.6
container_name: grafana
hostname: grafana
+ user: root
restart: unless-stopped
env_file: .env
- user: root
volumes:
- ./grafana_data:/var/lib/grafana
expose:
@@ -320,8 +321,8 @@ Including pushing information from windows powershell.
image: prom/prometheus:v2.42.0
container_name: prometheus
hostname: prometheus
- restart: unless-stopped
user: root
+ restart: unless-stopped
depends_on:
- cadvisor
command:
@@ -345,9 +346,9 @@ Including pushing information from windows powershell.
image: grafana/grafana:9.3.6
container_name: grafana
hostname: grafana
+ user: root
restart: unless-stopped
env_file: .env
- user: root
volumes:
- ./grafana_data:/var/lib/grafana
expose:
@@ -507,16 +508,27 @@ Including pushing information from windows powershell.
Loki
+ Loki is made by grafana team, who call it Prometheus for logs.
+ It is a **push** type monitoring, where an agent(**promtail**)
+ pushes logs on to a Loki instance.
+
+ What this example will set out to do is monitor logs of a
+ [minecraft server.](https://github.com/DoTheEvo/selfhosted-apps-docker/tree/master/minecraft)
+ Have some dashboards about logs and have an alert when a player joins.
+
+ To add to the existing setup:
+
- New container - `loki` added to the compose file. It stores logs and makes
them available for grafana to visualize.
- New container - `promtail` added to the compose file. Agent that pushes
- logs content to loki.
+ logs content to loki. This container needs access to logs to be pushed
+ and so docker host `/var/lib/docker/containers` directory is bind mounted.
- New file - `loki-docker-config.yml` bind mounted in the loki container.
The file is [all default](https://github.com/grafana/loki/tree/main/cmd/loki),
except for alertmanager url.
- New file - `promtail-config.yml` mounted in to promtail container
- This files defines source of logs. Any such source much be accessible
- by the promtail container.
+ This files defines source of logs. In here a minecraft container is set,
+ with some added labels for easier targeting of logs.
docker-compose.yml
@@ -529,8 +541,8 @@ Including pushing information from windows powershell.
image: prom/prometheus:v2.42.0
container_name: prometheus
hostname: prometheus
- restart: unless-stopped
user: root
+ restart: unless-stopped
depends_on:
- cadvisor
command:
@@ -554,9 +566,9 @@ Including pushing information from windows powershell.
image: grafana/grafana:9.3.6
container_name: grafana
hostname: grafana
+ user: root
restart: unless-stopped
env_file: .env
- user: root
volumes:
- ./grafana_data:/var/lib/grafana
expose:
@@ -626,6 +638,7 @@ Including pushing information from windows powershell.
image: grafana/loki:2.7.3
container_name: loki
hostname: loki
+ user: root
restart: unless-stopped
volumes:
- ./loki_data:/loki
@@ -642,8 +655,8 @@ Including pushing information from windows powershell.
image: grafana/promtail:2.7.3
container_name: promtail
hostname: promtail
- restart: unless-stopped
user: root
+ restart: unless-stopped
volumes:
- /var/log:/var/log:ro
- /var/lib/docker/containers:/var/lib/docker/containers:ro
@@ -724,8 +737,28 @@ Including pushing information from windows powershell.
```
+There was also an option to use loki logging driver to just get docker containers.
+But any container re-recreated with that setting set took always long time to down.
+
+Anyway, now with these files in place..
+
+* Loki needs to be added to in grafana as a datasource.
+ If everything works as it should, there should be no notice, down left side:
+ `Data source connected and labels found.`
+
+---
+---
+
+
+ Caddy monitoring
+
+
+
+---
+---
+
# Update
Manual image update: