====== Setting up a Munin server and nodes ======
===== Munin master setup =====
In this example, we are going to set up a munin server in its own [[LXC]] container instance. This container will run unprivileged under an ordinary user. We are using Devuan because regular Debian with systemd will not run in an unprivileged container AFAIK.

==== Creating a container ====
  - ''lxc-create -t download -n munin'' \\ ''(devuan, beowulf, arm64)''
  - ''lxc-attach munin passwd''

==== Basic container setup ====
  - ''lxc-attach munin''
  - ''apt update''
  - ''apt autoremove''
  - ''apt clean''
  - ''dpkg-reconfigure tzdata''

==== Installing munin ====
  - ''apt install munin''
  - Change these lines in ''/etc/munin/munin.conf'': <code ini>graph_strategy cgi
html_strategy cgi
htmldir /var/www/munin</code>
  - Add some nodes if you have any already: <code ini>[Node1]
address 1.2.3.4
use_node_name yes</code>
  - ''mkdir /var/www/munin''
  - ''chown munin:munin /var/www/munin''
  - ''service munin restart''

==== Setting up a web server ====
  - ''apt install apache2 libapache2-mod-fcgid libcgi-fast-perl''
  - ''vim /etc/apache2/sites-available/munin.conf'' <code apache><VirtualHost *:80>
    ServerName munin.example.org
    ServerAlias munin

    ServerAdmin  info@example.org

    DocumentRoot /var/www

    Alias /munin/static/ /etc/munin/static/

    <Location />
        AuthType Basic
        AuthName "Members Only"
        AuthUserFile /etc/apache2/.htpasswd
        require valid-user
    </Location>

    <Directory /etc/munin/static>
        Require all granted
    </Directory>

    ScriptAlias /munin-cgi/munin-cgi-graph /usr/lib/munin/cgi/munin-cgi-graph
    ScriptAlias /munin /usr/lib/munin/cgi/munin-cgi-html
    <Directory /usr/lib/munin/cgi>
        Require all granted
        <IfModule mod_fcgid.c>
            SetHandler fcgid-script
        </IfModule>
        <IfModule !mod_fcgid.c>
            SetHandler cgi-script
        </IfModule>
    </Directory>

    CustomLog /var/log/apache2/munin.example.org-access.log combined
    ErrorLog  /var/log/apache2/munin.example.org-error.log
</VirtualHost></code>
  - ''htpasswd -c /etc/apache2/.htpasswd''
  - ''a2ensite munin''
  - ''a2dissite 000-default''
  - ''a2enmod fcgid rewrite''
  - ''service apache2 restart''

===== Configuring nodes =====
==== Adding a node ====
To add nodes, all you need to do is this:
  - Install munin-node \\ ''apt install munin-node''
  - edit /etc/munin/munin and add a line allowing the master to connect, e.g.: \\ <code>allow ^10\.0\.3\.116$</code>
  - Optionally, make it listen only on localhost: \\ <code>host 127.0.0.1</code>
  - Restart the service. \\ ''systemctl restart munin-node''

On the master:
  - Add a new snippet to ''/etc/munin/munin.conf'', e.g.: <code>[Node1]
address 1.2.3.4
use_node_name yes</code>
  - Restart the munin master service. ''service munin restart''

===== Enabling/disabling plugins =====
On the nodes, enabled plugins are found in ''/etc/munin/plugins''.

Available plugins are found in ''/usr/share/munin/plugins''. Create symbolic links to the active configuration. Some of the plugins behave differently depending on how the links are named; the plugin files themselves should contain readable documentation for more info.

==== Plugin configuration ====
Plugins can be configured further via setting files in ''/etc/munin/plugin-conf.d''. Here are some examples:

=== diskstats ===
<code ini>[diskstats]
  env.include_only sda,sdb,...</code>

=== df ===
<code ini>[df]
  env.include_re ^/$ ^/var/log$ ^/web$
  env.exclude_re ^/dev/shm$ ^/run/lock$ ^/sys/fs/cgroup$
  
[df_inode]
  env.include_re ^/$ ^/var/log$ ^/web$
  env.exclude_re ^/dev/shm$ ^/run/lock$ ^/sys/fs/cgroup$</code>

===== Connect a host via ssh =====
A more secure way to connect a node to the master is via ssh. When connecting to a machine on the internet, ssh is a must have.

  - Create an SSH keypair on the host for user ''munin''
    - since the ''munin'' user can't login by default, create it as ''root'' and change ownership.
  - Move the ''.ssh'' folder to ''/var/lib/munin/ssh''
  - Make sure it's owned by ''munin'' and not accessible to anyone else. \\ <code># ls -la /var/lib/munin/ssh
total 20
drwx------  2 munin munin 4096 Mar  6 20:18 .
drwxr-xr-x 13 munin munin 4096 Mar  8 14:15 ..
-rw-------  1 munin munin  399 Mar  6 20:18 id_ed25519
-rw-r--r--  1 munin munin   92 Mar  6 20:18 id_ed25519.pub
-rw-r--r--  1 munin munin  666 Mar  6 20:21 known_hosts</code>
  - Add the key to the node's ''.ssh/authorized_keys''
  - configure ssh options in ''/etc/munin/munin.conf'': \\ <code>ssh_options -o IdentityFile=/var/lib/munin/ssh/id_ed25519 -o UserKnownHostsFile=/var/lib/munin/ssh/known_hosts -o User=remoteuser -o PreferredAuthentications=publickey</code>
  - A node section in ''/etc/munin/munin.conf'' might look like this: \\ <code ini>[node2.example.com]
    address ssh://node2.example.com/bin/nc 127.0.0.1 4949
    use_node_name yes</code>

===== (Alternative) init.d script to automatically create an SSH tunnel to a node =====
Though **I would recommend the way outlined in the previous section**, here's an example on how to automatically dig an SSH tunnel to a node system via an init.d script (since we have no systemd on Devuan).

  - Create a key pair on the munin master: \\ ''ssh-keygen -t ed25519''
  - Add the public key to the remote node's ''authorized_keys'', e.g.: \\ ''cat id_ed25519.pub >> .ssh/authorized_keys''
  - Create the script on the munin master: \\ ''vi ssh_example'' \\ <code bash>#!/bin/sh
### BEGIN INIT INFO
# Provides:          ssh_example
# Required-Start:    $all
# Required-Stop:
# Default-Start:     2 3 4 5
# Default-Stop:
# Short-Description: your description here
### END INIT INFO

PID_FILE="/var/run/ssh_example.pid"

case "$1" in
start)
    if [ -f $PID_FILE ]; then
        echo "SSH tunnel is already up apparently"
    else
        start-stop-daemon --start --exec /usr/bin/ssh --background --pidfile=$PID_FILE --make-pidfile -- -fNL 14949:localhost:4949 user@example.com
    fi
;;

stop)
    start-stop-daemon --stop --pidfile=$PID_FILE
    rm $PID_FILE
;;

restart)
    $0 stop
    $0 start
;;

*)
    echo "Usage: $0 {start|stop|restart}"
    exit 1
esac</code>
  - ''chmod +x ssh_example''
  - ''mv ssh_example /etc/init.d/''
  - ''insserv /etc/init.d/ssh_example''
  - ''update-rc.d ssh_example defaults''
  - ''update-rc.d ssh_example enable''
  - ''service ssh_example start''

==== Securing the node ====
  - Make sure the node only listens on the loopback interface by removing this line from ''/etc/munin/munin-node.conf'': \\ <code>host *</code>
  - and adding either of these instead: \\ <code>host 127.0.0.1
host ::1</code>
  - it should only listen to localhost: \\ <code>allow ^127\.0\.0\.1$
allow ^::1$</code>

==== Adding it to the master ====
This one might be obvious. Add a new section to ''/etc/munin/munin.conf'': \\ <code ini>[example.com]
    address 127.0.0.1
    port 14949
    use_node_name yes</code>

===== See also =====
  * [[LXC]]