Introduction
I have been running an uninterruptible power supply (UPS) for a few reasons.
- I want my router to always have power and be able to shutdown gracefully my servers during power outages
- I run a pihole on a raspberry pi that I want to always be running mostly to block tracking, phishing, malware, and ransom-ware sites
- I don’t want my servers to shutdown while I am transferring data
- I need surge protection from a voltage spike to avoid damaging any equipment
That said, I also want to be able to monitor the UPS’s battery, energy consumption and be able to compare with past data.
The Setup
My current setup is quite simple:
- Main Server : raspberry pi 3b+ running pihole
- Modem/Router providing Wifi and Lan
- Secondary Server : Running multiple VMs on Proxmox VE (AMD Ryzen 3600X, 16GB of RAM for now, few terabytes of storage)
- Maybe a third server in the future?
- UPS : APC Back-UPS XS 950U (Costs around 100€ on Amazon FR, Amazon DE)
The UPS has 4 outputs so I still have the ability to add an other server and most importantly is that it have a USB data port for management.
So the goal is to connect my main server to the UPS via the USB data port and share the data readings with the rest of the servers. For APC UPSs apcupsd package can be and used to extract the data and also configure the servers to shutdown when the battery reaches a certain level of charge.
The UPS has one USB data port so only one device can be connected directly to the UPS. the good news is that the apcupsd comes with a NIS server so instances of upcupsd on other machines can get connected using a polling mechanism via ethernet.
Installing apcupsd
On the primary server We will have to verify that the USB connection is functional.
$ lsusb Bus 001 Device 004: ID 051d:0002 American Power Conversion Uninterruptible Power Supply Bus 001 Device 005: ID 0424:7800 Standard Microsystems Corp. Bus 001 Device 003: ID 0424:2514 Standard Microsystems Corp. USB 2.0 Hub Bus 001 Device 002: ID 0424:2514 Standard Microsystems Corp. USB 2.0 Hub Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Install the apcupsd package
$ apt install apcupsd
Edit the apcupsd.conf file to configure the daemon with following parameters:
$ nano /etc/apcupsd/apcupsd.conf
| Parameter | Value | Comment |
|---|---|---|
| UPSTYPE | usb | Must be set to usb |
| UPSCABLE | usb | Must be set to usb |
| DEVICE | Should be blank | |
| NISIP | 192.168.1.26 | Must be a static IP address of the primary server |
| NISPORT | 3551 | Port that will be exposed by NIS on the primary server |
| UPSNAME | BX950U | Optional |
Edit apcupsd file and set ISCONFIGURED=yes
$ nano /etc/default/apcupsd
Start the service
$ service apcupsd restart
Verify that we can get the status of the UPS via USB using apcaccess command
$ apcaccess APC : 001,036,0878 DATE : 2020-10-31 23:32:40 +0100 HOSTNAME : mainserver VERSION : 3.14.14 (31 May 2016) debian UPSNAME : BX950U CABLE : USB Cable DRIVER : USB UPS Driver UPSMODE : Stand Alone STARTTIME: 2020-10-31 23:32:38 +0100 MODEL : Back-UPS XS 950U STATUS : ONLINE LINEV : 238.0 Volts LOADPCT : 9.0 Percent BCHARGE : 100.0 Percent TIMELEFT : 65.9 Minutes MBATTCHG : 5 Percent MINTIMEL : 3 Minutes MAXTIME : 0 Seconds SENSE : Medium LOTRANS : 155.0 Volts HITRANS : 280.0 Volts ALARMDEL : 30 Seconds BATTV : 13.4 Volts LASTXFER : Unacceptable line voltage changes NUMXFERS : 0 TONBATT : 0 Seconds CUMONBATT: 0 Seconds XOFFBATT : N/A SELFTEST : NO STATFLAG : 0x05000008 SERIALNO : 4B1952P18784 BATTDATE : 2019-12-28 NOMINV : 230 Volts NOMBATTV : 12.0 Volts NOMPOWER : 480 Watts FIRMWARE : 925.T2 .I USB FW:T2 END APC : 2020-10-31 23:32:41 +0100
Install the apcupsd package
$ apt install apcupsd
Edit the apcupsd.conf file to configure the daemon withe following parameters:
$ nano /etc/apcupsd/apcupsd.conf
| Parameter | Value | Comment |
|---|---|---|
| UPSCABLE | ether | Must be set to ether so we can connect to the primary sever |
| UPSTYPE | net | Must be set to net so we can connect to the primary sever |
| DEVICE | 192.168.1.26:3551 | <primary NIS IP>:<primary NIS PORT> |
| POLLTIME | 30 | Interval in seconds between polls |
$ nano /etc/default/apcupsd
Edit apcupsd file and set ISCONFIGURED=yes
$ service apcupsd restart
$ apcaccess APC : 001,037,0929 DATE : 2020-10-31 23:57:27 +0100 HOSTNAME : pve1 VERSION : 3.14.14 (31 May 2016) debian UPSNAME : BX950U CABLE : Ethernet Link DRIVER : NETWORK UPS Driver UPSMODE : Stand Alone STARTTIME: 2020-10-31 23:57:27 +0100 MASTERUPD: 2020-10-31 23:57:27 +0100 MASTER : 192.168.1.26:3551 MODEL : Back-UPS XS 950U STATUS : ONLINE SLAVE LINEV : 236.0 Volts LOADPCT : 9.0 Percent BCHARGE : 100.0 Percent TIMELEFT : 65.9 Minutes MBATTCHG : 5 Percent MINTIMEL : 3 Minutes MAXTIME : 0 Seconds SENSE : Medium LOTRANS : 155.0 Volts HITRANS : 280.0 Volts BATTV : 13.5 Volts LASTXFER : Unacceptable line voltage changes NUMXFERS : 0 TONBATT : 0 Seconds CUMONBATT: 0 Seconds XOFFBATT : N/A SELFTEST : NO STATFLAG : 0x05000408 SERIALNO : 4B1952P18784 BATTDATE : 2019-12-28 NOMINV : 230 Volts NOMBATTV : 12.0 Volts NOMPOWER : 480 Watts FIRMWARE : 925.T2 .I USB FW:T2 END APC : 2020-10-31 23:57:37 +0100
Collecting Metrics
I have created a simple bash script to parse the metrics and send them to the influxdb server. The metrics I am interested in are the following
| Metric | Comment | Unit |
|---|---|---|
| LOADPCT | Percentage of UPS load capacity used | Percent |
| BATTV | Current battery voltage | Volts |
| TIMELEFT | Remaining runtime left on battery as estimated by UPS | Minutes |
| BCHARGE | Current battery capacity charge percentage | Percent |
| NOMPOWER | Nominal power output in watts | Watts |
| LINEV | Current input line voltage | Volts |
| STATUS | UPS status (online, charging, on battery etc) | N/A |
| CUMONBATT | Cumulative seconds on battery since apcupsd startup | Seconds |
| LOADWATT | Computed by multiplying load by the nominal power | Watts |
$ nano /bin/export_apc_metrics.sh
#!/bin/bash
INFLUX_URL="192.168.168.26:8086"
DB_NAME="UPS"
metrics=`apcaccess -u`
LOADPCT=$(echo "$metrics" | grep "LOADPCT" | awk -F ": " '{print $2}' | xargs)
BATTV=$(echo "$metrics" | grep "^BATTV" | awk -F ": " '{print $2}' | xargs)
TIMELEFT=$(echo "$metrics" | grep "TIMELEFT" | awk -F ": " '{print $2}' | xargs)
BCHARGE=$(echo "$metrics" | grep "BCHARGE" | awk -F ": " '{print $2}' | xargs)
NOMPOWER=$(echo "$metrics" | grep "NOMPOWER" | awk -F ": " '{print $2}' | xargs)
LINEV=$(echo "$metrics" | grep "LINEV" | awk -F ": " '{print $2}' | xargs)
STATUS=$(echo "$metrics" | grep "STATUS" | awk -F ": " '{print $2}' | xargs)
CUMONBATT=$(echo "$metrics" | grep "CUMONBATT" | awk -F ": " '{print $2}' | xargs)
LOADWATT=$(awk '{print $1*$2/100}' <<<"${LOADPCT} ${NOMPOWER}")
FLOAT_METRIC_NAMES="LOADPCT LINEV BATTV TIMELEFT BCHARGE NOMPOWER LOADWATT CUMONBATT"
STRING_METRIC_NAMES="STATUS"
for metricName in $FLOAT_METRIC_NAMES
do
echo "$metricName ${!metricName}"
curl -i "http://$INFLUX_URL/write?db=$DB_NAME" --data-binary "APC,host=rp $metricName=${!metricName}"
done
for metricName in $STRING_METRIC_NAMES
do
echo "$metricName ${!metricName}"
curl -i "http://$INFLUX_URL/write?db=$DB_NAME" --data-binary "APC,host=rp $metricName=\"${!metricName}\""
done
I have also made the file executable
$ chmod +x /bin/export_apc_metrics.sh
Now the only thing I need to do is setup a Cron job to launch my script on a regular basis. I this case it will run every minute.
... ... * * * * * root /bin/export_apc_metrics.sh >/dev/null 2>&1
The Dashboard
The dashboard can be imported using the URL https://grafana.com/grafana/dashboards/13319 or the ID 13319. The data source has been parametrized for convenience.
This is obviously my first attempt at this dashboard and I still need to think more about it and what I actually will need to add, keep or remove. I will try to keep this post up to date with the changes.