Run triggers on systemd's journal messages
Running Arch Linux, I've now been using systemd's journal as logging engine for a little while. This isn't about how good/bad it is, or how it compares to other solutions (e.g. syslog).
I'm using the journal, and for a little while now I've been wondering about having a few things automatically triggered when certain messages are logged. I looked for a tool to do that, but couldn't find one.
There are a few tools that will do that sort of things, but they're made for very specific things, for instance listen for failed SSH login attempts, and will (temporary) ban an IP after a while or something.
This is pretty much exactly the sort of things I was looking to do, but I wanted to be able to define which messages to listen for, and what actions shall be triggered.
Which is why I wrote a little tool to do just that: listen to any new messages added to the journal, and whenever there's a match for a given "rule," trigger the associated action.
Identifying messages is done via so-called rules, which can be made of different group of conditions, used via boolean logic. Each group of conditions can be made of as many conditions as needed, all of which must match for the group to be a match.
A condition is simply a test on a given field of the message. Supported tests are exact match, pattern (glob-like) match, lesser/greater than tests which can be used when the value of the message is an integer. It is also possible to "negate" the test, so the condition match when the field is not a match.
Installed as a service, it runs in the background and whenever a matching messages is added to the journal, the corresponding action is triggered. Exactly what I was looking for.
For example, an error can sometimes occur leading to the network being down, as the interface goes down with a speed/duplex mismatch error. This doesn't happen very often and is easy enough to fix once the problem has been identified, but now I don't have to, thanks to a simple rule like the following:
[Rule] trigger=systemctl restart network.service [Filter] _KERNEL_DEVICE=+pci:0000:02:00.0 MESSAGE=sky2 0000:02:00.0 net0: speed/duplex mismatch
If you're interested, you can find out more about it from journal-triggerd. It is released under GNU GPL v3+ The source code is available on this Github repository.
And of course any bug report, suggestion or other constructive criticism is very much welcome.