13/Feb/2024 - Service Now
Yet another mostly-quiet day. Work I’m down to 44 days left in the contract, 28ish working days. I had some successes integrating our Alertmanager into a corporate tool called Service Now via its API, so we can send alerts in theory. I say in theory as no one’s defined any useful alerts in our eleventy-seven thousand metric points. Given we have no SRE, that’ll probably fall to me, and it’s really not my area of expertise. But, we do what we must. Made no progress on getting the Application Gateway / WAF into Staging. Will slip that in tomorrow and work to automate it a bit better with Terraform and Helm. Learned about the existence of something called Blackbox for Prometheus which will allow for synthetic checks to remote addresses. Certainly cheaper than monitoring from an external company. ...
