Make a Site Private but Allow Lets Encrypt

This is a pretty straightforward thing I’ve wanted to do for some time. Basically, I have a number of sites that I use internally that I wanted to get certificates via Let’s Encrypt, but I also wanted to keep them restricted to only a few IP addresses. The solution is quite simple and works perfectly. We accomplish this with two .htaccess files. One at the site root to restrict IP address that can access the site, the second to disable that restriction on the directory where the Let’s Encrypt challenge is stored....

May 22, 2017

Monitoring a Mount Point With Zabbix

A subtle issue I ran into was the issue that Proxmox VE would sometimes unmount a GlusterFS volume and would fail to backup. This issue was a bit sneaky though, since the PVE backup program wouldn’t execute it wouldn’t send an email notifying me of the failure. This would make it so the backups would fail silently for some time, until I happened to login and see the errors in the cluster’s log....

March 29, 2017

A Memory Leak Visualized

Graph of free memory on a node with a leaking piece of software.

March 9, 2017

MySQL (MariaDB) Galera Cluster Restart

This is a scary problem when you’re recovering from an outage of your database machines. If you’re running a Galera cluster and they all go offline, you’ll need to do a bit of work to restart the cluster and make it safe. Galera relies on the fact that there’s at least one node running in your cluster at all times. If your entire cluster goes offline, you won’t be able to start it again, even with the –wsrep-new-cluster option....

February 5, 2017

Zabbix MySQL (MariaDB) Monitoring

This is another one of those things that is pretty straightforward, but requires culminating information from a different sources in order to get things up and running. The goal here is to get Zabbix to monitor our MariaDB (MariaDB is a drop in replacement for MySQL, I’ll refer to either as MariaDB here) server’s status. There’s a built in template, but a few other files and settings need setup before you can get the juicy data flowing....

January 31, 2017

Proxmox 3 to 4 Upgrade Network Issue

This is a problem that showed itself when upgrading our Proxmox 3.2 Nodes up to Proxmox 4. About halfway through the upgrade, our network adapters suddenly stopped being able to communicate with any local addresses, but could still ping outside addresses. The cause was a minor config change that gets added in pretty stealthy. When this happens, simply add the following line to the bridge config in /etc/network/interfaces: bridge_vlan_aware yes To make the entire config section resemble:...

January 23, 2017

Installing Gluster on OpenVZ Container

Setting up OpenVZ containers to be able to use a FUSE filesystem is pretty simple, but it takes a bit to figure out exactly which steps you need to follow. There are a myriad of tutorials online (and here’s yet another), but this one focuses specifically on Gluster, a distributed network file system. With Storehouse, we use Gluster to act as our storage backend for most of our customer data. By default, our OpenVZ containers could not mount the volume directly, since FUSE is not enabled....

December 3, 2016

Resizing Lvm Partitions on Centos

One of the things I’ve done for my employer is resize partitions on a few CentOS machines that already had customer data on them. The default CentOS setup didn’t work exactly for our needs, so resizing the /root and /home partitions were our chosen course of action. Overall the process is pretty simple, but I’m writing it down here since it takes the concatenation of a two different processes to get things done....

October 26, 2016