dev-ops

This is something that bothered me when adding more than one server to PHPMyAdmin. The default configuration is very verbose, and largely isn’t needed for simple use cases. In my case, I boiled down the config options I use most frequently, and changed the format to be more inline. The format doesn’t look too good on the site, but it is pretty clean once in a wider editor. $cfg['Servers'] = [ 1 => [ 'host' => 'xxx', // MySQL hostname or IP address 'port' => '', // MySQL port - leave blank for default port 'socket' => '', // Path to the socket - leave blank for default socket 'ssl' => true, // Use SSL for connecting to MySQL server?...

After a long a deep think, I’ve decided to retire my physical hardware and migrate all of my machines to Amazon Web Services. It wasn’t an easy choice, and I feel like I need to spend some time explaining why, just get some things off my chest. Reason #1 - Our Hardware is End Of Life Most of the reasons we’re moving are related to our hardware approaching the end of its useful life....

Through some Google-fu and some other great tutorials, I’ve successfully setup a groups of Redis machines with automatic failover detection via keepalived and sentinel. This sounds mundane, but lets you setup another layer of protection for your Redis cluster without lots of extra configuration. Background – Redis Sentinel and Keepalived Redis Sentinel makes it pretty easy to setup a group of replicated Redis machines and elect new master nodes when others are offline....

Not too long ago in the Linux world, firewall rules were complex. iptables did its job very well, but managing rules was daunting for a newcomer. Debian 9 introduces some changes that make it pretty simple to add a firewall rule. Usually firewall rules are taken care of automatically, when you install a program it takes care of opening up the required ports for itself. In some cases, software can conflict and that is what happened in my case....

Today, The Storehouse experienced an outage that lasted approximately 12 hours. This was caused due to updates performed late the night before and services restarting during that process. Last night I ran upgrades of the servers that run The Storehouse, including our three ProxmoxVE nodes. When the upgrades on these nodes were complete, the nodes had an updated kernel version and needed to restart to use the new kernel Restarting is usually a painless process in our environment....

One of the new tools I’ve discovered is Chef to manage the configuration and software on Storehouse’s fleet of virtual machines. Chef makes it really handy to update and track config changes, since everything can be tracked using Git or similar. One issue we ran into was having chef-client run at the same time for multiple machines. This issue is kinda subtle, but makes a lot of sense when you think about it....

This is a pretty straightforward thing I’ve wanted to do for some time. Basically, I have a number of sites that I use internally that I wanted to get certificates via Let’s Encrypt, but I also wanted to keep them restricted to only a few IP addresses. The solution is quite simple and works perfectly. We accomplish this with two .htaccess files. One at the site root to restrict IP address that can access the site, the second to disable that restriction on the directory where the Let’s Encrypt challenge is stored....

A subtle issue I ran into was the issue that Proxmox VE would sometimes unmount a GlusterFS volume and would fail to backup. This issue was a bit sneaky though, since the PVE backup program wouldn’t execute it wouldn’t send an email notifying me of the failure. This would make it so the backups would fail silently for some time, until I happened to login and see the errors in the cluster’s log....