Pre Commit - This is *the* pipeline tool you're looking for

A while back in a company I worked for, we used to write code locally, commit it and then push it to the repo. The code was checked for the usual errors in the markdown, structure and so on using a BitBucket pipeline and if there were any issues, the build would fail and you would adjust the code locally, re-commit and re-push. So far so good. The problem was that the checks were only done when you pushed the code....

June 15, 2024 · 3 min · Morgan Conlon

MySQL Table Primary Key Clashes

Here’s an interesting scenario I happened across the other day. What if you had a distributed system that used twenty MySQL databases, each one located in one of 20 US cities. Same database on each server, but each server has a different set of data. Now, what if you wanted to merge all of the data from all of the servers into one database? You’d have a problem with primary key clashes....

June 7, 2024 · 3 min · Morgan Conlon

Postfix Autoreply

So, forever and a day I’ve been using a service at autorespond+dkim@dk.elandsys.com to provide a simple autoreply for testing connectivity to a mail server. Sadly, the service has been discontinued after many years and so I decided to set up my own with Postfix, the most venerable of mail servers. Here’s how I did that. Assumptions You’re using a fairly recent edition of Postfix. I’m using 3.7.10. You can find out your version using postconf -d mail_version....

May 3, 2024 · 6 min · Morgan Conlon

Svelte Dockerfile

Sooner or later you’re going to want to deploy your Svelte app to a server. That’s cool, but how do you do that? The simple answer is to use Docker(*) (realistically!) to create an image and then deploy it using AWS EKS or ECS or other container tech. But how? Here’s how. You can use Vercel which has much more integration with Svelte, but when I tried to use it, it didn’t work with a vanilla site....

March 14, 2024 · 5 min · Morgan Conlon

Github self hosted runners

Overview So, I’ve been using Github Actions for the longest time and the idea of using a self hosted runner has always piqued my interest but the cloud service from Github is so good, I’ve never had the need. Until now. I had a quick look through the documentation and, while overly long, it’s actually quite simple to set up. I’m going to walk you through the process. You’ll need a Github account obviously....

December 1, 2023 · 3 min · Morgan Conlon

For sale on Gumtree

So the other day, I was browsing through Gumtree because it’s interesting seeing what’s for sale in a 1 mile radius of where I live. I was thinking, I wonder how does the item count change over time? I wonder if I could use AWS to track the number of items for sale in a given area on Gumtree? With no clear API available, I had a look at the URL and realised quite quickly that the count of the number of items is in a singular H1 tag, which is very easy to snag, returning, as it does, the text…...

November 24, 2023 · 5 min · Morgan Conlon

Clusters on a budget

Clusters used to cost a lot of money. Now however, you can roll your own cluster for a couple of quid. Here’s an intro into how to do that. A cluster is a hub spoke model that uses a head node running a job management piece of software called a scheduler. The scheduler is responsible for taking jobs from users and assigning them to compute nodes. The compute nodes are the machines that do the actual work....

November 1, 2023 · 5 min · Morgan Conlon

Ansible toolbox

I’ve been meaning to set up a baseline Docker image for creating a complete Ansible and AWS environment to kick start your use of Ansible. As with all things, I won’t be the first, or even the last, but this should be some use if you want to just build an up-to-date Ansible image, use Ansible to run a container and fire off Ansible playbooks etc. and get on with your life....

September 20, 2023 · 3 min · Morgan Conlon

Costly cloud?

In business, never, ever, ever ignore costs. Cloud is just one aspect of business and so, like rents, salaries, taxes and so on, it can’t be ignored either. Cloud costs are tidal in nature and that’s perhaps why it gets unnoticed for a long time. S3 storage is a classic. Let’s say S3 costs £2,500 this month. Next month £2,750. Month after, £2,979. And yet, by the end of the year the organisation will be forking out over £5k in S3 costs with no decrease in sight....

August 7, 2023 · 7 min · Morgan Conlon