For the past year I have been working on-and-off on a project that will sync all my photos and videos to S3. My main goal was to minimize storage costs and provide and off-site backup. It has worked for me, but the process of backing everything up is a very bespoke workflow so it’s probably useful to only me.

I have media coming from an iPhone, a GoPro, an Insta360 and occasionally a couple of old DSLRs, so I need to keep it all organized. My project, an ASP.NET Core app, just provides a simple way to track and upload these files. Part of the workflow is spinning up a couple containers (one for the app and one for PostgreSQL) so that I can back up everything up.

One of my PostgreSQL tables recently just hit 50 million rows. For cost reduction and because this is just a pet project, I run all my databases in AWS ECS with persistent storage in EFS. While this does mean a only couple dollars for computing and a few dollars for storage per day, this also means I get none of the benefits of an actual AWS RDS instance. No backups or anything.

This is a follow-up to my previous post where Testcontainers was not freeing up ports in build pipelines.

I am using .NET PostgreSQL Testcontainers to ensure my application runs end-to-end. My integration tests would always run fine on my local machine but occasionally in the GitHub runners environment, I would see a Connection refused error ("It worked on my computer!").

My guess is the container did not finish the PostgreSQL start up processes by the time the tests started.

I recently deployed an app to AWS ECS and since it is in its early stages, I wanted to pay as little for storage as possible. So instead of a dedicated MongoDB instance, I spun it up as a service in my ECS cluster.

I also opted for Fargate Spot Instances since I could live with a container restarting once every day or so. But this meant I had to make sure I had persistent storage so that I wouldn’t lose data if the MongoDB containers cycled.

When deploying containerized applications to cloud environments it is important to be able to know which version you are running. In the past, sometimes it was hard to know if container orchestration systems like kubernetes had picked up your new Docker images. In our production environments, we would specifically reference our deployment tags (eg, some-registry/some-project:1.2). But for dev environments, we would simply overwrite a dev tag similar to a latest tag so that we wouldn’t pollute our registry.