If you’ve ever built a side project that organically grew into a real product, you know the siren song of major cloud providers. “Scale seamlessly to millions!” they promise. “High availability out of the box!”.
Is it a good idea or is there a better way to leverage your resources?
The Managed Cloud Trap
So, like any responsible engineers wanting a reliable setup, we listened. We had been building our platform for over two years in our free time—a non-trivial codebase spanning roughly 200,000 lines of code. The stack is modern and modular: about 20 Go microservices communicating via gRPC behind a REST API gateway, a PostgreSQL database, a React web frontend, and a second web application currently joining the fleet.
To host it properly, we turned to a major cloud provider and set up the full managed experience: managed Kubernetes clusters, high-availability managed PostgreSQL with three asynchronous replicas in production, managed S3 buckets, dedicated load balancers, cloud routers, and persistent storage. We had two isolated environments — Dev and Prod — ready to handle serious scale.
There was only one problem: our actual usage profile.
Our platform serves around 1,500 registered users with up to 10,000 visits per month. It’s a solid, stable project, but it certainly didn’t need the compute power of a mid-sized enterprise. Yet, every month, the invoice arrived with painful consistency: €500 to €600 for primary cloud infrastructure, split almost evenly between Kubernetes and managed high-availability database. On top of that, we were tacking on another €20/month across external tooling—including Docker Hub image hosting and extra GitHub Actions CI/CD build minutes. We were paying a steep premium for mostly idle resources.
Naturally, we tried to right-size it first. We created smaller nodes, adjusted configurations, and attempted to trim database tiers. But the previous provider’s platform mechanisms consistently resisted: automatic defaults and hard vendor minimums kept nudging resource sizes—and costs—right back to where they started.
We weren’t just over-provisioned; we were locked into managed infrastructure limits that made small-scale efficiency impossible. It was time to make a drastic move.
Reclaiming Control with K3s and Infrastructure as Code
Faced with rigid provider minimums, we made a radical call: ditch the managed abstractions and take back full control of our stack. We migrated to a simpler provider and built an architecture sized for our actual needs, not an imaginary hyper-growth curve.
Instead of paying for expensive managed Kubernetes control planes and dedicated HA database clusters, we shifted to lightweight Virtual Private Servers (VPS). We set up two isolated instances—one for Dev and one for Prod—and deployed K3s, a lightweight Kubernetes distribution, managed via Rancher.
To keep the setup deterministic and maintainable, everything was defined using Terraform. From network interfaces to storage volumes, our entire environment became fully reproducible from code.
Here is how we replaced the managed cloud components without sacrificing stability:
- Self-Hosted PostgreSQL: We moved off managed database instances to self-managed PostgreSQL directly on the VPS instances.
- Custom Backup Strategy: We wrote automated cron scripts to execute hourly local database snapshots and daily offsite syncs to a cheap S3-compatible bucket.
- In-House Container Registry: We canceled our paid Docker Hub plan and spun up a private registry backed directly by the current provider low-cost S3 storage.
- Lightweight Housekeeping: Automated cron tasks handle daily cleanup of dangling images and temporary storage, keeping disk usage lean.
The result was immediate. We gained direct SSH access, clear log visibility, and precise system metrics. Despite running 20 microservices, two frontends, and a database on single VPS instances, our CPU and RAM utilization sits comfortably below 60%. The entire infrastructure bill plummeted from €500–€600/month down to a meager €45–€50/month.
Of course, this trade-off meant exchanging one-click provider convenience for upfront engineering effort in Terraform and backup scripting. But for a predictable, low-traffic stack, trading €550/month of cloud overhead for a weekend of setup code was definitely worth the effort.
Cleaning Up the Peripheral Costs
Fixing the core cloud infrastructure addressed the bulk of our invoice, but we still had peripheral SaaS subscriptions and CI/CD pipelines to clean up.
Back when we were burning €600 a month on compute, paying an extra €20 for external tools felt like rounding error. But once our main bill dropped to €50, that same €20 suddenly represented a 40% tax on top of our entire monthly cloud budget. Paying for build overages and image hosting on an otherwise lean setup just didn’t make sense anymore.
With two active developers and up to five occasional contributors, our GitHub Actions pipelines regularly hit the free 2,000-minute monthly threshold around day 20. Between those overage charges, a paid Docker Hub subscription, and third-party monitoring, we had room to streamline.
Here is how we optimized our pipeline and observability down to zero additional cost:
- Selective Pipeline Triggers: We restructured our CI workflows so builds only execute for modified microservices or applications, rather than running blanket, full-repository builds on every commit.
- Hybrid CI Execution: We introduced a hybrid runner strategy, splitting execution between our own internal worker nodes and Blacksmith to take advantage of their 3.000 minutes free compute tier. Simple GitHub environment variables allow us to toggle execution targets or route specific heavy build tasks to specific machines as needed.
- Supply Chain Security Integration: We embedded our company’s in-house supply chain security guardrail directly into the pipeline, giving us automated dependency checks without adding external tool fees. You might want to learn more about our Supply Chain Guard 🔗
- Zero-Dollar Private Registry: By deploying a self-hosted container registry backed by the new provider’s affordable S3-compatible storage, we completely eliminated our Docker Hub subscription.
- Free-Tier Observability: For logging, tracing, and monitoring, we configured New Relic on its free tier. By tuning our log verbosity and dropping unnecessary debug noise, we comfortably keep our monthly log ingestion well under the free limit.
By making a few smart tweaks to our workflows and leveraging free tiers effectively, we shaved off the remaining subscription clutter. Our pipelines run faster, our logs are clean, and our peripheral software costs dropped to zero.
Future Roadmap, Decoupling, and Unlocking Real Scale
Moving to a single VPS per environment gave us an immediate 10x cost reduction, but we designed this architecture as a stepping stone—not a dead end. Running lean doesn’t mean backing ourselves into an architectural corner.
As the platform evolves, we have a clear roadmap to optimize resources further and prepare for growth:
- Self-Hosted CI/CD Runners: While our hybrid Blacksmith setup works well, we plan to spin up a dedicated GitHub Actions runner agent directly on a VPS. Moving all maintenance jobs, builds, and artifact handling completely in-house will boost pipeline performance even further and eliminate external runner management altogether.
- Isolating the Resource Hog: Our Go microservices and React frontends are ridiculously light, consuming under 1 GB of memory combined. However, we have a single specialized analysis service that draws roughly 1.5 GB of RAM all by itself—accounting for over 50% of the entire cluster’s memory footprint. Since Dev only uses this service for occasional manual testing, our next move is to decouple it onto a single, memory-optimized VPS shared between Dev and Prod. This will instantly liberate the core K3s clusters and keep compute costs razor-thin.
- Horizontal Scaling Without Cloud Lock-In: Our current single-node setup isn’t a technical limitation; it’s an intentional choice for our current traffic. Because everything is codified in Terraform and K3s, joining additional worker nodes to expand into a multi-node cluster is nearly as seamless as sliding a scale bar in a managed cloud console.
- The Managed Fallback Option: If traffic explodes or maintenance overhead ever outgrows our team’s bandwidth, current cloud also offers managed Kubernetes and database solutions. Transitioning back to managed services on the current provider would roughly double our bill to ~€100/month—a small price to pay, and still light-years away from the €600/month we used to dump into Original Cloud.
We didn’t build a fragile workaround to save a few bucks. We built a flexible, infrastructure-as-code baseline that scales up smoothly when—and only when—the application actually demands it.
Lessons Learned & Key Takeaways
Migrating away from managed hyperscaler services wasn’t just about shaving numbers off a monthly invoice—it was a reality check on modern software architecture and a perfect opportunity to put our infrastructure engineering skills to the test.
When you’re building a project from scratch, the industry pushes you toward enterprise-grade, multi-node managed infrastructure from day zero. But for a team managing 20 microservices with a few thousand active users, high-availability clusters and managed database replicas often add financial friction without providing practical benefits.
Here are our biggest takeaways from the migration:
- Right-Sizing Beats Convenience: Managed cloud abstractions are fantastic when you need automatic global scale, but they charge a massive convenience tax. If your traffic doesn’t justify auto-scaling, raw compute with lightweight orchestration will save you thousands per year.
- Infrastructure as Code Is Non-Negotiable: Moving off managed services doesn’t mean resorting to manual server setups. Defining our entire VPS, network, and storage setup in Terraform gave us the reproducibility of a managed cloud with the cost profile of bare metal.
- K3s Is a Game Changer for Small Teams: You don’t need a heavy control plane to run production Kubernetes. K3s gives you full API compatibility, easy local development, and seamless resource utilization on a fraction of the hardware footprint.
- Audit Your Secondary Stack: Secondary SaaS tools, registries, and CI/CD overages quickly stack up. Bringing image hosting in-house and tuning workflow execution trimmed a silent “40% tax” off our newly optimized baseline.
Over-provisioning is easy to ignore when you’re caught up in feature development, but reclaiming control of your stack puts money back into your pocket and gives you a much deeper understanding of your actual infrastructure needs.
If you have similar experience with hyperscalers or have thoughts/feedback/comments regarding the topic, or if you in a situation where you think you might benefit from a similar migration, do not hesitate to reach out.