Tech Stacks I Like: Free and Cheap Infrastructure for AI Experimentation
I work on AI: models, fine-tuning, user experience. The less time I spend on infrastructure, the better.
GPUs are expensive, and if anything, prices are going up. Managing cloud infrastructure as a solo developer is its own full-time job. For a long time, those two facts meant I shelved projects rather than deal with the overhead. But things have changed. There’s now a good set of free and cheap tools that let me skip the infrastructure work and focus on what I’m actually interested in: models and user experience. This post documents what I’m currently using. It’s a living document. I’ll update it as my stack evolves.
Computing Stack
When I need real GPU power for training or running models, I use two services depending on the workload.
Modal
Modal is where I do structured training work, fine-tuning LLM LoRAs for example. The free tier gives you $30/month in credits, which works out to roughly 30 hours of A100 time. That’s a lot of GPU compute for experimentation.
Storage (Modal Volumes) is currently free, though they’ve indicated this will change. When it does, I’ll likely mount Backblaze instead. Easy enough to do and keeps costs predictable.
The setup is slightly peculiar. There’s a specific way you define containers and jobs that takes some getting used to. I had Claude Code handle most of the initial configuration. I log everything to Weights & Biases and can iterate on experiments quickly.
I’m also looking at Modal for model deployment. They support scale-to-zero with sub-second cold starts, meaning I could serve a model without keeping a machine running permanently.
RunPod
For more general GPU work, like running ComfyUI or anything with a GUI, I use RunPod. They have pre-built Docker templates for common tools, so ComfyUI works out of the box.
The appeal over Modal is the wider range of GPUs (30+ types to choose from) and the flexibility for interactive workloads. Pricing is competitive: RTX 4090 from $0.34/hr, H100 from around $2/hr. Per-second billing, no ingress/egress fees.
The distinction in my head: Modal for structured, logged training jobs; RunPod for interactive experimentation and GUI-based tools.
Backblaze
For cloud storage, I use Backblaze B2 managed through rclone. This isn’t cold backup. It’s working storage that I actively pull from and push to.
Why Backblaze? The pricing is simple and cheap: around $6/TB/month, with egress free up to 3x your storage amount. The main reason, though, is speed. My home upload is slow, so having data in the cloud on their fat pipes makes everything faster when I need to move files between services.
Hosting
Cloudflare
Cloudflare has become the backbone of my hosting setup. I use Pages for static sites, including this one (built with Astro) and my side project Prominent Peaks. I use Workers for lightweight backend logic like routing and API endpoints. I’m experimenting with Vectorize for vector search and embeddings. And I use Zero Trust to protect endpoints I don’t want exposed to the public internet.
Why Cloudflare? The free tier is generous. Edge distribution means fast loading times everywhere, which matters to me. I have a bee in my bonnet about speed and latency. A fast site is a more pleasant experience for everyone. With Wrangler, I develop locally, test, and push when ready. Quick iteration, no fuss.
The practical upshot is that projects I’d previously shelved due to hosting costs are now viable. A Worker that handles occasional API calls costs essentially nothing. I might eventually move to the $5/month Workers Paid plan for some of my vector work. That unlocks bigger databases, more requests, and removes the daily limits that can interrupt you mid-experiment. But for now, the free tier covers everything.
Oracle Free Tier
Sometimes you need an actual server. For services where I can’t find a cheap serverless option, OpenCLIP for instance, I run them on Oracle’s always-free ARM VM (the Ampere A1). I also host a LibreChat instance there.
I was initially put off by the ARM64 chips, assuming compatibility would be a headache. They’re actually well-supported now. Most things just work.
One gotcha: I had trouble provisioning a free VM until I added a credit card to my account. They still haven’t billed me anything. It just seems to be how they gatekeep the free tier resources. Worth knowing if you’re stuck in provisioning limbo.
The setup pattern I’ve landed on: lock the VM down to SSH only on port 22, then tunnel everything out through Cloudflare Tunnel (cloudflared). Set it up once, forget about it. I periodically have Claude Code SSH in and audit the security configuration. It’s been stable.
The documentation isn’t as comprehensive as AWS’s or Azure’s, but it’s sufficient. And the price is hard to argue with: free, forever.
Model Access
OpenRouter
For LLM experimentation, I route everything through OpenRouter connected to a LibreChat instance. This gives me access to nearly all the models, large and small, from every major provider, through a single unified API.
Why not go directly to Anthropic, Google, OpenAI, etc.? Three reasons: I want a single place for everything, I want to easily switch between models without changing code, and I don’t want to manage a dozen different accounts and API keys.
Yes, there’s a small premium over going directly. It’s worth it for the convenience.
Agentic Coding Tools
Claude Code
For agentic coding, I use Claude Code. It handles the back-and-forth of implementation while I focus on direction and review.
GitHub Spec Kit
I pair Claude Code with GitHub Spec Kit. The advantage of Spec Kit is that it allows longer, more reliable, hands-off implementation. By providing structure to the work (specs, task breakdowns, checkpoints), it reduces the risk of going off-track. I can kick off a larger piece of work and trust it to make meaningful progress without constant intervention.
The Point
The infrastructure tax that used to kill side projects is basically gone. Between generous free tiers and cheap pay-as-you-go pricing, there’s very little stopping you.
You don’t need to be an IT person to run this stuff. You just need to pick tools that stay out of your way. These are the ones that work for me.