I’ve been using LLMs for lots of things for the past 4 years now, but I rarely have blogged about any of it. It’s such a fast-moving space that I figured I should start being more diligent about tracking the evolution of AI and my use of it.
Quick Historical Summary
I started using LLMs in 2022 with the release of ChatGPT. I was pretty quickly fascinated by it and my mind exploded with ideas and questions. By early 2023 I was training my own model, because I discovered the term “training” before I knew what “RAG” was. Lots of manual labor, embeddings, vector databases, laptops were crashed… I loved every minute of it. Improvements to every single part of the AI ecosystem seemed to happen overnight. Between 2023 and early 2026 I tried to build the same IOS app three times. The first time I was cut & pasting code from ChatGPT into XCode, there were no coding agents, the code was pretty bad, the training data cutoff was forever ago, etc. The third time, the training cutoff was just a couple months ago, the code was pretty good, everything from code implementation to pushing PRs was done by an agent, and that attempt actually tested out and I put it in the app store. In the same period, I went from AI being forbidden at one job, to it being required at another job, to now being at a place that trusts engineers to make things happen, which turns out to be the most effective approach I’ve seen. Maybe I should write about that.
Today
I am a software engineer who started their career focused on systems/network/data infrastructure. I co-authored a book about Linux system administration, and one about Python development, to give an idea about the sort of things I like to get into. I’m lucky to have been able to mold a career around not just being either a sysadmin or a developer, but landing in environments where I was able to wear lots of hats and add value in both of those spaces. My current environment allows for that, too.
I’m using AI in a lot of different contexts as a result. I use it in monorepos and small repos. I use it for infrastructure, software engineering, data engineering, research & ideation, and any other category of task that comes up. I use it with Golang, Python, Typescript, Javascript, Terraform (HCL), bash & zsh, awk & sed, regexes, SQL, Scala, and jq. I use it against various SaaS APIs, I use it to draft communication, create slide decks, etc.
I have also used it to obviate the need to use external vendors for various things, and I expect that to continue. I find that to be the most interesting result of using AI: the idea that it’s possible to skip having expense report line items for github repo backups or security awareness training, because you can now build those things using some terraform, or Google Workspace tools, in a matter of hours.
Tooling
TL;DR – I have used Claude Code since it came out. I started using Codex when that came out, but it hasn’t been able to pull me over from Claude Code as my daily driver. However, Sol 5.6 just came out a week or so ago, and it is pretty amazing. Just in time as Anthropic is about to (probably) make Fable prohibitively expensive. I use some other stuff for various non-daily tasks, but these are really my main tools.
The AI tooling landscape is just breathtaking. The speed with which things have appeared, disappeared, merged with other things, spawned whole new cottage industries, etc., is astounding. If everyone actually made use of only those tools specific to their role and using AI within it, we would run out of space on our hard drives for tools.
I use relatively few tools, and I think this is due to having used AI for so long and having built up a reasonably mature set of markdown files (hooks, rules, skills, etc) that dictate how AI tools will behave. I do try hard to keep up with the evolution of tooling, and I do adopt new tools for various one-off projects, PoCs, and small tasks, only to find myself thinking that they’re kind of redundant, don’t work very well, use up too many tokens, etc. I think I’ve gotten the most benefit from:
- Iterating on my local configurations (skills, rules, AGENTS.md, etc)
- Keeping up with the Claude Code and Codex changelogs
- Experimenting with open weight models in LM Studio
- Talking to friends in other industries, companies, with other side projects, etc about their own use of AI
As I’m writing, I guess I’m also concluding that an awful lot of the AI tooling space consists of noise from fledgling efforts to ride the wave, sell you something, and achieve an exit. For open source tooling, some genuinely want to solve a problem and sit on the same side of the table as their users, but others quickly evolved into commercial offerings that were then bought out by those whose motives are less clear and whose track records make them less trustworthy to me.
What I’m Missing
Two really critical things:
- A really good way to manage many agents across many separate, independent projects that each require a diverse set of tasks (setting up a database, validating a schema, creating a new github repo, writing code across multiple repos, implementing terraform for new infrastructure, writing documentation, etc). I have a visual model for this in mind that I tried to implement with AI, but it didn’t go great. That was a year ago, though. It might be a no-brainer with today’s models, which is crazy to think about.
- Infrastructure in particular is a challenge, because the model for infrastructure changes doesn’t naturally align with how code changes happen. Code can be implemented locally, and then have a suite of thousands of tests that can locally confirm that not only does your new code do what you expect, but it doesn’t cause any existing code to break. And those tests can run locally too. And typically if those tests all pass, surprises after deploying to prod should be exceedingly rare. And, if there is an issue, you can revert to the last known-good version of the code in a flash. The whole coding ecosystem is built around making this easy. Almost none of this is true for infrastructure, so I find that infrastructure tasks require more bot-sitting, I trust everything a lot less, I have to review every little thing, it’s very difficult to account for everything up front in the prompt, or even in my precious markdown rules/skills, etc.
I’m doing some work on both of these things, but they aren’t really my primary job, so time on them is limited. But even though there is tooling that seems to want to solve these problems, I just don’t find them to be very… desirable, for one reason or another.
For #1 what I really want is something that kinda looks like Linear but when I move a ticket from ‘Planned’ to ‘In Progress’, that triggers an agent to pick up the task and execute on it, and it moves the ticket to ‘ready for review’ when it’s done, so I can review it (for now – later I’d have the option of delegating review to a bot that could be authorized to go all the way through merging and deployment, potentially).
For #2, I shouldn’t really need something as heavy as Atlantis or SpaceLift, but at the same time, the current features of GitHub are an awkward fit — it’s just always obvious that GitHub was built for code and not infrastructure, so from that perspective I understand why these large, heavyweight tools exist. Terraform’s own quirks don’t help, either, though it’s very obviously light years beyond where Chef & Puppet left off, but now they’re putting some useful stuff in their cloud-only product, etc. OpenTofu is definitely in my future.
The Future
The rest of my year is probably going to be spent:
- Building AI infrastructure in AWS
- Building infrastructure that’s not AI-specific but is used for projects in the AI space
- Preparing for scale, mostly focused on the infrastructure space
- Preparing for frontier model pricing to become kind of insane (so, experimenting with replicating existing functionality using non-frontier models, which I’m already doing, it’s just lower on the priority list for the moment)
If anyone wants to connect on any of this let me know. If you just want me to write more about any of this, let me know. If you just think I should stop sleeping on your favorite tool, let me know that too! It’s certainly more than a small possibility that I don’t know about every tool and I’m happy to hear about more!