Where Do I Fit Into AI?
Estimated read time: 7 min
The Opening Salvo
I used to fear AI. Not in the science fiction sense, but in the way any professional fears obsolescence. I watched models go from generating mediocre code snippets to producing full-scale APIs. Agentic coding tools are starting to author better and more complete code than is possible for the vast majority of human software engineers. Fear is an expected response when your livelihood is threatened by something you don't fully understand. My fear broke when I stopped resisting and started building with AI. I went from skeptic to fully enamored. The power of frontier AI models in an agentic coding harness is truly awe-inspiring. That awe brought forth a new anxiety. If models can write code better and faster than me, then where do I fit?
Initially, I framed the AI landscape as two camps. In the research engineering camp, people are building frontier models. Training runs, novel architectures, alignment research. In the infrastructure engineering camp, people are serving those models at scale. GPU clusters, inference optimization, serving frameworks, quantization. Neither side really matches my pedigree. I don't have a PhD in machine learning. I haven't published papers on transformer architectures. I'm not optimizing CUDA kernels for inference throughput. Everyone wants to be the next Ilya Sutskever or Andrej Karpathy, but the sexy research tier is insanely competitive. I don't have numbers, but I would guess there are maybe ~1,000 research engineers and ~3,500 infrastructure engineers building the systems that deliver inference at scale. These specialized roles at one of a dozen major labs are hyper-competitive with enormous barriers to entry.
How does one of ~30m run of the mill software engineers reconcile the fact that coding agents outperform them in almost every way?
It's easy to lose your compass in a situation like this. I wrote this article to provide hope to those who are struggling to adopt AI, for those who fear obsolescence, and for those who believe in the singularity. First, the singularity is already here... learn to deal with it. How? Dive in.
Pore over research papers. Attempt to learn linear algebra and probability theory. Build your own homelab server or buy a beefy Mac Studio (with as much RAM as you can afford). Prompt AI 10,000 times through chat, terminal, coding agents, voice mode, and any other modality available. Press the model into corners, hurry them, coddle them, yell at them, write long and flowy prose, short and trite prompts. Tell them your life depends on their quality, ask for absurd outputs and generally just stress test the model. Learn the limitations of AI. In this process you'll realize that research engineers and infrastructure providers are the originators of commoditized inference. They build the engine and they keep it running, but large language models are just fixed sets of numerical weights which are vectors and matrices. These static bundles are exposed to the world via API to provide inference. They do nothing more than provide inference. They are idle without an outside force acting upon them. And coding agents, even as powerful as they are, have no internal drive. They have no mission. They're waiting for an operator.
The Operator is the Missing Piece
Enter SWE 2.0: The Harness Engineer. We've discussed the domain of research engineers and infrastructure engineers, but not that of the harness engineer. Harness Engineers put AI to work. They give models drive, mission, and purpose.
You're no longer an author of code. You're a wrangler of agents. You're also a system thinker. You design systems at the behavioral level. You're a person who understands a desired behavior, designs a system to achieve that behavior, and identifies the appropriate place to sprinkle the inference that makes that behavior possible. You employ inference at scale and scaffold safeguards against all the ways it can go wrong.
This is where the overwhelming majority of software engineers belong. And the field is wide open. There is no established playbook for harness engineering (though it is arguably in its infancy). The people doing this work are writing the playbook as they go. This is precisely where you can secure your seat on the AI train and carve out your own legacy.
Understanding the Landscape
I think of AI systems in terms of the API boundary. There is a left side and a right side: Research/Models → Infrastructure → | API | ← Harness ← End User
The inference provider (left side): You're not building the left side, you're harnessing it, and to harness it properly, you must understand it. If I were starting from zero I would read Andriy Burkov's The Hundred-Page Machine Learning Book and The Hundred-Page Language Models Book. Understanding the fundamentals outlined in his books will transform how you view inference from black box to mathematical probability machine.
The inference harness (right side): I would start by setting up Ollama to run models locally. An effective harness engineer must experiment the same way a research engineer experiments in the lab. Build an agent with a framework like Mastra, feed it data, evaluate the output, root out the unexpected, and add controls. This is how you learn scaffolding and the practical limitations of models. I would do this before paying for an API inference provider.
Traditional Engineering is the Foundation of the Harness
Practical AI engineering is fundamentally traditional software engineering with inference layered into operations that require dynamic decision-making. Inference becomes necessary not strictly where decision trees max out, but where the input is too unstructured for discrete logic, or where the cost of enumerating and maintaining explicit rules exceeds the cost of delegating to a learned model. The fundamentals of software engineering are not replaced by large language models. Only new capabilities are added.
Scaffolding: The Critical Differentiator
These new capabilities bring new challenges. This is where harness engineering diverges from traditional software engineering. The scaffolding is what separates a demo from a production system.
A solid foundation for scaffolding is what Geoffrey Huntley calls The Ralph Loop. This loop feeds context into a model, evaluates the output, and retries with failure context when evals fail. Mastering this loop is the single most important skill a harness engineer can develop. At the heart of that loop is context engineering: giving the model exactly what it needs to get the job done. Too little context forces hallucination, too much overwhelms and confuses. Getting it right means learning prompt engineering, building a tooling layer that gives models agency through function calling and code execution, and implementing retrieval augmented generation (RAG) to feed proprietary knowledge at inference time.
Scaffolding also extends into something engineers tend to overlook: governance, risk, and compliance (GRC). When you deploy AI into regulated industries, you're accountable for what the model produces, which means audit trails, bias evaluation, data sovereignty, privacy controls, and compliance frameworks. Cybersecurity demands equal attention, because AI introduces attack surfaces that didn't exist before: prompt injection, data poisoning, model extraction, adversarial inputs. Cost and performance round out the picture. Inference isn't free, and understanding token economics, caching strategies, and model selection is part of the job. Not every problem needs a frontier model. Sometimes a rules engine, a database query, or a pure function is the right answer.
The Hard Truth
The role of "person who writes code" is being absorbed by AI at an accelerating rate. If your identity as an engineer is tied to the act of typing code into an editor, you're gonna have a bad time.
Software engineering as a career isn't over, but it's changed.
The next generation of software engineers are harness engineers. They don't just code. They design systems at the behavioral level, manage agents, employ inference at scale, and scaffold around AI to make it safe, reliable, and useful. The foundation is traditional software engineering. The differentiator is understanding how to harness AI effectively and responsibly.
You don't need a PhD. You don't need to publish papers. You need to put in the reps. Prompt, read, reason, repeat. Run models locally. Build agents. Break things. Learn the boundaries. Understand the left side well enough to make informed decisions on the right. Use coding agents to accelerate your learning. Let the agent write the code and just review the output.
Are you afraid AI will take your job? If your job is just coding then it already has.
Go find your place. The world is waiting for harness engineers.