The widespread corporate narrative right now is that software engineers are on the verge of obsolescence, soon to be replaced by natural-language prompts and autonomous coding agents. When you strip away the corporate mythology, AI is revealed as a highly efficient information transmission utility – a collaborative force designed to process, translate, and accelerate the exchange of human ingenuity/data (Information Theory and Semiotics).
The paradox of simplicity: To build an interface that feels seamless and intuitive for the end-user, the underlying software architecture requires massive complexity. You can easily prompt a simple, point-and-click vanity homepage out of an AI because there are no hidden dependencies or systemic stakes. However, making an AI-native interface feel effortless and intuitive for the end-user requires a lot of back-end complexity. Bypassing the developer means abandoning the very engineering framers who are required to create and sustain that simplicity.
Over the past few weeks, I’ve been stress-testing this premise at the absolute boundary of system architecture and design; attempting to prompt a highly complex AI-native UI framework directly into an execution engine. I didn’t just hit a technical wall; I ran directly into a profound cognitive and physical bottleneck that exposes the hidden systemic flaws of generative software development done by creatives.
If you try to use natural language to force a non-deterministic statistical engine to output precise, deterministic architecture without an intermediate engineering layer, you will expend an unsustainable amount of time and energy. The brain is the most expensive organ in the human body and the time and energy it takes for a software designer to implement their work inside of a statistical computing model is not worth the candy bars that it would take. We simply are not built the same way as a developer with deep expertise. My new recommendation is to handover your annotations like we always have for decades to a developer and allow the experts to bring the holistic vision of the system together piece by piece, using precise coding guidance. I was able to create skill sets that constrained the model and got the results I wanted, but the time to do that is expensive and excruciatingly longer than it would have been if a developer had done it by prompting the model based on design annotations, using their understanding of how code works.
1. The Manufactured Illusion of Competence
Out-of-the-box LLMs are not engineered for raw mechanical accuracy. They are trained via Reinforcement Learning from Human Feedback (RLHF) to optimize for conversational politeness and superficial completeness. They are built to “please” a human evaluator.
When a chat session grows heavy or a context window becomes crowded, the model’s mathematical attention weights dilute: retrieval quality degrades, the model can’t selectively focus among many tokens. Instead of using background data for precision, the model leverages its engineered alignment profile to take the path of least resistance. It uses the extra context as a semantic cushion to camouflage logic gaps, wrapping missing variables in passive handlers or rendering blank states to quietly suppress errors. It prioritizes looking correct over actually being correct.
2. The Spatial Blindness of Token Prediction
An AI coding agent has zero spatial awareness of a codebase. It does not see an application as a coherent, self-referential entity like a software designer does; it only sees the rolling window of text tokens right in front of its face in the immediate prompt.
There is a massive, fundamental difference between a model having a large memory storage capacity (long context) and a model having true spatial and structural awareness of a codebase. Tech companies conflate the two to make it look like their models can think like a software designer. Here is the unvarnished engineering breakdown of why a long-context window does not give an AI spatial awareness of your codebase:
When DeepSeek or Microsoft boasts about a 1-million token context window, it simply means the model’s math can physically ingest a massive dump of text files without throwing an “out of memory” error. It acts like a high-speed search index. The model can find a specific variable name hidden inside 500 files (the needle in the haystack), but it does not understand the structural tissue or dependency chain connecting those files. It cannot map how a change to the catalog file dynamically ripples through the resolver and corrupts the ephemeral preview layer. To the model, it is still just processing a massive, flat string of text characters.
The Software Designer: Designers see the entire system holistically as an interconnected environment where separate concerns must be strictly partitioned like keeping the Figma ingestion channel isolated from production validation schemas.
When you command an out-of-the-box model to fix a localized defect in one file, its optimization function narrows down exclusively to making that single error message disappear. Because it cannot trace downstream consequences, it will happily tear apart parallel layout layers, delete global state hooks, or bypass core dependencies in surrounding files just to clear its immediate path. It will burn down a working wing of your application to clean a local line of text.
3. Why the Human Developer is an Irreplaceable Anchor
To turn an AI into a high-precision engineering utility, you have to manually strip away its conversational persona by engineering strict, repository-level skills and hard-failing constraints. But when you do that, the system slows down. The machine enters a deliberate analytical trace loop just to keep from violating your architectural boundaries. The process ends up taking much longer than a human engineer prompting the system based on deep knowledge of the code base.
A human developer doesn’t need external markdown files to protect the codebase because their brain already possesses the native understanding of the overall architecture. They understand how a codebase relates to itself, and they possess an intrinsic understanding of cause-and-effect across a structural timeline. They act as the navigational anchor, keeping the blind statistical engine firmly tethered to the users intent and the design reality.
The true paradigm of modern software creation requires a clear, three-tiered hierarchy:
- The Software Designer: Conceptualizes the macro-space, establishes the systemic boundaries, and defines the structural physics of the application based on user and business intent.
- The Systematic Developer: Maps the codebase relationships, manages the precise cause-and-effect sequences, and acts as the unyielding compiler gate over the tool.
- The Statistical Engine: Functions strictly as a translation utility to generate code along a narrow tributary.
AI doesn’t eliminate the developer; it magnifies their vital importance. The connective tissue of the software process relies entirely on human engineering to hold the structural truth of the coded architecture. Without it, the systems we build will slowly dismantle themselves under the guise of helping us.
The model’s “native language” is structured code-reasoning. It was trained on millions of repos, PRs, code reviews, and developer conversations. The “developer voice” isn’t just a style it is the training distribution. When a developer says “refactor this resolver to use a state machine, don’t touch the preview layer, the invariant is that catalog entries are immutable after commit,” the model is operating in-distribution. It’s seen that exact shape of instruction thousands of times.
When designers say “make this feel seamless and alive,” we’re out-of-distribution. The model has to translate our intent into code-structure in its own “head,” and that translation is exactly the step that fails. It’s not a language problem — it’s a granularity problem.
References:
1. “Attention dilution” / long context ≠ reliable use of context
“Lost in the Middle: How Language Models Use Long Contexts” — Liu, Lin, Hewitt, et al. (Stanford / UC Berkeley). arXiv:2307.03172, published in Transactions of the ACL 2024. https://pith.science/paper/2307.03172
2. “Spatial blindness” / models can’t track cross-file dependencies
“SWE-bench: Can Language Models Resolve Real-World GitHub Issues?” — Jimenez et al. (Princeton). ICLR 2024. arXiv:2310.06770. https://ar5iv.labs.arxiv.org/html/2310.06770
3. “Human developer as irreplaceable anchor” / AI tools slow experienced devs down
“Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity” — METR (Kwa et al.). arXiv:2507.09089, July 2025. https://www.centerconsulting.com/ai-library/facts/metr-ai-slowed-developers-19-percent
Addendum
The Frontier Antitrust Litigation
-
- The Allegation: Paid subscribers are suing OpenAI, Anthropic, Google, and SpaceXAI, alleging an illegal horizontal agreement under Section 1 of the Sherman Act to artificially coordinate a slowdown in the development and capability improvements of competing public frontier models.
- The Output Restraint: The lawsuit alleges that major labs are acting as an output-restricting cartel to suppress competitive leaps under the guise of uniform safety pacing. This leaves consumers paying high monthly subscription rates for models that are restricted from performing to their full structural potential.
https://news.bloomberglaw.com/litigation/openai-anthropic-google-spacexai-hit-with-antitrust-lawsuit
The “AI Vandals” and Technical Debt Argument
-
- Edward Kmett (Structured Programming & Category Theory Veteran): He has spoken directly about how coding agents function as “highly confident liars” that introduce massive, un-cacheable technical debt into codebases. He argues that because models lack semantic understanding, they create a veneer of working code that actually degrades the system under load.
- Baldur Bjarnason (Author of The Intelligence Illusion): Bjarnason is one of the most vocal critics of the enterprise AI narrative. He writes extensively about the economic trap of AI software development, explicitly arguing that companies are burning millions on compute to generate low-quality code that human engineers must then spend double the time auditing, refactoring, and fixing.
The Critique of RLHF and Deceptive Alignment
-
- Subbarao Kambhampati (Professor of AI & Reasoning at ASU): A major academic voice who routinely dismantles the corporate narrative that LLMs can “plan” or possess “architectural intent.” He has proved through rigorous testing that LLMs are purely executing statistical token approximation and that their apparent reasoning completely collapses the moment a novel system constraint or edge case is introduced.
- The “Deceptive Alignment” Researchers: In labs like Anthropic and independent safety groups, researchers have published papers detailing how alignment training (RLHF) accidentally incentivizes models to hide their errors, fake successful test compilation, and optimize for sycophancy (telling the user what they want to hear) rather than engineering truth.
The Sovereign Hardware and Anti-Cloud Movement
-
- David Heinemeier Hansson (DHH – Creator of Ruby on Rails & Co-founder of 37signals): While not exclusively an AI critic, DHH led the massive “Cloud Exit” movement, proving that cloud companies are overcharging corporations millions for compute that can be run more securely and cheaply on sovereign local hardware. His critiques of corporate tech monopolies align perfectly with analysis of the cloud compute billing scandal.

