“Write code faster”. Managers see cycle time. Developers are divided. Some see glorified autocomplete; others see “amateur hour”. I’m in the camp that sees the beginning of engineers as leaders: less time typing code, more time defining intent and managing risk - and if you generate 1,000 lines of code in a few seconds, you have not just solved a problem, you have created a maintenance obligation.

AI is part of my everyday workflow now. At its best, it scales my work: it can pick up a half-finished thread after a week away, and it gives me a kind of semantic map of a codebase.

Not “search” as in grep. More like: trace how concept X flows through the system, show me every call site that can trigger behaviour Y, where is this validated / transformed / logged, and what else would a change here plausibly break. Questions like that used to take a ticket and a block of time. Now it’s seconds.

And separately: it turns ideas into something I can look at. For me as an (aphant), that externalising step is a super productivity boost.

At its worst, it is confidently wrong in ways that look tidy. I spend a lot of time steering it away from dodgy abstractions and subtle breakage. I’m able to spot that because I’ve made those mistakes myself.

I have written before about the economic and environmental costs of the infrastructure itself. But the other bill, the one your team pays directly, arrives as review time, churn, and security flaws.

The defining challenge: epistemic debt

Epistemic debt is what happens when you ship code you can’t explain. Not technical debt from shortcuts - this is the illusion of competence. Code that looks professional, tests that pass, but without the understanding to maintain it.

I have caught myself here. Deep in the flow, the model starts doing more than filling in syntax: it begins choosing the shape of the solution. You still feel like you are “writing code” because you’re nudging and iterating, but your role has quietly shifted from driving to supervising.

The danger is not that you’re rubber-stamping. It’s that the handover of control is gradual enough that you only notice later - when you need to explain the change or debug a weird edge case, and you realise you haven’t built the mental model.

If you can’t explain it, you didn’t ship a solution - you shipped a mystery.

Bugs happen in hand-written code too. The difference is recovery cost. When you write the change, you usually build a mental model as you go: what it assumes, what it touches, and where it is fragile.

With AI, the work shifts. You become the reviewer of a draft you did not author. That can be fine, but only if someone still does the ‘authoring work’ - internalising the change.

Shipping code is taking responsibility for its behaviour. If you can’t explain what a change does, what it assumes, and how it can fail, you don’t really own it. AI doesn’t remove that obligation - it increases the amount of code you can produce, which makes the ownership requirement more important, not less.

Why epistemic debt amplifies every other risk

Epistemic debt matters because when you don’t understand what you’ve shipped, you can’t spot problems lurking beneath the surface. It makes every traditional software risk worse:

1. The churn

GitClear’s AI code quality research looks at real codebase change patterns rather than anecdotes. Their headline signal is a shift in how code changes happen.

They found that “moved” code is down (often a proxy for refactoring), while copy/paste changes are up (duplicate logic - no respect for the rule of three).

The problem, is the path of least resistance. It’s easier to ask a model to “add a function that does X” (append-only) than to ask it to “refactor the existing class to support X generically” (edit-in-place). Without strict guidance, the default outcome is a codebase that grows faster than it improves.

Why epistemic debt makes this worse: If you don’t deeply understand the change, you won’t recognize when you’re duplicating existing logic. The AI draft looks reasonable, tests pass, so you merge it - never realizing you just created the third variation of the same pattern.

What this costs in practice: duplicated logic, inconsistent behaviour, and more code to maintain. This is something I have seen firsthand.

2. You can’t catch the security flaw

Models are getting better at security. But the research highlights a human problem, not a machine problem: overconfidence.

Because the code arrives formatted, commented, and syntactically perfect, our brains lower their guard. We assume competence in syntax equals competence in logic.

Why epistemic debt makes this worse: It is not always a blatant injection flaw. It is often a subtle logic assumption that looks reasonable in review, passes tests, then fails in production because the real world violates an unstated precondition. If you don’t understand the implementation deeply, you won’t spot the hidden assumption.

What this costs in practice: Security incidents and production failures that could have been caught in review - if the author or reviewer had built a proper mental model of how the code actually works.

3. You can’t diagnose stability issues

DORA’s reporting captures something many teams are feeling: AI can lift individual throughput, while trust, coordination, and stability become the limiting factors.

My shorthand for this is the vacuum hypothesis:

Time saved on authoring is vacuumed up by review, debugging, coordination, and operational work.

Why epistemic debt makes this worse: When something breaks in production, the person who wrote the code can’t explain how it was supposed to work. The team has to reverse-engineer intent from the diff, logs, and history - paying twice to understand what should have been understood before merge.

That is not an argument against AI. It is an argument against treating it as free speed.

Opting out is not realistic

Even if you accept every risk above, I don’t believe that a full opt-out is viable. Some work genuinely becomes cheaper, especially scaffolding and first drafts - and if you refuse that leverage entirely, someone else will ship faster.

This is the uncomfortable part: if AI gives even a 20% advantage in certain loops, the teams that take it will deliver features faster, and markets will reward them. The market does not care if your code was artisanally hand-crafted, if you ship six months late.

So the question is not “AI or no AI”. It is “what operating model prevents AI from turning speed into debt”.

Where epistemic debt shows up in teams

• LGTM in good faith: the reviewer wants to review properly, but the author cannot explain intent, assumptions, or failure modes. Review degrades into “tests pass and it looks tidy”.
• Silent contracts: the implementation bakes in an unstated assumption (ordering, nullability, idempotency, tenancy, time zones). The tests mirror the assumption, so nothing fails until production violates it.
• Dependency drift timebomb: AI pulls in a library or pattern that is slightly off your ecosystem (licensing posture, security posture, operational footprint). Months later you hit a vulnerability advisory or an upgrade break, and nobody can answer why it is there.

One caveat: decent models do now produce good comments and a neat explanation of what they did. That helps. But comments are not comprehension. The debt is paid down only when a human can explain the change, including what it assumes and how it can fail.

The review bottleneck: 10 seconds to generate, 10 minutes to review

AI changes the shape of the pipeline:

• Drafting got a lot cheaper.
• Review, validation, and operational confidence did not.

This is where “LGTM syndrome” comes from. Reviewers are asked to absorb more change than attention allows, so they start pattern-matching for obvious issues and miss deeper problems: intent mismatches, edge cases, and system impact.

The solution is to embrace, but validate

This is the pivot: the total cost of AI is only high if you treat it as magic.

The analogy I keep coming back to is aviation. Modern systems can automate an enormous amount of the routine work, and they do it with superhuman consistency.

But the operating model still assumes a pilot in control of aircraft: someone who understands what the system is trying to do, monitors for when reality diverges from the model, and takes over when it matters. The risk is not that the automation is stupid - it’s that it is good enough to lull you into not noticing when you’re no longer fully in control.

Capability is not the same thing as responsibility.

Treat AI like a junior engineer: fast, useful, sometimes brilliant, often overconfident, and in need of constraints and checks.

Shift the mindset from “write code” to “own intent”

If you use AI as autocomplete, you get faster typing.

If you use AI as an agent, you can move faster without losing intent, but only if you add guardrails.

I wrote about this in Autocomplete Was Never the Point: I believe the true the leverage is delegation, not predictive completion.

The bottom line

The illusion of competence is the real risk. AI produces code that looks professional, tests that pass, diffs that seem reasonable - but without the understanding you need to maintain it.

If you treat AI as free speed, you will pay interest later in churn, security incidents, slower reviews, and epistemic debt: working code that nobody on the team can fully explain.

If you treat AI as a fast intern, and invest in solid specs, tests, and validation gates, you can keep the speed without bankrupting the codebase.

Use AI. Get good at it. But keep your hands on the wheel. If you can’t explain the change in plain language, it isn’t speed - it’s epistemic debt, and the bill will arrive when you least expect it.

Sources

• GitClear: Coding on Copilot (AI code quality research) - change-pattern analysis (code movement vs copy/paste signals)
• DORA: Accelerate State of DevOps Report 2024 - delivery performance framing, with notes on generative AI and outcomes
• Your AI Coding Assistant Is Quietly Creating a New Kind of Technical Debt - Stanislav Komarovsky, Medium (2025)
• The Illusion of Competence: Defining ‘Epistemic Debt’ in the Era of LLM-Assisted Software Engineering - Ludovic Ngabang, viXra (2026)
• Epistemic Debt: A Concept and Measure of Technical Ignorance - Ionescu et al. (2019) (The original academic definition in manufacturing)
• Do Users Write More Insecure Code with AI Assistants? - Perry et al., arXiv (Stanford/Cornell), 2022
• Assessing the Security of GitHub Copilot’s Generated Code - Majdinasab et al., arXiv, 2023 (replicating and extending Copilot security analyses)
• DORA: State of AI-assisted Software Development (2025) - DORA research on how AI interacts with software delivery outcomes

I’m talking about tools like ChatGPT and GitHub Copilot used as delegated, parallel workers, not just autocomplete.

Autocomplete - “Intellisense on crack”

The version of AI I’m not excited about is predictive autocomplete that confidently guesses whole functions, control flow, or behaviour from a few lines of context.

When I’m choosing to do the coding myself, I want to think: shape the solution deliberately, reason about constraints, edge cases, and failure modes. Having an LLM jump ahead and guess my intent often interrupts that process more than it helps.

I am a fan of mechanical shortcuts. What I don’t want is intent guessed for me.

It was initially how AI-assisted coding was framed - less boilerplate and smarter in-editor suggestions. The problem is that this framing has lingered, and it obscures where the leverage actually is now.

Code review as an example

I’m seeing the same sales pitch around code review: treat review as a throughput problem, then sell AI as the fix.

That’s risky, because review is one of the places we most need human attention. It’s where we check intent and system impact: what are we changing, why, what could break, and what does this do to everything around it.

If a draft was generated in seconds, review can accidentally become the first time a human brain really processes the change. That is a massive shift in responsibility, and it’s not a shift you want by default.

I do use AI to assist code reviews, but more for broad coverage, like automated tests or static analysis. It catches obvious issues and asks awkward questions early, so I can spend time on the high-value part. Even here, model choice matters: fast models for surface-level checks; stronger reasoning models when I want it to challenge risk and trade-offs. (See my quick picks.)

AI-in-review has to be intentional.

I’ll pull the diff down, ask targeted questions, and treat it as one of the steps in my review. Always-on analysis in a PR risks becoming ambient noise. I want AI to sharpen attention, not dilute it.

The mode that actually changed how I work

For the moment, we’ve settled into a different shape of collaboration with AI, and it is not autocomplete. It is agentic chat: chat, plus delegated tasks that can read code, suggest edits across files, run checks, and iterate.

Instead, delegation: working the way I already work with a team, just faster and more parallel. Delegation also requires more discipline than doing it yourself.

This feels natural if your day already involves coordinating work, managing risk, and holding the system in your head, rather than staying deep in one implementation thread.

On non-trivial work there are always multiple threads: explore an approach, test assumptions, read docs, check edge cases, and do setup/testing. Agentic AI lets me run those threads in parallel. I’ll put one agent on a migration strategy, another on a data model, another on a library I don’t fully trust yet. I question the answers, push back, redirect, discard a lot, and I’m still the one steering: scope, priorities, and what “done” means.

That doesn’t feel like automation. It feels like leadership.

Why this feels familiar to experienced engineers

Agentic AI clicked for me quickly because it doesn’t require a new way of thinking about engineering.

Many of the practices we rely on now, exist to reduce risk, not maximise speed: pairing, design review, parallelising work, challenging assumptions. Agentic AI fits into this perfectly.

Double down on what good teams already do

This is why the joking around “AI-assisted juniors” misses the point. When code is cheap, unclear intent is expensive, and you feel it faster. The answer isn’t to reject the tool. It’s to double down on what good teams already do: strong review, explicit assumptions, and real ownership.

Agentic AI makes it much faster to try ideas, but it does not remove the need to do the work engineers have always done: understanding requirements, thinking through design, checking assumptions, and considering system impact.

Experience compounds here. The hard part was never typing code. It was knowing what mattered, what could go wrong, and where to be careful.

Where things actually go wrong

What I’m seeing in on the ground now is not about seniority. It is about mode selection.

• Overuse without discipline: the AI does good local work, but the developer stops leading the system. Scope, constraints, and checks aren’t made explicit.
• Underuse out of principle: treating failure as proof it has no place in serious work. We wouldn’t judge a human collaborator this way.
• Confusion about responsibility: oscillating between over-trust and total dismissal, because no one has a clear model for what humans must still own.

The elephant in the room - our future as devs

In the very long term, if the rate of improvement continues, it’s reasonable to ask whether we still need as many developers in the trenches 😢

But my bigger concern is messy middle phase: the months/years (maybe longer) where the tools are powerful enough to create a convincing demo, but not reliable enough to run a business-critical system without human engineering behind it.

This is also where the incentives get weird. If AI makes it look like software is now “just prompts”, some organisations will try to route feature delivery around engineering: PMs, analysts, or vendor teams driving changes directly.

We’ve seen this movie before in waves of “we don’t need developers”: low-code/no-code platforms, heavily outsourced feature teams, third-party “service accelerators”. The pitch is always the same: the business can move faster without waiting for engineers.

The outcome is often the same too:

• It works for simple cases, then hits hard limits (it can’t do X or Y).
• The platform/support burden is underestimated (observability, security, upgrades, incident response, data migrations). Or overlooked completely and it just gets handed over to the Platform team with no warning!
• The best developers leave or refuse to touch it because it feels like downskilling.
• The remaining work becomes mostly glue code and escape hatches, until you end up with a Franken-system: part platform, part bespoke, hard to change, and hard to reason about.

That’s why I’m pushing the “delegation” framing for now. If you treat AI like a shortcut around engineering, you get fragile systems. If you treat it as a force-multiplier inside engineering, you can get real speed without losing ownership of intent.

Where the real cost shows up

Idea → What/Why (spec) → Draft (AI or human) → Validate (tests/checks) → Review (PR) → Merge

The engineering pipeline doesn’t change. AI just compresses the Draft step, which increases pressure on Spec and Validation.

AI makes exploration cheap and parallel. If you don’t do the normal engineering work (what’s changing, why, what might break, how you’ll know, and how you’ll roll it out), the cost shows up later.

There’s another cost that’s easy to miss: the developer may ship changes without coming away with much understanding of what they just built. That knowledge doesn’t transfer automatically just because the code exists.

This is the shape of what’s now being called epistemic debt: working code with missing understanding, where the bill arrives later when you need to change it, debug it, or defend it in review. I wrote more about it here: Epistemic Debt: the hidden cost of AI speed.

Example 1: One common failure mode is brittle tests: AI can generate “good looking” UI tests from a snapshot of what it sees today, baking in incidental details. They pass once, then become a maintenance trap.

Example 2: Subtle logic drift: the developer steps back, the model decides the shape, and the code can look tidy, pass the wrong tests, and still be wrong.

And yes, it also shows up as larger diffs, slower reviews, fuzzier intent, and a gradual erosion of trust. That is where guardrails become necessary.

The guardrails I rely on

These aren’t rules. They’re the defaults I try to fall back to when the pace goes up and it becomes easy to lose intent. Think of them as team-friendly expectations: the kind of thing you can say out loud in review without it sounding personal.

No explanation, no merge. If the author can’t explain what changed, why it changed, what it assumes, and how it can fail, it shouldn’t ship yet. That’s the simplest way I know to avoid epistemic debt.

Write the intent down before you “improve” the code. For any non-trivial change, we want a tiny written spec somewhere (often just the PR description):

• What behaviour is changing?
• What is explicitly not changing?
• What assumptions does this rely on?
• How did we validate it (tests, checks, manual verification)?

If the model drafts a neat explanation, it’s worth rewriting it in your own words. If you can’t, that’s a signal the change isn’t understood yet.

Keep diffs reviewable. Large diffs collapse multiple decisions into one review moment. If something small turns into something big, it’s usually a sign to split it: isolate refactors from behaviour changes, and isolate scaffolding from logic.

Treat tests as product code. This is where I’ve been bitten. AI generates plausible UI tests from “what the page looks like today”, and they pass once before turning brittle. Aim for tests that assert intent (what must be true), not incidental structure.

If it feels too easy, pause. Switching models, changing behaviour (not just refactoring), touching multiple concerns, or widening scope should trigger a deliberate check-in. If everything feels effortless, it’s worth assuming something important is being skipped and slowing down on purpose.

I’m also thinking about lightweight “guardrails for agents” kept close to the code (a short repo rules file). If it works, it will be because it’s short, specific, and enforced by review and tooling.

How I work right now

Sometimes I code by hand because I want to think. Sometimes I delegate aggressively because I want to think and create in parallel.

Doing this well in a team needs a bit of humour. AI will occasionally do something brilliant, and occasionally do something confidently bizarre. I would rather we can say “that was the model” without embarrassment than pretend none of this is happening. The goal is not secrecy, it is shared habits.

Make AI usage visible, or you can’t manage the risk.

The worst outcome is when people feel they have to hide it. If reviewers don’t know where AI was used heavily, they can’t dial up the right kind of scrutiny (intent, assumptions, tests, and edge cases). Keeping it in the open also lets teams get better together: what prompts worked, what failed, and what guardrails actually helped.

Working with agentic AI looks similar to how I work as a lead developer: I hand off scoped work, review what comes back, and step in to keep the delivery connected, so intent does not get lost as things move faster.

I am definitely not advocating any plan to replace engineers. I am trying to make sure their attention is spent where it pays off most.

What I’m watching next: The systems thinking gap

This model works best for people who already know how to keep a system coherent: what matters, what can break, what to validate, and what to roll back. My biggest question is how a new generation learns that judgement when the act of drafting has become near-instant.

You always had to care what happens after shipping. The difference is you now hit that part much sooner, because code is cheap.

Systems thinking is the habit of asking “what else does this touch?”, other services, data, users, operational load, and failure modes.

Part of that is learning the end-to-end lifecycle of a change. For example: Requirements → Design → Tests → Rollout → Ops

Agentic AI doesn’t remove responsibility; it concentrates it.

Delegation is a senior skill - start teaching it deliberately.

The teams that win won’t just be the ones with the best tools. They’ll be the ones that realise delegation is a senior skill and start teaching it deliberately.

Why would you go local?

• Cost - No API fees, no usage limits
• Latency - No network round-trip (faster autocomplete)
• Privacy - Code never leaves your machine
• Offline - Works on planes, in secure environments

The trade-off: you need decent hardware, and local models still lag behind the best cloud models.

The hardware reality

Running LLMs locally requires GPU VRAM. The “B” in model names (7B, 14B, 32B) means billions of parameters (what’s this?) - the learnable weights that define the model. More parameters generally means smarter, but also more VRAM needed.

A 7B and 32B model are typically trained on the same code/data - the difference is brain capacity, not knowledge.

7B (seven billion) sounds massive, but for context: GPT-4 is estimated at 1.7 trillion parameters, and even that makes mistakes. A 7B model can absolutely write correct code, but it holds fewer patterns (what’s this?) in its head simultaneously. On straightforward tasks (“write a function that sorts this list”), 7B and 32B perform similarly. The gap shows up when juggling multiple concerns at once: “refactor this authentication flow while maintaining backward compatibility with the legacy API and handling the edge case where tokens expire mid-request.”

VRAM requirements by GPU

A note on “quantized” models: (what’s this?) When you download a local model, you’ll see options like “Q4” or “Q5”. These are compressed versions that use less VRAM (roughly half) with only minor quality loss. The VRAM numbers in this article assume Q4 - if you see “FP16” or “full precision”, expect to need roughly double the VRAM. Just stick with Q4/Q5 and you’ll be fine.

Local coding models as of December 2025

Qwen 2.5 Coder - the current champion

Alibaba’s Qwen 2.5 Coder series currently dominates local coding benchmarks for realistic hardware. It is a “standard” instruct model: you ask for code, and it gives you code immediately.

For context: Claude 3.5 Sonnet scores 84.2%, while GPT-4o scores 72.9% on the same Aider benchmark.

The “Reasoning” Trap: R1 vs. Qwen

You will see DeepSeek R1 mentioned loads. Note that while R1 Distill 32B is the same size as Qwen 32B, it is a reasoning model (like OpenAI o1).

• Qwen 32B (Instruct): Fast, direct. Good for “write this function” or “fix this error”.
• DeepSeek R1 (Reasoning): Slow, verbose. It “thinks” via an internal chain-of-thought before outputting code.

If you just want to have something chosen for you and get started, use Qwen as your default coding assistant. Use R1 when you are stuck on a hard bug and need a “second brain”.

The other options

Setup: Ollama + Continue.dev (easiest path)

This is the stack I use. It effectively replaces the proprietary backend of Copilot with a local API server.

1. Install Ollama

# macOS/Linux
curl -fsSL https://ollama.com/install.sh | sh

# Windows: download from https://ollama.com/download

2. Pull a model

ollama pull qwen2.5-coder:7b    # 8GB VRAM
ollama pull qwen2.5-coder:14b   # 16GB VRAM  
ollama pull qwen2.5-coder:32b   # 24GB VRAM

Don’t overthink it.

If you only have 8GB, the 7B model is still useful. It handles autocomplete, small refactors, and “write a test for this” perfectly well.

Where you’ll notice limits is on larger changes: multi-file refactors, architectural questions, or subtle logic bugs. For those, bigger models hold context better. But you do not need 24GB to get value.

3. Install Continue.dev in VS Code

1. Install the Continue extension
2. Open settings and add your Ollama model

That’s it. Autocomplete and chat now run locally.

Realistic expectations: local vs cloud

Let’s be honest about the trade-offs:

When local makes sense

• Autocomplete - Local latency wins. A 7B model handles tab completion well, and the instant response feels snappier than cloud.
• High volume - If you’re making hundreds of requests daily, local saves real money over time.
• Privacy requirements - Regulated industries, proprietary code, air-gapped environments.
• You already have the hardware - If you have a 4090 for gaming/ML, local coding is nearly free.

Air-gapped = safe and secure?

“Privacy” is the headline benefit of local models - no data leaves your machine - and this sounds like a silver bullet. But it’s not so simple.

What air-gapping does solve

Running locally solves “where is my data going?” - your code never hits OpenAI’s servers, never gets logged by Anthropic, never crosses network boundaries. But we still have the “what’s inside the box?” problem.

The supply chain risk

Consider Qwen - currently the best local coding model. It’s developed by Alibaba, a Chinese company. For most developers, this is irrelevant. For some industries…

Backdoors: Could a model be fine-tuned to introduce subtle vulnerabilities when it recognises certain patterns? A buffer overflow here, a “sleeper” logic branch there - triggered by military-specific terminology or classified project structures.

Training data poisoning: If the training set included insecure code patterns, the model will confidently suggest those patterns as “best practices.” It’s not malicious - it’s just pattern matching on bad examples.

This isn’t paranoia - it’s standard supply chain security thinking applied to a new domain. And it’s not theoretical: JFrog researchers found around 100 malicious models on Hugging Face in 2024, including PyTorch models with working reverse shell backdoors. Simply loading the model - not even running inference - executed the payload.

Code quality risks

Even without supply chain concerns, local models have inherent limitations:

Weaker reasoning: Smaller models (under 70B parameters) don’t reason as deeply as the major cloud frontier models. They’re more likely to suggest code that “works” without considering security or stability implications.

Insecure patterns: LLMs replicate patterns from training data, including vulnerable ones (SQL injection, command injection, path traversal). The model doesn’t know these are vulnerabilities; they’re just patterns it learned.

Stale knowledge: Open-source models are frozen at release. Qwen 2.5 Coder was trained on data up to mid-2024; cloud models like Claude and GPT-4 are updated more frequently. A local model might suggest a library with a known CVE or a deprecated API because its training predates the vulnerability disclosure.

Risk comparison

Practical mitigations

If you’re using local models in secure environments:

1. Choose provenance carefully

Consider models with transparent training lineages. Llama-based models from Meta, or fine-tunes from domestic contractors, may be more appropriate than Qwen for sensitive work - even if benchmarks are slightly lower.

2. Never skip code review

LLM-generated code should go through the same (or stricter) review process as human code. Automated static analysis tools like SonarQube or Snyk (offline versions exist) can catch common vulnerabilities.

3. Sandbox everything

Test AI-suggested code in isolated containers before integration. Assume it’s untrusted until proven otherwise.

The honest take

Local models solve the data sovereignty problem. They don’t solve the “can I trust AI-generated code?” problem - that requires the same rigour you’d apply to any external code, plus awareness that LLMs pattern-match rather than reason about security.

For most secure environments, local models are viable with appropriate guardrails. But “local” doesn’t mean “safe” - it means “different threat model.”

When cloud still wins

• Complex reasoning - For architecture decisions, multi-file refactors, or debugging tricky issues, Sonnet/GPT-5 still outperforms local.
• You value your time - Setup, model management, and troubleshooting VRAM issues add friction.
• Laptop users - Integrated graphics won’t cut it. Even a MacBook Pro with M3 Pro (18GB) is limited to ~14B models.

The hybrid approach

Use both!

1. Local for autocomplete - Qwen 7B or 14B via Ollama + Continue. Instant, no cost.
2. Local for simple tasks - Bash scripts, CLI automation, file manipulation, quick utility functions. A 14B model handles these fine.
3. Cloud for complex reasoning - Multi-file refactors, architecture decisions, debugging subtle issues. This is where Sonnet/GPT-5 earns its cost.

The sweet spot: use local for the high-volume, low-complexity stuff (autocomplete, simple scripts), and cloud for the tasks where quality really matters.

Bottom line - is local realistic in 2026?

• For autocomplete: Yes, if you have 8GB+ VRAM
• For chat/reasoning: Yes, if you have a 4090 or M2 Max+
• For matching cloud quality: Not quite - but the gap is closing

Qwen 2.5 Coder 32B running locally genuinely competes with GPT-4o. That was unthinkable two years ago. At this pace, local models matching Sonnet-class quality are probably going to happen by the end of 2026.

If you’re curious, start with Ollama + Qwen 7B. It takes 10 minutes to set up and costs nothing to try.

Sources

• Qwen 2.5 Coder Technical Report - Alibaba, September 2024
• Qwen 2.5 Coder on Hugging Face
• Aider LLM Leaderboards - Paul Gauthier, updated regularly
• Ollama Model Library
• Can it run LLM? - VRAM calculator
• Malicious Hugging Face ML Models with Silent Backdoor - JFrog Security Research, February 2024

The Financial Reality Is Investor Subsidies

OpenAI

OpenAI’s financial position in September 2024, according to reporting by the New York Times:

• Monthly revenue: $300 million (August 2024)
• Projected 2024 revenue: $3.7 billion
• Estimated 2024 losses: approximately $5 billion
• Revenue growth: 1,700% since early 2023
• Funding raised: approximately $7 billion at a $150 billion valuation (reported across multiple outlets, October 2024)

That equates to roughly $1.35 lost for every $1 earned. The revenue does not cover the compute, infrastructure, research, and staffing costs required to operate frontier models at scale.

Anthropic

Anthropic’s growth follows a similar pattern. Public disclosures through 2024 show rapid enterprise adoption and aggressive fundraising, including multi-billion-dollar rounds led by strategic partners.

The company does not publish detailed profit and loss figures, but the scale and pace of investment strongly suggest ongoing operating losses as it prioritises growth, capacity, and model development over near-term profitability.

What Does This Mean for Pricing?

The prices developers see today, whether Copilot at £19 per month or Claude Pro at £17 per month when billed annually, are almost certainly below the true cost of delivery.

The pattern is familiar:

1. Subsidise usage to acquire users
2. Build habits, workflows, and switching costs
3. Raise prices once market position stabilises

We saw this with Uber. Early pricing felt implausibly cheap. It was. Once subsidies receded, prices adjusted to economic reality.

API Pricing: A Subsidised Race

API pricing makes this dynamic clearer. In 2023, a typical GPT-4 coding task (around 50K input and 10K output tokens) could cost roughly $2.00. By late 2025, comparable tasks are far cheaper.

These figures are illustrative rather than exact equivalence, but the trend is clear.

Has efficiency really improved fifty to sixty-seven times in two years? Possibly. But providers are also competing aggressively on price, with investors absorbing much of the cost. No one wants to lose ground during a land grab.

Won’t Efficiency Fix This?

Improving efficiency absolutely helps, and it is happening. But lower cost per request does not automatically mean lower total cost.

As inference becomes cheaper, it becomes easier to justify using AI more often, in more places, by more people. Features that once felt extravagant become default. Background agents appear. Always-on assistance creeps in. Total system load rises faster than unit costs fall.

Developers have seen this pattern before. Cheaper resources change behaviour. Unless usage is deliberately constrained, efficiency gains tend to increase demand rather than cap it.

The Environmental Cost (for Developers)

From a software development perspective, AI usage behaves less like running a compiler and more like making a remote call to a large, continuously operating data centre. The cost is externalised, but it is still real.

Every prompt triggers compute across clusters of GPUs, backed by power delivery, cooling, redundancy, and networking. The abstraction is clean. The system underneath is not.

Energy Consumption

The exact energy cost per prompt is not public, but the direction of travel is clear.

We often fixate on the training costs—the massive energy spike required to build a model. While those numbers are huge (Strubell et al. showed way back in 2019 that training one model could emit as much CO₂ as five cars), they are essentially a one-off ‘capital expenditure’.

The real, ongoing cost is inference.

This is the energy used every time the model answers a question. Unlike training, which ends once the model is released, inference runs 24/7. Every time you hit Tab or ask for a refactor, you aren’t retrieving a cached file; you are triggering a fresh, complex calculation on a GPU. It scales linearly with every user. It’s the difference between the energy to build a car and the fuel needed to drive it every day.

The International Energy Agency’s Electricity Report 2024 projects that global data centre electricity demand will roughly double by 2026. AI workloads are a primary driver. In some regions, additional demand is being met by keeping older fossil-fuel plants online rather than by new renewable capacity.

From a systems point of view, this is a classic scaling problem: latency and reliability expectations increase faster than clean energy supply can adapt.

Water and Cooling

Compute density creates heat. Heat must be removed.

Large data centres rely heavily on water-intensive cooling, particularly in warm or water-stressed regions. Microsoft’s 2025 Sustainability Report highlights that newer direct-to-chip cooling systems can save over 125 million litres of water per facility each year.

That figure is useful precisely because it implies how much water traditional cooling consumes.

For engineers, the key point is that cooling is not an optimisation detail. It is a hard constraint that determines rack density, data centre placement, and how fast AI infrastructure can scale at all.

Carbon Offsets Are a Lagging Fix

Major providers have made serious sustainability commitments:

• Microsoft aims to be carbon negative by 2030
• Google targets 24/7 carbon-free energy by 2030

These programmes involve real investment and real progress. They also exist because current operations generate a substantial footprint.

Offsets and renewable contracts operate at organisational timescales. Inference demand grows at product timescales. The gap between the two is where environmental cost accumulates.

What Should Developers Do?

1. Factor in Future Pricing

Assume today’s prices are temporary.

Ask yourself:

• What productivity gains are you actually seeing?
• Could your workflow survive if prices doubled?
• Are you building dependencies that would be painful to unwind?

2. Match the Model to the Task

Using a large reasoning model for trivial completions is like driving a lorry to pick up groceries.

In practice:

• Smaller models for autocomplete and boilerplate
• Mid-tier models for everyday development
• Top-tier models only for genuinely complex reasoning and large code reviews

This is a cost decision and an efficiency decision.

See my quick picks to match model to task. Updated monthly.

3. Consider Local Models

For high-volume work or privacy-sensitive code, local models can be compelling. Tools such as Qwen 2.5 Coder (32B) achieve strong coding benchmarks with zero per-query cost once running.

See my article on the current state of local coding AI models.

4. Stay Informed

AI pricing and capabilities will keep changing. What is subsidised today may not be tomorrow.

Personally, I am finding Copilot’s pricing multipliers and broad model selection useful, because they make cost explicit at the point of use, directly inside my IDE.

The Bottom Line

AI coding tools are remarkable, and remarkably subsidised.

OpenAI’s multi-billion-dollar annual losses and Anthropic’s aggressive fundraising are not a stable end state. Even with continued efficiency gains, always-on inference at scale will not be free.

At some point, the system must rebalance through some combination of:

1. Further efficiency gains
2. Higher prices or stricter usage limits
3. Consolidation as weaker players exit

None of this means developers should avoid AI tools. They offer real value today. But understanding the true costs helps you decide how deeply to depend on them.

The electricity powering your prompts, the water cooling the data centres, the billions of investor dollars underwriting each response: those are the real costs of AI. Today, you are not paying them directly. But someone is.

Sources

• New York Times, “OpenAI Is Growing Fast and Burning Through Piles of Money” (September 2024)
• Anthropic public fundraising disclosures (2023–2024)
• Microsoft Environmental Sustainability Report 2025
• IEA Electricity Report 2024
• Strubell, Ganesh, McCallum, “Energy and Policy Considerations for Deep Learning in NLP” (ACL 2019)
• Google Sustainability Reports

I was recently invited to be a guest on the Day 2 Cloud podcast (part of the Packet Pushers network), that covers DevOps and cloud-ops topics. The conversation was quite engaging, and although the episodes typically run for 40 minutes, we managed to fill an entire hour!

In the podcast, we discussed OpenTelemetry and observability, which are not only areas of personal interest but should absolutely be something we are looking at for our long-term tech strategies. For those of you who have already heard me discuss these topics, I think this podcast presents a more dynamic and in-depth exploration.

You can listen to the episode here: Day Two Cloud 192: OpenTelemetry - Getting From Visibility To Observability With Ben Hall - Packet Pushers.

Or get a link to your favourite podcast player: Day 2 Cloud (pod.link)

I hope you find the conversation insightful and enjoyable. Please feel free to share your thoughts, questions, or feedback in the comments or with me directly.

In Part 1, we looked at the observability challenges for modern systems - microservices, composable architectures, event-driven, shared components - all that good stuff solving problems well. But they’re really hard to support. We emphasised the importance of the right data being instrumented and we concluded that there are many good observability solutions, both open-source and commercial. But they all do things their own way, greatly reducing our ability to flex and adapt - letting our old opponent vendor lock-in through the door.

In this article, we’ll look at how OpenTelemetry bridges the gap between our code and the observability backend options, supporting a move away from vendor-specific tooling for generating and exporting telemetry data.

A Quick Refresher

Before we get started, let’s refresh our terminology from Part 1:

• Instrumentation is what produces the telemetry. When we talk about instrumenting, we mean to add libraries and/or code to our applications to collect telemetry such as logs, metrics and traces.
• Telemetry are insights into the behaviour or inner workings of a system – the data that the system emits about what’s happening inside the ‘black box’, collected via instrumentation. You may also hear these being called signals.

A Vendor-Neutral Standard

At its core, OpenTelemetry (OTel) is a vendor-neutral standard for telemetry across languages. You only need to instrument your code once and can then easily change to other observability backends when your needs change.

It is also means you can stick with your backend of choice while being able to change the language you use - all supported languages will adhere to the same API specification for instrumentation.

More Than A Vendor-Neutral Standard

OTel also provides a complete end-to-end implementation for generating, emitting, collecting, processing and exporting telemetry data to any supported observability back-end. For each language currently supported there core libraries, automatic instrumentation libraries, and exporters.

Core libraries

The API is the bare bones interface for instrumentation - no actual telemetry data will be sent to a backend. It’s been done this way because third party libraries wishing to instrument their code should not need to be concerned with how the consuming application implements its OpenTelemetry.

The SDK is the complete language library that provides implementations of the API so we can instrument our code manually. It is what we pull directly into our applications. It doesn’t implement exporters, which are separate libraries that take a dependency on the SDK. Alternative instrumentation are available in separate libraries implementing the API.

Typically, any application that we wish to hook up to a observability backend with OTel will take dependencies on both of these indirectly, through dependencies on instrumentation libraries and exporters.

Instrumentation Libraries

Plugin-and-go for many common libraries and frameworks, such as those for web and databases. For example, we can generate telemetry data automatically for ASP.NET Core incoming web requests and outgoing Entity Framework requests. The only steps required are adding the two packages:

• OpenTelemetry.Instrumentation.AspNetCore
• OpenTelemetry.Contrib.Instrumentation.EntityFrameworkCore

And adding them as services in code:

 public void ConfigureServices(IServiceCollection services)
{
    services.AddOpenTelemetryTracing((builder) => builder
        .AddAspNetCoreInstrumentation()
        .AddEntityFrameworkCoreInstrumentation()
        // Add your exporter(s) here
    );
}

The OTel project maintains instrumentation and exporter packages for key OSS projects and compliance with the OTel specification, in its main repository for each language. For .NET this is open-telemetry/opentelemetry-dotnet. Many others are then available from a sister contrib repository, which for .NET is open-telemetry/opentelemetry-dotnet-contrib.

Exporters

Exporters are libraries to send the instrumented telemetry data to backends. This can be through the OpenTelemetry Protocol (OTLP) itself - and that is the long-term goal*, or vendor-specific such as Jaegar or Azure Monitor, where the exporter library would translate from the OpenTelemetry data in memory, to the format used by a vendor tool. The availability of these types of exporters makes it much easier to plan and execute a transition to OTLP.

*Vendors are increasingly offering full support for direct ingress of native OTLP. See this list for updates. The Jaeger docs now even state:

As of 2022, the Jaeger SDKs are no longer supported, and all users are advised to migrate to OpenTelemetry.

Vendor-Neutral Tracing

One of the biggest challenges I’ve faced when debugging distributed systems is getting visibility the complete end-to-end journey of a transaction through all the services involved.

In part 1 we introduced distributed tracing as a key part of any observability solution to solve just this problem. We acknowledged that there are many open-source and commercial products that do this already by propagating a trace ID between all operations in a transaction to tie everything together.

But more often than not, we end up with an incomplete picture that the business just accept as a risk, despite the increased effort for support. Some common causes of this include:

• Services on mixed tech stacks require different solutions and resource constraints prevent us from instrumenting all of them
• Vendor lock-in, where our chosen backend does not support the tech stack of some of our services
• Clients and agents to instrument and send telemetry are built in-house and resource constraints prevent us from maintaining them for all service tech stacks

As a single standard for tech stacks and backends, OTel eliminates these challenges. It brings a complete set of tools that really do simply plug in, to greatly reduce the resourcing required, and because it is an open standard, very little resources are required to switch to (or add) other backends.

How OpenTelemetry Does Tracing

OTel represents each operation in a trace across a system with the concept of a span. Amongst other things defined in the OTel specification, each span will hold:

• A unique trace ID for the whole ‘journey’ of the current transaction through the system
• A unique ID for the span itself
• A unique ID for the parent span so we can reconstruct the journey in an observability backend

Each span represents a single operation in the overall transaction through a system. Most backends will represent these against time, in a way you might be familiar with from Gantt charts.

In this simplified example:

1. Span A: request comes into Service A for user details. This is the root span and of course, parent to span B. Most mainstream web frameworks will be able to auto-instrument this, meaning it will automatically create a trace span for this incoming request.
2. Span B:Service A calls Service B to get a user’s details. We might get this from a library to auto-instrument HTTP calls or we could manually add a trace signal ‘around’ the code that makes the call.
3. Span C: Service B receives the request.
4. Span D: Inside the code to handle the request, a child span represents an SQL query to the database for the user details.
5. Span E: We manually instrument a signal for adding then user if it is not found.
6. Span F: Service B has returned the user details and Service A returns that data to the caller, which is auto-instrumented by our web framework.

Vendor-Neutral Context Propagation

OTel uses the concept of baggage, which standardises the format of a shared context for values across programming languages and platforms, removing the need for teams to develop their own custom solutions for propagating shared data.

For example, if only service A sees a user’s ID, we could put in the baggage so that service B could access in when called as a child/dependency.

The Collector: A Vendor-Neutral Proxy

The star of the show in my opinion, is the collector. While most languages have exporters to send telemetry data directly to back-ends, which is fine during development, it is beneficial to offload data quickly to a collector to handle the resource intensive tasks, such as:

• Exporting to multiple places
• Enhancing with metadata
• Batching
• Encryption
• Buffering and retries when the back-ends are unresponsive
• Control flow to back-ends to allow service to continue its work

The collector can receive and export data in multiple formats, to and from observability tools using the OpenTelemetry Procotol (OTLP) and proprietary ones. It does his by acting as a pipeline through receivers, processors and exporters.

Receivers

Receivers are how telemetry signals are ingressed from sources into the collector before being processed and sent on to observability backends.

The core receiver is for telemetry using OTLP but there are many more available from the contrib repository to support vendor formats and translate these to OTLP:

• A file log receiver that will tail and parse logs from files.
• A Jaeger Receiver, if you have an existing services exporting traces to Jaegar, these can be redirected to your collector and exported to new backends alongside the existing Jaegar instance.
• A Kubernetes cluster receiver that collects cluster-level metrics from the k8s API server.

Processors

Processors can be run on data between receiving and exporting. Some examples:

• A redaction processor that can be configured to prevent sensitive information leaking into traces.
• A k8s attributes processor that can automatically tag telemetry passing through the collector with k8s metadata.
• A batch processor to reduce the number of outgoing connections used to transmit to backends.

Exporters

Similar to the libraries used directly by applications, these are used to send data to observability backends or other destinations from the collector.

As well as the core OTLP exporter, there are many available from the contrib repository

The Sidecar Pattern

Below is a snapshot of a typical use-case. We have a .NET web app exporting logs, metrics and traces using OTLP to a collector. It also shows how we might choose to transition while continuing to output file logs, pulling them into the collector using the file log receiver.

This method of deployment depicted is where a collector is deployed alongside each service on a one-to-one basis, with a shared lifetime with the service. This tends to put ownership of the collector with the development team(s) and is useful if you need to do processing specific to a service.

Our collector could then be setup as a pipeline to export those to the back-ends of our choosing.

The configuration for that pipeline would like this (complete config file):

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [otlp, elasticsearch/trace]
    logs:
      receivers: [otlp]
      exporters: [otlp, elasticsearch/log, file/rotation_with_default_settings]
    metrics:
      receivers: [otlp]
      exporters: [otlp, prometheus]

As a Single Gateway

It’s also possible to deploy the collector as standalone gateway component, to be scaled independently of the services. This tends to mean ownership of the collector is with platform teams who can then centrally manage policies and permission.

The ideal setup is for each service or collection of services to have dedicated collectors as agents, owned by the development team. These would export to an instance run as a gateway by the ops teams to apply their own processes such as authentication or metadata.

Summary

Observability solutions, both open-source and commercial, have done a good job in catching up with the demands of supporting complex modern architectures. The big unsolved problem was the lack of standards, which reduced our ability to flex and adapt, resulting in some level of vendor lock-in.

OpenTelemetry is the missing ingredient, which is set to change the landscape significantly. It is very easy to adopt with a real plug-and-play feel to it, and there is good transition support with libraries to translate between proprietary protocols and OpenTelemetry.

For the .NET readers, fire up my demo repository and have a go at using OpenTelemetry with .NET and Azure Monitor.

EDIT April 2023: Listen to me on the Day Two Cloud, a Packet Pushers podcast, talking about OpenTelemetry. Use this direct link or listen on your favourite podcast platform via pod.link.

Debugging an application in a single process tends to be easy 💪 and most of what goes wrong with our monoliths will already be recorded as known problems, with standard solutions. We may even have automated their fixes. But if not, we know there are finite routes through the application, and that tracing the cause of most problems is relatively trivial.

But the complexity of interactions in large, modern microservice architectures produces novel, one-off errors, that cross boundaries in space and time. Factor in components being built on a broad mix of tech stacks, and we have a proper challenge supporting them.

Imagine just a small number of services like those represented in the diagram below. We have a mix of synchronous REST communication and asynchronous* events. If we get an error in service H because of something that happened in service B, this would be non-trivial to investigate - without the right information.

* Synchronous HTTP calls will request and receive a response at the same time. Of course sometimes it might take a few seconds, but quick enough so that we can easily link it to any side effects it might have. Asynchronous communications, like events, won’t necessarily trigger an instant response. It could even be a day before an event added to some queue is processed! Clearly, it won’t be as easy to assign blame for some side effect to an event.

Visibility is not Observability

Observability and monitoring are two different things. We’re usually pretty good at monitoring, or if that fails, the users will shout at us. Either way, we tend to know when there is a problem. We don’t always do so well at answering the why questions.

That, for me, is the core observability problem: Why is the system exhibiting this behaviour?

And if you’re doing observability right, you should be able to answer that question without having to redeploy with additional logs or ‘prints’.

If you can only answer questions about predictable problems - that’s monitoring, not observability.

Telemetry and the Three Pillars of Observability

• Your car dashboard displaying how much fuel is left in the tank
• When you filter log output from your code in a tool like Kibana
• When you’ve looked at a histogram of events passing through your servers

These are all examples of telemetry – insights into the behaviour or inner workings of a system – the data that the system emits about what’s happening inside the ‘black box’. You may also hear these being called signals.

And instrumentation is what produces the telemetry. When we talk about instrumenting, we mean to add libraries and/or code to our applications that collects telemetry such as logs, metrics and traces.

In software engineering, we often to simplify telemetry to three distinct things:

Logs: More specifically, centralised logging – shipping all the text log entries from our components to a single data source to easily draw correlations with searches. Logs typically describe in some detail, what is happening, so are expensive to store a long time. They are not always tied to a specific user/request and thus lack context, but nonetheless, are still heavily relied on to understand system behaviour.

Metrics: Typically focusing on reliability, the type of stuff discussed for service level indicators (SLIs) and service level agreements (SLAs). As they are largely numerical data like counters and rates, they use significantly less storage than logging. This means it can be kept for longer, allowing for better analysis of trends over time. Caution though, size can increase with complexity – the cardinality – how much extra data we attach to each metric to make them more useful.

Traces: More specifically, distributed tracing. Think of this as a stack trace across multiple components (and boundaries) of a distributed or large system. It offers some of the analytical benefits of metrics but with the cardinality of logging. Distributed tracing also relies on context propagation, a component to move shared information and identifiers used to associate traces, between the services.

With tracing instrumented across all our services, properly correlated with logs, then in our fictional system we could see the route our imaginary transaction has taken, leading us to the problem service(s), which in our imagined scenario, is some dodgy data in service F and the code in service B not gracefully handling it, such that the error didn’t surface until service H.

A typical observability tool view of traces and logs, if an exception was thrown in service B after it was called from service A would be like the image below. With it, we can see exactly what call from service A made the request and drill down into it further to see associated logs and get a complete picture of the overall transaction.

We’ll look at traces in more detail for part 2, on OpenTelemetry.

Ultimately, logs, metrics, and traces are the well-established ‘3 pillars’ of observability, but there are others, like health checks. Whichever way we choose to break it down, we often have raw telemetry data in varying formats and structures that need to be processed and correlated to answer questions about the behavior of our system.

What is the Right Telemetry?

Shift away from that monitoring state of mind where you’re looking for problems you can predict, and make sure your system emits enough of the right telemetry to allow you to troubleshoot the problems you cannot foresee, because microservice architectures in a production environment will provide plenty of those! Emitting the right telemetry is what makes a system observable.

What actually stops internal users working or external customers getting what they need done?

What’s the business value of each service?

Keen an open dialogue with your colleagues in the business to understand what questions about the system matter to them. Use this to decide what is the right data to be instrumented.

This turns observability into a more powerful tool that can be use for what..if scenarios and forecasting.

The Technology Problem

We have our applications built in a mix of technologies (and more) here on the left, doing stuff we want to know about (this is intentionally getting more complicated as we purposefully shift to composed architectures and smaller microservices).

And we have all these options on the right (and more) that we might want to use to collect and analyse what the things on the left are doing.

The reality will be a bit of all of it. Some files, some custom database, maybe one team has gone their own way with elastic on prem., another in the cloud. All emitting telemetry signals but not necessarily in the same format, which means it becomes really difficult to get a joined up picture of what is going on in side the black box. For example:

• You’re not going to be able to aggregate logs and run queries across the system if they’re not using a shared format. I’ve seen log shipping agents written to reformat logs but maintaining the agents and clients to do this, across tech stacks, uses a lot of resources.
• I’ve witnessed confusion between teams, each trying to find out what they should be adding to some common context in a transaction across services owned by a number of teams. A gap that can be easily be filled by establishing a standard but standards take time and effort to build in-house.

Vendor Lock-in

It’s okay to roll our own database or file logging and utilise the abstractions provided by our language frameworks. But at scale, we need the powerful features of dedicated self-hosted or SaaS observability solutions. The problem, is that they instrument code in their own way, which makes it a non-trivial piece of work to switch to a different solution.

We call this ‘vendor lock-in’ and its effects go both ways:

• Our services all depend on the logging solution they are now plumbed into
• The logging solution will only support X and Y tech stacks, limiting our freedom to change these for our services

Sure, if you have a handful of C# web applications and that’s all you ever make, then maybe it is okay to lock into specific observability backends:

But we can’t predict the future and we don’t want to create future legacy.

Summary

To resolve problems in a large microservice system:

• We must be able to find out why they have occurred.
• For this, our system must be observable.
• To be observable, a system needs to be instrumented such that the code emits telemetry, which is typically logs, traces and metrics.
• This telemetry must be sent to a backend that supports joining up that telemetry data and answering questions about the system’s behaviour.

There are lots of open-source and commercial backends, available as self-hosted or SaaS, many of which are really good at this. The problem, is that they all do things in their own ways - there is no standard. Plumbing our logging, metrics and traces into all our services is a big task, meaning when you have done it once for any vendor, you probably won’t be doing it again any time soon, greatly reducing your freedom to be flexible and adapt.

OpenTelemetry (OTeL) is the evolution of ideas from the OpenTracing and OpenCensus projects, to build a single, vendor-neutral standard. We’ll look at this in Part 2: OpenTelemetry, The Missing Ingredient.

EDIT April 2023: Listen to me on the Day Two Cloud, a Packet Pushers podcast, talking about OpenTelemetry. Use this direct link or listen on your favourite podcast platform via pod.link.

We’re working hard in our little software engineering corner at Homes England to build a strong, permanent development capability, and to establish Digital as a directorate that can deliver larger products. We’re not doing this so that we can build everything. But rather, to avoid passing a tipping point, outsourcing more, where developers and testers either leave, or their skillsets become obsolete. Then not only do we lose the knowledge to oversee outsourced work, but we are also not able to support it.

Xiaopu Fung, who works in the Canadian public sector, captured it well:

We need to build and ship in order to keep flexing our in-house capacity muscles, otherwise we atrophy.

All three were published on Code Project and can be read through the links below:

• Introduction to GitOps. This article takes a look at GitOps — its origins, what it actually is, and how it relates to the term IaC and the pipeline concepts of CI and CD.
• GitOps with Terraform and GitHub. In this article, you will see a hands-on tutorial that shows how to take a simple Terraform plan that deploys some simple Azure infrastructure (such as VMs), and commits the code to a GitHub repository. Then, you will learn how to make changes to the code in a fork or branch, commit the changes, and do code review on the changes using a pull request.
• GitOps Provisioning with GitHub Actions. This article gives you hands-on experience in constructing a CI/CD pipeline that provisions changes that have been merged into a Terraform plan via a pull request. You will also see how to use automated checks to support the code review process in ‘gating’ deployments, and review the end-to-end process for making changes and seeing those changes in the live Azure resources.

Well, don’t panic! It is still possible to get up and running in Windows without Docker Desktop. Why? Because we’re developers with super powers and there is nothing we cannot do when we’re given the time to solve a problem.

Notice I said ‘given the time’ - just because we can, doesn’t mean we should all of the time. It’s complicated and comes with an ongoing commitment to maintaining it all - not ideal for productivity, and your security team won’t be pleased if you don’t keep everything patched.

That said, there have been occasions when Docker Desktop has not been available to me and it has been useful to know how to use Docker in Windows, nonetheless.

In this article I’ll walk you through the process that I follow with Windows Subsystem for Linux (WSL) 2 that lives in a small executable, wsl.exe, in c:\windows\system32. It will allow us to run Linux containers natively on Windows, without emulation.

Prerequisites

These are shared with Docker Desktop so, if you had previously setup Docker Desktop, you may not need to complete these.

1. Install Windows 10 version 1903 or higher or Windows 11.
2. Enable the WSL 2 feature on Windows.
3. Download and install the Linux kernel update package.

Creating a WSL 2 Distribution

1. First, install a Linux distro on WSL 2 - either from the Microsoft Store or from the command line. I’m using Ubuntu in this example.

wsl --install -d Ubuntu

2. When the installation completes, an automatic process to create a non-root user begins. Other distros may require you to configure a non-root user and sudo access yourself.

3. You should also confirm the command installed WSL 2 (not v1):

wsl --list --verbose

4. And if it did not, run the following command to upgrade it to v2:

wsl --set-version Ubuntu 2

5. Remember, you will be managing this VM yourself, so now is a good time to update the Ubuntu distro. Switch to your Ubuntu distro’s WSL 2 Bash shell:

wsl --distribution Ubuntu

And run:

sudo apt update && sudo apt upgrade

Next, install Docker manually by setting up Docker Engine and launching dockerd. Refer to the complete official guide for Ubuntu if you get stuck, but the steps below should get you started.

Setting up the Docker Repository

Continue in your VM Bash shell.

1. Install packages to allow apt to use a repository over HTTPS:

sudo apt-get install \
    ca-certificates \
    curl \
    gnupg \
    lsb-release

2. Add Docker’s official GPG key:

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

3. Use the stable repository:

echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

Installing Docker Engine on the WSL 2 VM

Continue in your VM Bash shell.

1. Update the apt package index next:

sudo apt-get update

2. For this example, install the latest version of Docker Engine and containerd with the following command:

sudo apt-get install docker-ce docker-ce-cli containerd.io

3. Test it with the following command:

sudo docker run hello-world

You have now manually installed the Docker Engine on an Ubuntu WSL 2 VM. However, since you’re managing it yourself, there’s still more to do.

Additional Setup and Challenges

To manage Docker as a non-root user, rather than using sudo to run Docker commands, you must create a group, add a user, and possibly modify some file permissions. I won’t go through the details here, but you can follow the post-installation steps for Linux if you’ve got the time to do a proper job.

The post-installation steps also offer guidance on other challenges you might face while managing your own Docker VM, such as:

• Configuring Docker to start on boot
• Logging
• Accepting connections to Docker daemon from remote hosts
• Configuring remote access
• Enabling IPv6
• Fixing IP forwarding problems

Troubleshooting: Cannot Connect to the Docker Daemon

You were warned - Docker Engine and WSL 2 VM setup is a complex operation to take on. You’re now the admin of a Linux system and responsible for solving any problems. For example, you may have received the following error when you tried to run hello-world:

You received this error because you are not working with a Debian-based distribution and must start the Docker service manually using systemctl or the service command. You probably won’t have systemctl out-of-the-box with the Ubuntu WSL distro, so use the service command:

sudo service docker start

When you rerun the hello-world test, you should now get a “Hello from Docker!”.