Ask an AI assistant to write you a Python script and it will usually run first time. Ask the same tool for a PowerShell script that audits mailbox permissions or disables dormant user accounts, and the odds shift noticeably. The script will look right. The indentation will be immaculate, the comments helpful, the structure sensible. And somewhere in the middle there will be a command that has never existed, or a parameter that was quietly retired three years ago. For anyone keeping a small business network running, the gap between “looks right” and “is right” is worth understanding, because it changes how these tools should be used.
The failure that does not look like a failure
When an AI assistant gets Python wrong, you usually find out immediately: the script throws an error and stops. When it gets PowerShell wrong, the failure is often quieter. A misspelled cmdlet or an invented parameter fails loudly enough — PowerShell rejects both outright. The dangerous cases are the quiet ones. Under PowerShell’s default settings, a filter that refers to a property which does not exist will often match nothing at all rather than raise an error, so a script that should have found forty mailboxes reports a tidy run against none. A command whose behaviour differs between Windows PowerShell 5.1 and PowerShell 7 can do exactly the wrong thing without complaint. And an -ErrorAction SilentlyContinue added to keep the output clean will suppress precisely the failure you needed to see.
Much of the best PowerShell was never published
Part of the explanation is what these models were able to read. Coding assistants are trained, at least in part, on very large quantities of publicly available code, and Python benefits from an unusually rich public ecosystem of libraries, tutorials and complete applications. PowerShell has a substantial public presence too — the language itself is open source, and the PowerShell Gallery holds thousands of modules — but a great deal of the most valuable production administration code never leaves the organisation that wrote it. Tenant migration tooling, Active Directory remediation, monitoring libraries and deployment systems sit inside managed service providers, internal IT departments and consultancies. What remains publicly visible therefore skews more heavily towards one-off snippets, forum answers and blog posts, a good many written for products and modules that have since changed considerably.
Hardly anybody keeps score
There is a second, more structural reason, and it comes down to measurement. The industry judges AI coding ability using a handful of standard tests, and those tests shape where the engineering effort goes. MultiPL-E, one of the better-known multilingual benchmarks, translated the HumanEval and MBPP tests from Python into eighteen further languages: C++, TypeScript, Shell, C#, Go, Java, Lua, JavaScript, PHP, Perl, Racket, R, Rust, Scala, Swift, Ruby, D and Julia. Shell scripting made the list. PowerShell did not. Specialist evaluations have begun to appear — a January 2026 research preprint introduced SecGenEval-PS to assess PowerShell generation, security analysis and repair, and found that more than sixty per cent of the scripts produced by GPT-4o and o3-mini were insecure without structured guidance. That is a preprint rather than settled literature, and both models tested are now a generation behind, but it measures something close to the problem described here: output that is fluent, plausible and nonetheless not safe to run. Such work carries nowhere near the visibility of the benchmarks used to advertise general coding ability, and capability tends to follow whatever is routinely measured.
Testing PowerShell is genuinely hard
There is a third factor. One contributor to recent improvements in AI coding has been training methods that run generated code automatically and reward the model when it works. That is straightforward for Python: spin up a container, run the tests, score the result, repeat at enormous scale. PowerShell itself is not the obstacle — PowerShell 7 runs perfectly well on Linux, so the language can be executed in an ordinary container. The difficulty is reproducing the environments in which useful PowerShell actually operates. A realistic test may need Windows-only modules, a populated Active Directory domain, an Exchange organisation, or a Microsoft 365 tenant with the right licences, permissions and object history. Those are slow, costly and awkward to recreate safely, millions of times over.
A decade of moving targets
Then there is fragmentation, which affects even the material that does exist. Windows PowerShell 5.1 and PowerShell 7 differ in cmdlet availability, module compatibility and the underlying .NET runtime. The MSOnline and AzureAD modules are now deprecated in favour of the Microsoft Graph PowerShell SDK, and Microsoft is explicit that migrating between them is more than renaming cmdlets — parameters, filters and returned objects can all differ. Exchange Online management has been through several generations of module with its own connection method each time. The same task therefore has several mutually incompatible correct answers depending on what is installed, and a decade of that is layered through the public record with nothing to indicate which era any given example belongs to.
How to get value from it anyway
None of this means you should stop using AI assistants for PowerShell. It means using them for the right part of the job. They are excellent at structure — error handling, logging, parameter blocks, loops, formatting output — and that is genuinely most of the tedium. Treat the specific commands as suggestions rather than facts. Verify them with Get-Command and Get-Help, bearing in mind that a freshly built server may need Update-Help before the examples are there to read. Run the finished script through PSScriptAnalyzer, Microsoft’s static checker, which will flag uninitialised variables, compatibility problems and departures from good practice before anything executes. Consider Set-StrictMode -Version 2.0 while testing, so missing properties and uninitialised variables raise errors instead of passing silently. Where it is supported, use -WhatIf so PowerShell reports what it intends to change — but treat it as a check rather than a guarantee, because it only protects operations that implement PowerShell’s ShouldProcess mechanism properly, and a wrapper script passes it down to the commands inside only if it was written to do so. Then test against a handful of known objects before pointing anything at the whole organisation.
The irony worth noticing
There is something almost comic about all this, because PowerShell is unusually good at explaining itself. The verb-noun naming is consistent, Get-Command reports exactly what is installed and reads that information from the command itself, Get-Help returns the documentation for the version in front of you, and objects can be inspected directly at the console. Much of what an assistant needs in order to stop guessing about commands, parameters and returned objects is available live on the machine — it is simply thin in the static text these tools learned from. It cannot tell them your intended business logic or how a remote service will behave, but it removes most of the invention. This is why assistants able to run commands while they work tend to produce noticeably more reliable PowerShell than those writing from memory.
Our view
For the small businesses we support, the practical lesson is about where the risk sits rather than whether to adopt the technology. Automation is how a small IT team achieves the consistency of a large one, and AI assistants meaningfully lower the barrier to writing it — we would encourage anyone to use them. But PowerShell is the language that changes things, often with elevated privileges and across every user at once, and it is precisely the language these tools are least reliable at. That combination deserves respect rather than avoidance. Keep a human review step between generation and execution, put a static check in the way, and build the habit of verifying commands rather than trusting well-formatted output. Used that way, an AI assistant is a genuine efficiency gain. Used as an oracle, it is an outage waiting for a quiet Friday.