RepDex
Detectors

AI Code Detectors: Can Anyone Really Tell Who Wrote the Code?

RDRepDex Editorial Team
12 min read
Share:

Somewhere in the last two years, a quiet panic settled over two very different rooms. In one room sit computer science instructors, staring at a batch of submitted assignments that all somehow arrive at the same elegant solution, and wondering how many of their students actually wrote the loops in front of them. In the other room sit engineering hiring managers, holding a stack of flawless take-home tests and whiteboard follow-ups that don't quite match the polish of the submitted code, wondering whether the candidate they're about to hire can actually do the job. Both rooms are asking the same question, and both are hoping there's a tool that answers it: can anyone really tell who wrote the code — a person, or a machine?

The uncomfortable, honest answer is that code is the single hardest thing to detect AI authorship in. Not moderately harder than essays. Categorically, structurally harder — hard in a way that isn't going to be solved by a better model or a cleverer heuristic, because the difficulty lives in the nature of code itself. If you have spent any time reading confident marketing copy about "AI code detection," this article is going to feel like cold water. That's intentional. We think the interesting story isn't which detector to trust; it's why the whole category struggles, and what educators and employers should be doing instead of chasing a number.

Why prose leaves fingerprints and code doesn't

To understand why code detection fails, you first have to understand why prose detection sometimes works — and even that "works" deserves heavy air quotes. Human writing is astonishingly high in what you might loosely call stylistic entropy. When two people describe the same sunset, they will not produce the same sentence. They won't even produce sentences that are structurally similar. One writer front-loads a subordinate clause, another leans on short declaratives, a third can't resist a semicolon. There are effectively infinite valid ways to express any given idea in natural language, and each writer occupies a small, idiosyncratic slice of that space. That variability is the raw material detectors feed on.

Now ask two competent programmers to write a function that reverses a linked list. Here is the problem: there are only so many correct ways to do that. The algorithm is fixed. The idiomatic implementation in a given language is close to canonical. Both programmers will declare a prev pointer, walk the list, flip the next references, and return the new head. They might name a variable current versus node, but the skeleton is nearly identical because the problem constrains the solution. Correct code converges. Correct prose diverges. That single asymmetry is the root of nearly everything that follows, and it's the thing detector vendors are least eager to talk about.

Code is written to satisfy a compiler and a specification, not to express a personality. Syntax is rigid — a misplaced brace isn't a stylistic choice, it's a build error. Naming conventions are enforced by linters and team style guides that deliberately flatten individual quirks. The whole culture of software engineering pushes toward convergence: we teach juniors to write code that looks like everyone else's code, because readable, idiomatic, conventional code is the goal. In prose, sounding like everyone else is a failure. In code, it's the point. So the very forces that make writing detectable — personal idiosyncrasy, unconstrained choice, stylistic signature — are the forces that professional software culture spends enormous energy trying to eliminate.

The perplexity problem, now in a language machines actually speak

Most text-based AI detectors lean, one way or another, on a statistical idea called perplexity: roughly, how surprised a language model is by the next token in a sequence. AI-generated text tends to be low-perplexity — smooth, predictable, hugging the high-probability path — because that's exactly what the generating model was optimizing for. Human text tends to be spikier, with unexpected word choices and uneven rhythm. If you want the full picture of how this machinery works, we've written about perplexity, burstiness, and watermarks separately, and it's worth reading before you trust any detector's verdict.

Here's what happens when you point that same machinery at source code. Code is already low-perplexity, no matter who wrote it. Programming languages have a tiny vocabulary compared to English, a rigid grammar, and enormous local predictability. After you type for (int i = 0; a model can guess the rest of the line with near-certainty, because there are only a handful of things that legally and idiomatically come next. Well-written human code and AI-generated code both live in this low-perplexity basement. The statistical gap that detectors exploit in prose — the difference between "surprising human" and "smooth machine" — collapses almost to nothing when the medium is code. Both look like the machine, because good code is the predictable path.

It gets worse. The large language models that write code today were trained on essentially the entire public corpus of human-written code — GitHub, Stack Overflow, open-source projects, tutorials. When a model like the one behind GitHub Copilot suggests an implementation, it is reproducing patterns learned directly from millions of human programmers. The AI isn't generating alien code that a detector could flag as unnatural; it's generating the most human-typical code possible, because typicality is what it learned to reward. A detector trying to separate the two is looking for a difference that the training process deliberately erased. The AI's whole competitive advantage is that its output is indistinguishable from a skilled human's. Detection is asking it to fail at the one thing it's best at, and it won't oblige.

What the few code detectors attempt — and where they crack

There are tools that claim to detect AI-written code. Some are extensions of general-purpose detectors like GPTZero that were built for prose and now offer a code mode; others are purpose-built academic prototypes. Broadly, they try a few strategies, and it's worth walking through each because the failure modes are instructive.

The first strategy is to apply perplexity scoring directly to the code tokens, treating the source like text. We've already covered why this drowns: the baseline perplexity of legitimate human code is so low that the signal-to-noise ratio is hopeless. A tidy human implementation and a Copilot completion produce nearly identical scores, and the detector is left guessing.

The second strategy is to look at "meta" signals — the stuff around the code rather than the logic itself. This is where things get slightly more interesting and a lot more fragile. AI-generated code often arrives with a particular texture: unusually thorough docstrings, comments that explain the obvious (# increment the counter above counter += 1), consistent and slightly over-formal naming, exhaustive edge-case handling that a rushing student wouldn't bother with, and a suspiciously clean structure with no dead code, no commented-out experiments, no half-finished refactor. These are real tendencies. They are also trivially defeated — strip the comments, rename a few variables, introduce one messy shortcut, and the tell evaporates. They also produce brutal false positives against exactly the people you'd want to reward: the meticulous student who writes clean, well-commented code by habit looks maximally guilty. A signal that punishes good practice is not a signal you can build policy on. If you want to understand why this class of tool generates so many wrong accusations, our piece on AI-detector false positives lays out the mechanism in detail.

The third strategy is training a classifier on paired datasets — human code versus AI code — and hoping it learns some latent boundary. This overfits ferociously. A model trained on Python solutions to introductory exercises may hit respectable accuracy on that exact distribution and then collapse the moment it sees a different language, a different problem domain, a different AI model, or code that's been lightly edited by a human after generation. And "lightly edited by a human after generation" describes the overwhelming majority of real-world AI-assisted code. Almost nobody pastes raw Copilot output and ships it untouched. The realistic scenario — human and AI collaborating, tab-completing, accepting some suggestions and rewriting others — is precisely the scenario these classifiers were never trained on and cannot untangle. The clean binary of "human wrote this" versus "AI wrote this" doesn't survive contact with how people actually use tools like Cursor and Copilot, where authorship is genuinely, irreducibly blended. There is often no ground truth for the detector to find, because the honest label is "both."

Plagiarism detection is a different animal, and the confusion is dangerous

A lot of the confidence around "detecting AI code" is borrowed, illegitimately, from a mature and genuinely effective technology: code similarity detection. Tools like MOSS (Measure of Software Similarity), which has been used in CS departments for decades, do something real and well-understood. They compare submissions against each other and against a corpus, looking for structural overlap — the same logic with variables renamed, the same unusual algorithmic choice appearing in two supposedly independent submissions, the same distinctive bug reproduced verbatim. MOSS is robust precisely because it ignores surface changes and compares the underlying structure. When it flags two submissions as suspiciously similar, that flag means something concrete and checkable.

But notice exactly what MOSS answers. It answers "is submission A structurally similar to submission B (or to something in this reference set)?" It does not answer "was this written by an AI?" Those are different questions, and conflating them causes real harm. If ten students in a class each independently ask ChatGPT to solve the same assignment, the AI may hand them ten similar solutions — and MOSS will correctly light up, because the submissions really are similar to each other. That's a genuine signal worth investigating. But if one student uses AI and produces a solution that resembles nothing else in the class, MOSS sees nothing, because there's nothing to match against. And if a student writes an original solution by hand that happens to converge on the canonical approach — remember, code converges — MOSS might flag them against a classmate who did the same thing honestly.

The distinction that matters is similarity versus authorship. Similarity detection is comparative and grounded: it needs at least two things to compare and it tells you they overlap. AI-authorship detection is absolute and speculative: it looks at one artifact in isolation and tries to divine its origin. The first is a solved, useful problem. The second is the hard problem this entire article is about. When a vendor cites decades of plagiarism-detection heritage to sell you an AI-code detector, they are trading on the credibility of a tool that answers a completely different question. Keep the two firmly separate in your head, because institutions that blur them end up making authorship accusations backed by evidence that only ever spoke to similarity.

The classroom reality: auto-graders, honor codes, and the temptation of a magic number

Walk into a modern CS course and the pressure is obvious. Enrollments are large, staff is thin, and assignments are graded by automated test suites that check whether the code produces correct output. Auto-graders are wonderful for scale and terrible for provenance: they were built to answer "does this code work?" and they have no opinion whatsoever on "did this student write it?" A submission that passes every test tells you the solution is correct. It tells you nothing about who — or what — produced it. Into that gap, an AI-code detector promising a percentage looks like salvation. Paste the submission, get a number, done.

Please resist this. The number is not reliable enough to sit at the center of an academic-integrity process, and academic integrity is exactly the context where a wrong answer does the most damage. A false positive here isn't a minor annoyance — it's an accusation of cheating against a student who did the work, with consequences that can follow them through a program. Given everything above about how thin and defeatable the underlying signal is, staking a disciplinary case on a code detector's output is indefensible. The tool cannot distinguish the diligent student who writes clean, commented code from the one who pasted it, and it cannot see the blended reality of a student who used AI to scaffold and then genuinely understood and rewrote the result. If you want the general grounding on what these systems can and can't do before you set policy, how AI detection actually works is the right starting point.

The instructors getting this right have mostly stopped trying to catch AI at the artifact and started designing courses where the artifact isn't the whole story. They weight in-person or proctored components. They ask students to explain their own code aloud, or to extend it live in ways that require actual understanding. They use version-control history to see whether a solution grew organically through commits and debugging or arrived fully formed in a single paste. They build assignments where the reasoning and the iteration are the graded object, not just the final passing submission. None of this is a detector. All of it is more effective, and none of it produces false accusations against honest students, because it's measuring understanding directly instead of trying to reverse-engineer origin from surface statistics.

The hiring reality: take-homes, HackerRank, and the arms race nobody wins

Hiring has the same disease with higher financial stakes. The take-home coding assignment and the timed platform test — HackerRank, Codility, and their peers — were already imperfect signals of engineering ability before generative AI, and they've become dramatically noisier since. A candidate working from home on a take-home has unrestricted access to Copilot, ChatGPT, Cursor, and every tool a working engineer uses daily. The clean submission you receive may reflect the candidate's skill, or the AI's, or — most likely and most confusingly — some genuine blend where the candidate directed the tool competently, which is itself a real and valuable skill.

Some platforms have bolted on "AI detection" and proctoring features in response, and it's worth being blunt about the incentives: they are selling reassurance to nervous buyers, and reassurance sells even when the underlying detection doesn't work. The same structural problems apply — code converges, perplexity signals are meaningless on source, blended authorship has no clean label — with the added twist that experienced engineers are exactly the population most able to make AI-assisted code look hand-written, because they know what hand-written code looks like. The detector is best at catching the unsophisticated and worst at catching the sophisticated, which is precisely backwards from what you'd want. We've written more broadly about the gap between what employers fear and what they can actually verify in our look at whether employers use AI detectors, and the code case is the sharpest version of that gap.

There's also a deeper strategic question hiding here that the detection framing obscures entirely. In 2026, an engineer who uses AI tools well is often a better hire than one who refuses to, not a worse one. Real engineering work now happens with these tools in the loop. A take-home that a candidate aced using Copilot may be telling you something genuinely useful about how they'll perform on the job, where Copilot will be sitting right there in their editor. Trying to detect and penalize AI use in the assessment can mean penalizing exactly the modern workflow you'll want them to bring on day one. The question worth answering is not "did they use AI?" but "can they reason about the problem, defend their choices, debug under pressure, and produce working systems?" — and no static analysis of a code artifact answers that. Only interaction does.

The tells that exist, and why leaning on them is a trap

Because someone always asks: yes, there are soft signals. AI-generated code does have tendencies, and an experienced reviewer will sometimes get a hunch. The comments are a common one — machine-generated code frequently over-explains, narrating trivial operations with comments no working engineer would bother writing, or producing docstrings of uniform, slightly airless thoroughness. Variable naming can skew toward the generic-but-correct: result, data, temp, output, technically appropriate and utterly personality-free. The code is often too clean — no scaffolding left behind, no commented-out dead ends, no idiosyncratic helper the author clearly wrote in a previous project and pasted in, none of the archaeological layers that real human development deposits. Sometimes there's a mismatch between an ambitious, sophisticated solution and a simple prompt, or between polished code and a candidate who can't explain a line of it.

Every one of these is real, and every one of these is a terrible foundation for a decision. They are probabilistic hunches, not evidence. They punish tidiness and reward mess, which is exactly the wrong incentive for a craft that prizes clean code. They are trivially defeated by anyone who knows they exist — and the moment "AI code has generic variable names" becomes common knowledge, people generating AI code start renaming variables, and the tell dies. They produce false positives against naturally meticulous humans and false negatives against anyone making the slightest effort to obscure. A hunch is a fine reason to look closer and a catastrophic reason to accuse. The correct use of these signals is as a prompt to have a conversation — to ask the author to walk you through their reasoning — never as a verdict delivered to the author about themselves.

Where this leaves the honest observer

The through-line of every honest answer in this space is the same, and it's frustrating for anyone who wanted a tool: you verify authorship through process and interaction, not through post-hoc analysis of the finished artifact. Code is the domain where this is most starkly true, because code is the domain where the finished artifact carries the least information about its origin. The statistics that catch AI in prose are nearly silent on source code. The plagiarism tools that genuinely work answer a different question — similarity, not authorship. The soft tells are hunches that punish good practice and dissolve the moment they're known. There is no number you can paste your way to.

What actually works isn't glamorous and isn't a product. In the classroom it's proctored components, oral defenses, version-control archaeology, and assignments designed so that understanding — not just a passing submission — is the graded object. In hiring it's live technical conversation, pair-programming sessions, asking candidates to extend and debug their own submitted code in real time, and questions that probe reasoning rather than output. In both cases the shift is from interrogating the artifact to observing the person, from "prove this wasn't AI" to "show me you understand this." That shift also happens to be better pedagogy and better hiring on its own merits, independent of AI, which is a strong hint that it's the right move rather than a reluctant workaround.

If you came here hoping repdex.net would point you to the one code detector that finally cracks the problem, we won't do it, because it doesn't exist and pretending otherwise would be exactly the dishonesty this site exists to push back against. The panic in both rooms — the CS department and the hiring committee — is real, and the anxiety underneath it is legitimate. But the answer to "can anyone tell who wrote the code?" is not a better scanner. It's a better question, asked directly to the person, in a setting where they have to answer it themselves. The code will never confess. The author, given the right conversation, usually will — one way or the other — and that conversation is the only detector that has ever actually worked.

Frequently Asked Questions

Can AI code detectors actually tell if code was written by ChatGPT or Copilot?+
Not reliably. Code is the hardest domain for AI detection because correct code converges on a small number of idiomatic solutions, and source code is inherently low-perplexity no matter who writes it. The statistical gap that detectors exploit in prose nearly disappears in code, so human-written and AI-generated code look statistically almost identical. Any tool claiming confident code-authorship detection is overselling a signal that barely exists.
Isn't MOSS an AI code detector?+
No. MOSS (Measure of Software Similarity) is a code similarity detector, not an AI-authorship detector. It compares submissions against each other and a reference corpus to find structural overlap, answering 'is this similar to that?' It does not answer 'was this written by an AI?' Those are different questions. MOSS may catch multiple students who all used the same AI and got similar output, but it sees nothing when a single student produces an original-looking AI solution.
Why is detecting AI-written code harder than detecting AI-written essays?+
Human writing has high stylistic entropy: there are effectively infinite ways to express an idea, and each writer occupies an idiosyncratic slice of that space. Code is the opposite. Syntax is rigid, algorithms are fixed, linters and style guides deliberately flatten individual quirks, and professional culture pushes everyone toward the same idiomatic patterns. Code is written to satisfy a compiler, not to express personality, so it leaves far fewer authorship fingerprints than prose.
What signals do people use to guess if code is AI-generated?+
Common soft tells include over-explanatory comments narrating trivial operations, uniformly thorough docstrings, generic-but-correct variable names like result, data, and temp, suspiciously clean structure with no dead code or half-finished experiments, and a mismatch between polished code and an author who can't explain it. These are real tendencies but terrible evidence: they punish tidy, well-commented work, produce false positives against meticulous humans, and disappear the moment someone renames a few variables.
If detectors don't work, how should schools and employers handle AI-assisted code?+
By verifying through process and interaction rather than analyzing the finished artifact. In classrooms that means proctored components, oral code defenses, version-control history review, and assignments where understanding is the graded object. In hiring it means live technical conversation, pair programming, and asking candidates to extend or debug their own submitted code in real time. These approaches measure understanding directly, avoid false accusations, and happen to be better pedagogy and hiring regardless of AI.

Related Articles