Hiring Signal vs. Hiring Noise: What a Take-Home Actually Tells You

Two take-home submissions landed in the same week for the same role. The first was immaculate. Clean architecture, full test coverage, a tidy README, every edge case handled, the kind of submission you could drop into a portfolio. The second was rougher. It solved the core problem well, left two edge cases explicitly unhandled with a note explaining why, and the README spent most of its length on what the author would do differently with more time and which trade-off they were unsure about.

I made an offer to the second candidate. On the raw artifact, that looks like the wrong call. On the thing a take-home is actually supposed to measure, it was not close. And the reason I could tell them apart at all is the whole point of this post, because most teams are still grading the artifact, and in 2026 the artifact is mostly noise.

I have run a lot of hiring loops with take-homes in them. I have also killed take-homes from loops where they were producing nothing but false confidence. The tool is neither good nor bad. It is a measurement instrument, and like any instrument it captures the signal along with a large cloud of noise, and if you cannot tell the two apart you will hire on the noise. What follows is where the signal actually lives, why most of what you are looking at is not it, and how the calculation changed now that every candidate has a model that writes the code for them.

What people think a take-home measures

The unexamined belief is that a take-home measures whether someone can build the thing. You give them a problem, they build a solution, you look at the solution, and a good solution means a good engineer. Clean, right?

It was never quite that clean, and it is not that at all anymore. Even before AI tools were everywhere, "can build the thing" was contaminated by a dozen variables that have nothing to do with the candidate's ability to do the job. How much free time they had that week. Whether they have kids or a demanding current job eating their evenings. How much they were willing to gold-plate a throwaway exercise to impress a stranger. How closely the problem happened to match something they had built before. A polished submission could mean a strong engineer, or it could mean an average engineer with a free weekend and a similar project in their recent past. A rough submission could mean a weak engineer, or a strong one with two kids and a full-time job who spent their strictly-budgeted two hours on the parts that mattered and told you honestly about the rest.

That contamination was always there. Teams mostly ignored it, graded the artifact, and quietly selected for candidates with time and a willingness to over-invest, which is not the same as selecting for the best engineers and is often actively opposed to it. Then AI tools arrived and turned the contamination from a background problem into the entire signal.

The artifact is now mostly a measure of tooling

Here is the uncomfortable thing every hiring manager needs to sit with. In 2026, a competent engineer with a good AI setup can produce that immaculate first submission in a fraction of the time it would have taken two years ago, and a less competent engineer can produce something that looks nearly identical. The raw code, the test coverage, the tidy structure, all of the surface features teams have been grading for a decade, are exactly the things current models are best at generating.

This does not make take-homes worthless. It makes the old way of reading them worthless. If you are still scoring the submission on code cleanliness, test presence, and whether the edge cases are handled, you are now largely measuring which candidate had the better tools and prompted them well. That is not nothing, using your tools well is a real skill, but it is not what most teams think they are measuring, and it is a thin thing to hire on by itself. Two candidates can hand you near-identical polished artifacts and be a strong engineer and a weak one respectively, and the artifact will not tell you which is which.

Pretending the AI is not there does not work either. Banning it is unenforceable and also insane, because you are hiring people to do a job they will do with these tools every day. The only sensible move is to assume the code was written with heavy AI assistance, because it was, and to stop treating the code as the primary signal. Which raises the obvious question: if not the code, then what.

Where the signal actually lives

The signal was never really in the code. It was in the decisions around the code, and those are exactly the things AI does not make for the candidate. The model will write whatever it is told to write. It does not decide what to build, what to leave out, which trade-off to make, what to be honest about, or how to reason when the spec is ambiguous. Those decisions are taken by the candidate, and they are what a well-designed take-home should be surfacing.

Concretely, this is the stuff that carries signal and survives the AI era intact.

What they chose not to do. A strong engineer working against a time box makes deliberate cuts and tells you about them. "I did not handle concurrent writes because the spec did not mention them and it would have doubled the scope, here is how I would approach it if it mattered." That single sentence tells you more than a thousand lines of handled edge cases, because it shows judgment about scope, honesty about limitations, and the ability to reason about what was not asked. The candidate who silently handles everything, or worse, silently handles nothing, is not showing you judgment. The one who draws the line explicitly and defends it is.

How they handled the ambiguity you left in the spec. If your take-home is fully specified, you have removed the most valuable thing it could measure, because real work is never fully specified. A good take-home has deliberate gaps, and the signal is what the candidate does with them. Do they make a reasonable assumption and state it? Do they ask? Do they pick the interpretation that makes the problem easier and hide that they did? The interview post I wrote about senior engineers made this same point about live design, and it is even truer on a take-home, where you get to see the assumptions written down.

The reasoning in the README. This is where the real candidate is, and it is the part most teams skim. A README that walks through why this approach and not that one, what the author was uncertain about, what they would change with more time, and what they explicitly traded away is a window into how the person thinks. The model can write the code. It is much harder to fake a coherent, honest account of the trade-offs behind it, and a candidate who cannot produce that account probably does not understand the code they submitted, which is itself the most important thing you learned.

Whether the thing actually addresses the real problem. Underneath all of it, did they solve the problem that was asked, or a more impressive adjacent problem they would rather show off? Over-engineering is a negative signal, not a positive one, and it is more tempting than ever now that scaffolding a complex solution is cheap. The candidate who built the simple thing that solves the actual problem, and can explain why they did not build the elaborate thing, is showing you exactly the judgment you want on the team.

The single highest-signal move: make them talk about it

Everything above becomes far easier to read if you add one step that most take-home processes skip, and it is close to free. Do not grade the submission in isolation. Use it as the material for a conversation.

A thirty-minute call where the candidate walks you through their submission and defends their decisions collapses the entire AI problem, because you are no longer evaluating the artifact, you are evaluating the person's understanding of it. Ask them why they made a particular choice. Ask what they would change. Ask how they would extend it to handle the case they left out. Push on a decision and see if they can defend it or if they fold immediately. Point at a specific function and ask them to explain what it does and why it is written that way.

A candidate who did the thinking will light up in this conversation. They will defend the real decisions, concede the shaky ones honestly, and reason fluently about extensions because they actually understand what they built, regardless of how much of the typing the model did. A candidate who leaned entirely on the tool without understanding the output will stall the moment you go one level below the surface, because the understanding was never theirs. This is the same principle as drilling into a claimed area of depth in a live interview. You keep going down until you hit either real substance or the bottom of a script, and the take-home gives you a rich, specific thing to drill into.

There is a fairness bonus here too. The walkthrough helps the candidate with two kids and no free weekend, because a rough submission with a sharp defense now clearly beats a polished one its author cannot explain. You stop rewarding the person who had the most time and start rewarding the person who has the most judgment, which is the entire thing you were trying to measure.

This one step is what separated my two candidates. On paper the immaculate submission won easily. In the conversation, its author could not explain why several of their own choices were made, got vague when I pushed on a trade-off, and clearly did not understand parts of what they had submitted. The author of the rough submission defended every cut they had made, immediately identified the weakest part of their own design before I could, and reasoned cleanly about how they would handle the edge cases they had skipped. One of them understood their work. The other had a good weekend and good tools. The conversation made it obvious, and nothing about the artifacts did.

Design the take-home for the signal you want

If the signal is judgment and reasoning, the take-home has to be designed to surface those, and most are not. A few principles that follow directly from everything above.

Keep it short, and hold the time limit hard. A take-home that really needs eight hours is both unfair and useless. Unfair, because it favors whoever has a free weekend. Useless, because the judgment you are looking for gets buried under hours of grinding. Two to three hours, hard-capped, stated clearly, and designed so that a good candidate makes visible trade-offs to fit inside the box. The trade-offs are the point. A problem small enough that everyone finishes it completely measures nothing interesting. A problem where everyone has to cut something, and you get to see what they cut, measures exactly the right thing.

Leave deliberate ambiguity in the spec, and say that you did. Tell the candidate up front that the spec is intentionally incomplete and that how they handle the gaps is part of what you are looking at. This is not a trick. It is honesty about what the job is, and it frees the candidate to show you their judgment instead of guessing at whether they are allowed to.

Ask explicitly for the reasoning. Do not hope the README exists. Require a short write-up: what you assumed, what you traded off, what you would do with more time, what you are least sure about. You are telling the candidate where the signal is, which is fair to them and useful to you, and it makes the AI question mostly moot because you are grading the part the AI cannot do for them.

Assume AI was used, and consider leaning into it. Some of the best take-homes I have seen recently make the AI use explicit: build this with whatever tools you would normally use, and be ready to explain and defend every decision as if it were a production change you were putting your name on. That framing is honest about how the work will actually be done, and it moves the evaluation to exactly the right place, which is whether the candidate owns and understands the output, not whether they typed it.

Review it blind, and against a rubric

Even with a well-designed take-home, the review step is where good signal quietly turns back into noise, because reviewers are human and they anchor on the wrong things. Two failures show up again and again, and both are fixable.

The first is style bias dressed up as quality judgment. A reviewer opens a submission, sees a code style or a library choice that is not the one they personally favor, and marks the candidate down for it, even though the choice was perfectly reasonable and the reviewer would defend the opposite choice with equal conviction. This is not evaluation, it is preference, and it punishes candidates for failing to guess the reviewer's taste. The fix is a rubric that names what you actually care about before anyone reads a single submission. Not "is the code good," which is a vibe, but specific dimensions: did they solve the real problem, did they make and state sensible trade-offs, did they handle the spec's ambiguity well, do they understand what they submitted. When the rubric is explicit, "I would have used a different library" stops being a scoring event, because it is not on the list of things that matter.

The second failure is context leaking into the evaluation. If the reviewer knows the candidate is a senior engineer from a famous company, they read the submission generously and fill in gaps with the benefit of the doubt. If they know the candidate is a career-switcher from a bootcamp, they read the same submission looking for reasons it falls short. The work is identical; the score is not. Reviewing submissions blind, with names and backgrounds stripped, removes a bias you will otherwise apply without ever noticing you are applying it. It also has a useful side effect: it forces you to evaluate the work in front of you rather than the resume behind it, which is the entire reason you asked for a work sample in the first place.

Have more than one person review independently before they compare notes, too. A single reviewer's read is one noisy sample, and reviewers who discuss a submission before scoring it converge on whoever spoke first with the most confidence. Independent scores against a shared rubric, compared afterward, surface the disagreements that are worth talking through, and the disagreements are usually where the interesting signal is hiding.

When not to use a take-home at all

The honest final point is that for some candidates and some markets, the right number of take-homes is zero. A strong senior or staff candidate with options will often, reasonably, decline to spend their evenings on unpaid homework for one of several companies courting them, and the best of them are exactly the ones most able to walk. If your take-home is filtering out your strongest candidates while retaining the ones with the most free time, it is an anti-filter, and no amount of good design fixes that.

For senior hires especially, a live paired session where you work through a realistic problem together often produces more signal than a take-home and costs the candidate an hour of scheduled time instead of a weekend of their own. You get the reasoning, the ambiguity-handling, and the collaboration signal in real time, and you get it from candidates who would never have done the homework. Reserve take-homes for where they genuinely fit, and know that "this is how we have always done it" is not a reason to make a scarce senior candidate prove themselves on spec.

What you are really measuring

Strip it all back and a take-home was never a test of whether someone can produce code. It is a probe for judgment, and the code was only ever the medium the judgment showed up in. That was true before the models could write the medium for free, and it is the only framing that survives now that they can.

So stop grading the artifact. The clean architecture, the test coverage, the handled edge cases, those are the noise now, generated cheaply and near-identically by strong and weak candidates alike. The signal is in the decisions the candidate owns and the model cannot make for them: what they cut, what they assumed, what they traded away, what they were honest about not knowing, and whether they actually understand the thing they handed you. You surface that by leaving room for it in the design and by making them talk you through the result.

My immaculate submission and my rough one looked, on the artifact, like an easy call in one direction. On the thing that actually predicts whether someone will be good at the job, it was an easy call in the other. The whole skill of using a take-home well is being able to see that difference, and in 2026 that skill matters more than it ever has, because the artifact has never been easier to fake and the judgment underneath it has never been harder.