This is definitely going to require changes in interview practices for a lot of places. For those looking for an alternative, may I suggest pair programming?
I've been doing pairing interviews for years. These days I have a standardized, practical problem, something that's reasonably like the work. (E.g., let's use APIs A and B to build features X, Y, and Z.) I let them pick their preferred language and tooling, so that I can see them at their best. And then we spend a fixed period diving in on the problem, with me getting them started, answering questions, and getting them unstuck.
I like this because not only do I get to see code, I get to see how they think about code, how they deal with problems, and how they collaborate. They get to spend time building things, not doing mensa puzzles, posturing, or other not-very-like-the-work things. And they can't bluff their way through, and it's pretty hard to cheat.
If some places need to change their interview practices, I think it means they've been doing it wrong all this time. You ask them to write some code, then to explain and justify it. Whether or not it has been mostly written by an LLM really shouldn't matter... (leaving aside edge cases preventing to use some tools)
The emergence of this problem is exactly why I don’t like giving people programming exercises that are overly explicit.
Most coding tests just tell people what to write and then have them write it. Real world problems are more complicated. Instead, tell your candidates what your problem is and then ask them for a solution. Let them write their own requirements. It’s a lot harder for language models (and developers with poor problem solving skills) to solve these kinds of questions as well.
Exactly. At points during a pairing interview when I get asks for more details on requirements, I'll make sure to reply with something like, "Which do you think is better for the user?" It turns out most developers have a pretty good sense for this, even if they're not used to being able to use it.
I've been doing pairing interviews for years. These days I have a standardized, practical problem, something that's reasonably like the work. (E.g., let's use APIs A and B to build features X, Y, and Z.) I let them pick their preferred language and tooling, so that I can see them at their best. And then we spend a fixed period diving in on the problem, with me getting them started, answering questions, and getting them unstuck.
I like this because not only do I get to see code, I get to see how they think about code, how they deal with problems, and how they collaborate. They get to spend time building things, not doing mensa puzzles, posturing, or other not-very-like-the-work things. And they can't bluff their way through, and it's pretty hard to cheat.