The Pulse #146: How AI is changing tech interviews

Before we start: we are doing research on MCP server best pratices. Do you use MCP servers at work, or did you build one/more? We’d like to hear from you. You can send details via this form. We’ll share findings with people sending details – and, of course, in the newsletter as well, at a later date. Thanks for your help!

The Pulse is a series covering events, insights, and trends within Big Tech and startups.

Today, we cover:

1. How AI is changing tech interviews. New details about harder questions for candidates, growing cheating concerns, and the rise of AI-assisted interview processes, from 63 engineers in Big Tech scaleups, and startups, who are interviewers on the mock interviewing platform, interviewing.io.

2. New trend: devs vibe coding internal tools. An increasingly popular use case among software engineers is using AI agents to vibe code simple tools that would normally take hours or days to write.

3. Industry pulse. Security nightmare at npm, Microsoft joins the “three days in the office” club, Klarna goes public, Meta to close down small engineering offices, and more.

1. How AI is changing tech interviews

The team at interviewing.io has surveyed interviewers on their platform from Big Tech and startups; asking how AI is changing interviews in their workplaces. They received 63 detailed responses, and this newsletter is the first to publish the findings.

Special thanks to Aline Lerner, who shared data with me. Aline is founder of interviewing.io, co-author of Beyond Cracking the Coding Interview, and creator of the Whiteboard Confidential Podcast.

Demographics

Most respondents work in Big Tech (Apple, Google, Meta, Microsoft or Netflix), or in New Big Tech (Uber, Stripe, DoorDash, and others), while others work at startups:

How big a problem is cheating in remote interviews? According to the survey responses, it is widespread:

• 81% of interviewers at Big Tech and New Big Tech said they’ve suspected candidates of using AI tools to cheat during interviews

• 31% of interviewers at Big Tech and New Big Tech have caught a candidate cheating in an interview; that is, definitely using AI tools to provide answers which they claim as their own

Meta is the only Big Tech making company-wide changes to combat cheating, based on this research. From a Meta interviewer at interviewing.io:

“Cheating prevention is pretty front-and-center at Meta right now. We now have to mark whether we suspect a candidate of cheating across nearly all interview types/levels, and provide justification if so. Previously, it was only coding interviews. We also are requiring candidates to share their entire screen and turn off all background filters (including blur) in most interviews as well.”

Another interviewer at Microsoft mentioned that their team is taking steps to detect cheating tools, but it’s not company-wide. Other interviewers said their workplaces have not changed anything. One engineer at a New Big Tech giant said:

“My company is moving slowly, despite numerous incidents of cheating and other issues being raised.”

So, why is AI cheating a matter of concern for interviewers, but many companies don’t respond? My take is that this is down to a few things:

• At Big Tech, changing company-wide processes like interviewing is slow. Until there are major incidents around cheating, there’s probably no trigger for change

• Most Big Tech interviews have worked the same way for 15 years, so reform is inevitably sluggish

• Counter-intuitively, it might be easier to bring back in-person interviews than to roll out “anti-cheating” policies. Google is likely getting ready to do more in-person interviews because more candidates are using AI tools in interviews, and Google wants to ensure that “the fundamentals are there”, as CEO Sundar Pichai said in June

Big Tech interview questions really are tougher

Meanwhile, Interviewers seem to be taking matters into their own hands and battling cheats with AI tools by asking harder questions. 58% of Big Tech interviewers said they have adjusted the kind of questions they ask in an effort to combat potential AI “cheat tool” usage. Here’s what survey respondents at various companies told us (each point is a single interviewer’s approach):

Meta:

• More open-ended questions which probe thinking, rather than applying a known pattern

• Add variation to popular questions because this often catches cheats off guard – at least, for now it does

• Ask questions with much longer descriptions and more complicated setup steps, which are harder for AI tools to perform well at

Microsoft:

• Using coding problems that are more complex, to which LLM solutions often contain clues that reveal their usage

• Asking questions that require candidates to explain concepts fluently, for which reading an AI-summarized answer won’t work as well

• Testing how candidates can expand on existing logic using class structure, instead of