Getty Images; Alyssa Powell/BI
Bosses everywhere are scrambling to figure out if and how to grade their employees’ AI use.
At Gusto, a platform for human resources, managers slot workers into one of five AI archetypes during quarterly reviews. They may begin as “observers,” those who aren’t yet using AI on their own. “Integrators” have made AI part of their regular work day and see “better outcomes consistently” as a result. The most advanced are “amplifiers,” who find new ways to use AI and share what they’ve learned with coworkers.
The intent is less to rule by the carrot or the stick, and more to standardize how the company implements an emerging technology, says Scott Helmes, Gusto’s chief people officer. “We want to be able to have a coaching conversation,” he tells me. The next step is to look more directly at the impact workers make with AI. “The introduction of this, I think, has really helped us have a consistent way to talk about AI fluency across the organization.”
The employee AI grading rubric started with tokenmaxxing, but the backlash quickly arrived. Companies like Amazon and Uber tried leaderboards for tracking AI use, but have since pivoted away from maxing out AI. Duolingo walked back plans to evaluate AI use in employee performance reviews. Meta faces a lawsuit from laid-off workers alleging that the company used AI to rank workers’ productivity, including how often they used AI tools, and then singled out lower performers for cuts without considering medical and parental leaves as reasons for lower usage. The company has denied the allegations.
Workplaces can’t take macro leaps of AI adoption until they sort out what’s happening on the micro level and individual workers’ desks as they corral agents and chatbots. Attempts to suss out those who shun AI from the superusers has increasingly become part of performance reviews, but with many companies still looking to unearth AI’s potential, that can leave workers clambering to meet expectations that managers haven’t fully formed. “There’s a lot of drive to push that responsibility onto frontline workers in a way that’s not particularly fair,” says Richard Landers, a professor of industrial-organizational psychology at the University of Minnesota. Some companies have made the false equivalence of rewarding AI use even if it doesn’t achieve the goal of better work, he says. “You can’t really reward AI performance because we don’t really know what that even is yet, and it also looks very different for different jobs.”
AI might be a box to check in your next performance review. But when success with the technology remains rare and subjective, it’s an elusive category to ace.
Companies have largely dropped subscriptions and logins to AI systems onto workers and urged them to figure out how to best use them. The lack of training and brute force adoption has led to a deluge of workslop, security risks, and a confusing sprawl of sprawl workflows. Yet, managers increasingly want to evaluate how people use AI. General Assembly, the tech skills training provider, found in a recent survey of 500 business leaders in the US and UK that nearly half had incorporated AI into employee reviews, looking particularly at reports of AI tool usage, improvements to performance attributable to AI, and anecdotes about improved efficiency.
Much of that is subjective, and unlike past innovations where companies bought software for specific use cases and trained workers on it, there’s an expectation at some companies that the workers lead.
Without strict quantitative benchmarks like tokens spent, lines of code generated, prompts written, and agents deployed, many evaluate workers’ AI use by tracking that they’re doing more of what they did before. Stefan Camilleri, vice president of engineering at the software company Typeform, says he asks his engineers: “Now that I’ve given you a new tool, how much faster are you doing it?” But speed alone doesn’t cut it. “I’m expecting more ambitious output from fewer people.” Shensi Ding, the cofounder and CEO of the AI software company Merge, says she reviews employees based on the quality of their AI use. For example, someone in accounting automated payment tracking and increased the team’s productivity, which Ding saw as highly effective. The highest praise comes for those who have figured out a good use case and passed it along to their colleagues. “If you’re teaching other people, then that’s a level above what we would expect, and it’s such a big impact and compounds over time in the business that people are really rewarded for that,” Ding says.
For startups, there’s pressure to do more with fewer people equipped with AI, but large companies are also now closely evaluating workers’ AI use. Meta announced to employees late last year that it would add “AI driven impact” as a core part of their performance review process. Accenture reportedly began monitoring AI logins when considering workers for top-level promotions earlier this year. My colleague Hugh Langley reported earlier this year that non-technical employees at Google were told that AI use could show up in their performance reviews. The company says that managers can evaluate AI proficiency in reviews, but it’s not a requirement.
Many companies are still in an experimental phase, focusing on how AI can add value rather than who uses the most. Lee Senderov, chief transformation officer at Travelport, a retail platform for travel agencies, says her company has built a dashboard for engineers that uses AI to analyze code and determine the tools used to write it. Managers can see if an engineer isn’t using particular AI features, and then use those insights to have conversations about what particular workers do or don’t understand, and how they’re coding. AI, Senderov says, is case by case for different functions, and the mandatory push or rollout may not lend itself as well in some places as others. “We want this to happen organically and happen through inspiration,” she says.
Across corporate America, the lack of clarity over what AI materially means for performance reviews has led evaluations to feel uneven and wonky. A July Harvard Business Review article called for reviews to focus not on output, but on how workers can evaluate the accuracy of AI and then use judgement to override errors, if a worker uses AI to increase the productivity of their team, and how well they adapt to new technology, policy changes, and workflows.
Many companies haven’t standardized these changes. A January Deloitte report found that 84% of companies have not redesigned work to meet the capabilities of AI; the emphasis remains on educating workers and pushing them to experiment with AI, rather than altering workflows and career paths to match the tech changes. “Leaders are realizing that this usage conversation isn’t having the impact that they were looking for,” says Lori Moffatt, managing director of executive recruitment and company culture advisory firm Waterstone’s Leadership and Culture Advisory Services. Anyone can prompt AI or find a task ripe for automation, fewer have found real problems that merit an AI solution. “AI is meant to be transformational. And yet managing it the ways that they’re managing today is not creating the transformation that they’re looking for.”
Marc Cenedella, founder of the career site Ladders, says that while AI advances rapidly, people need more time to fully discover and stretch its technical feats into meaningful work. He encourages workers to experiment with AI, to fail — even if one day we look back and see these early efforts as “goofing around.” “It kind of doesn’t matter if it’s useless or useful at this point,” Cenedella says. If AI had obvious benefits, industries would streamline it. “If this was measurable, that would mean it’s not an amazing technology.”
Amanda Hoover is a senior correspondent at Business Insider covering the tech industry. She writes about the biggest tech companies and trends.