How's your team dealing with AI right now? It's probably 50/50.

How's your team dealing with AI right now? It's probably 50/50.
Photo by Headway / Unsplash

I was teaching a 3-day MBA class to 30+ students a few days ago. It was, as always, a great time, not just because they were from diverse backgrounds (age, experience, nationality), but also because we got to explore many challenging ideas about business and innovation together.

During this class, a key part of my curriculum is cracking, making subgroups of students crack another student's business case in real time. As we don't have time to prepare with real data beforehand, this year we finally got to use generative AI tools for quick-and-dirty market research (no real-life consequences; it's just a very realistic business game to learn and deploy tools and rewire the student mindset about innovation).

And I was a bit... surprised.

Cue this paper I read this morning:

What the paper is saying in a nutshell is as follows:

  1. GenAI doesn't lift everyone: in hard, open-ended tasks under time pressure, it pulls top performers down while raising weak ones. Scores converge at the middle.
  2. The mechanism is cognitive load inversion: GenAI removes the burden weak students can't carry, while burying strong students in output they have to filter and integrate on top of their normal thinking process.
  3. Task type is everything. Clear, structured tasks = GenAI helps everyone. Ambiguous, judgment-heavy tasks = the effect flips. Most real professional work is the second kind.
  4. Under pressure, everyone ends up copying the output. High performers intended to collaborate iteratively with the tool. They ran out of time and pasted anyway. Most used 1-2 prompts, ~15 words each.
  5. Grades now measure GenAI access, not ability. Weak students look more competent than they are. Strong ones look average. Any credential earned without a carefully staged assessment design is essentially meaningless.

In my experience, point 3. was the most obvious:

Ambiguous, judgment-heavy tasks = the effect flips. Most real professional work is the second kind.

(My words.)

What happened then?

A third of the class was misusing AI so badly it was painful to watch, a third was doing OK (not yet on top of it, though), and a last third wasn't really paying attention to the tool:

  • The ones I suspect will be the top performers in the class were not touching AI if they were senior enough. They could ballpark numbers and guestimate around the complexity of the tasks quite efficiently.
  • More junior ones were among those using AI very efficiently to push further their work and fine-tune hypotheses and results.
  • The ones that were also doing OK were part of the junior students struggling a bit more, and somehow covering lack of skills with a baseline of AI output, which wasn't even remotely interesting, but was providing passable work (i.e., they didn't squeeze the juice out of the class, but they'll get their grade).
  • A fair share of the students weren't even really trying to use AI at all (which, again, in my class was OK to do, if not entirely required, yet).
  • Lastly, the ones struggling a bit more were a mix of students clearly using AI in a business context for the first time, and a) prompting without much details, anything, and b) trusting the output blindly.

All in all, it feels like a year of transition.

GenAI is percolating into our lives at a rate far slower than what is reported or sold to us. Sure, almost everyone has access now, but very few are getting value out of it. And my non-scientific assessment is that it's what your teams are currently facing.