How to Harness the Power of Generative AI for Creative Problem-Solving at Work

Generative AI tools like ChatGPT have taken the world by storm with their ability to quickly generate ideas, content, and solutions. But are they really the creativity silver bullet many are hoping for in the workplace? A recent experiment dug into this question, and the results were eye-opening.

The study, conducted by researchers Kian Gohar and Jeremy Utley in partnership with four U.S. and European companies, put AI-assisted creative problem-solving to the test. Teams at each company were tasked with brainstorming solutions to a real business challenge, like improving B2B sales or developing better training resources. Some teams, the control group, tackled the problem without any AI assistance. Others, the experimental group, were given access to ChatGPT to aid their ideation process.

Going in, the researchers expected the AI-assisted teams to blow the others out of the water in terms of both quality and quantity of ideas generated. But that’s not quite what happened. The teams using ChatGPT did produce 8% more ideas on average. However, when the ideas were evaluated by the “owner” of each business problem, the AI-assisted teams actually got 2% fewer top-tier “A” ideas scored as “highly compelling.” They got 8% more middle-of-the-road “B” ideas (“interesting but needs development”) and about the same amount of “C” ideas (“needs significant development”). The AI did help them come up with fewer total duds — 7% fewer “D” ideas that were “not worth pursuing.”

So what happened? Why didn’t having a superhuman brainstorming partner translate to vastly better results? There are a few key factors at play:

1. The “average answer” effect. The large language models underpinning tools like ChatGPT are designed to identify the highest probability combination of words and spit out the “most likely” response. Ask it to finish the sentence “I bark like a…” and it will say “dog” every time. This is great for predictable queries, but not so much for creative problem-solving where out-of-the-box thinking is required.

2. Lack of context. Generative AI doesn’t have the deep contextual understanding of your company, industry, and unique challenges that comes from months or years on the job. It can only work with the information it’s given. Many teams didn’t spend enough time “training” the AI on relevant background information before setting it loose.

3. The oracle fallacy. People often expect generative AI to be an all-knowing oracle, able to divine brilliant solutions from minimal input. Teams that simply fed the tool their broadly worded problem statement and expected it to spit out the perfect answer were sorely disappointed.

4. Lack of back-and-forth. The teams that got the best results engaged in multiple rounds of interaction with ChatGPT — refining, questioning, and building upon its initial suggestions. But most teams simply took the first response and ran with it. They succumbed to the Einstellung effect, a cognitive bias that makes us latch onto early, familiar answers instead of exploring further.

5. Overconfidence. Surveys showed the AI-assisted teams gained 21% more confidence in their creative problem-solving abilities compared to the control group. But clearly, much of that confidence was misplaced, given the lackluster results. A little knowledge can be a dangerous thing.

So does this mean generative AI tools are useless for creativity and ideation in a business setting? Far from it. But it does mean teams need to learn how to leverage them effectively. Here’s how:

First, get specific about the problem you’re trying to solve. Don’t settle for a vague, high-level description. Spell out the context and criteria in detail. Instead of “How can we improve customer satisfaction?” try something like “Our customer onboarding currently involves steps X, Y, Z. What changes can we make to step X to improve retention by 10%?” The more context you can feed the AI model, the more relevant and useful its suggestions will be.

Second, carve out time for team members to brainstorm individually, without the AI, before convening as a group. This ensures you’re collecting a wide range of original ideas instead of just echoing the chatbot’s first take. Diversity of thought is key.

Third, treat the AI as a conversational thought partner, not an oracle to consult once for the “right answer.” Question its suggestions, point out drawbacks, and ask for refinements and alternatives. The more you iterate, the more it will learn and the richer the possibilities it will generate. But this takes patience, practice, and a willingness to dig deeper.

Finally, when it comes time to converge around a solution, designate a team member (or better yet an outside facilitator) to consolidate the ideas the team has generated with and without AI assistance. Then prompt the AI to look for common themes, spot potential issues, poke holes, and suggest ways to combine and build upon the ideas. This process of exploring and evaluating ideas from multiple angles is where the real creative magic happens.

Of course, none of this is easy. Getting the most out of generative AI for creative problem-solving requires rethinking old processes and learning new skills. We’re in uncharted territory and still very much in the learning phase of human-AI creative collaboration.

One thing is clear: slapping ChatGPT onto the same old brainstorming approach and expecting transformative results is a recipe for disappointment. As Joe Riesberg, CIO of EMC Insurance, put it after participating in the experiment, “The immediate efficiencies from the technology probably won’t be as impressive as people hope. But the improvements from iterating with it — speed, productivity, creativity — will be tremendous in the long term.”

The key is to approach generative AI tools not as a crutch or an easy button, but as a powerful partner in a creative conversation. With the right structure, skills, and mindset, companies that learn to leverage these tools effectively will be able to take their ideation and problem-solving capabilities to a whole new level. In a rapidly changing world, that creative edge is only going to become more valuable. But it won’t happen magically overnight just by plugging in a chatbot. As with any new technology, realizing its full potential will be a journey of experimentation, learning, and growth.

For more information about the study — Check out the March-April 2024 issue of Harvard Business Review.