We ran a pilot to explore the potential of AI tools for qualitative research analysis. We used data from an actual project and compared work by AI tools with that of a seasoned researcher. We used a variety of AI tools – Forelight, Dovetail, Gemini.
Planning Discussion Guide
We tried various tools for interview transcriptions. Their quality and accuracy has improved considerably – for English conversations (even when accented), the quality is excellent. For other languages, it still has some way to go. However, they can be used at crunch times to get a rough draft and then manually corrected.
Tools have become sophisticated on coding the data as per your needs. We were easily able to code different parts of the conversation such as ‘profile’ ‘relationship with category’, ‘purchase process’, ‘compromises etc. However, this coding took time as the research conversations were semi-structured and allowed for many spontaneous digressions.
Verdict: AI tools add immense value in transcriptions. They save an incredible amount of time and avoid errors due to human fatigue and boredom!
Coding of data via AI is still work in progress. It requires human oversight and intervention but may work well for simple, structured research. Think ad and product testing. Exploratory, and complicated research projects will still need a great deal of human depth and insight.
Collating and Coding Data
We tried various tools for interview transcriptions. Their quality and accuracy has improved considerably – for English conversations (even when accented), the quality is excellent. For other languages, it still has some way to go. However, they can be used at crunch times to get a rough draft and then manually corrected.
Tools have become sophisticated on coding the data as per your needs. We were easily able to code different parts of the conversation such as ‘profile’ ‘relationship with category’, ‘purchase process’, ‘compromises etc. However, this coding took time as the research conversations were semi-structured and allowed for many spontaneous digressions.
Verdict: AI tools add immense value in transcriptions. They save an incredible amount of time and avoid errors due to human fatigue and boredom!
Coding of data via AI is still work in progress. It requires human oversight and intervention but may work well for simple, structured research. Think ad and product testing. Exploratory, and complicated research projects will still need a great deal of human depth and insight.
Data Analysis & Insights
AI tools are getting pretty good at generating initial themes from the research data. They may not be deep or exhaustive but they were not wrong.
For instance one of the information areas that we needed from the data was parent’s relationship with their children’s health. The AI tool provided the following insight:
“Parents desire for children’s physical well-being, but have limited knowledge about preventive measures like vaccinations. They have concerns about side effects and lack of trust in sources other than doctors. They focus on healthy diet, hygiene, and traditional remedies. They have limited awareness about mental health and emotional well-being.”
This insight is not wrong but it is generic and surface-level. It provides marketers with information but no stimulus for action.
This is where the labour intensive and time consuming human analysis scores, assuming they have the right experience and skills. As researchers and strategists we oscillate between big picture thinking and building up from details as we immerse ourselves in the data. We also have the benefit of having been on the field and having received multiple non-verbal inputs to make sense of the words that are being spoken. The AI tools are unable to do that, as of now.
As I persisted with the AI interrogating the data, however, it was able to answer any question I threw at it. The answers were again generic but correct. It was even able to pull out quotes to substantiate any point. These quotes were accurate even though not the best ones to substantiate the point. We spend a lot of time going through data to pull out the right quotes. I see AI playing a considerable role in this in the future, as it even can provide timestamps easily for us to revisit the right data.
We (humans) were able to bring in emotion, storytelling and a narrative to the insights that make the customer world come alive in the minds of the listeners and readers. We are also able to read the room and orient our presentations for meaningful communication.
Verdict: Humans trump qualitative data analysis but AI tools can help speed up certain aspects like identifying quotes and data source, check for any insights we may have overlooked and even provide a starting point for key themes.
Conclusion
AI tools are ready to be used as assistants in qualitative research data analysis. They can help speed up some cumbersome tasks like transcriptions, identifying data sources and quotes. AI’s ability to pull out details can complement a seasoned, strategic researcher’s big picture thinking well.
It is not yet time to rely solely on AI tools though. One should understand what AI can or cannot do and use the tools smartly. We are moving towards a hybrid model with seasoned researchers using AI tools to make their own work smart and more effective.
As Jim Collins wrote in his seminal work ‘Built to Last’ – Builders of greatness reject the “Tyranny of the OR” and embrace the “Genius of the AND.”
