ONLINE WEBINAR : Using GenAI to apply frameworks in QDA [Jan 29, 2025]

Online Webinar ║ 29 January 2025

Hosts:

Introductions

Jiten: Good morning/ afternoon/ evening, depending on where you are. I’m Jiten and I have Ushma with me. We are going to be your hosts tonight. Ushma, over to you…

Ushma: So hello, everybody. First of all, thank you so much for taking out the time. It's midweek. So, depending on how your week is going, we hope to make it better for you. For those of you who have, you know, it's the first time you're interacting with us… we are a platform designed by and for qualitative researchers. This is how we love to express ourselves. The entire objective being to deliver a seamless qualitative research experience for researchers, so that you can focus more on the joy of insight discovery rather than doing the tedium that typically goes along with collecting, analyzing and reporting qualitative data. A very quick word on Jiten and me. Jiten has been actually on both sides of the proverbial table. He has been a qualitative researcher, then moved on to the client side. In that sense has a 360-degree view of the industry because he's played different stakeholders, at different points in time. Generally speaking, he is our GenAI champion. In fact, he alternatively calls GenAI his, you know, everything… but I think his spouse, partner, friend; sometimes his, you know, comic book, whatever. He's extremely wedded to GenAI and you will see why actually in a few minutes. I started off as a qualitative researcher and then developed alternative skill sets which are somewhat related. So, I've been into Social Media Analytics from the lens of Consumer Insights, not so much other use cases that social media is used for; across India, Southeast Asia, and Middle Eastern Africa most recently.

Agenda

Ushma: What we are going to do today is hopefully take you through some work that you will find as inspiring as we did, when we had the opportunity to do it for clients. The starting point really for this exercise was the fact that we have clients who are at different stages in their journey of interaction with qualitative data analysis tools. So, we have some who are just picking this up, in fact, Flowres is the first tool that they've ever really adopted. Then there are some who are seasoned and there are some who are lying in between. It's a spectrum. What we heard them tell us was that hey, everybody is, you know, more or less, all right when it comes to sorting or organizing raw data. But where GenAI comes in and what is the real potential that it can bring to the table when it does the rest of the work… because qualitative analysis is a longish process as most of you who are practitioners would agree.  So, that's where the client need inspired us to relook at GenAI, push the envelope and say that - Hey, can we look at how GenAI is actually going to enable us to take out some of the tedium from the qualitative data analysis process, while not compromising on the rigor that a researcher applies to the process, the human intervention that they bring in. And this is where we said, hey, can we look at actually conversing or talking to qualitative data; using prompts, using different techniques, see if we can massage the data a little better or faster than we are used to massaging it? So, the result of all of that is what we are hoping to take you through in the next 30 minutes or so. We are hoping to keep it crisp, yet interesting for you. What we have done is although we have a lot of material, we are going to try and focus on two of these… one is the Kapferer Brand Prism framework, the other is Maslow's Hierarchy of Needs framework. Both are fairly established frameworks, very popularly used by qualitative researchers across multiple markets. That's the reason for choosing these two for today’s session. There is work also done around other frameworks, which if time permits, we will get into and we would love to share with you and hear your views on.

Recap of frameworks & research objectives to which they were applied

Framework 1: Kapferer's Brand Prism x Whiskey category

Jiten: All right. LLMs have vast knowledge, including a lot of mental models and analysis frameworks. And I think one opportunity that currently seems to be under-leveraged is taking advantage of that knowledge and getting new perspectives; and what we’re hoping to do is inspire new ways of looking at the data. So, this is a study on whiskey as Ushma mentioned and what I have done is I have given a prompt and then developed it further, to get more nuanced results. Let me let me let me take you through the prompt and show you know what it threw up. I have done it already, but if time permits, we can even show it live on flowres; with other similar frameworks. So, yeah, the prompt reads like this… (reading prompt)…

Ushma: Julie asks us - Are these frameworks built into the tool with the prompts to guide customizing to the themes of insights we want to explore?

Jiten: The prompts are not built into the tool. We don't need to build the prompt into the tool. It would be fancy for me to say that these frameworks are built in. Essentially, what we are doing is we are using an application layer on an LLM. We use Gemini as a LLM in the backend, and what you see is flowres as an application layer. Gemini essentially understands all the models… Kapferer's, Maslow's… you name it, it understands. Because the LLM has been trained on the vast set of data, right? And that's why, you know, when I started, we said that while I'm showing this on flowres, you should be able to do this with ChatGPT, Claude or Gemini or for that matter, anything else. What flowres provides is an easy application layer, where some of the grunt work that you need to do elsewhere, will be saved.

Ushma: Yeah, let's move ahead.

Jiten: With that, if you look at the findings, it basically showed me the findings where it gave me the different aspects and identified the key theme. For brand A, the physical aspect is associated with premiumness, sophistication and visually appealing presentation and so on and so forth. If you go to the personality, it gave some quotes because I had asked it to give quotes. It gave me a key theme and so on and so forth for all the aspects. And then also it gave me, you know, I had asked for the comparison within my prompt… so it also gave me the comparison.

Jiten: So, what I did then is I said that, you know, please give me a brief summary of each aspect and then support it with verbatims. So, it then actually gave me the summaries first and then gave me the verbatims.

Jiten: Subsequently, I actually wanted to get the responses of brand loyalist and competition loyalist. Now It gave me response only for brand loyalist, right? There was a very good reason why it was not giving me competition loyalist analysis, because the competition data was diffused. There were several competition brands… so, it didn't have very strong image and that's why it started telling me that competition brands are not as clearly defined

Jiten: So, you know the other thing about flowres is that it will not hallucinate. It's a walled garden, it will not go outside the dataset and produce something which is not part of the data. So, wherever there wasn't any data; it told me so.

Jiten: So yeah, there is a summary table at the last in terms of brand A versus competition across all the dimensions. All right, I think that's on Kapferer.

Ushma: Right, so while you're fishing that one out, I see another question about sample prompts. So, clearly prompts are something or prompt construction is something that people are naturally curious about. So, maybe once you're done with this next example, you could spend a couple of minutes just giving those pointers on how to construct prompts.

Jiten: Absolutely.

Framework 2: Maslow's Needs Hierarchy x Real Estate category

Jiten: So, let me go to the Maslow's hierarchy one and you will see a pattern in the way I have constructed prompts. The first thing I usually do is - I anthropomorphize AI, asking it to take a certain role. I tested a few variations (of prompts) starting with just saying – “Give me a Maslow's hierarchy of needs on this data” to then gradually building better prompts; and I started getting better results. Also, I did provide the business objective, which does make a difference. I can put more as well, something like research objective… or maybe I want to add a certain instruction saying – “Always give me answer in bullet points”.

Jiten: What I did is - the first prompt was a generic prompt, where I didn't ask it to apply the Maslow's hierarchy in the context of category. Instead, I just said – “Analyze this transcript using Maslow's hierarchy of needs, categorized into the five levels of Maslow's hierarchy”, gave those levels. And then, the standard prompt which is ask comparison across segments and traceability of all the original verbatims.

Jiten: So, what I realize is - whenever you ask for verbatim examples, it will go respondent by respondent, and it will tell you, for each respondent, what each of the level means. So there are 10 respondents that we have data for, and for each respondent it has given the examples of what it means in terms of the physiological need, the safety need, the love and belonging need, esteem needs and self-actualization.

Jiten: I had asked for male and female segments differences, I mean comparisons, so it did gave me that. I was not too happy with the differences at first. So, what I did is, I asked it to compare. I said – “These differences don't seem to be telling me anything different for each of the segments”. So, maybe there no differences in the segments or maybe my prompt was not right. So, I created another prompt where I said – “Highlight differences and if you find them similar, just mention that they are same”. So, that worked well and it told me that for physiological needs, both have the same needs, there isn't any significant difference. For safety, the needs are same, the expression may not differ. Male emphasizes financial security versus females emphasize on sort of immediate physical security of home and family. Even in terms of the love and belonging, the focus differs. For me, it is more friends and social network versus for female, it was more family units, immediate community within the premises. Even for self-actualization… it showed that males are more inclined towards aspirations versus females are more towards fulfilment of their responsibility.

Jiten: Then I said, okay, this looks good, but you know, can you do it in a tabular format? So it just formatted it nicely and gave me a table, which kind of emphasized on the differences.

Jiten: So yeah, I think I'll stop here now. Ushma, are there any more questions in terms of anything related to Maslow? Otherwise I can talk about the construct that you mentioned.

Top tips: How to construct prompts that work?

Jiten: Let's say, you are doing this on ChatGPT, maybe you want to mention - this is the study, where this was the business objective, this was the research objective, these are the segments that we collected the data from; and then say that this is what I want you to do. If you look at the previous questions, I didn't just say apply Maslow's Hierarchy of Needs. Instead, it’s better to tell the LLM that I want it in these five levels of Maslow's hierarchy and then add whatever other instruction regarding the format you want to give.

Jiten: And remember, your prompts need not be one-line prompts or two-line prompts.

Jiten: You also don't want to over-engineer anything, because I think I've also seen when you over engineer a prompt, it might go into a different direction. What you want to do essentially is you want to have a very clear-cut set of instructions which it can follow through and produce the results.

Jiten: As you start constructing the prompts, maybe you realize that this is not working and then you can add a little layer, you can ask a follow-up question. All of us are qualitative researchers, so we are naturally good at probing and asking good questions and I think prompt is nothing but just asking good, clear questions.

Jiten: And yes, avoid any sort of jargons, just to ensure that we don't mess up the LLM thinking in any way.

Ushma: I also heard you talking about how prompts can actually be in the role plays. So, it's almost like asking the LLM to behave in a certain way, to put a thinking cap on of a certain profession or person or whatever the case may be. So, they need not be straight-off prompts and instructive prompts, but even role-playing prompts will help.

Jiten: Absolutely. So, basically, you are giving LLM a human form, asking it to play different personalities for your different needs.

Q&A : Is my data secure on flowres? Does flowres use my data to train itself?

Jiten: Right. So, flowres is ISO 27001 certified, which is gold standard information security. We are also GDPR compliant. We use fairly robust infrastructure of AWS and Microsoft across as a foundational layer or infrastructure layer on our product. So, in terms of the data security, it's fairly secure.

Jiten: Now in terms of what do we do with your data… so, your data is in your control. flowres is just hosting the data. All the data is deleted after three months unless we have different agreement with the client. Sometimes, clients do want us to hold the data for some time for their own compliance purposes. But otherwise, the data gets deleted in three months or sooner if you prefer. Plus, you can delete the data yourself, you are in control. We are not developing any LLM. So, we are not training any LLM, we are not using this data for any sort of sort of training. So in a nutshell, your data is perfectly secure. We work with large companies who run pretty strict information security protocols, audit us and we do just fine.

Q&A : How is flowres' AI different from generic LLMs?

Ushma: Okay, so we have Alex - Can you please give us a rundown on what you have done in the application layer to make this unique compared to a generic LLM?

Jiten: All right. So, you know, essentially, there are a few things which your ChatGPT or Claude cannot do which flowres can do. One is the citation part - where you can ask for citations. flowres gives citations on each and every data. Whenever you ask, you can go back to the transcript for that quote, basically. This is a tool that is built by researchers, we understand how important data traceability is for researchers. You wouldn't want to trust the conclusions that LLM makes. LLM tend to be very mean-heavy in terms of how they draw their conclusions. Sometimes, the insights are lying in qualitative parlance, so you want to see the data yourself. So, that's point number one that flowres allows, which you can't get through the generic LLMs.

Jiten: The second one is comparing and contrasting the data. You can't basically add a metadata to your generic LLM. While you can upload the transcripts and get the answers, you can't say that this transcript is of male segment and this transcript is of female segment and middle-aged and younger and so on and so forth. In flowres you can add as many tags as you like; and then design prompts accordingly. So, as I was just showing, give me differences between male and female and so on and so forth. So, whatever tags you have given you can design prompts accordingly. So, that's the difference number two.

Jiten: And then the difference number three is we have two options. One is the chat, and the second one is the analysis grid. What analysis grid does is it adds a little bit of convenience because you ask a question and it will get you the response for each and every transcript in one go; and then it will also show you the summary for each.

Ushma: So meanwhile there’s a shout out from Dean, he says – “Guaranteeing data traceability, data security and not defaulting to the mean are three very important factors that make it better than using generic LLMs”. Thanks, Dean!

Q&A : Can flowres manage terminologies relevant to specialist domains?

Ushma: Right. So, Jiten, before you move on to other examples, there are a couple of questions which we might like to take. The first one is... Can you provide overarching project context? For example, I do healthcare research. I often have terminology that is drug names, that have a drug name and a brand name, and those fall into categories. For example, biologics. Is it possible to provide that info? So, that when I ask a question about frequency of biologics use, it can find that without only looking for the word biologic?

Jiten: Right, so if I am understanding this right, whenever there is a study-specific context, especially in terms of the terminologies… you want to feed that into flowres; to ensure better analysis. Gone are the days where the LLM is actually counting words and making sense of the words literally. I believe all of you would have heard about Deepseek and I know you would have heard about O1 and O3. O3 is even better than O1. So the reasoning or the thinking of LLMs is becoming better and better. They're not just the stochastic parrot they were, in earlier days. You can actually upload all your study-specific acronyms, jargons, terminologies as a Word document or just copy-paste them in flowres when you upload your audio. And then, your automated transcription will be better; because it will make fewer mistakes in terms of the proper nouns.

Q&A : Are GenAI tools gender-biased?

Ushma: So, we also have a question from Dean. He says – “I have heard or read somewhere that GenAI tools have gender biases because their training data is inherently gender biased”. And he's given what he calls a crude example… “If you ask a GenAI tool to recommend a career to high-school students based on class transcripts, when the class transcripts are identical and the only difference is the name of the student, the GenAI tools are more likely to recommend STEM jobs or careers to males and Creative or Non-STEM jobs or careers to females”. So, his question therefore is – “Does flowres’ AI prevent this kind of gender bias when using qualitative research transcripts that include the name of the respondent”?

Jiten: So, that's an interesting question, which keeps coming up again and again. It's a well-known fact that the training data that most of the LLM's were fed was predominantly Western. And to answer your question in short – No, flowres’ AI doesn't prevent that, but I think more importantly, flowres’ AI doesn't need to prevent it… because we are not giving you any generic answers. You are not really asking advice in terms of what career a male should choose versus a female should choose. What flowres’ AI is giving you is giving you is the answer from the data that you have provided. Now if that data has inherent bias in it, when it was collected; then it will come up. If there isn't such bias, it will not come up. But essentially, when you have a walled-garden approach, where you are looking at only the data that has been fed as a source to ask the questions to; this question becomes moot in that sense. I hope that answers your question?

Jiten: I hope that answers your question.

Ushma: So, yes, Dean says it is the answer he was hoping for.

Jiten: Thanks Dean, do we have any other questions? I can very quicky show you Eisenhower if there is....

Framework 3: Eisenhower's Matrix

Ushma: Right. So, we have 10 minutes. I think we can go ahead Jiten, if there are more questions they will keep coming through in the chat and we will pick them up last.

Jiten: Super, this shouldn't take time, this is very quick. So, I said – “These are transcripts about home-buying preferences. Now, give me insights and identify trends and give me practical recommendations that I can go back to my client with”.

Jiten: So basically it first summarized the insights for me and then it did give me recommendations around different areas. There are interesting ways to ask this question. You can say – “Give me thought-provoking recommendations”. You can say – “Give me something in line with the research objectives”.

Jiten: What you can then do is – “Act like a property management expert who is familiar with Eisenhower’s Matrix and give me recommendations in those four quadrants and then dive into the strategic implications of each”. If you look at it, out of all the recommendations that it had given already, it told me what are the first things that the builder should do first - ensure timely delivery of the project, address water availability concerns and construction quality issues… these are the absolute musts, need to be done first. And then, what are the things which are important but not urgent, were things like amenities and catering to diverse needs; then there is urgent but not important, which will move the needle like customer service.

Jiten: So now, of course, as a researcher, you would be more context aware. You would know more about what is right and what is not right for your client. But this is an interesting way to kickstart your thinking and sort of move from there.

Jiten: And yes, it can give also in terms of the tabular format. So yeah, that's very quickly, you know, one interesting way of looking at your recommendations at the end of the study.

Jiten: All right, I think I have more examples like Hofstede's analysis and Pareto analysis and so on and so forth. I think we are almost at the fag end of the time and....

Ushma: Right. So, what we can do is anybody who would like specific frameworks to be discussed or any information on how we applied them, we can easily cover that one-to-one. I am just going to leave our website and Jiten's LinkedIn profile...

Ushma: Thank you so much everybody, for your time, have a great week ahead!

Jiten: Thank you so much, thanks everyone, have a great day ahead!