Posted by Sentiment analysis is another analytical technique that can now be integrated into Qlik. This time, the solution is GPT and its API, which can be connected to Qlik using the Qlik REST Connector. See the results and learn how it works.
Why?
Customer inquiries about the possibility of sentiment analysis in Qlik Sense are returning like a boomerang. Up until now, the answer has been:
- “Yes, it’s possible, but a statistical model needs to be trained, which will cost (a lot).”
- “Yes, it’s possible, and there are dedicated tools for that, outside of Qlik.”
Why GPT?
The beauty of using GPT for sentiment analysis in Qlik is:
- It can be launched immediately – GPT is a universal conversational model that has been trained on such huge datasets that it can essentially be asked any question and expected to return a reasonable response. It handles sentiment analysis questions very well.
- No need of additional programming skills (machine learning) or the need to learn new tools – we “program” GPT by asking the appropriate question (more on that below).
- Low entry level – both in terms of hardware (Qlik Sense in any environment is sufficient) and cost – analyzing multiple texts costs cents.
- Works in all languages – without additional programming. GPT naturally handles texts in dozens of languages. Even if the input is a mix of several different languages.
How?
The secret is to ask the right question (the “prompt”). In the case of sentiment analysis, the following scheme works well:
- Tell the tool “what it is” – for example, “Act as a trained ML model that estimates sentiment.“
- Tell it “what the input data will be” – for example, “I will provide you a list of opinions in rows.“
- Tell it “what the task is” – for example, “Evaluate their sentiment.“
- Tell it “how the data is prepared” – for example, “Each opinion has an associated Id.“
- Be precise in specifying what you expect as output – for example, “Return the Id and Sentiment only.” If you are not specific, GPT might return not only the sentiment but… an explanation of each classification result.
This way the prompt will look like:
Act as a trained ML model which estimates sentiment. I will give you a list of opinions in rows. Every Opinion has an Id. Evaluate their sentiment. Return the Id and Sentiment only.
+ list of opinions
The above is just an example. If you encounter any exceptions or unexpected behavior, you can instruct GPT to adjust – simply by using words!
Example – evaluation of reviews of Amazon products
To test the solution, I loaded several thousand real reviews of Amazon products, downloaded from Kaggle.
The result is very satisfying.
- Reviews classified as “Positive” or “Negative” align well with the numerical rating available for each review, for example:
- “Neutral“ opinions are well recognized. What’s more (!) there was an opinion which was classified as “Mixed” – which actually corresponds exactly to what it is (“my husband loves these, but I do not”):
Technicalities
- A REST Connector in Qlik is used to access GPT’s API.
- The prompt passed to GPT is a concatenated question and the list of opinions to be evaluated.
- The prompt needs to be passed as JSON.
- The prompt (JSON) cannot contain certain special characters so a cleanup of strings is needed. Example:
- There is a limit of “tokens” processed by GPT in one request so for a longer list we need to iterate through it with the For Loop.
- The result is one value, which then needs to be parsed to a table – using the Qlik’s From_Field Load.
Next steps
Use Qlik’s Facebook connector to download comments on your Fanpage and then assess them with ChatGPT.
Use Qlik’s Twitter connector to get twitts with specific filter and assess them too.
Using the results, calculate the Net Promoters Score.
You can test other GPT models, which might be even cheaper and working in the simple task of sentiment analysys just as good as the newest one.
If you have any comments or questions – do not hesitate to contact me!