Large language models (LLMs), with their sophisticated natural language understanding capabilities, have significantly enhanced the functionality of chatbots, enabling them to conduct more meaningful and contextually relevant conversations with customers. These advanced chatbots can manage a wide range of customer service tasks, from answering FAQs to providing personalised recommendations, thus improving efficiency and scalability in customer service operations.

However, despite the advancements, a critical component of human interaction often missing in digital customer service is the nuanced understanding of the customer’s emotional state. Real-time sentiment analysis seeks to bridge this gap. By detecting subtle cues in text that indicate a customer’s mood or emotional state, businesses can provide more empathetic and tailored responses. This not only helps in resolving issues more effectively but also aids in building stronger customer relationships.

The integration of LLMs with sentiment analysis models, further optimised by OpenVINO, addresses this need. OpenVINO toolkit accelerates the inference process of AI models, allowing the sentiment analysis to occur instantaneously as the conversation unfolds. This enables a company to automate nuanced responses or escalate issues to live agents, depending on the emotional tone detected in customer communications. This not only streamlines the response process but also ensures that customers with pressing or sensitive issues receive the human attention they require.

Here is an example of the chatbot with a positive sentiment, that is, the user sentiment is close to 1.0:

On the other hand, the following is an example of a negative sentiment, that is, the user sentiment is close to -1.0:

You can see how a company can use these opportunities to engage with their customer exactly at these times, when the relationship is being put to the test. They could for example connect the user with a human operator just when the sentiment turned negative instead of simply leaving the chatbot continue interacting with them.

In this post I will show you how you can create such an application using a Large Language Model, and real time sentiment analysis, both models using OpenVINO for fast performance.

The first thing you need to do is to make sure you have your system up to date, and some basic libraries installed:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install python3-venv build-essential python3-dev git-all

We need to get the sample source code, setup a Python environment for this project, and install the requirements:

git clone https://github.com/samontab/llm_sentiment.git
cd llm_sentiment
python3 -m venv llm_sentiment
source llm_sentiment/bin/activate
python -m pip install --upgrade pip
pip install -r requirements.txt

We are now ready to have a look at the code. Start a jupyter lab with the llm_sentiment notebook, which is available at the current folder. You can do that by simply executing this:

jupyter lab llm_sentiment.ipynb

Now you should be able to see the code in your browser. If you click on the fast forward button on the top right, you will be able to execute all the code.

After clicking that button, there will be a popup asking you to restart the kernel. Click on Restart and wait for a few minutes.

You will be able to access the chat interface at the end of the notebook either directly there, or in another tab by clicking on the link after this text:

Now you can test it for yourself.

To get more details about the code, make sure to checkout the llm_sentiment notebook included in the repository:

You can have a look at the code at https://github.com/samontab/llm_sentiment