Text mining and Sentiment Analysis Help Improve Customer Service Performance
The company is a financial services firm that provides a broad range of solutions and services to a global customer base. The company has a comprehensive network of facilities around the world, with over 5000 associates assisting their customers. Customers lodge service requests by telephone, email or through an online chat interface.
As a B2C service provider, the company strives to maintain high standards for effective communication between their associates and customers, and tries to monitor customer interactions at every opportunity. The broad objective of this service performance monitoring is to maintain satisfactory quality of service over time and across the organization. To this end, the company has devised a set of standards for service excellence, to which all customer interactions are expected to adhere. These standards comprise of different qualitative measures of service levels, e.g., associates should use clear and understandable language, associates should always maintain a professional and friendly demeanor, etc. Associates’ performances are measured based on compliance with these quality standards. Organizational units at different levels, like teams, departments, and the company as a whole, also receive scores based on associate performances. The evaluations and remunerations of not only the associates but also of management are influenced by these service performance scores.
Challenge
Continually monitoring service levels is essential for service quality control. Customer surveys are an excellent way of gathering feedback about service levels. An even richer source of information is the corpus of associate-customer interactions. Historically the company manually evaluated a sample of associate-customer interactions and survey responses for compliance with excellence standards. This approach, in addition to being subjective and error-prone, was time- and labor-intensive. Advances in machine learning and computational linguistics offer an opportunity to objectively evaluate all customer interactions in a timely manner.
The company needs a system for (1) automatically evaluating associate-customer interactions for compliance with quality standards and (2) analyzing survey responses to extract positive and negative feedback. The analysis must be able to account for the wide diversity of expression in natural language, e.g., pleasant and reassuring tone, acceptable language, appropriate abbreviations, addressing all of the customers’ issues, etc.
Solution
PolyAnalyst 6.5™ by Megaputer Intelligence is a data mining and analysis platform that provides a comprehensive set of tools for analyzing structured and unstructured data. PolyAnalyst’s text analysis tools are used for extracting complex word patterns, grammatical and semantic relationships, and expressions of sentiment. The results of these text analyses are then classified into context-specific themes to identify actionable issues, which can be assigned to relevant individuals responsible for their resolution. The system can be programmed to provide feedback in case of insufficient classification so that analyses can be modified or amended. The relationships between structured fields and text analysis results are also established in order to identify patterns and interactions. The system publishes the results of analyses through graphical, interactive web-based reports. Users create analysis scenarios using a drag and drop graphical user interface (GUI). These scenarios are reusable solutions that can be programmed to automate the analysis and report generation process.
A set of specific criteria were designed to capture and automatically detect compliance with the company’s Quality Standards. The figure below displays an example of an associate’s response, as well as the quality criteria that it succeeds or fails to match.
As illustrated above, this comment matches several criteria while failing to match one, and contributes accordingly to the associate’s performance score. These scores are then automatically calculated and aggregated across various organizational units. It is relatively easy to modify the system in case of changes in quality standards, and the changes can be quickly applied to historical data. The system also has an integrated case
management system, which generates email alerts in case of drops in service quality and allows users to track the progress of issue resolution.
1. Completely automated analysis; saves time.
2. Analysis of entire dataset (> 1 million records per year); no need for sampling.
3. 45% cost savings over traditional analysis.
4. Weekly processing. In the case of traditional analysis, data could only be processed monthly due to time and resource constraints.
5. Analysis not subjective to the analyst.
a. Increased accuracy.
b. Increased uniformity.
6. Greater accountability. Associates can review the analysis and raise concerns in case of discrepancies.
Future Directions
Currently the corpus of associate-customer interactions does not include transcripts of phone conversations. By incorporating speech recognition capability, the system can become a one-stop destination for analyzing all customer interactions. The system could also potentially be used in real-time, instead of periodic analyses.
• How did the financial services firm used text mining and text analytics to improve its customer service performance?
• What were the challenges, the proposed solution, and the obtained results?