N-GRAM DRIVEN SMS BASED FAQ RETRIEVAL SYSTEM

Abstract

In the present scenario, everyone is looking for a better, efficient and easy way to access information. Resource availability and its user friendliness are directly encouraging a large group of people to access information more conveniently. Short Messaging Service (SMS) is one of the most popularly used services that provide information access to the people having mobile phones. In India alone, there are around 811 million mobile subscribers and still growing with a fast rate [3]. So, SMS based Question Answering (QA) services can be one of the cheapest and easiest ways to provide information access to the mobile users on move. However, there are several significant challenges in order to process a SMS query automatically. Humans have tendency to use abbreviations and shortcuts in their SMSes. We call these inconsistencies as noise in the SMSes. Existing SMS services such as service to access Examination result requires user to type the message in some specific format. These are the unnecessarily constraints to the users who generally feel it convenient to type a query in a “texting” language (i.e. including abbreviations and the shortcuts). Some businesses such as “ChaCha” [5] allow their users to make query through the SMSes without using any specific format. But these services are not automatic and the SMSes are handled by human experts. Though this kind of system provides independence to the users in writing the SMS query but this approach is not an efficient way to handle user‟s queries because the system is limited to handle a small number of queries proportional to the number of human experts on the business side. The approach can be efficient if we have a system which automatically handles the query at business side. In this thesis work, I presented a novel approach to handle these inconsistencies in the SMSes efficiently. This approach for SMS based FAQ retrieval system took N-gram counts into consideration while calculating the score for the question in the corpus. The experimental results demonstrates that this approach is significantly improves the accuracy of previous SMS based QA system as proposed by L. Venkata Subramaniam et al., August 2009 [1]. I demonstrate my algorithm over many real-life FAQ-datasets from different domains (e.g. Agriculture, Bank, Health, Insurance and Telecom etc.).