AUTOMATIC TEXT SUMMARIZATION USING SOFT-COSINE SIMILARITY AND CENTRALITY MEASURES

Abstract

Automatic text summarization is one of the major problems in the field of machine learning. The approach used in this project is significantly different from all previous works done in this field in the respect that it uses two major concept called soft-cosine similarity and centrality measures. Soft cosine similarity takes into account the semantic relationship between the words thereby reducing the ambiguity caused by words with similar meanings. It also helps to realize how similar two sentences are which could be used to reduce redundancy and hence improve the quality of the final summary produced. There are dictionaries present to get the semantic relations between words. We are using WordNet which is an English linguistic dictionary containing 8 different relation types. Centrality measures is another widely used concept for graph-based approaches. We have discussed 40 different centrality measures and analyzed there impact and usage in 30 different real world networks. Finally studied 4 basic and most widely used centrality measures in order to decide which measure derives best results. EigenVector has shown to outperform other centrality measures. We have used two types of datasets single text documents from BBC news articles and and multi-text documents from DUC 2007 dataset. We have used the renowned ROUGE measure to compare the results and found that our approach performs better than all other state-of-the-art automatic text summarization methods namely TextRank, LexRank, Luhn and LSA.