It is no doubt that the web is growing at an incredible pace. And as the most documents of the web consist of the text, the applications of text analytics or text mining are getting more use. In such applications the textual data are used for extracting intelligence from a large collection of documents. Here are 7 ideas for building this type of applications. Later on during this year 2016 some online working demo examples on this site will be built to test the ideas. The focus is on applications for personal use. Business applications of text mining can be found in [1]
1. Trending is collecting historical data in order to find pattern or predict future. If the usage of word phrase “python programming” is going up from month to month then it is good signal for paying attention to this. There is the tool Google Trends is a public web facility of Google Inc., based on Google Search, that shows how often a particular search-term is entered relative to the total search-volume across various regions of the world, and in various languages. The horizontal axis of the main graph represents time (starting from 2004), and the vertical is how often a term is searched for relative to the total number of searches, globally. [2]
However what if we more focused on the future and we want know what terms will be popular in the future. For example “data science” or “big data” are now popular search terms but back to the time when the usage was the lowest – if the tool can predict high increase of usage in such situation – that would be very useful.
2. Building news feed is another example of application for text mining. Newly published web based content if it matches the user interest has a great value. So the application should allow for user to set the topics for the desired content.
For the same topic the user might be interesting in the ability to set other filters such as source of content, type of content and some other characteristics of content.
Over the time the user interests will change and so the application should learn and adapt to user interests too.
3.Post editing. While someone online is typing article for blog or paper there is always the need to find something related to topic on the web. Imagine that it will automatically show an additional text box with similar content from the web. This would eliminate switching back to search engine and also can bring something new that the author even did not think about.
4. Automatic creation of data reports is very helpful for people who needs data for their research or business needs. There a lot of data and information freely available on the web however it is tedious go to online on weekly or monthly basis and manually extract data and put in some file or database for further analysis. Such task often will include such things as formatting data, merging the information from different sources and some other processing operations.
5. Sentiment analysis (also known as opinion mining) refers to the use of natural language processing, text analysis and computational linguistics to identify and extract subjective information in source materials. Sentiment analysis is widely applied to reviews and social media for a variety of applications, ranging from marketing to customer service. [3]
Almost each online text analytics web based service is offering sentiment analysis option for users. There are also many online examples how to do sentiment analysis using python or R as programming language. [4]
Sentiment analysis application would be useful for predicting some financial event or collecting opinion about some product.
6. Content Organizing application can organize documents into groups by topic, keywords or by some other means.
We can subscribe and receive email notifications about site news, latest post or new article. The application that is saving links that we liked or decided to review later can help us be more productive. In addition to links it would save some information about links like keywords, topics or description. Such application could also group information by topic or keyword and automatically assign additional keywords.
Text document clustering and classification would be used a lot for this application.
7. Topic detection application can be used for automatic text categorization, for understanding what people are talking about, for automatic processing or preprocessing emails or user submitted online articles, comments.
The task of topic detection might also require the
development of approaches related to the presentation
of topics: topic ranking, relevant image retrieval, title
and keyword extraction. One of the example of using topic detection is shown in [5]
Obviously one document can consist of several segments on different topics. In one of researches simple clustering algorithm was used to group the semantically-related sentences. The distance between two sentences was calculated based on the distance between
all nouns that appear in the sentences. The distance between two nouns was calculated using the Wordnet thesaurus. [6],[7]
References
1. Text Mining and its Business Applications http://www.codeproject.com/Articles/822379/Text-Mining-and-its-Business-Applications
2. https://en.wikipedia.org/wiki/Google_Trends Google Trends
From Wikipedia
3. https://en.wikipedia.org/wiki/Sentiment_analysis Sentiment analysis
From Wikipedia
4. https://support.sas.com/resources/papers/proceedings14/1288-2014.pdf
Analysis of Unstructured Data: Applications of Text Analytics
and Sentiment Mining, Dr. Goutam Chakraborty, Murali Krishna Pagolu
5. http://ceur-ws.org/Vol-1150/petkos.pdf
Two-level message clustering for topic detection in
Twitter. Georgios Petkos ,Symeon Papadopoulos, Yiannis Kompatsiaris
6. A Non-Linear Topic Detection Method for Text
Summarization Using Wordnet
Click to access Wksp-Tec-Info-Ling-Silla-2003.pdf
Carlos N. Silla Jr. , Celso A. A. Kaestner, Alex A. Freitas
7. https://www.uni-weimar.de/medien/webis/events/tir-08/tir08-papers-final/wartena08-topic-detection-by-clustering-keywords.pdf
Topic Detection by Clustering Keywords. Christian Wartena and Rogier Brussee