{"id":426,"date":"2016-07-29T19:48:19","date_gmt":"2016-07-29T19:48:19","guid":{"rendered":"http:\/\/intelligentonlinetools.com\/blog\/?p=426"},"modified":"2016-08-08T00:35:22","modified_gmt":"2016-08-08T00:35:22","slug":"bio-inspired-optimization-for-text-mining-1","status":"publish","type":"post","link":"http:\/\/intelligentonlinetools.com\/blog\/2016\/07\/29\/bio-inspired-optimization-for-text-mining-1\/","title":{"rendered":"Bio-Inspired Optimization for Text Mining-1"},"content":{"rendered":"

Motivation<\/strong>
\nOptimization problem studies maximizing or minimizing some function
\ny<\/b>=f(x<\/b>) with some range of choices available for x<\/b>. Biologically inspired (bio-inspired) algorithms for optimization problems are now widely used. A few examples of such optimization are:
\nparticle swarm optimization (PSO) that is based on the swarming behavior of fish and birds,
\nfirefly algorithm (FA) that is based on the flashing behavior of swarming fireflies,
\nant colony optimization (ACO) that is based on the interaction of social insects (e.g., ants)
\nbee algorithms are all based on the foraging behavior of honey bees. [1]<\/p>\n

Recently on different forums there were questions how this optimization technique can be applied to text mining tasks such as classification or clustering. Those problems usually use algorithms that come from data mining or machine learning fields such as k-means clustering, SVM (support vector machine), naive Bayes.<\/p>\n

How can we apply bio-inspired algorithms for clustering? <\/p>\n

Let’s take a python package inspyred which has many bio-inspired algorithms including evolutionary computation, swarm intelligence, and immunocomputing [2]. This package has examples for specific predetermined functions that are used as benchmarks.<\/p>\n

However the code also can be customized to run for user defined functions. <\/p>\n

Below is the example of customized code for finding minimum of function f(x)=(x-1)**2, in this code function evaluate was added to specify how we calculate fitness. In this example we use PSO algorithm.<\/p>\n

We run this code and we get what we expected to get. Below you can find also the final output of the program.<\/p>\n

So now we understand what we need to change to fit optimization to different problems. Specifically we need to modify fitness function. Additionally we need to modify range and observer.
\nIn the next post Bio-Inspired Optimization for Text Mining-2 Numerical One Dimensional Example<\/a> we will continue to modify source code for solving clustering problem.<\/p>\n

\r\n# f(x)=(x-1)**2 minimize  on 0...2\r\n## http:\/\/pythonhosted.org\/inspyred\/examples.html#ant-colony-optimization\r\n## https:\/\/aarongarrett.github.io\/inspyred\/reference.html#benchmarks-benchmark-optimization-functions\r\n\r\nfrom time import time\r\nfrom random import Random\r\nimport inspyred\r\n\r\ndef my_observer(population, num_generations, num_evaluations, args):\r\n    best = max(population)\r\n    print('{0:6} -- {1} : {2}'.format(num_generations, \r\n                                      best.fitness, \r\n                                      str(best.candidate)))\r\n\r\n\r\ndef generate(random, args):\r\n     return [random.uniform(0, 2)]\r\n    \r\ndef evaluate(candidates, args):\r\n     fitness = []\r\n     for cand in candidates:\r\n          fit = [(c-1)**2 for c in cand]\r\n          fitness.append(fit)\r\n     return fitness\r\n\r\n\r\ndef main(prng=None, display=False):\r\n    if prng is None:\r\n        prng = Random()\r\n        prng.seed(time()) \r\n    \r\n   \r\n    \r\n    \r\n    ea = inspyred.swarm.PSO(prng)\r\n    ea.observer = my_observer\r\n    ea.terminator = inspyred.ec.terminators.evaluation_termination\r\n    ea.topology = inspyred.swarm.topologies.ring_topology\r\n    final_pop = ea.evolve(generator=generate,\r\n                          evaluator=evaluate, \r\n                          pop_size=100,\r\n                          bounder=inspyred.ec.Bounder(0, 2),\r\n                          maximize=False,\r\n                          max_evaluations=30000, \r\n                          neighborhood_size=2)\r\n\r\n    if display:\r\n        best = max(final_pop) \r\n        print('Best Solution: \\n{0}'.format(str(best)))\r\n    return ea\r\n\r\nif __name__ == '__main__':\r\n    main(display=True)\r\n<\/code><\/pre>\n
The output of program:<\/p>\n
\r\nBest Solution: \r\n[1.0] : [0.0]\r\n<\/code><\/pre>\n
References<\/strong>
\n1. A Brief Review of Nature-Inspired Algorithms for Optimization<\/a>
\n2. inspyred: Bio-inspired Algorithms in Python<\/a>
\n3. Bio-Inspired Optimization for Text Mining-2 Numerical One Dimensional Example <\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
Motivation Optimization problem studies maximizing or minimizing some function y=f(x) with some range of choices available for x. Biologically inspired (bio-inspired) algorithms for optimization problems are now widely used. A few examples of such optimization are: particle swarm optimization (PSO) that is based on the swarming behavior of fish and birds, firefly algorithm (FA) that … Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":[]},"categories":[2,9,12,10],"tags":[],"jetpack_publicize_connections":[],"yoast_head":"\nBio-Inspired Optimization for Text Mining-1 - Machine Learning Applications<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"http:\/\/intelligentonlinetools.com\/blog\/2016\/07\/29\/bio-inspired-optimization-for-text-mining-1\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Bio-Inspired Optimization for Text Mining-1 - Machine Learning Applications\" \/>\n<meta property=\"og:description\" content=\"Motivation Optimization problem studies maximizing or minimizing some function y=f(x) with some range of choices available for x. Biologically inspired (bio-inspired) algorithms for optimization problems are now widely used. A few examples of such optimization are: particle swarm optimization (PSO) that is based on the swarming behavior of fish and birds, firefly algorithm (FA) that ... Read more\" \/>\n<meta property=\"og:url\" content=\"http:\/\/intelligentonlinetools.com\/blog\/2016\/07\/29\/bio-inspired-optimization-for-text-mining-1\/\" \/>\n<meta property=\"og:site_name\" content=\"Machine Learning Applications\" \/>\n<meta property=\"article:published_time\" content=\"2016-07-29T19:48:19+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2016-08-08T00:35:22+00:00\" \/>\n<meta name=\"author\" content=\"owygs156\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"owygs156\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/2016\/07\/29\/bio-inspired-optimization-for-text-mining-1\/\",\"url\":\"http:\/\/intelligentonlinetools.com\/blog\/2016\/07\/29\/bio-inspired-optimization-for-text-mining-1\/\",\"name\":\"Bio-Inspired Optimization for Text Mining-1 - Machine Learning Applications\",\"isPartOf\":{\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/#website\"},\"datePublished\":\"2016-07-29T19:48:19+00:00\",\"dateModified\":\"2016-08-08T00:35:22+00:00\",\"author\":{\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/7a886dc5eb9758369af2f6d2cb342478\"},\"breadcrumb\":{\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/2016\/07\/29\/bio-inspired-optimization-for-text-mining-1\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"http:\/\/intelligentonlinetools.com\/blog\/2016\/07\/29\/bio-inspired-optimization-for-text-mining-1\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/2016\/07\/29\/bio-inspired-optimization-for-text-mining-1\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"http:\/\/intelligentonlinetools.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Bio-Inspired Optimization for Text Mining-1\"}]},{\"@type\":\"WebSite\",\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/#website\",\"url\":\"http:\/\/intelligentonlinetools.com\/blog\/\",\"name\":\"Machine Learning Applications\",\"description\":\"Artificial intelligence, data mining and machine learning for building web based tools and services.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"http:\/\/intelligentonlinetools.com\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/7a886dc5eb9758369af2f6d2cb342478\",\"name\":\"owygs156\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"http:\/\/2.gravatar.com\/avatar\/b351def598609cb4c0b5bca26497c7e5?s=96&d=mm&r=g\",\"contentUrl\":\"http:\/\/2.gravatar.com\/avatar\/b351def598609cb4c0b5bca26497c7e5?s=96&d=mm&r=g\",\"caption\":\"owygs156\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Bio-Inspired Optimization for Text Mining-1 - Machine Learning Applications","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"http:\/\/intelligentonlinetools.com\/blog\/2016\/07\/29\/bio-inspired-optimization-for-text-mining-1\/","og_locale":"en_US","og_type":"article","og_title":"Bio-Inspired Optimization for Text Mining-1 - Machine Learning Applications","og_description":"Motivation Optimization problem studies maximizing or minimizing some function y=f(x) with some range of choices available for x. Biologically inspired (bio-inspired) algorithms for optimization problems are now widely used. A few examples of such optimization are: particle swarm optimization (PSO) that is based on the swarming behavior of fish and birds, firefly algorithm (FA) that ... Read more","og_url":"http:\/\/intelligentonlinetools.com\/blog\/2016\/07\/29\/bio-inspired-optimization-for-text-mining-1\/","og_site_name":"Machine Learning Applications","article_published_time":"2016-07-29T19:48:19+00:00","article_modified_time":"2016-08-08T00:35:22+00:00","author":"owygs156","twitter_card":"summary_large_image","twitter_misc":{"Written by":"owygs156","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"http:\/\/intelligentonlinetools.com\/blog\/2016\/07\/29\/bio-inspired-optimization-for-text-mining-1\/","url":"http:\/\/intelligentonlinetools.com\/blog\/2016\/07\/29\/bio-inspired-optimization-for-text-mining-1\/","name":"Bio-Inspired Optimization for Text Mining-1 - Machine Learning Applications","isPartOf":{"@id":"http:\/\/intelligentonlinetools.com\/blog\/#website"},"datePublished":"2016-07-29T19:48:19+00:00","dateModified":"2016-08-08T00:35:22+00:00","author":{"@id":"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/7a886dc5eb9758369af2f6d2cb342478"},"breadcrumb":{"@id":"http:\/\/intelligentonlinetools.com\/blog\/2016\/07\/29\/bio-inspired-optimization-for-text-mining-1\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["http:\/\/intelligentonlinetools.com\/blog\/2016\/07\/29\/bio-inspired-optimization-for-text-mining-1\/"]}]},{"@type":"BreadcrumbList","@id":"http:\/\/intelligentonlinetools.com\/blog\/2016\/07\/29\/bio-inspired-optimization-for-text-mining-1\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"http:\/\/intelligentonlinetools.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Bio-Inspired Optimization for Text Mining-1"}]},{"@type":"WebSite","@id":"http:\/\/intelligentonlinetools.com\/blog\/#website","url":"http:\/\/intelligentonlinetools.com\/blog\/","name":"Machine Learning Applications","description":"Artificial intelligence, data mining and machine learning for building web based tools and services.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"http:\/\/intelligentonlinetools.com\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/7a886dc5eb9758369af2f6d2cb342478","name":"owygs156","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/image\/","url":"http:\/\/2.gravatar.com\/avatar\/b351def598609cb4c0b5bca26497c7e5?s=96&d=mm&r=g","contentUrl":"http:\/\/2.gravatar.com\/avatar\/b351def598609cb4c0b5bca26497c7e5?s=96&d=mm&r=g","caption":"owygs156"}}]}},"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p7h1IJ-6S","jetpack-related-posts":[{"id":450,"url":"http:\/\/intelligentonlinetools.com\/blog\/2016\/08\/03\/bio-inspired-optimization-for-text-mining-2\/","url_meta":{"origin":426,"position":0},"title":"Bio-Inspired Optimization for Text Mining-2","date":"August 3, 2016","format":false,"excerpt":"Numerical One Dimensional Example In the previous code Bio-Inspired Optimization for Text Mining-1 Motivation we implemented source code for optimization some function using bio-inspired algorithm. Now we need to put actual function for clustering. In clustering we want to group our clusters in such way that the distance from each\u2026","rel":"","context":"In "Data Mining"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":521,"url":"http:\/\/intelligentonlinetools.com\/blog\/2016\/08\/26\/bio-inspired-optimization-for-text-mining-4\/","url_meta":{"origin":426,"position":1},"title":"Bio-Inspired Optimization for Text Mining-4","date":"August 26, 2016","format":false,"excerpt":"Clustering Text Data In previous post Bio-Inspired Optimization was applied for clustering of numerical data. In this post text data will be used for clustering. So python source code will be modified for clustering of text data. This data will be initialized in the beginning of this python script with\u2026","rel":"","context":"In "Machine Learning"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":498,"url":"http:\/\/intelligentonlinetools.com\/blog\/2016\/08\/13\/bio-inspired-optimization-for-text-mining-3\/","url_meta":{"origin":426,"position":2},"title":"Bio-Inspired Optimization for Text Mining-3","date":"August 13, 2016","format":false,"excerpt":"Clustering Numerical Multidimensional Data In this post we will implement Bio Inspired Optimization for clustering multidimensional data. We will use two dimensional data array \"data\" however the code can be used for any reasonable size of array. To do this parameter num_dimensions should be set to data array dimension. We\u2026","rel":"","context":"In "Data Mining"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":875,"url":"http:\/\/intelligentonlinetools.com\/blog\/2017\/01\/28\/iris-plant-classification-using-neural-network-online-experiments-with-normalization-and-other-parameters\/","url_meta":{"origin":426,"position":3},"title":"Iris Plant Classification Using Neural Network – Online Experiments with Normalization and Other Parameters","date":"January 28, 2017","format":false,"excerpt":"Do we need to normalize input data for neural network? How differently will be results from running normalized and non normalized data? This will be explored in the post using Online Machine Learning Algorithms tool for classification of iris data set with feed-forward neural network. Feed-forward Neural Network Feed-forward neural\u2026","rel":"","context":"In "Artificial Intelligence"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/01\/delta-error-graph-from-iris-data-set-36-neurons-300x231.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":510,"url":"http:\/\/intelligentonlinetools.com\/blog\/2016\/08\/19\/getting-data-from-wikipedia-using-python\/","url_meta":{"origin":426,"position":4},"title":"Getting Data From Wikipedia Using Python","date":"August 19, 2016","format":false,"excerpt":"Recently I come across python package Wikipedia which is a Python library that makes it easy to access and parse data from Wikipedia. Using this library you can search Wikipedia, get article summaries, get data like links and images from a page, and more. Wikipedia wraps the MediaWiki API so\u2026","rel":"","context":"In "API Programming"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":9,"url":"http:\/\/intelligentonlinetools.com\/blog\/2016\/01\/30\/general-regression-neural-networks\/","url_meta":{"origin":426,"position":5},"title":"General Regression Neural Networks","date":"January 30, 2016","format":false,"excerpt":"General Regression Neural Networks (GRNN) is one of the type of neural networks with a one pass learning algorithm. The simplicity of algorithm for GRNN is one of the advantage of this neural network and many researchers use it. In one of recent paper was proposed algorithm that is using\u2026","rel":"","context":"Similar post","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/posts\/426"}],"collection":[{"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/comments?post=426"}],"version-history":[{"count":20,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/posts\/426\/revisions"}],"predecessor-version":[{"id":490,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/posts\/426\/revisions\/490"}],"wp:attachment":[{"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/media?parent=426"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/categories?post=426"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/tags?post=426"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}