{"id":1422,"date":"2017-11-02T02:09:42","date_gmt":"2017-11-02T02:09:42","guid":{"rendered":"http:\/\/intelligentonlinetools.com\/blog\/?p=1422"},"modified":"2017-11-30T02:02:08","modified_gmt":"2017-11-30T02:02:08","slug":"data-visualization-of-word-correlations-with-networkx","status":"publish","type":"post","link":"http:\/\/intelligentonlinetools.com\/blog\/2017\/11\/02\/data-visualization-of-word-correlations-with-networkx\/","title":{"rendered":"Data Visualization of Word Correlations with NetworkX"},"content":{"rendered":"<p>This is a continuation of my previous post, found here <a href=http:\/\/intelligentonlinetools.com\/blog\/2017\/10\/15\/scraping\/>Combining Machine Learning and Data Scraping<\/a>.    Data visualization is added to show correlations between words. The graph was built using NetworkX python library.<br \/>\nThe input for the graph is the array corr_data with 3 columns : pair of words and correlation between them. This was calculated in the previous post.<\/p>\n<p>In this post are added two functions:<br \/>\n<em>build_graph_for_all<\/em> &#8211; it is taking words from matrix for the first N rows and adding to the graph.<br \/>\nThe graph is shown below.<br \/>\n<img data-attachment-id=\"1427\" data-permalink=\"http:\/\/intelligentonlinetools.com\/blog\/2017\/11\/02\/data-visualization-of-word-correlations-with-networkx\/networkx_graph\/#main\" data-orig-file=\"http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/11\/NetworkX_graph.png\" data-orig-size=\"1105,776\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"NetworkX_graph\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/11\/NetworkX_graph-300x211.png\" data-large-file=\"http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/11\/NetworkX_graph-1024x719.png\" decoding=\"async\" loading=\"lazy\" src=\"http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/11\/NetworkX_graph-300x211.png\" alt=\"\" width=\"300\" height=\"211\" class=\"alignnone size-medium wp-image-1427\" srcset=\"http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/11\/NetworkX_graph-300x211.png 300w, http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/11\/NetworkX_graph-768x539.png 768w, http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/11\/NetworkX_graph-1024x719.png 1024w, http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/11\/NetworkX_graph.png 1105w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/p>\n<p>The Second function <em>build_graph<\/em> is taking specific word and adding to graph only edge that have this word. The process is repeating but now it is adding edges to other words on the graph. This is recursive function. Below in the python code are shown these functions.<\/p>\n<p>Python computer code:<\/p>\n<pre><code>\r\nimport networkx as nx\r\nimport matplotlib.pyplot as plt\r\nG=nx.Graph()\r\n\r\nexisting_edges = {}\r\n\r\ndef build_graph(w, lev):\r\n  if (lev > 5)  :\r\n      return\r\n  for z in corr_data:\r\n     ind=-1 \r\n     if z[0] == w:\r\n         ind=0\r\n         ind1=1\r\n     if z[1] == w:\r\n         ind ==1\r\n         ind1 =0\r\n         \r\n     if ind == 0 or ind == 1:\r\n         if  str(w) + \"_\" + str(corr_data[ind1]) not in existing_edges :\r\n            \r\n             G.add_node(str(corr_data[ind]))\r\n             existing_edges[str(w) + \"_\" + str(corr_data[ind1])] = 1;\r\n             G.add_edge(w,str(corr_data[ind1]))\r\n            \r\n             build_graph(corr_data[ind1], lev+1)\r\n\r\n\r\nexisting_nodes = {}\r\ndef build_graph_for_all():\r\n    count=0\r\n    for d in corr_data:\r\n        if (count > 40) :\r\n            return\r\n        if  d[0] not in existing_edges :\r\n             G.add_node(str(d[0]))\r\n        if  d[1] not in existing_edges :     \r\n             G.add_node(str(d[1]))\r\n        G.add_edge(str(d[0]), str(d[1]))     \r\n        count=count + 1\r\n\r\n\r\nbuild_graph_for_all()\r\n\r\nprint (G.nodes(data=True))\r\nplt.show()\r\nnx.draw(G, width=2, with_labels=True)\r\nplt.savefig(\"path1.png\")\r\n\r\n\r\nw=\"design\"\r\nG.add_node(w)\r\nbuild_graph(w, 0)\r\n \r\nprint (G.nodes(data=True))\r\nplt.show()\r\nnx.draw(G, width=2, with_labels=True)\r\nplt.savefig(\"path.png\")\r\n<\/code><\/pre>\n<p>In this post we created script that can be used to draw plot of connections between the words. In the near future I am planning to apply this technique to real problem. Below is the full source code. <\/p>\n<pre><code>\r\n# -*- coding: utf-8 -*-\r\n\r\nimport numpy as np\r\nimport nltk\r\nimport csv\r\nimport re\r\nfrom scipy.stats.stats import pearsonr   \r\n\r\ndef remove_html_tags(text):\r\n        \"\"\"Remove html tags from a string\"\"\"\r\n        clean = re.compile('<.*?>')\r\n        return re.sub(clean, '', text)\r\n\r\n\r\nfn=\"C:\\\\Users\\\\Owner\\\\Desktop\\\\A\\\\Scrapping\\\\craigslist\\\\result-jobs-multi-pages-content.csv\"\r\n\r\ndocs=[]\r\ndef load_file(fn):\r\n         start=1\r\n         file_urls=[]\r\n         \r\n         strtext=\"\"\r\n         with open(fn, encoding=\"utf8\" ) as f:\r\n            csv_f = csv.reader(f)\r\n            for i, row in enumerate(csv_f):\r\n               if i >=  start  :\r\n                 file_urls.append (row)\r\n                \r\n                 strtext=strtext + replaceNotNeeded(str(stripNonAlphaNum(row[5])))\r\n                 docs.append (str(stripNonAlphaNum(row[5])))\r\n                \r\n         return strtext  \r\n     \r\n\r\n# Given a text string, remove all non-alphanumeric\r\n# characters (using Unicode definition of alphanumeric).\r\n\r\ndef stripNonAlphaNum(text):\r\n    import re\r\n    return re.compile(r'\\W+', re.UNICODE).split(text)\r\n\r\ndef replaceNotNeeded(text):\r\n    text=text.replace(\"'\",\"\").replace(\",\",\"\").replace (\"''\",\"\").replace(\"'',\",\"\")\r\n    text=text.replace(\" and \", \" \").replace (\" to \", \" \").replace(\" a \",\" \").replace(\" the \",\" \").replace(\" of \",\" \").replace(\" in \",\" \").replace(\" for \", \" \").replace(\" or \", \" \")\r\n    text=text.replace(\" will \", \" \").replace (\" on \", \" \").replace(\" be \",\" \").replace(\" with \",\" \").replace(\" is \",\" \").replace(\" as \",\" \")\r\n    text=text.replace(\"    \",\" \").replace(\"   \",\" \").replace(\"  \",\" \")\r\n    return text\r\n\r\ntxt=load_file(fn)\r\nprint (txt)\r\n\r\ntokens = nltk.wordpunct_tokenize(str(txt))\r\n\r\nmy_count = {}\r\nfor word in tokens:\r\n    try: my_count[word] += 1\r\n    except KeyError: my_count[word] = 1\r\n\r\nprint (my_count)\r\n\r\n\r\ndata = []\r\n\r\nsortedItems = sorted(my_count , key=my_count.get , reverse = True)\r\nitem_count=0\r\nfor element in sortedItems :\r\n       if (my_count.get(element) > 3):\r\n           data.append([element, my_count.get(element)])\r\n           item_count=item_count+1\r\n           \r\n\r\n\r\nN=5\r\ntopN = []\r\ncorr_data =[]\r\nfor z in range(N):\r\n    topN.append (data[z][0])\r\n\r\n\r\n\r\n\r\nwcount = [[0 for x in range(500)] for y in range(2000)] \r\ndocNumber=0     \r\nfor doc in docs:\r\n    \r\n    for z in range(item_count):\r\n        \r\n        wcount[docNumber][z] = doc.count (data[z][0])\r\n    docNumber=docNumber+1\r\n\r\nprint (\"calc correlation\")        \r\n     \r\nfor ii in range(N-1):\r\n    for z in range(item_count):\r\n       \r\n        r_row, p_value = pearsonr(np.array(wcount)[:, ii], np.array(wcount)[:, z])\r\n        print (r_row, p_value)\r\n        if r_row > 0.6 and r_row < 1:\r\n               corr_data.append ([topN[ii],  data[z][0], r_row])\r\n        \r\nprint (\"correlation data\")\r\nprint (corr_data)\r\n\r\n\r\nimport networkx as nx\r\nimport matplotlib.pyplot as plt\r\nG=nx.Graph()\r\n\r\nexisting_edges = {}\r\n\r\ndef build_graph(w, lev):\r\n  if (lev > 5)  :\r\n      return\r\n  for z in corr_data:\r\n     ind=-1 \r\n     if z[0] == w:\r\n         ind=0\r\n         ind1=1\r\n     if z[1] == w:\r\n         ind ==1\r\n         ind1 =0\r\n         \r\n     if ind == 0 or ind == 1:\r\n         if  str(w) + \"_\" + str(corr_data[ind1]) not in existing_edges :\r\n            \r\n             G.add_node(str(corr_data[ind]))\r\n             existing_edges[str(w) + \"_\" + str(corr_data[ind1])] = 1;\r\n             G.add_edge(w,str(corr_data[ind1]))\r\n            \r\n             build_graph(corr_data[ind1], lev+1)\r\n\r\n\r\nexisting_nodes = {}\r\ndef build_graph_for_all():\r\n    count=0\r\n    for d in corr_data:\r\n        if (count > 40) :\r\n            return\r\n        if  d[0] not in existing_edges :\r\n             G.add_node(str(d[0]))\r\n        if  d[1] not in existing_edges :     \r\n             G.add_node(str(d[1]))\r\n        G.add_edge(str(d[0]), str(d[1]))     \r\n        count=count + 1\r\n\r\n\r\nbuild_graph_for_all()\r\n\r\nprint (G.nodes(data=True))\r\nplt.show()\r\nnx.draw(G, width=2, with_labels=True)\r\nplt.savefig(\"path5.png\")\r\n\r\nw=\"design\"\r\n\r\nG.add_node(w)\r\n\r\nbuild_graph(w, 10)\r\n\r\nprint (G.nodes(data=True))\r\nplt.show()\r\nnx.draw(G, width=2, with_labels=True)\r\nplt.savefig(\"path.png\")\r\n<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>This is a continuation of my previous post, found here Combining Machine Learning and Data Scraping. Data visualization is added to show correlations between words. The graph was built using NetworkX python library. The input for the graph is the array corr_data with 3 columns : pair of words and correlation between them. This was &#8230; <a title=\"Data Visualization of Word Correlations with NetworkX\" class=\"read-more\" href=\"http:\/\/intelligentonlinetools.com\/blog\/2017\/11\/02\/data-visualization-of-word-correlations-with-networkx\/\">Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":[]},"categories":[2,6,9,10],"tags":[29,18,27],"jetpack_publicize_connections":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v20.4 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Data Visualization of Word Correlations with NetworkX - Machine Learning Applications<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"http:\/\/intelligentonlinetools.com\/blog\/2017\/11\/02\/data-visualization-of-word-correlations-with-networkx\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Data Visualization of Word Correlations with NetworkX - Machine Learning Applications\" \/>\n<meta property=\"og:description\" content=\"This is a continuation of my previous post, found here Combining Machine Learning and Data Scraping. Data visualization is added to show correlations between words. The graph was built using NetworkX python library. The input for the graph is the array corr_data with 3 columns : pair of words and correlation between them. This was ... Read more\" \/>\n<meta property=\"og:url\" content=\"http:\/\/intelligentonlinetools.com\/blog\/2017\/11\/02\/data-visualization-of-word-correlations-with-networkx\/\" \/>\n<meta property=\"og:site_name\" content=\"Machine Learning Applications\" \/>\n<meta property=\"article:published_time\" content=\"2017-11-02T02:09:42+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2017-11-30T02:02:08+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/11\/NetworkX_graph-300x211.png\" \/>\n<meta name=\"author\" content=\"owygs156\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"owygs156\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/2017\/11\/02\/data-visualization-of-word-correlations-with-networkx\/\",\"url\":\"http:\/\/intelligentonlinetools.com\/blog\/2017\/11\/02\/data-visualization-of-word-correlations-with-networkx\/\",\"name\":\"Data Visualization of Word Correlations with NetworkX - Machine Learning Applications\",\"isPartOf\":{\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/#website\"},\"datePublished\":\"2017-11-02T02:09:42+00:00\",\"dateModified\":\"2017-11-30T02:02:08+00:00\",\"author\":{\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/7a886dc5eb9758369af2f6d2cb342478\"},\"breadcrumb\":{\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/2017\/11\/02\/data-visualization-of-word-correlations-with-networkx\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"http:\/\/intelligentonlinetools.com\/blog\/2017\/11\/02\/data-visualization-of-word-correlations-with-networkx\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/2017\/11\/02\/data-visualization-of-word-correlations-with-networkx\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"http:\/\/intelligentonlinetools.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Data Visualization of Word Correlations with NetworkX\"}]},{\"@type\":\"WebSite\",\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/#website\",\"url\":\"http:\/\/intelligentonlinetools.com\/blog\/\",\"name\":\"Machine Learning Applications\",\"description\":\"Artificial intelligence, data mining and machine learning for building web based tools and services.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"http:\/\/intelligentonlinetools.com\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/7a886dc5eb9758369af2f6d2cb342478\",\"name\":\"owygs156\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"http:\/\/2.gravatar.com\/avatar\/b351def598609cb4c0b5bca26497c7e5?s=96&d=mm&r=g\",\"contentUrl\":\"http:\/\/2.gravatar.com\/avatar\/b351def598609cb4c0b5bca26497c7e5?s=96&d=mm&r=g\",\"caption\":\"owygs156\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Data Visualization of Word Correlations with NetworkX - Machine Learning Applications","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"http:\/\/intelligentonlinetools.com\/blog\/2017\/11\/02\/data-visualization-of-word-correlations-with-networkx\/","og_locale":"en_US","og_type":"article","og_title":"Data Visualization of Word Correlations with NetworkX - Machine Learning Applications","og_description":"This is a continuation of my previous post, found here Combining Machine Learning and Data Scraping. Data visualization is added to show correlations between words. The graph was built using NetworkX python library. The input for the graph is the array corr_data with 3 columns : pair of words and correlation between them. This was ... Read more","og_url":"http:\/\/intelligentonlinetools.com\/blog\/2017\/11\/02\/data-visualization-of-word-correlations-with-networkx\/","og_site_name":"Machine Learning Applications","article_published_time":"2017-11-02T02:09:42+00:00","article_modified_time":"2017-11-30T02:02:08+00:00","og_image":[{"url":"http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/11\/NetworkX_graph-300x211.png"}],"author":"owygs156","twitter_card":"summary_large_image","twitter_misc":{"Written by":"owygs156","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"http:\/\/intelligentonlinetools.com\/blog\/2017\/11\/02\/data-visualization-of-word-correlations-with-networkx\/","url":"http:\/\/intelligentonlinetools.com\/blog\/2017\/11\/02\/data-visualization-of-word-correlations-with-networkx\/","name":"Data Visualization of Word Correlations with NetworkX - Machine Learning Applications","isPartOf":{"@id":"http:\/\/intelligentonlinetools.com\/blog\/#website"},"datePublished":"2017-11-02T02:09:42+00:00","dateModified":"2017-11-30T02:02:08+00:00","author":{"@id":"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/7a886dc5eb9758369af2f6d2cb342478"},"breadcrumb":{"@id":"http:\/\/intelligentonlinetools.com\/blog\/2017\/11\/02\/data-visualization-of-word-correlations-with-networkx\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["http:\/\/intelligentonlinetools.com\/blog\/2017\/11\/02\/data-visualization-of-word-correlations-with-networkx\/"]}]},{"@type":"BreadcrumbList","@id":"http:\/\/intelligentonlinetools.com\/blog\/2017\/11\/02\/data-visualization-of-word-correlations-with-networkx\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"http:\/\/intelligentonlinetools.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Data Visualization of Word Correlations with NetworkX"}]},{"@type":"WebSite","@id":"http:\/\/intelligentonlinetools.com\/blog\/#website","url":"http:\/\/intelligentonlinetools.com\/blog\/","name":"Machine Learning Applications","description":"Artificial intelligence, data mining and machine learning for building web based tools and services.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"http:\/\/intelligentonlinetools.com\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/7a886dc5eb9758369af2f6d2cb342478","name":"owygs156","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/image\/","url":"http:\/\/2.gravatar.com\/avatar\/b351def598609cb4c0b5bca26497c7e5?s=96&d=mm&r=g","contentUrl":"http:\/\/2.gravatar.com\/avatar\/b351def598609cb4c0b5bca26497c7e5?s=96&d=mm&r=g","caption":"owygs156"}}]}},"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p7h1IJ-mW","jetpack-related-posts":[{"id":966,"url":"http:\/\/intelligentonlinetools.com\/blog\/2017\/02\/18\/building-decision-trees-in-python\/","url_meta":{"origin":1422,"position":0},"title":"Building Decision Trees in Python","date":"February 18, 2017","format":false,"excerpt":"A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm. Decision trees are commonly used in operations research, specifically in decision analysis, to\u2026","rel":"","context":"In &quot;Artificial Intelligence&quot;","img":{"alt_text":"Decision Tree","src":"https:\/\/i0.wp.com\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/02\/dt_post1_N_CTQ_Cost_regr1-2-use-this-300x103.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":1857,"url":"http:\/\/intelligentonlinetools.com\/blog\/2018\/02\/10\/how-to-create-data-visualization-for-association-rules-in-data-mining\/","url_meta":{"origin":1422,"position":1},"title":"How to Create Data Visualization for Association Rules in Data Mining","date":"February 10, 2018","format":false,"excerpt":"Association rule learning is used in machine learning for discovering interesting relations between variables. Apriori algorithm is a popular algorithm for association rules mining and extracting frequent itemsets with applications in association rule learning. It has been designed to operate on databases containing transactions, such as purchases by customers of\u2026","rel":"","context":"In &quot;Data Mining&quot;","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2018\/02\/association_rules_scatter_plot_for_retailer_dataset.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":1385,"url":"http:\/\/intelligentonlinetools.com\/blog\/2017\/10\/15\/scraping\/","url_meta":{"origin":1422,"position":2},"title":"Combining Machine Learning and Data Scraping","date":"October 15, 2017","format":false,"excerpt":"I often come across web posts about extracting data (data scraping) from websites. For example recently in [1] Scrapy tool was used for web scraping with Python. Once we get scraping data we can use extracted information in many different ways. As computer algorithms evolve and can do more, the\u2026","rel":"","context":"In &quot;Data Mining&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":551,"url":"http:\/\/intelligentonlinetools.com\/blog\/2016\/09\/05\/getting-wordnet-information-and-building-and-building-graph-with-python-and-networkx\/","url_meta":{"origin":1422,"position":3},"title":"Getting WordNet Information and Building Graph with Python and NetworkX","date":"September 5, 2016","format":false,"excerpt":"WordNet and Wikipedia are often utilized in text mining algorithms for enriching short text representation [1] or for extracting additional knowledge about words. [2] WordNet's structure makes it a useful tool for computational linguistics and natural language processing.[3] In this post we will look how to pull information from WordNet\u2026","rel":"","context":"In &quot;Python Scripts&quot;","img":{"alt_text":"Wordnet_graph","src":"https:\/\/i0.wp.com\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2016\/09\/Wordnet_graph-300x180.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":1095,"url":"http:\/\/intelligentonlinetools.com\/blog\/2017\/03\/25\/3-most-useful-examples-to-add-interactivity-to-graph-data-using-bokeh-library\/","url_meta":{"origin":1422,"position":4},"title":"3 Most Useful Examples to Add Interactivity to Graph Data Using Bokeh Library","date":"March 25, 2017","format":false,"excerpt":"Bokeh is a Python library for building advanced and modern data visualization web applications. Bokeh allows to add interactive controls like slider, buttons, dropdown menu and so on to the data graphs. Bokeh provides a variety of ways to embed plots and data into HTML documents including generating standalone HTML\u2026","rel":"","context":"In &quot;Data Visualization&quot;","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/03\/bokeh-and-other-data-viz-300x138.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":256,"url":"http:\/\/intelligentonlinetools.com\/blog\/2016\/06\/05\/using-python-for-mining-data-from-twitter-visualization-and-other-enchancements\/","url_meta":{"origin":1422,"position":5},"title":"Using Python for Data Visualization of Clustering Results","date":"June 5, 2016","format":false,"excerpt":"In one of the previous post http:\/\/intelligentonlinetools.com\/blog\/2016\/05\/28\/using-python-for-mining-data-from-twitter\/ python source code for mining Twitter data was implemented. Clustering was applied to put tweets in different groups using bag of words representation for the text. The results of clustering were obtained via numerical matrix. Now we will look at visualization of clustering\u2026","rel":"","context":"In &quot;Artificial Intelligence&quot;","img":{"alt_text":"Data Visualization for Clustering Results","src":"https:\/\/i0.wp.com\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2016\/06\/data-visualization1-300x220.png?resize=350%2C200","width":350,"height":200},"classes":[]}],"_links":{"self":[{"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/posts\/1422"}],"collection":[{"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/comments?post=1422"}],"version-history":[{"count":11,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/posts\/1422\/revisions"}],"predecessor-version":[{"id":1563,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/posts\/1422\/revisions\/1563"}],"wp:attachment":[{"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/media?parent=1422"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/categories?post=1422"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/tags?post=1422"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}