{"id":2194,"date":"2018-08-11T02:31:22","date_gmt":"2018-08-11T02:31:22","guid":{"rendered":"http:\/\/intelligentonlinetools.com\/blog\/?p=2194"},"modified":"2018-08-16T22:51:48","modified_gmt":"2018-08-16T22:51:48","slug":"applied-machine-learning-classification-for-decision-making","status":"publish","type":"post","link":"http:\/\/intelligentonlinetools.com\/blog\/2018\/08\/11\/applied-machine-learning-classification-for-decision-making\/","title":{"rendered":"Applied Machine Learning Classification for Decision Making"},"content":{"rendered":"

Making the good decision is the challenge that we often have. So, in this post, we will look at how applied machine learning classification can be used for the process of decision making<\/b>.<\/p>\n

The simple and quick approach to make decision is follow our past experience of similar situations. Usually we use compiling a list of pros and cons, asking someone for help or searching on the web. According to [1] we have two systems in our brain: logical and intuitive system<\/b>: <\/p>\n

“With every decision you take, every judgement you make, there is a battle in your mind – a battle between intuition and logic.”<\/p>\n

“Most of the beliefs or opinions you have come from an automatic response. But then your logical mind invents a reason why you think or believe something.”
\n<\/em><\/p><\/blockquote>\n

Most of the time our intuitive system is working efficiently, taking charge of all the thousands of decisions we make each day. But our intuitive system can be biased in many ways. For example it can be biased toward the latest unsuccessful outcome. <\/p>\n

Besides this our memory can not remember a lot of information so we can not use efficiently all our past information and experience. That’s why people created tools like decision matrix<\/b> that can help to improve decision making. In the next section we will look at techniques that facilitates using our rational decision making.<\/p>\n

\"Decision<\/p>\n

Decision Matrix<\/h2>\n

More advanced approach for making decision is score each possible option. In this approach we score each option against some criteria or feature. For example for the candidate product that we need to buy we are looking at price, quality, service and safety features. This approach results in creating decision matrix for analysis of possible options.<\/p>\n

Below is an example of decision matrix<\/b> [3] for choosing strategy for building some software project. Here we score 4 options based on the time to build and cost. The score is on the scale 0 (worst) – 100 (best). After we score each cell for time and cost rows, we can get sum of scores, rank and then make our choice (see last 3 rows)<\/p>\n

\"Decision
Decision Matrix Example<\/figcaption><\/figure>\n

There are also other, similar to decision matrix tools: belief decision matrix[3], Pugh Matrix[4].
\nThese tools allow do comparison analysis of available options vs features and this enforces our mind logically evaluate and rank as much as possible pros and cons based on some numerical metrics. <\/p>\n

However there are some limitations<\/b> also. Incorrect selection criteria will obviously lead to the wrong conclusion. Poorly defined criteria can have multiple interpretations. For example too low can mean different things. With many rows or columns it becomes labor intensive to fill out the matrix.<\/p>\n

Machine Learning Approach for Making Decision<\/h2>\n

Machine learning techniques can also help us improve decision making and even solve some of the above limitations. For example with Feature Engineering we can evaluate what criteria are important for decision. <\/p>\n

In machine learning<\/b> making decision can be viewed as assigning or predicting correct label (for example buy, not buy) based on data for the item features. In the field of machine learning or AI this is known as classification problem. <\/p>\n

Classification algorithms<\/b> learn correct decisions from data. Below is the example of training data that we input to machine learning classification algorithm. (Xij represent some numerical values)<\/p>\n

\"\"
Classification problem<\/figcaption><\/figure>\n

Our options (decisions) are now represented by class label (most right column), criteria are represented by features. So we now switched columns with rows. Using training data like above we train classifier and then use it to choose the class (make decision) for new data.<\/p>\n

There are different classification algorithms such as decision tree, SVM, Naive Bayes, neural network classification. In this post we will look at classification with neural network.
\nWe will use Keras neural network with 2 dense layers. <\/p>\n

Per Keras documentation[5] Dense layer<\/b> implements the operation:
\n output = activation(dot(input, kernel) + bias)
\nwhere activation is the element-wise activation function passed as the activation argument,
\nkernel is a weights matrix created by the layer,
\nand bias is a bias vector created by the layer (only applicable if use_bias is True).<\/p>\n

So we can see that the dense layer is performing similar math that we were doing in decision matrix.<\/p>\n

Python Source Code for Neural Network Classification Algorithms<\/h2>\n

Finally, below is the python source code for classification<\/b> problem. To test this code we will use iris dataset. This dataset has 3 classes and 4 features. Our task here to make the classifier able to assign correct label class.
\n<\/p>\n

\r\n# -*- coding: utf-8 -*-\r\n\r\nfrom keras.utils import to_categorical\r\nfrom sklearn import datasets\r\n\r\niris = datasets.load_iris()\r\n\r\n# Create feature matrix\r\nX = iris.data\r\nprint (X)\r\n\r\n# Create target vector\r\ny = iris.target\r\ny = to_categorical(y, num_classes=3)\r\nprint (y)\r\n\r\nfrom sklearn.model_selection import train_test_split\r\n\r\nx_train, x_test, y_train, y_test = train_test_split( X, y, test_size=0.33, random_state=42)\r\n\r\nprint (x_train)\r\nprint (y_train)\r\n\r\nfrom keras.models import Sequential\r\nfrom keras.layers import Dense\r\n\r\nmodel = Sequential()\r\nmodel.add(Dense(32, input_shape=(4,), activation='relu',  name='L1'))\r\nmodel.add(Dense(3, activation='softmax', name='L2'))\r\nmodel.compile(optimizer='rmsprop',\r\n              loss='categorical_crossentropy',\r\n              metrics=['accuracy'])\r\n\r\nprint('Model Summary:')\r\nprint(model.summary())\r\n\r\nmodel.fit(x_train, y_train, verbose=2, batch_size=10, epochs=100)\r\noutput = model.evaluate(x_test, y_test)\r\n\r\nprint('Final test loss: {:4f}'.format(output[0]))\r\nprint('Final test accuracy: {:4f}'.format(output[1]))\r\n<\/pre>\n

\nBelow are results of neural network run.<\/p>\n
\"Results
Results of neural network run<\/figcaption><\/figure>\n
As we can see our classifier is able to make correct decisions<\/b> with 98% accuracy.<\/p>\n
Thus we investigated different approaches for making decision. We saw how machine learning can be applied to this too. Specifically we looked at neural network classification algorithm for selecting correct label.
\nI would love to hear what types of decision making tools do you use for making decisions? Also feel free to provide feedback or suggestions. <\/p>\n
References<\/strong>
\n1. How do we really make decisions?<\/a>
\n2. What Is a Decision Matrix? Definition and Examples<\/a>
\n3. Decision matrix  From Wikipedia, the free encyclopedia<\/a>
\n4. The Systems Engineering Tool Box<\/a>
\n5. Keras Documentation<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
Making the good decision is the challenge that we often have. So, in this post, we will look at how applied machine learning classification can be used for the process of decision making. The simple and quick approach to make decision is follow our past experience of similar situations. Usually we use compiling a list … Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":[]},"categories":[5,88,9],"tags":[87,85,84,86,27],"jetpack_publicize_connections":[],"yoast_head":"\nApplied Machine Learning Classification for Decision Making - Machine Learning Applications<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/intelligentonlinetools.com\/blog\/2018\/08\/11\/applied-machine-learning-classification-for-decision-making\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Applied Machine Learning Classification for Decision Making - Machine Learning Applications\" \/>\n<meta property=\"og:description\" content=\"Making the good decision is the challenge that we often have. So, in this post, we will look at how applied machine learning classification can be used for the process of decision making. The simple and quick approach to make decision is follow our past experience of similar situations. Usually we use compiling a list ... Read more\" \/>\n<meta property=\"og:url\" content=\"https:\/\/intelligentonlinetools.com\/blog\/2018\/08\/11\/applied-machine-learning-classification-for-decision-making\/\" \/>\n<meta property=\"og:site_name\" content=\"Machine Learning Applications\" \/>\n<meta property=\"article:published_time\" content=\"2018-08-11T02:31:22+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2018-08-16T22:51:48+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2018\/08\/signs-1172211_640-e1534384498570.jpg\" \/>\n<meta name=\"author\" content=\"owygs156\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"owygs156\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/intelligentonlinetools.com\/blog\/2018\/08\/11\/applied-machine-learning-classification-for-decision-making\/\",\"url\":\"https:\/\/intelligentonlinetools.com\/blog\/2018\/08\/11\/applied-machine-learning-classification-for-decision-making\/\",\"name\":\"Applied Machine Learning Classification for Decision Making - Machine Learning Applications\",\"isPartOf\":{\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/#website\"},\"datePublished\":\"2018-08-11T02:31:22+00:00\",\"dateModified\":\"2018-08-16T22:51:48+00:00\",\"author\":{\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/7a886dc5eb9758369af2f6d2cb342478\"},\"breadcrumb\":{\"@id\":\"https:\/\/intelligentonlinetools.com\/blog\/2018\/08\/11\/applied-machine-learning-classification-for-decision-making\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/intelligentonlinetools.com\/blog\/2018\/08\/11\/applied-machine-learning-classification-for-decision-making\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/intelligentonlinetools.com\/blog\/2018\/08\/11\/applied-machine-learning-classification-for-decision-making\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"http:\/\/intelligentonlinetools.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Applied Machine Learning Classification for Decision Making\"}]},{\"@type\":\"WebSite\",\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/#website\",\"url\":\"http:\/\/intelligentonlinetools.com\/blog\/\",\"name\":\"Machine Learning Applications\",\"description\":\"Artificial intelligence, data mining and machine learning for building web based tools and services.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"http:\/\/intelligentonlinetools.com\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/7a886dc5eb9758369af2f6d2cb342478\",\"name\":\"owygs156\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"http:\/\/2.gravatar.com\/avatar\/b351def598609cb4c0b5bca26497c7e5?s=96&d=mm&r=g\",\"contentUrl\":\"http:\/\/2.gravatar.com\/avatar\/b351def598609cb4c0b5bca26497c7e5?s=96&d=mm&r=g\",\"caption\":\"owygs156\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Applied Machine Learning Classification for Decision Making - Machine Learning Applications","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/intelligentonlinetools.com\/blog\/2018\/08\/11\/applied-machine-learning-classification-for-decision-making\/","og_locale":"en_US","og_type":"article","og_title":"Applied Machine Learning Classification for Decision Making - Machine Learning Applications","og_description":"Making the good decision is the challenge that we often have. So, in this post, we will look at how applied machine learning classification can be used for the process of decision making. The simple and quick approach to make decision is follow our past experience of similar situations. Usually we use compiling a list ... Read more","og_url":"https:\/\/intelligentonlinetools.com\/blog\/2018\/08\/11\/applied-machine-learning-classification-for-decision-making\/","og_site_name":"Machine Learning Applications","article_published_time":"2018-08-11T02:31:22+00:00","article_modified_time":"2018-08-16T22:51:48+00:00","og_image":[{"url":"http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2018\/08\/signs-1172211_640-e1534384498570.jpg"}],"author":"owygs156","twitter_card":"summary_large_image","twitter_misc":{"Written by":"owygs156","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/intelligentonlinetools.com\/blog\/2018\/08\/11\/applied-machine-learning-classification-for-decision-making\/","url":"https:\/\/intelligentonlinetools.com\/blog\/2018\/08\/11\/applied-machine-learning-classification-for-decision-making\/","name":"Applied Machine Learning Classification for Decision Making - Machine Learning Applications","isPartOf":{"@id":"http:\/\/intelligentonlinetools.com\/blog\/#website"},"datePublished":"2018-08-11T02:31:22+00:00","dateModified":"2018-08-16T22:51:48+00:00","author":{"@id":"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/7a886dc5eb9758369af2f6d2cb342478"},"breadcrumb":{"@id":"https:\/\/intelligentonlinetools.com\/blog\/2018\/08\/11\/applied-machine-learning-classification-for-decision-making\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/intelligentonlinetools.com\/blog\/2018\/08\/11\/applied-machine-learning-classification-for-decision-making\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/intelligentonlinetools.com\/blog\/2018\/08\/11\/applied-machine-learning-classification-for-decision-making\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"http:\/\/intelligentonlinetools.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Applied Machine Learning Classification for Decision Making"}]},{"@type":"WebSite","@id":"http:\/\/intelligentonlinetools.com\/blog\/#website","url":"http:\/\/intelligentonlinetools.com\/blog\/","name":"Machine Learning Applications","description":"Artificial intelligence, data mining and machine learning for building web based tools and services.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"http:\/\/intelligentonlinetools.com\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/7a886dc5eb9758369af2f6d2cb342478","name":"owygs156","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/image\/","url":"http:\/\/2.gravatar.com\/avatar\/b351def598609cb4c0b5bca26497c7e5?s=96&d=mm&r=g","contentUrl":"http:\/\/2.gravatar.com\/avatar\/b351def598609cb4c0b5bca26497c7e5?s=96&d=mm&r=g","caption":"owygs156"}}]}},"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p7h1IJ-zo","jetpack-related-posts":[{"id":966,"url":"http:\/\/intelligentonlinetools.com\/blog\/2017\/02\/18\/building-decision-trees-in-python\/","url_meta":{"origin":2194,"position":0},"title":"Building Decision Trees in Python","date":"February 18, 2017","format":false,"excerpt":"A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm. Decision trees are commonly used in operations research, specifically in decision analysis, to\u2026","rel":"","context":"In "Artificial Intelligence"","img":{"alt_text":"Decision Tree","src":"https:\/\/i0.wp.com\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/02\/dt_post1_N_CTQ_Cost_regr1-2-use-this-300x103.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":975,"url":"http:\/\/intelligentonlinetools.com\/blog\/2017\/02\/18\/building-decision-trees-in-python-handling-categorical-data\/","url_meta":{"origin":2194,"position":1},"title":"Building Decision Trees in Python – Handling Categorical Data","date":"February 18, 2017","format":false,"excerpt":"In the post Building Decision Trees in Python we looked at the decision tree with numerical continuous dependent variable. This type of decision trees can be called also regression tree. But what if we need to use categorical dependent variable? It is still possible to create decision tree and in\u2026","rel":"","context":"In "Artificial Intelligence"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/02\/dt-235x300.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":1516,"url":"http:\/\/intelligentonlinetools.com\/blog\/2017\/11\/23\/regression-and-classification-decision-trees-building-with-python-and-running-online\/","url_meta":{"origin":2194,"position":2},"title":"Regression and Classification Decision Trees – Building with Python and Running Online","date":"November 23, 2017","format":false,"excerpt":"According to survey [1] Decision Trees constitute one of the 10 most popular data mining algorithms. Decision trees used in data mining are of two main types: Classification tree analysis is when the predicted outcome is the class to which the data belongs. Regression tree analysis is when the predicted\u2026","rel":"","context":"In "Data Mining"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/11\/decision_tree_11_2017-300x283.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":875,"url":"http:\/\/intelligentonlinetools.com\/blog\/2017\/01\/28\/iris-plant-classification-using-neural-network-online-experiments-with-normalization-and-other-parameters\/","url_meta":{"origin":2194,"position":3},"title":"Iris Plant Classification Using Neural Network – Online Experiments with Normalization and Other Parameters","date":"January 28, 2017","format":false,"excerpt":"Do we need to normalize input data for neural network? How differently will be results from running normalized and non normalized data? This will be explored in the post using Online Machine Learning Algorithms tool for classification of iris data set with feed-forward neural network. Feed-forward Neural Network Feed-forward neural\u2026","rel":"","context":"In "Artificial Intelligence"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/01\/delta-error-graph-from-iris-data-set-36-neurons-300x231.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":1178,"url":"http:\/\/intelligentonlinetools.com\/blog\/2017\/05\/14\/time-series-prediction-with-convolutional-neural-networks\/","url_meta":{"origin":2194,"position":4},"title":"Forecasting Time Series Data with Convolutional Neural Networks","date":"May 14, 2017","format":false,"excerpt":"Convolutional neural networks(CNN) is increasingly important concept in computer science and finds more and more applications in different fields. Many posts on the web are about applying convolutional neural networks for image classification as CNN is very useful type of neural networks for image classification. But convolutional neural networks can\u2026","rel":"","context":"In "Artificial Intelligence"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/05\/time-series-LSTM-300x164.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":65,"url":"http:\/\/intelligentonlinetools.com\/blog\/2016\/02\/09\/cloud-computing\/","url_meta":{"origin":2194,"position":5},"title":"Cloud Computing","date":"February 9, 2016","format":false,"excerpt":"Cloud computing is the use of computing resources (hardware and software) that are delivered as a service over a network (typically the Internet). The name comes from the use of a cloud-shaped symbol as an abstraction for the complex infrastructure it contains in system diagrams. Cloud computing entrusts remote services\u2026","rel":"","context":"Similar post","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/posts\/2194"}],"collection":[{"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/comments?post=2194"}],"version-history":[{"count":48,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/posts\/2194\/revisions"}],"predecessor-version":[{"id":2246,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/posts\/2194\/revisions\/2246"}],"wp:attachment":[{"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/media?parent=2194"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/categories?post=2194"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/tags?post=2194"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}