{"id":1257,"date":"2017-06-17T19:38:02","date_gmt":"2017-06-17T19:38:02","guid":{"rendered":"http:\/\/intelligentonlinetools.com\/blog\/?p=1257"},"modified":"2017-12-05T01:50:32","modified_gmt":"2017-12-05T01:50:32","slug":"time-series-prediction-with-convolutional-neural-networks-and-keras","status":"publish","type":"post","link":"http:\/\/intelligentonlinetools.com\/blog\/2017\/06\/17\/time-series-prediction-with-convolutional-neural-networks-and-keras\/","title":{"rendered":"Time Series Prediction with Convolutional Neural Networks and Keras"},"content":{"rendered":"<p>A convolutional neural network (CNNs) is a type of network that has recently<br \/>\ngained popularity due to its success in classification problems (e.g. image recognition<br \/>\nor time series classification) [1].   One of the working examples how to use Keras CNN for time series can be found at  <a href=https:\/\/gist.github.com\/jkleint\/1d878d0401b28b281eb75016ed29f2ee target=\"_blank\">this link<\/a>[2].  This example allows to predict for single time series and multivariate time series.<\/p>\n<p>Running the code from this link, it was noticed that sometimes the prediction error has very high value, may be because optimizer gets stuck in local minimum. (See Fig1. The error is on axis Y and is very high for run 6) So I updated the script to run several times and then remove results with high error. (See Fig2 The Y axis showing small error values). Here is the summary of all changes:<\/p>\n<ul>\n<li>Created multiple runs that can allow to filter bad results based on error. Training CNN is running 10 times and for each run error data and some other associated data is saved. Error is calculated as square root of sum if squared errors for last 10 predictions during the training.<\/li>\n<li>Added also plot to see error over multiple runs. <\/li>\n<li>In the end of script added one plot that showing errors for each run (See Fig1.) , and another plot showing errors only for runs that did not have high error (See Fig2.).<\/li>\n<li>Added saving keras model to file for each run and then loading it from file for model that showed best results (min error). See [3] for more information on saving and loading keras models.<\/li>\n<li>Added ability to load time series from csv file.<\/li>\n<\/ul>\n<p><figure id=\"attachment_1271\" aria-describedby=\"caption-attachment-1271\" style=\"width: 290px\" class=\"wp-caption alignnone\"><img data-attachment-id=\"1271\" data-permalink=\"http:\/\/intelligentonlinetools.com\/blog\/2017\/06\/17\/time-series-prediction-with-convolutional-neural-networks-and-keras\/cnn_error_total_all_runs\/#main\" data-orig-file=\"http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/06\/CNN_error_total_all_runs.png\" data-orig-size=\"389,258\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"CNN_error_total_all_runs\" data-image-description=\"&lt;p&gt;Error for all runs&lt;\/p&gt;\n\" data-image-caption=\"&lt;p&gt;Error for all runs&lt;\/p&gt;\n\" data-medium-file=\"http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/06\/CNN_error_total_all_runs-300x199.png\" data-large-file=\"http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/06\/CNN_error_total_all_runs.png\" decoding=\"async\" loading=\"lazy\" src=\"http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/06\/CNN_error_total_all_runs-300x199.png\" alt=\"Error for all runs\" width=\"300\" height=\"199\" class=\"size-medium wp-image-1271\" srcset=\"http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/06\/CNN_error_total_all_runs-300x199.png 300w, http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/06\/CNN_error_total_all_runs.png 389w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><figcaption id=\"caption-attachment-1271\" class=\"wp-caption-text\">Error for all runs<\/figcaption><\/figure>Fig 1.<\/p>\n<p><figure id=\"attachment_1272\" aria-describedby=\"caption-attachment-1272\" style=\"width: 290px\" class=\"wp-caption alignnone\"><img data-attachment-id=\"1272\" data-permalink=\"http:\/\/intelligentonlinetools.com\/blog\/2017\/06\/17\/time-series-prediction-with-convolutional-neural-networks-and-keras\/cnn22\/#main\" data-orig-file=\"http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/06\/CNN22.png\" data-orig-size=\"369,261\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"CNN22\" data-image-description=\"&lt;p&gt;Error chart after removing runs with high value error&lt;\/p&gt;\n\" data-image-caption=\"&lt;p&gt;Error chart after removing runs with high value error&lt;\/p&gt;\n\" data-medium-file=\"http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/06\/CNN22-300x212.png\" data-large-file=\"http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/06\/CNN22.png\" decoding=\"async\" loading=\"lazy\" src=\"http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/06\/CNN22-300x212.png\" alt=\"Error chart after removing runs with high value error\" width=\"300\" height=\"212\" class=\"size-medium wp-image-1272\" srcset=\"http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/06\/CNN22-300x212.png 300w, http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/06\/CNN22.png 369w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><figcaption id=\"caption-attachment-1272\" class=\"wp-caption-text\">Error chart after removing runs with high value error<\/figcaption><\/figure>Fig 2.<\/p>\n<p>Below is the full code<\/p>\n<pre><code>\r\n#!\/usr\/bin\/env python\r\n\"\"\"\r\nThis code is based on convolutional neural network model from below link\r\ngist.github.com\/jkleint\/1d878d0401b28b281eb75016ed29f2ee\r\n\"\"\"\r\n\r\nfrom __future__ import print_function, division\r\n\r\nimport numpy as np\r\nfrom keras.layers import Convolution1D, Dense, MaxPooling1D, Flatten\r\nfrom keras.models import Sequential\r\nfrom keras.models import model_from_json\r\n\r\nimport matplotlib.pyplot as plt\r\nimport csv\r\n\r\n__date__ = '2017-06-22'\r\n\r\nerror_total =[]\r\nresult=[]\r\ni=0\r\n\r\ndef make_timeseries_regressor(window_size, filter_length, nb_input_series=1, nb_outputs=1, nb_filter=4):\r\n    \"\"\":Return: a Keras Model for predicting the next value in a timeseries given a fixed-size lookback window of previous values.\r\n\r\n    The model can handle multiple input timeseries (`nb_input_series`) and multiple prediction targets (`nb_outputs`).\r\n\r\n    :param int window_size: The number of previous timeseries values to use as input features.  Also called lag or lookback.\r\n    :param int nb_input_series: The number of input timeseries; 1 for a single timeseries.\r\n      The `X` input to ``fit()`` should be an array of shape ``(n_instances, window_size, nb_input_series)``; each instance is\r\n      a 2D array of shape ``(window_size, nb_input_series)``.  For example, for `window_size` = 3 and `nb_input_series` = 1 (a\r\n      single timeseries), one instance could be ``[[0], [1], [2]]``. See ``make_timeseries_instances()``.\r\n    :param int nb_outputs: The output dimension, often equal to the number of inputs.\r\n      For each input instance (array with shape ``(window_size, nb_input_series)``), the output is a vector of size `nb_outputs`,\r\n      usually the value(s) predicted to come after the last value in that input instance, i.e., the next value\r\n      in the sequence. The `y` input to ``fit()`` should be an array of shape ``(n_instances, nb_outputs)``.\r\n    :param int filter_length: the size (along the `window_size` dimension) of the sliding window that gets convolved with\r\n      each position along each instance. The difference between 1D and 2D convolution is that a 1D filter's \"height\" is fixed\r\n      to the number of input timeseries (its \"width\" being `filter_length`), and it can only slide along the window\r\n      dimension.  This is useful as generally the input timeseries have no spatial\/ordinal relationship, so it's not\r\n      meaningful to look for patterns that are invariant with respect to subsets of the timeseries.\r\n    :param int nb_filter: The number of different filters to learn (roughly, input patterns to recognize).\r\n    \"\"\"\r\n    model = Sequential((\r\n        # The first conv layer learns `nb_filter` filters (aka kernels), each of size ``(filter_length, nb_input_series)``.\r\n        # Its output will have shape (None, window_size - filter_length + 1, nb_filter), i.e., for each position in\r\n        # the input timeseries, the activation of each filter at that position.\r\n        Convolution1D(nb_filter=nb_filter, filter_length=filter_length, activation='relu', input_shape=(window_size, nb_input_series)),\r\n        MaxPooling1D(),     # Downsample the output of convolution by 2X.\r\n        Convolution1D(nb_filter=nb_filter, filter_length=filter_length, activation='relu'),\r\n        MaxPooling1D(),\r\n        Flatten(),\r\n        Dense(nb_outputs, activation='linear'),     # For binary classification, change the activation to 'sigmoid'\r\n    ))\r\n    model.compile(loss='mse', optimizer='adam', metrics=['mae'])\r\n    # To perform (binary) classification instead:\r\n    # model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['binary_accuracy'])\r\n    return model\r\n\r\n\r\ndef make_timeseries_instances(timeseries, window_size):\r\n    \"\"\"Make input features and prediction targets from a `timeseries` for use in machine learning.\r\n\r\n    :return: A tuple of `(X, y, q)`.  `X` are the inputs to a predictor, a 3D ndarray with shape\r\n      ``(timeseries.shape[0] - window_size, window_size, timeseries.shape[1] or 1)``.  For each row of `X`, the\r\n      corresponding row of `y` is the next value in the timeseries.  The `q` or query is the last instance, what you would use\r\n      to predict a hypothetical next (unprovided) value in the `timeseries`.\r\n    :param ndarray timeseries: Either a simple vector, or a matrix of shape ``(timestep, series_num)``, i.e., time is axis 0 (the\r\n      row) and the series is axis 1 (the column).\r\n    :param int window_size: The number of samples to use as input prediction features (also called the lag or lookback).\r\n    \"\"\"\r\n    timeseries = np.asarray(timeseries)\r\n    assert 0 < window_size < timeseries.shape[0]\r\n    X = np.atleast_3d(np.array([timeseries[start:start + window_size] for start in range(0, timeseries.shape[0] - window_size)]))\r\n    y = timeseries[window_size:]\r\n    q = np.atleast_3d([timeseries[-window_size:]])\r\n    return X, y, q\r\n\r\n\r\ndef evaluate_timeseries(timeseries, window_size):\r\n    \"\"\"Create a 1D CNN regressor to predict the next value in a `timeseries` using the preceding `window_size` elements\r\n    as input features and evaluate its performance.\r\n\r\n    :param ndarray timeseries: Timeseries data with time increasing down the rows (the leading dimension\/axis).\r\n    :param int window_size: The number of previous timeseries values to use to predict the next.\r\n    \"\"\"\r\n    filter_length = 5\r\n    nb_filter = 4\r\n    timeseries = np.atleast_2d(timeseries)\r\n    if timeseries.shape[0] == 1:\r\n        timeseries = timeseries.T       # Convert 1D vectors to 2D column vectors\r\n\r\n    nb_samples, nb_series = timeseries.shape\r\n    print('\\n\\nTimeseries ({} samples by {} series):\\n'.format(nb_samples, nb_series), timeseries)\r\n    model = make_timeseries_regressor(window_size=window_size, filter_length=filter_length, nb_input_series=nb_series, nb_outputs=nb_series, nb_filter=nb_filter)\r\n    print('\\n\\nModel with input size {}, output size {}, {} conv filters of length {}'.format(model.input_shape, model.output_shape, nb_filter, filter_length))\r\n    model.summary()\r\n\r\n    error=[]\r\n    \r\n    X, y, q = make_timeseries_instances(timeseries, window_size)\r\n    print('\\n\\nInput features:', X, '\\n\\nOutput labels:', y, '\\n\\nQuery vector:', q, sep='\\n')\r\n    test_size = int(0.01 * nb_samples)           # In real life you'd want to use 0.2 - 0.5\r\n    X_train, X_test, y_train, y_test = X[:-test_size], X[-test_size:], y[:-test_size], y[-test_size:]\r\n    model.fit(X_train, y_train, nb_epoch=25, batch_size=2, validation_data=(X_test, y_test))\r\n\r\n    # serialize model to JSON\r\n    model_json = model.to_json()\r\n    with open(\"model\"+str(i)+\".json\", \"w\") as json_file:\r\n          json_file.write(model_json)\r\n    # serialize weights to HDF5\r\n    model.save_weights(\"model\"+str(i)+\".h5\")\r\n    print(\"Saved model to disk\")\r\n    global i\r\n    i=i+1\r\n\r\n    pred = model.predict(X_test)\r\n    print('\\n\\nactual', 'predicted', sep='\\t')\r\n    error_curr=0\r\n    for actual, predicted in zip(y_test, pred.squeeze()):\r\n        print(actual.squeeze(), predicted, sep='\\t')\r\n        tmp = actual-predicted\r\n        sum_squared = np.dot(tmp.T , tmp)\r\n        error.append ( np.sqrt(sum_squared) )\r\n        error_curr=error_curr+ np.sqrt(sum_squared)\r\n    print('next', model.predict(q).squeeze(), sep='\\t')\r\n    result.append  (model.predict(q).squeeze())\r\n    error_total.append (error_curr)\r\n    print (error)\r\n\r\ndef read_file(fn):\r\n    '''\r\n    Reads the CSV file \r\n    -----\r\n    RETURNS:\r\n        A matrix with the file contents\r\n    '''\r\n\r\n    vals = []\r\n    with open(fn, 'r') as csvfile:\r\n        tsdata = csv.reader(csvfile, delimiter=',')\r\n        for row in tsdata:\r\n             vals.append(row) \r\n\r\n    # removing title row\r\n    vals = vals[1:]\r\n    y = np.array(vals).astype(np.float) \r\n    return y\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\ndef main():\r\n    \"\"\"Prepare input data, build model, eval uate.\"\"\"\r\n    np.set_printoptions(threshold=25)\r\n    ts_length = 1000\r\n    window_size = 50\r\n    number_of_runs=10\r\n    error_max=200\r\n    \r\n    print('\\nSimple single timeseries vector prediction')\r\n    timeseries = np.arange(ts_length)                   # The timeseries f(t) = t\r\n    # enable below line to run this time series\r\n    #evaluate_timeseries(timeseries, window_size)\r\n\r\n    print('\\nMultiple-input, multiple-output prediction')\r\n    timeseries = np.array([np.arange(ts_length), -np.arange(ts_length)]).T      # The timeseries f(t) = [t, -t]\r\n    # enable below line to run this time series\r\n    ##evaluate_timeseries(timeseries, window_size)\r\n\r\n    print('\\nMultiple-input, multiple-output prediction')\r\n    timeseries = np.array([np.arange(ts_length), -np.arange(ts_length), 2000-np.arange(ts_length)]).T      # The timeseries f(t) = [t, -t]\r\n    # enable below line to run this time series\r\n    #evaluate_timeseries(timeseries, window_size)\r\n    \r\n    timeseries = read_file('ts_input.csv')\r\n    print (timeseries)\r\n    \r\n    for i in range(number_of_runs):\r\n        evaluate_timeseries(timeseries, window_size)\r\n        \r\n    error_total_new=[]    \r\n    for i in range(number_of_runs):\r\n        if (error_total[i] < error_max):  \r\n            error_total_new.append (error_total[i])\r\n    \r\n    plt.plot(error_total)\r\n    plt.show()\r\n    print (result)\r\n    \r\n    plt.plot(error_total_new)\r\n    plt.show()\r\n    print (result)\r\n    \r\n    best_model=np.asarray(error_total).argmin(axis=0)\r\n    print (\"best_model=\"+str(best_model))\r\n   \r\n    \r\n    \r\n     \r\n    json_file = open('model'+str(best_model)+'.json', 'r')\r\n    loaded_model_json = json_file.read()\r\n    json_file.close()\r\n    loaded_model = model_from_json(loaded_model_json)\r\n    \r\n    \r\n    # load weights into new model\r\n    loaded_model.load_weights(\"model\"+str(best_model)+\".h5\")\r\n    print(\"Loaded model from disk\")\r\n \r\n     \r\n    \r\n    \r\n    \r\nif __name__ == '__main__':\r\n    main()\r\n\r\n<\/code><\/pre>\n<p><strong>References<\/strong><br \/>\n1. <a href=https:\/\/arxiv.org\/pdf\/1703.04691.pdf target=\"_blank\">Conditional Time Series Forecasting with Convolutional Neural Networks<\/a><br \/>\n2. <a href=https:\/\/gist.github.com\/jkleint\/1d878d0401b28b281eb75016ed29f2ee target=\"_blank\">Example of using Keras to implement a 1D convolutional neural network (CNN) for timeseries prediction<\/a><br \/>\n3. <a href=http:\/\/machinelearningmastery.com\/save-load-keras-deep-learning-models\/ target=\"_blank\">        Save and Load Your Keras Deep Learning Models<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>A convolutional neural network (CNNs) is a type of network that has recently gained popularity due to its success in classification problems (e.g. image recognition or time series classification) [1]. One of the working examples how to use Keras CNN for time series can be found at this link[2]. This example allows to predict for &#8230; <a title=\"Time Series Prediction with Convolutional Neural Networks and Keras\" class=\"read-more\" href=\"http:\/\/intelligentonlinetools.com\/blog\/2017\/06\/17\/time-series-prediction-with-convolutional-neural-networks-and-keras\/\">Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":[]},"categories":[5,9,10],"tags":[15,17,18,14,16],"jetpack_publicize_connections":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v20.4 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Time Series Prediction with Convolutional Neural Networks and Keras - Machine Learning Applications<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"http:\/\/intelligentonlinetools.com\/blog\/2017\/06\/17\/time-series-prediction-with-convolutional-neural-networks-and-keras\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Time Series Prediction with Convolutional Neural Networks and Keras - Machine Learning Applications\" \/>\n<meta property=\"og:description\" content=\"A convolutional neural network (CNNs) is a type of network that has recently gained popularity due to its success in classification problems (e.g. image recognition or time series classification) [1]. One of the working examples how to use Keras CNN for time series can be found at this link[2]. This example allows to predict for ... Read more\" \/>\n<meta property=\"og:url\" content=\"http:\/\/intelligentonlinetools.com\/blog\/2017\/06\/17\/time-series-prediction-with-convolutional-neural-networks-and-keras\/\" \/>\n<meta property=\"og:site_name\" content=\"Machine Learning Applications\" \/>\n<meta property=\"article:published_time\" content=\"2017-06-17T19:38:02+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2017-12-05T01:50:32+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/06\/CNN_error_total_all_runs-300x199.png\" \/>\n<meta name=\"author\" content=\"owygs156\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"owygs156\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"9 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/2017\/06\/17\/time-series-prediction-with-convolutional-neural-networks-and-keras\/\",\"url\":\"http:\/\/intelligentonlinetools.com\/blog\/2017\/06\/17\/time-series-prediction-with-convolutional-neural-networks-and-keras\/\",\"name\":\"Time Series Prediction with Convolutional Neural Networks and Keras - Machine Learning Applications\",\"isPartOf\":{\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/#website\"},\"datePublished\":\"2017-06-17T19:38:02+00:00\",\"dateModified\":\"2017-12-05T01:50:32+00:00\",\"author\":{\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/7a886dc5eb9758369af2f6d2cb342478\"},\"breadcrumb\":{\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/2017\/06\/17\/time-series-prediction-with-convolutional-neural-networks-and-keras\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"http:\/\/intelligentonlinetools.com\/blog\/2017\/06\/17\/time-series-prediction-with-convolutional-neural-networks-and-keras\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/2017\/06\/17\/time-series-prediction-with-convolutional-neural-networks-and-keras\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"http:\/\/intelligentonlinetools.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Time Series Prediction with Convolutional Neural Networks and Keras\"}]},{\"@type\":\"WebSite\",\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/#website\",\"url\":\"http:\/\/intelligentonlinetools.com\/blog\/\",\"name\":\"Machine Learning Applications\",\"description\":\"Artificial intelligence, data mining and machine learning for building web based tools and services.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"http:\/\/intelligentonlinetools.com\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/7a886dc5eb9758369af2f6d2cb342478\",\"name\":\"owygs156\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"http:\/\/2.gravatar.com\/avatar\/b351def598609cb4c0b5bca26497c7e5?s=96&d=mm&r=g\",\"contentUrl\":\"http:\/\/2.gravatar.com\/avatar\/b351def598609cb4c0b5bca26497c7e5?s=96&d=mm&r=g\",\"caption\":\"owygs156\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Time Series Prediction with Convolutional Neural Networks and Keras - Machine Learning Applications","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"http:\/\/intelligentonlinetools.com\/blog\/2017\/06\/17\/time-series-prediction-with-convolutional-neural-networks-and-keras\/","og_locale":"en_US","og_type":"article","og_title":"Time Series Prediction with Convolutional Neural Networks and Keras - Machine Learning Applications","og_description":"A convolutional neural network (CNNs) is a type of network that has recently gained popularity due to its success in classification problems (e.g. image recognition or time series classification) [1]. One of the working examples how to use Keras CNN for time series can be found at this link[2]. This example allows to predict for ... Read more","og_url":"http:\/\/intelligentonlinetools.com\/blog\/2017\/06\/17\/time-series-prediction-with-convolutional-neural-networks-and-keras\/","og_site_name":"Machine Learning Applications","article_published_time":"2017-06-17T19:38:02+00:00","article_modified_time":"2017-12-05T01:50:32+00:00","og_image":[{"url":"http:\/\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/06\/CNN_error_total_all_runs-300x199.png"}],"author":"owygs156","twitter_card":"summary_large_image","twitter_misc":{"Written by":"owygs156","Est. reading time":"9 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"http:\/\/intelligentonlinetools.com\/blog\/2017\/06\/17\/time-series-prediction-with-convolutional-neural-networks-and-keras\/","url":"http:\/\/intelligentonlinetools.com\/blog\/2017\/06\/17\/time-series-prediction-with-convolutional-neural-networks-and-keras\/","name":"Time Series Prediction with Convolutional Neural Networks and Keras - Machine Learning Applications","isPartOf":{"@id":"http:\/\/intelligentonlinetools.com\/blog\/#website"},"datePublished":"2017-06-17T19:38:02+00:00","dateModified":"2017-12-05T01:50:32+00:00","author":{"@id":"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/7a886dc5eb9758369af2f6d2cb342478"},"breadcrumb":{"@id":"http:\/\/intelligentonlinetools.com\/blog\/2017\/06\/17\/time-series-prediction-with-convolutional-neural-networks-and-keras\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["http:\/\/intelligentonlinetools.com\/blog\/2017\/06\/17\/time-series-prediction-with-convolutional-neural-networks-and-keras\/"]}]},{"@type":"BreadcrumbList","@id":"http:\/\/intelligentonlinetools.com\/blog\/2017\/06\/17\/time-series-prediction-with-convolutional-neural-networks-and-keras\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"http:\/\/intelligentonlinetools.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Time Series Prediction with Convolutional Neural Networks and Keras"}]},{"@type":"WebSite","@id":"http:\/\/intelligentonlinetools.com\/blog\/#website","url":"http:\/\/intelligentonlinetools.com\/blog\/","name":"Machine Learning Applications","description":"Artificial intelligence, data mining and machine learning for building web based tools and services.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"http:\/\/intelligentonlinetools.com\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/7a886dc5eb9758369af2f6d2cb342478","name":"owygs156","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"http:\/\/intelligentonlinetools.com\/blog\/#\/schema\/person\/image\/","url":"http:\/\/2.gravatar.com\/avatar\/b351def598609cb4c0b5bca26497c7e5?s=96&d=mm&r=g","contentUrl":"http:\/\/2.gravatar.com\/avatar\/b351def598609cb4c0b5bca26497c7e5?s=96&d=mm&r=g","caption":"owygs156"}}]}},"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p7h1IJ-kh","jetpack-related-posts":[{"id":1178,"url":"http:\/\/intelligentonlinetools.com\/blog\/2017\/05\/14\/time-series-prediction-with-convolutional-neural-networks\/","url_meta":{"origin":1257,"position":0},"title":"Forecasting Time Series Data with Convolutional Neural Networks","date":"May 14, 2017","format":false,"excerpt":"Convolutional neural networks(CNN) is increasingly important concept in computer science and finds more and more applications in different fields. Many posts on the web are about applying convolutional neural networks for image classification as CNN is very useful type of neural networks for image classification. But convolutional neural networks can\u2026","rel":"","context":"In &quot;Artificial Intelligence&quot;","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/05\/time-series-LSTM-300x164.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":1446,"url":"http:\/\/intelligentonlinetools.com\/blog\/2017\/11\/06\/10-new-top-resources-on-machine-learning-from-around-the-web\/","url_meta":{"origin":1257,"position":1},"title":"10 New Top Resources on Machine Learning from Around the Web","date":"November 6, 2017","format":false,"excerpt":"For this post I put new and most interesting machine learning resources that I recently found on the web. This is the list of useful resources in such areas like stock market forecasting, text mining, deep learning, neural networks and getting data from Twitter. Hope you enjoy the reading. 1.\u2026","rel":"","context":"In &quot;Machine Learning&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1783,"url":"http:\/\/intelligentonlinetools.com\/blog\/2018\/01\/19\/machine-learning-stock-prediction-lstm-keras\/","url_meta":{"origin":1257,"position":2},"title":"Machine Learning Stock Prediction with LSTM and Keras","date":"January 19, 2018","format":false,"excerpt":"In this post I will share experiments on machine learning stock prediction with LSTM and Keras with one step ahead. I tried to do first multiple steps ahead with few techniques described in the papers on the web. But I discovered that I need fully understand and test the simplest\u2026","rel":"","context":"In &quot;Machine Learning&quot;","img":{"alt_text":"Forecasting one step ahead LSTM 60","src":"https:\/\/i0.wp.com\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2018\/01\/Forecasting-one-step-ahead-LSTM-60-3_1_2018.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":1754,"url":"http:\/\/intelligentonlinetools.com\/blog\/2018\/02\/27\/time-series-prediction-lstm-keras\/","url_meta":{"origin":1257,"position":3},"title":"Time Series Prediction with LSTM and Keras for Multiple Steps Ahead","date":"February 27, 2018","format":false,"excerpt":"In this post I will share experiment with Time Series Prediction with LSTM and Keras. LSTM neural network is used in this experiment for multiple steps ahead for stock prices data. The experiment is based on the paper [1]. The authors of the paper examine independent value prediction approach. With\u2026","rel":"","context":"In &quot;Machine Learning&quot;","img":{"alt_text":"Multiple step prediction with separate neural networks","src":"https:\/\/i0.wp.com\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2018\/02\/multiple-step-prediction.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":875,"url":"http:\/\/intelligentonlinetools.com\/blog\/2017\/01\/28\/iris-plant-classification-using-neural-network-online-experiments-with-normalization-and-other-parameters\/","url_meta":{"origin":1257,"position":4},"title":"Iris Plant Classification Using Neural Network &#8211; Online Experiments with Normalization and Other Parameters","date":"January 28, 2017","format":false,"excerpt":"Do we need to normalize input data for neural network? How differently will be results from running normalized and non normalized data? This will be explored in the post using Online Machine Learning Algorithms tool for classification of iris data set with feed-forward neural network. Feed-forward Neural Network Feed-forward neural\u2026","rel":"","context":"In &quot;Artificial Intelligence&quot;","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2017\/01\/delta-error-graph-from-iris-data-set-36-neurons-300x231.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":1995,"url":"http:\/\/intelligentonlinetools.com\/blog\/2018\/04\/17\/lstm-neural-network-training-techniques-tuning-hyperparameters\/","url_meta":{"origin":1257,"position":5},"title":"LSTM Neural Network Training &#8211; Few Useful Techniques for Tuning Hyperparameters and Saving Time","date":"April 17, 2018","format":false,"excerpt":"Neural networks are among the most widely used machine learning techniques.[1] But neural network training and tuning multiple hyper-parameters takes time. I was recently building LSTM neural network for prediction for this post Machine Learning Stock Market Prediction with LSTM Keras and I learned some tricks that can save time.\u2026","rel":"","context":"In &quot;Machine Learning&quot;","img":{"alt_text":"LSTM NN Training Value Loss Charts with High Number and Adjusted","src":"https:\/\/i0.wp.com\/intelligentonlinetools.com\/blog\/wp-content\/uploads\/2018\/04\/LSTM-NN-Training-Value-Loss-with-High-Number-and-adjusted-e1524097124417.png?resize=350%2C200","width":350,"height":200},"classes":[]}],"_links":{"self":[{"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/posts\/1257"}],"collection":[{"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/comments?post=1257"}],"version-history":[{"count":18,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/posts\/1257\/revisions"}],"predecessor-version":[{"id":1603,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/posts\/1257\/revisions\/1603"}],"wp:attachment":[{"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/media?parent=1257"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/categories?post=1257"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/intelligentonlinetools.com\/blog\/wp-json\/wp\/v2\/tags?post=1257"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}