|
2 | 2 | "cells": [ |
3 | 3 | { |
4 | 4 | "cell_type": "markdown", |
5 | | - "metadata": { |
6 | | - "deletable": true, |
7 | | - "editable": true |
8 | | - }, |
| 5 | + "metadata": {}, |
9 | 6 | "source": [ |
10 | 7 | "# Ch `10`: Concept `02`" |
11 | 8 | ] |
12 | 9 | }, |
13 | 10 | { |
14 | 11 | "cell_type": "markdown", |
15 | | - "metadata": { |
16 | | - "deletable": true, |
17 | | - "editable": true |
18 | | - }, |
| 12 | + "metadata": {}, |
19 | 13 | "source": [ |
20 | 14 | "## Recurrent Neural Network" |
21 | 15 | ] |
22 | 16 | }, |
23 | 17 | { |
24 | 18 | "cell_type": "markdown", |
25 | | - "metadata": { |
26 | | - "deletable": true, |
27 | | - "editable": true |
28 | | - }, |
| 19 | + "metadata": {}, |
29 | 20 | "source": [ |
30 | 21 | "Import the relevant libraries:" |
31 | 22 | ] |
32 | 23 | }, |
33 | 24 | { |
34 | 25 | "cell_type": "code", |
35 | 26 | "execution_count": 1, |
36 | | - "metadata": { |
37 | | - "collapsed": false, |
38 | | - "deletable": true, |
39 | | - "editable": true |
40 | | - }, |
41 | | - "outputs": [], |
| 27 | + "metadata": {}, |
| 28 | + "outputs": [ |
| 29 | + { |
| 30 | + "name": "stderr", |
| 31 | + "output_type": "stream", |
| 32 | + "text": [ |
| 33 | + "/Users/anastasiia/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n", |
| 34 | + " from ._conv import register_converters as _register_converters\n" |
| 35 | + ] |
| 36 | + } |
| 37 | + ], |
42 | 38 | "source": [ |
43 | 39 | "import numpy as np\n", |
44 | 40 | "import tensorflow as tf\n", |
|
47 | 43 | }, |
48 | 44 | { |
49 | 45 | "cell_type": "markdown", |
50 | | - "metadata": { |
51 | | - "deletable": true, |
52 | | - "editable": true |
53 | | - }, |
| 46 | + "metadata": {}, |
54 | 47 | "source": [ |
55 | 48 | "Define the RNN model:" |
56 | 49 | ] |
57 | 50 | }, |
58 | 51 | { |
59 | 52 | "cell_type": "code", |
60 | 53 | "execution_count": 2, |
61 | | - "metadata": { |
62 | | - "collapsed": true, |
63 | | - "deletable": true, |
64 | | - "editable": true |
65 | | - }, |
| 54 | + "metadata": {}, |
66 | 55 | "outputs": [], |
67 | 56 | "source": [ |
68 | 57 | "class SeriesPredictor:\n", |
|
92 | 81 | " :param W: matrix of fully-connected output layer weights\n", |
93 | 82 | " :param b: vector of fully-connected output layer biases\n", |
94 | 83 | " \"\"\"\n", |
95 | | - " cell = rnn.BasicLSTMCell(self.hidden_dim)\n", |
| 84 | + " cell = rnn.BasicLSTMCell(self.hidden_dim, reuse=tf.get_variable_scope().reuse)\n", |
96 | 85 | " outputs, states = tf.nn.dynamic_rnn(cell, self.x, dtype=tf.float32)\n", |
97 | 86 | " num_examples = tf.shape(self.x)[0]\n", |
98 | 87 | " W_repeated = tf.tile(tf.expand_dims(self.W_out, 0), [num_examples, 1, 1])\n", |
|
123 | 112 | }, |
124 | 113 | { |
125 | 114 | "cell_type": "markdown", |
126 | | - "metadata": { |
127 | | - "deletable": true, |
128 | | - "editable": true |
129 | | - }, |
| 115 | + "metadata": {}, |
130 | 116 | "source": [ |
131 | 117 | "Now, we'll train a series predictor. Let's say we have a sequence of numbers `[a, b, c, d]` that we want to transform into `[a, a+b, b+c, c+d]`. We'll give the RNN a couple examples in the training data. Let's see how well it learns this intended transformation:" |
132 | 118 | ] |
133 | 119 | }, |
134 | 120 | { |
135 | 121 | "cell_type": "code", |
136 | 122 | "execution_count": 3, |
137 | | - "metadata": { |
138 | | - "collapsed": false, |
139 | | - "deletable": true, |
140 | | - "editable": true |
141 | | - }, |
| 123 | + "metadata": {}, |
142 | 124 | "outputs": [ |
143 | 125 | { |
144 | 126 | "name": "stdout", |
145 | 127 | "output_type": "stream", |
146 | 128 | "text": [ |
147 | | - "0 92.1852\n", |
148 | | - "100 61.1175\n", |
149 | | - "200 27.0341\n", |
150 | | - "300 13.9523\n", |
151 | | - "400 9.39037\n", |
152 | | - "500 7.08643\n", |
153 | | - "600 5.50997\n", |
154 | | - "700 4.12571\n", |
155 | | - "800 3.12016\n", |
156 | | - "900 2.42311\n", |
| 129 | + "0 96.78678\n", |
| 130 | + "100 61.329662\n", |
| 131 | + "200 18.419907\n", |
| 132 | + "300 7.646343\n", |
| 133 | + "400 4.7979555\n", |
| 134 | + "500 3.2019987\n", |
| 135 | + "600 2.2661102\n", |
| 136 | + "700 1.6707231\n", |
| 137 | + "800 1.2424115\n", |
| 138 | + "900 0.9125628\n", |
157 | 139 | "Model saved to model.ckpt\n", |
| 140 | + "INFO:tensorflow:Restoring parameters from ./model.ckpt\n", |
158 | 141 | "\n", |
159 | 142 | "Lets run some tests!\n", |
160 | 143 | "\n", |
161 | 144 | "When the input is [[1], [2], [3], [4]]\n", |
162 | 145 | "The ground truth output should be [[1], [3], [5], [7]]\n", |
163 | | - "And the model thinks it is [ 0.96018004 2.76944828 5.35826826 7.3706851 ]\n", |
| 146 | + "And the model thinks it is [1.037468 2.519481 4.514736 6.729595]\n", |
164 | 147 | "\n", |
165 | 148 | "When the input is [[4], [5], [6], [7]]\n", |
166 | 149 | "The ground truth output should be [[4], [9], [11], [13]]\n", |
167 | | - "And the model thinks it is [ 4.17302942 9.161376 11.13204765 11.64120388]\n", |
| 150 | + "And the model thinks it is [ 4.5689063 9.189994 11.679442 12.760409 ]\n", |
168 | 151 | "\n" |
169 | 152 | ] |
170 | 153 | } |
|
211 | 194 | "name": "python", |
212 | 195 | "nbconvert_exporter": "python", |
213 | 196 | "pygments_lexer": "ipython3", |
214 | | - "version": "3.5.2" |
| 197 | + "version": "3.6.5" |
215 | 198 | } |
216 | 199 | }, |
217 | 200 | "nbformat": 4, |
|
0 commit comments