You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
270 lines
10 KiB
Plaintext
270 lines
10 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Handwritten Number Recognition with TFLearn and MNIST\n",
|
|
"\n",
|
|
"In this notebook, we'll be building a neural network that recognizes handwritten numbers 0-9. \n",
|
|
"\n",
|
|
"This kind of neural network is used in a variety of real-world applications including: recognizing phone numbers and sorting postal mail by address. To build the network, we'll be using the **MNIST** data set, which consists of images of handwritten numbers and their correct labels 0-9.\n",
|
|
"\n",
|
|
"We'll be using [TFLearn](http://tflearn.org/), a high-level library built on top of TensorFlow to build the neural network. We'll start off by importing all the modules we'll need, then load the data, and finally build the network."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"collapsed": false
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Import Numpy, TensorFlow, TFLearn, and MNIST data\n",
|
|
"import numpy as np\n",
|
|
"import tensorflow as tf\n",
|
|
"import tflearn\n",
|
|
"import tflearn.datasets.mnist as mnist"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Retrieving training and test data\n",
|
|
"\n",
|
|
"The MNIST data set already contains both training and test data. There are 55,000 data points of training data, and 10,000 points of test data.\n",
|
|
"\n",
|
|
"Each MNIST data point has:\n",
|
|
"1. an image of a handwritten digit and \n",
|
|
"2. a corresponding label (a number 0-9 that identifies the image)\n",
|
|
"\n",
|
|
"We'll call the images, which will be the input to our neural network, **X** and their corresponding labels **Y**.\n",
|
|
"\n",
|
|
"We're going to want our labels as *one-hot vectors*, which are vectors that holds mostly 0's and one 1. It's easiest to see this in a example. As a one-hot vector, the number 0 is represented as [1, 0, 0, 0, 0, 0, 0, 0, 0, 0], and 4 is represented as [0, 0, 0, 0, 1, 0, 0, 0, 0, 0].\n",
|
|
"\n",
|
|
"### Flattened data\n",
|
|
"\n",
|
|
"For this example, we'll be using *flattened* data or a representation of MNIST images in one dimension rather than two. So, each handwritten number image, which is 28x28 pixels, will be represented as a one dimensional array of 784 pixel values. \n",
|
|
"\n",
|
|
"Flattening the data throws away information about the 2D structure of the image, but it simplifies our data so that all of the training data can be contained in one array whose shape is [55000, 784]; the first dimension is the number of training images and the second dimension is the number of pixels in each image. This is the kind of data that is easy to analyze using a simple neural network."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"collapsed": false
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Retrieve the training and test data\n",
|
|
"trainX, trainY, testX, testY = mnist.load_data(one_hot=True)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Visualize the training data\n",
|
|
"\n",
|
|
"Provided below is a function that will help you visualize the MNIST data. By passing in the index of a training example, the function `show_digit` will display that training image along with it's corresponding label in the title."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"collapsed": false
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Visualizing the data\n",
|
|
"import matplotlib.pyplot as plt\n",
|
|
"%matplotlib inline\n",
|
|
"\n",
|
|
"# Function for displaying a training image by it's index in the MNIST set\n",
|
|
"def show_digit(index):\n",
|
|
" label = trainY[index].argmax(axis=0)\n",
|
|
" # Reshape 784 array into 28x28 image\n",
|
|
" image = trainX[index].reshape([28,28])\n",
|
|
" plt.title('Training data, index: %d, Label: %d' % (index, label))\n",
|
|
" plt.imshow(image, cmap='gray_r')\n",
|
|
" plt.show()\n",
|
|
" \n",
|
|
"# Display the first (index 0) training image\n",
|
|
"show_digit(0)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"collapsed": false
|
|
},
|
|
"source": [
|
|
"## Building the network\n",
|
|
"\n",
|
|
"TFLearn lets you build the network by defining the layers in that network. \n",
|
|
"\n",
|
|
"For this example, you'll define:\n",
|
|
"\n",
|
|
"1. The input layer, which tells the network the number of inputs it should expect for each piece of MNIST data. \n",
|
|
"2. Hidden layers, which recognize patterns in data and connect the input to the output layer, and\n",
|
|
"3. The output layer, which defines how the network learns and outputs a label for a given image.\n",
|
|
"\n",
|
|
"Let's start with the input layer; to define the input layer, you'll define the type of data that the network expects. For example,\n",
|
|
"\n",
|
|
"```\n",
|
|
"net = tflearn.input_data([None, 100])\n",
|
|
"```\n",
|
|
"\n",
|
|
"would create a network with 100 inputs. The number of inputs to your network needs to match the size of your data. For this example, we're using 784 element long vectors to encode our input data, so we need **784 input units**.\n",
|
|
"\n",
|
|
"\n",
|
|
"### Adding layers\n",
|
|
"\n",
|
|
"To add new hidden layers, you use \n",
|
|
"\n",
|
|
"```\n",
|
|
"net = tflearn.fully_connected(net, n_units, activation='ReLU')\n",
|
|
"```\n",
|
|
"\n",
|
|
"This adds a fully connected layer where every unit (or node) in the previous layer is connected to every unit in this layer. The first argument `net` is the network you created in the `tflearn.input_data` call, it designates the input to the hidden layer. You can set the number of units in the layer with `n_units`, and set the activation function with the `activation` keyword. You can keep adding layers to your network by repeated calling `tflearn.fully_connected(net, n_units)`. \n",
|
|
"\n",
|
|
"Then, to set how you train the network, use:\n",
|
|
"\n",
|
|
"```\n",
|
|
"net = tflearn.regression(net, optimizer='sgd', learning_rate=0.1, loss='categorical_crossentropy')\n",
|
|
"```\n",
|
|
"\n",
|
|
"Again, this is passing in the network you've been building. The keywords: \n",
|
|
"\n",
|
|
"* `optimizer` sets the training method, here stochastic gradient descent\n",
|
|
"* `learning_rate` is the learning rate\n",
|
|
"* `loss` determines how the network error is calculated. In this example, with categorical cross-entropy.\n",
|
|
"\n",
|
|
"Finally, you put all this together to create the model with `tflearn.DNN(net)`."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"**Exercise:** Below in the `build_model()` function, you'll put together the network using TFLearn. You get to choose how many layers to use, how many hidden units, etc.\n",
|
|
"\n",
|
|
"**Hint:** The final output layer must have 10 output nodes (one for each digit 0-9). It's also recommended to use a `softmax` activation layer as your final output layer. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"collapsed": false
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Define the neural network\n",
|
|
"def build_model():\n",
|
|
" # This resets all parameters and variables, leave this here\n",
|
|
" tf.reset_default_graph()\n",
|
|
" \n",
|
|
" #### Your code ####\n",
|
|
" # Include the input layer, hidden layer(s), and set how you want to train the model\n",
|
|
" \n",
|
|
" # This model assumes that your network is named \"net\" \n",
|
|
" model = tflearn.DNN(net)\n",
|
|
" return model"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"collapsed": false
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Build the model\n",
|
|
"model = build_model()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Training the network\n",
|
|
"\n",
|
|
"Now that we've constructed the network, saved as the variable `model`, we can fit it to the data. Here we use the `model.fit` method. You pass in the training features `trainX` and the training targets `trainY`. Below I set `validation_set=0.1` which reserves 10% of the data set as the validation set. You can also set the batch size and number of epochs with the `batch_size` and `n_epoch` keywords, respectively. \n",
|
|
"\n",
|
|
"Too few epochs don't effectively train your network, and too many take a long time to execute. Choose wisely!"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"collapsed": false
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Training\n",
|
|
"model.fit(trainX, trainY, validation_set=0.1, show_metric=True, batch_size=100, n_epoch=20)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Testing\n",
|
|
"After you're satisified with the training output and accuracy, you can then run the network on the **test data set** to measure it's performance! Remember, only do this after you've done the training and are satisfied with the results.\n",
|
|
"\n",
|
|
"A good result will be **higher than 95% accuracy**. Some simple models have been known to get up to 99.7% accuracy!"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"collapsed": false
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Compare the labels that our model predicts with the actual labels\n",
|
|
"\n",
|
|
"# Find the indices of the most confident prediction for each item. That tells us the predicted digit for that sample.\n",
|
|
"predictions = np.array(model.predict(testX)).argmax(axis=1)\n",
|
|
"\n",
|
|
"# Calculate the accuracy, which is the percentage of times the predicated labels matched the actual labels\n",
|
|
"actual = testY.argmax(axis=1)\n",
|
|
"test_accuracy = np.mean(predictions == actual, axis=0)\n",
|
|
"\n",
|
|
"# Print out the result\n",
|
|
"print(\"Test accuracy: \", test_accuracy)"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"anaconda-cloud": {},
|
|
"kernelspec": {
|
|
"display_name": "Python [default]",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.5.2"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
}
|