commit f3e1c191ef3dddee51b2fcb16c8dc0952b9d1bf1 Author: Chakib Benziane Date: Tue Sep 5 08:37:33 2017 +0200 initial commit diff --git a/dlnd_language_translation.ipynb b/dlnd_language_translation.ipynb new file mode 100644 index 0000000..e953b13 --- /dev/null +++ b/dlnd_language_translation.ipynb @@ -0,0 +1,4740 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "collapsed": true + }, + "source": [ + "# Language Translation\n", + "In this project, you’re going to take a peek into the realm of neural network machine translation. You’ll be training a sequence to sequence model on a dataset of English and French sentences that can translate new sentences from English to French.\n", + "## Get the Data\n", + "Since translating the whole language of English to French will take lots of time to train, we have provided you with a small portion of the English corpus." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "\"\"\"\n", + "DON'T MODIFY ANYTHING IN THIS CELL\n", + "\"\"\"\n", + "import helper\n", + "import problem_unittests as tests\n", + "\n", + "source_path = '/data/small_vocab_en'\n", + "target_path = '/data/small_vocab_fr'\n", + "source_text = helper.load_data(source_path)\n", + "target_text = helper.load_data(target_path)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Explore the Data\n", + "Play around with view_sentence_range to view different parts of the data." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Dataset Stats\n", + "Roughly the number of unique words: 227\n", + "Number of sentences: 137861\n", + "Average number of words in a sentence: 13.225277634719028\n", + "\n", + "English sentences 0 to 10:\n", + "new jersey is sometimes quiet during autumn , and it is snowy in april .\n", + "the united states is usually chilly during july , and it is usually freezing in november .\n", + "california is usually quiet during march , and it is usually hot in june .\n", + "the united states is sometimes mild during june , and it is cold in september .\n", + "your least liked fruit is the grape , but my least liked is the apple .\n", + "his favorite fruit is the orange , but my favorite is the grape .\n", + "paris is relaxing during december , but it is usually chilly in july .\n", + "new jersey is busy during spring , and it is never hot in march .\n", + "our least liked fruit is the lemon , but my least liked is the grape .\n", + "the united states is sometimes busy during january , and it is sometimes warm in november .\n", + "\n", + "French sentences 0 to 10:\n", + "new jersey est parfois calme pendant l' automne , et il est neigeux en avril .\n", + "les états-unis est généralement froid en juillet , et il gèle habituellement en novembre .\n", + "california est généralement calme en mars , et il est généralement chaud en juin .\n", + "les états-unis est parfois légère en juin , et il fait froid en septembre .\n", + "votre moins aimé fruit est le raisin , mais mon moins aimé est la pomme .\n", + "son fruit préféré est l'orange , mais mon préféré est le raisin .\n", + "paris est relaxant en décembre , mais il est généralement froid en juillet .\n", + "new jersey est occupé au printemps , et il est jamais chaude en mars .\n", + "notre fruit est moins aimé le citron , mais mon moins aimé est le raisin .\n", + "les états-unis est parfois occupé en janvier , et il est parfois chaud en novembre .\n" + ] + } + ], + "source": [ + "view_sentence_range = (0, 10)\n", + "\n", + "\"\"\"\n", + "DON'T MODIFY ANYTHING IN THIS CELL\n", + "\"\"\"\n", + "import numpy as np\n", + "\n", + "print('Dataset Stats')\n", + "print('Roughly the number of unique words: {}'.format(len({word: None for word in source_text.split()})))\n", + "\n", + "sentences = source_text.split('\\n')\n", + "word_counts = [len(sentence.split()) for sentence in sentences]\n", + "print('Number of sentences: {}'.format(len(sentences)))\n", + "print('Average number of words in a sentence: {}'.format(np.average(word_counts)))\n", + "\n", + "print()\n", + "print('English sentences {} to {}:'.format(*view_sentence_range))\n", + "print('\\n'.join(source_text.split('\\n')[view_sentence_range[0]:view_sentence_range[1]]))\n", + "print()\n", + "print('French sentences {} to {}:'.format(*view_sentence_range))\n", + "print('\\n'.join(target_text.split('\\n')[view_sentence_range[0]:view_sentence_range[1]]))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Implement Preprocessing Function\n", + "### Text to Word Ids\n", + "As you did with other RNNs, you must turn the text into a number so the computer can understand it. In the function `text_to_ids()`, you'll turn `source_text` and `target_text` from words to ids. However, you need to add the `` word id at the end of `target_text`. This will help the neural network predict when the sentence should end.\n", + "\n", + "You can get the `` word id by doing:\n", + "```python\n", + "target_vocab_to_int['']\n", + "```\n", + "You can get other word ids using `source_vocab_to_int` and `target_vocab_to_int`." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Tests Passed\n" + ] + } + ], + "source": [ + "def text_to_ids(source_text, target_text, source_vocab_to_int, target_vocab_to_int):\n", + " \"\"\"\n", + " Convert source and target text to proper word ids\n", + " :param source_text: String that contains all the source text.\n", + " :param target_text: String that contains all the target text.\n", + " :param source_vocab_to_int: Dictionary to go from the source words to an id\n", + " :param target_vocab_to_int: Dictionary to go from the target words to an id\n", + " :return: A tuple of lists (source_id_text, target_id_text)\n", + " \"\"\"\n", + " \n", + " # Process source text\n", + " words = [[word for word in line.split()] for line in source_text.split('\\n')]\n", + " source_word_ids = [[source_vocab_to_int.get(word, source_vocab_to_int['']) for word in line.split()] for line in source_text.split('\\n')] # use get to replace ignored/unknown characters by \n", + " \n", + " \n", + " # Process target text\n", + " target_word_ids = [[target_vocab_to_int.get(word, target_vocab_to_int['']) for word in line.split()] + [target_vocab_to_int['']] for line in target_text.split('\\n')]\n", + " \n", + " \n", + " return source_word_ids, target_word_ids\n", + "\n", + "\"\"\"\n", + "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n", + "\"\"\"\n", + "tests.test_text_to_ids(text_to_ids)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Preprocess all the data and save it\n", + "Running the code cell below will preprocess all the data and save it to file." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "\"\"\"\n", + "DON'T MODIFY ANYTHING IN THIS CELL\n", + "\"\"\"\n", + "helper.preprocess_and_save_data(source_path, target_path, text_to_ids)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Check Point\n", + "This is your first checkpoint. If you ever decide to come back to this notebook or have to restart the notebook, you can start from here. The preprocessed data has been saved to disk." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "\"\"\"\n", + "DON'T MODIFY ANYTHING IN THIS CELL\n", + "\"\"\"\n", + "import numpy as np\n", + "import helper\n", + "import problem_unittests as tests\n", + "\n", + "(source_int_text, target_int_text), (source_vocab_to_int, target_vocab_to_int), _ = helper.load_preprocess()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Check the Version of TensorFlow and Access to GPU\n", + "This will check to make sure you have the correct version of TensorFlow and access to a GPU" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "TensorFlow Version: 1.2.1\n", + "Default GPU Device: /gpu:0\n" + ] + } + ], + "source": [ + "\"\"\"\n", + "DON'T MODIFY ANYTHING IN THIS CELL\n", + "\"\"\"\n", + "from distutils.version import LooseVersion\n", + "import warnings\n", + "import tensorflow as tf\n", + "from tensorflow.python.layers.core import Dense\n", + "\n", + "# Check TensorFlow Version\n", + "assert LooseVersion(tf.__version__) >= LooseVersion('1.1'), 'Please use TensorFlow version 1.1 or newer'\n", + "print('TensorFlow Version: {}'.format(tf.__version__))\n", + "\n", + "# Check for a GPU\n", + "if not tf.test.gpu_device_name():\n", + " warnings.warn('No GPU found. Please use a GPU to train your neural network.')\n", + "else:\n", + " print('Default GPU Device: {}'.format(tf.test.gpu_device_name()))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Build the Neural Network\n", + "You'll build the components necessary to build a Sequence-to-Sequence model by implementing the following functions below:\n", + "- `model_inputs`\n", + "- `process_decoder_input`\n", + "- `encoding_layer`\n", + "- `decoding_layer_train`\n", + "- `decoding_layer_infer`\n", + "- `decoding_layer`\n", + "- `seq2seq_model`\n", + "\n", + "### Input\n", + "Implement the `model_inputs()` function to create TF Placeholders for the Neural Network. It should create the following placeholders:\n", + "\n", + "- Input text placeholder named \"input\" using the TF Placeholder name parameter with rank 2.\n", + "- Targets placeholder with rank 2.\n", + "- Learning rate placeholder with rank 0.\n", + "- Keep probability placeholder named \"keep_prob\" using the TF Placeholder name parameter with rank 0.\n", + "- Target sequence length placeholder named \"target_sequence_length\" with rank 1\n", + "- Max target sequence length tensor named \"max_target_len\" getting its value from applying tf.reduce_max on the target_sequence_length placeholder. Rank 0.\n", + "- Source sequence length placeholder named \"source_sequence_length\" with rank 1\n", + "\n", + "Return the placeholders in the following the tuple (input, targets, learning rate, keep probability, target sequence length, max target sequence length, source sequence length)" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "ERROR:tensorflow:==================================\n", + "Object was never used (type ):\n", + "\n", + "If you want to mark it as used call its \"mark_used()\" method.\n", + "It was originally created here:\n", + "['File \"/usr/local/lib/python3.5/runpy.py\", line 193, in _run_module_as_main\\n \"__main__\", mod_spec)', 'File \"/usr/local/lib/python3.5/runpy.py\", line 85, in _run_code\\n exec(code, run_globals)', 'File \"/usr/local/lib/python3.5/site-packages/ipykernel_launcher.py\", line 16, in \\n app.launch_new_instance()', 'File \"/usr/local/lib/python3.5/site-packages/traitlets/config/application.py\", line 658, in launch_instance\\n app.start()', 'File \"/usr/local/lib/python3.5/site-packages/ipykernel/kernelapp.py\", line 477, in start\\n ioloop.IOLoop.instance().start()', 'File \"/usr/local/lib/python3.5/site-packages/zmq/eventloop/ioloop.py\", line 177, in start\\n super(ZMQIOLoop, self).start()', 'File \"/usr/local/lib/python3.5/site-packages/tornado/ioloop.py\", line 888, in start\\n handler_func(fd_obj, events)', 'File \"/usr/local/lib/python3.5/site-packages/tornado/stack_context.py\", line 277, in null_wrapper\\n return fn(*args, **kwargs)', 'File \"/usr/local/lib/python3.5/site-packages/zmq/eventloop/zmqstream.py\", line 440, in _handle_events\\n self._handle_recv()', 'File \"/usr/local/lib/python3.5/site-packages/zmq/eventloop/zmqstream.py\", line 472, in _handle_recv\\n self._run_callback(callback, msg)', 'File \"/usr/local/lib/python3.5/site-packages/zmq/eventloop/zmqstream.py\", line 414, in _run_callback\\n callback(*args, **kwargs)', 'File \"/usr/local/lib/python3.5/site-packages/tornado/stack_context.py\", line 277, in null_wrapper\\n return fn(*args, **kwargs)', 'File \"/usr/local/lib/python3.5/site-packages/ipykernel/kernelbase.py\", line 283, in dispatcher\\n return self.dispatch_shell(stream, msg)', 'File \"/usr/local/lib/python3.5/site-packages/ipykernel/kernelbase.py\", line 235, in dispatch_shell\\n handler(stream, idents, msg)', 'File \"/usr/local/lib/python3.5/site-packages/ipykernel/kernelbase.py\", line 399, in execute_request\\n user_expressions, allow_stdin)', 'File \"/usr/local/lib/python3.5/site-packages/ipykernel/ipkernel.py\", line 196, in do_execute\\n res = shell.run_cell(code, store_history=store_history, silent=silent)', 'File \"/usr/local/lib/python3.5/site-packages/ipykernel/zmqshell.py\", line 533, in run_cell\\n return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)', 'File \"/usr/local/lib/python3.5/site-packages/IPython/core/interactiveshell.py\", line 2698, in run_cell\\n interactivity=interactivity, compiler=compiler, result=result)', 'File \"/usr/local/lib/python3.5/site-packages/IPython/core/interactiveshell.py\", line 2808, in run_ast_nodes\\n if self.run_code(code, result):', 'File \"/usr/local/lib/python3.5/site-packages/IPython/core/interactiveshell.py\", line 2862, in run_code\\n exec(code_obj, self.user_global_ns, self.user_ns)', 'File \"\", line 22, in \\n tests.test_model_inputs(model_inputs)', 'File \"/output/problem_unittests.py\", line 106, in test_model_inputs\\n assert tf.assert_rank(lr, 0, message=\\'Learning Rate has wrong rank\\')', 'File \"/usr/local/lib/python3.5/site-packages/tensorflow/python/ops/check_ops.py\", line 617, in assert_rank\\n dynamic_condition, data, summarize)', 'File \"/usr/local/lib/python3.5/site-packages/tensorflow/python/ops/check_ops.py\", line 571, in _assert_rank_condition\\n return control_flow_ops.Assert(condition, data, summarize=summarize)', 'File \"/usr/local/lib/python3.5/site-packages/tensorflow/python/util/tf_should_use.py\", line 170, in wrapped\\n return _add_should_use_warning(fn(*args, **kwargs))', 'File \"/usr/local/lib/python3.5/site-packages/tensorflow/python/util/tf_should_use.py\", line 139, in _add_should_use_warning\\n wrapped = TFShouldUseWarningWrapper(x)', 'File \"/usr/local/lib/python3.5/site-packages/tensorflow/python/util/tf_should_use.py\", line 96, in __init__\\n stack = [s.strip() for s in traceback.format_stack()]']\n", + "==================================\n", + "ERROR:tensorflow:==================================\n", + "Object was never used (type ):\n", + "\n", + "If you want to mark it as used call its \"mark_used()\" method.\n", + "It was originally created here:\n", + "['File \"/usr/local/lib/python3.5/runpy.py\", line 193, in _run_module_as_main\\n \"__main__\", mod_spec)', 'File \"/usr/local/lib/python3.5/runpy.py\", line 85, in _run_code\\n exec(code, run_globals)', 'File \"/usr/local/lib/python3.5/site-packages/ipykernel_launcher.py\", line 16, in \\n app.launch_new_instance()', 'File \"/usr/local/lib/python3.5/site-packages/traitlets/config/application.py\", line 658, in launch_instance\\n app.start()', 'File \"/usr/local/lib/python3.5/site-packages/ipykernel/kernelapp.py\", line 477, in start\\n ioloop.IOLoop.instance().start()', 'File \"/usr/local/lib/python3.5/site-packages/zmq/eventloop/ioloop.py\", line 177, in start\\n super(ZMQIOLoop, self).start()', 'File \"/usr/local/lib/python3.5/site-packages/tornado/ioloop.py\", line 888, in start\\n handler_func(fd_obj, events)', 'File \"/usr/local/lib/python3.5/site-packages/tornado/stack_context.py\", line 277, in null_wrapper\\n return fn(*args, **kwargs)', 'File \"/usr/local/lib/python3.5/site-packages/zmq/eventloop/zmqstream.py\", line 440, in _handle_events\\n self._handle_recv()', 'File \"/usr/local/lib/python3.5/site-packages/zmq/eventloop/zmqstream.py\", line 472, in _handle_recv\\n self._run_callback(callback, msg)', 'File \"/usr/local/lib/python3.5/site-packages/zmq/eventloop/zmqstream.py\", line 414, in _run_callback\\n callback(*args, **kwargs)', 'File \"/usr/local/lib/python3.5/site-packages/tornado/stack_context.py\", line 277, in null_wrapper\\n return fn(*args, **kwargs)', 'File \"/usr/local/lib/python3.5/site-packages/ipykernel/kernelbase.py\", line 283, in dispatcher\\n return self.dispatch_shell(stream, msg)', 'File \"/usr/local/lib/python3.5/site-packages/ipykernel/kernelbase.py\", line 235, in dispatch_shell\\n handler(stream, idents, msg)', 'File \"/usr/local/lib/python3.5/site-packages/ipykernel/kernelbase.py\", line 399, in execute_request\\n user_expressions, allow_stdin)', 'File \"/usr/local/lib/python3.5/site-packages/ipykernel/ipkernel.py\", line 196, in do_execute\\n res = shell.run_cell(code, store_history=store_history, silent=silent)', 'File \"/usr/local/lib/python3.5/site-packages/ipykernel/zmqshell.py\", line 533, in run_cell\\n return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)', 'File \"/usr/local/lib/python3.5/site-packages/IPython/core/interactiveshell.py\", line 2698, in run_cell\\n interactivity=interactivity, compiler=compiler, result=result)', 'File \"/usr/local/lib/python3.5/site-packages/IPython/core/interactiveshell.py\", line 2808, in run_ast_nodes\\n if self.run_code(code, result):', 'File \"/usr/local/lib/python3.5/site-packages/IPython/core/interactiveshell.py\", line 2862, in run_code\\n exec(code_obj, self.user_global_ns, self.user_ns)', 'File \"\", line 22, in \\n tests.test_model_inputs(model_inputs)', 'File \"/output/problem_unittests.py\", line 107, in test_model_inputs\\n assert tf.assert_rank(keep_prob, 0, message=\\'Keep Probability has wrong rank\\')', 'File \"/usr/local/lib/python3.5/site-packages/tensorflow/python/ops/check_ops.py\", line 617, in assert_rank\\n dynamic_condition, data, summarize)', 'File \"/usr/local/lib/python3.5/site-packages/tensorflow/python/ops/check_ops.py\", line 571, in _assert_rank_condition\\n return control_flow_ops.Assert(condition, data, summarize=summarize)', 'File \"/usr/local/lib/python3.5/site-packages/tensorflow/python/util/tf_should_use.py\", line 170, in wrapped\\n return _add_should_use_warning(fn(*args, **kwargs))', 'File \"/usr/local/lib/python3.5/site-packages/tensorflow/python/util/tf_should_use.py\", line 139, in _add_should_use_warning\\n wrapped = TFShouldUseWarningWrapper(x)', 'File \"/usr/local/lib/python3.5/site-packages/tensorflow/python/util/tf_should_use.py\", line 96, in __init__\\n stack = [s.strip() for s in traceback.format_stack()]']\n", + "==================================\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Tests Passed\n" + ] + } + ], + "source": [ + "def model_inputs():\n", + " \"\"\"\n", + " Create TF Placeholders for input, targets, learning rate, and lengths of source and target sequences.\n", + " :return: Tuple (input, targets, learning rate, keep probability, target sequence length,\n", + " max target sequence length, source sequence length)\n", + " \"\"\"\n", + " \n", + " input_text = tf.placeholder(tf.int32, [None, None], name='input')\n", + " targets = tf.placeholder(tf.int32, [None, None], name='targets')\n", + " lr = tf.placeholder(tf.float32, name='learning_rate' )\n", + " keep = tf.placeholder(tf.float32, name='keep_prob')\n", + " target_seq_len = tf.placeholder(tf.int32, (None,), name='target_sequence_length')\n", + " max_target_seq_len = tf.reduce_max(target_seq_len, name='max_target_len')\n", + " source_seq_len = tf.placeholder(tf.int32, (None,), name='source_sequence_length')\n", + "\n", + " return input_text, targets, lr, keep, target_seq_len, max_target_seq_len, source_seq_len \n", + "\n", + "\n", + "\"\"\"\n", + "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n", + "\"\"\"\n", + "tests.test_model_inputs(model_inputs)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Process Decoder Input\n", + "Implement `process_decoder_input` by removing the last word id from each batch in `target_data` and concat the GO ID to the begining of each batch." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Tests Passed\n" + ] + } + ], + "source": [ + "def process_decoder_input(target_data, target_vocab_to_int, batch_size):\n", + " \"\"\"\n", + " Preprocess target data for encoding\n", + " :param target_data: Target Placehoder\n", + " :param target_vocab_to_int: Dictionary to go from the target words to an id\n", + " :param batch_size: Batch Size\n", + " :return: Preprocessed target data\n", + " \"\"\"\n", + " # TODO: Implement Function\n", + " ending = tf.strided_slice(target_data, [0,0], [batch_size, -1], [1,1])\n", + " dec_input = tf.concat([tf.fill([batch_size, 1], target_vocab_to_int['']), ending], 1)\n", + " \n", + " return dec_input\n", + "\n", + "\"\"\"\n", + "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n", + "\"\"\"\n", + "tests.test_process_encoding_input(process_decoder_input)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Encoding\n", + "Implement `encoding_layer()` to create a Encoder RNN layer:\n", + " * Embed the encoder input using [`tf.contrib.layers.embed_sequence`](https://www.tensorflow.org/api_docs/python/tf/contrib/layers/embed_sequence)\n", + " * Construct a [stacked](https://github.com/tensorflow/tensorflow/blob/6947f65a374ebf29e74bb71e36fd82760056d82c/tensorflow/docs_src/tutorials/recurrent.md#stacking-multiple-lstms) [`tf.contrib.rnn.LSTMCell`](https://www.tensorflow.org/api_docs/python/tf/contrib/rnn/LSTMCell) wrapped in a [`tf.contrib.rnn.DropoutWrapper`](https://www.tensorflow.org/api_docs/python/tf/contrib/rnn/DropoutWrapper)\n", + " * Pass cell and embedded input to [`tf.nn.dynamic_rnn()`](https://www.tensorflow.org/api_docs/python/tf/nn/dynamic_rnn)" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "scrolled": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Tests Passed\n" + ] + } + ], + "source": [ + "from imp import reload\n", + "reload(tests)\n", + "\n", + "def encoding_layer(rnn_inputs, rnn_size, num_layers, keep_prob, \n", + " source_sequence_length, source_vocab_size, \n", + " encoding_embedding_size):\n", + " \"\"\"\n", + " Create encoding layer\n", + " :param rnn_inputs: Inputs for the RNN\n", + " :param rnn_size: RNN Size\n", + " :param num_layers: Number of layers\n", + " :param keep_prob: Dropout keep probability\n", + " :param source_sequence_length: a list of the lengths of each sequence in the batch\n", + " :param source_vocab_size: vocabulary size of source data\n", + " :param encoding_embedding_size: embedding size of source data\n", + " :return: tuple (RNN output, RNN state)\n", + " \"\"\"\n", + " \n", + " enc_embed_input = tf.contrib.layers.embed_sequence(rnn_inputs, source_vocab_size, encoding_embedding_size)\n", + " \n", + " #Rnn cell\n", + " def make_cell(rnn_size):\n", + " cell = tf.contrib.rnn.LSTMCell(rnn_size,\n", + " initializer=tf.random_uniform_initializer(-0.1, 0.1, seed=2))\n", + " # add dropout layer\n", + " enc_cell = tf.contrib.rnn.DropoutWrapper(cell, output_keep_prob=keep_prob)\n", + " return enc_cell\n", + " \n", + " enc_cell = tf.contrib.rnn.MultiRNNCell([make_cell(rnn_size) for _ in range(num_layers)])\n", + " \n", + " enc_output, enc_state = tf.nn.dynamic_rnn(enc_cell, enc_embed_input, sequence_length=source_sequence_length, dtype=tf.float32)\n", + "\n", + " \n", + " return enc_output, enc_state\n", + "\n", + "\"\"\"\n", + "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n", + "\"\"\"\n", + "tests.test_encoding_layer(encoding_layer)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Decoding - Training\n", + "Create a training decoding layer:\n", + "* Create a [`tf.contrib.seq2seq.TrainingHelper`](https://www.tensorflow.org/api_docs/python/tf/contrib/seq2seq/TrainingHelper) \n", + "* Create a [`tf.contrib.seq2seq.BasicDecoder`](https://www.tensorflow.org/api_docs/python/tf/contrib/seq2seq/BasicDecoder)\n", + "* Obtain the decoder outputs from [`tf.contrib.seq2seq.dynamic_decode`](https://www.tensorflow.org/api_docs/python/tf/contrib/seq2seq/dynamic_decode)" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Tests Passed\n" + ] + } + ], + "source": [ + "\n", + "def decoding_layer_train(encoder_state, dec_cell, dec_embed_input, \n", + " target_sequence_length, max_summary_length, \n", + " output_layer, keep_prob):\n", + " \"\"\"\n", + " Create a decoding layer for training\n", + " :param encoder_state: Encoder State\n", + " :param dec_cell: Decoder RNN Cell\n", + " :param dec_embed_input: Decoder embedded input\n", + " :param target_sequence_length: The lengths of each sequence in the target batch\n", + " :param max_summary_length: The length of the longest sequence in the batch\n", + " :param output_layer: Function to apply the output layer\n", + " :param keep_prob: Dropout keep probability\n", + " :return: BasicDecoderOutput containing training logits and sample_id\n", + " \"\"\"\n", + " # Question: Why are we receiving keep_prob here\n", + " # Where would we add dropout layer here\n", + " \n", + " # Helper for the training process. Used by BasicDecoder to read inputs.\n", + " training_helper = tf.contrib.seq2seq.TrainingHelper(inputs=dec_embed_input,\n", + " sequence_length=target_sequence_length,\n", + " time_major=False)\n", + "\n", + "\n", + " # Basic decoder\n", + " training_decoder = tf.contrib.seq2seq.BasicDecoder(dec_cell,\n", + " training_helper,\n", + " encoder_state,\n", + " output_layer) \n", + "\n", + " # Perform dynamic decoding using the decoder\n", + " training_decoder_output = tf.contrib.seq2seq.dynamic_decode(training_decoder,\n", + " impute_finished=True,\n", + " maximum_iterations=max_summary_length)[0]\n", + " \n", + " \n", + " return training_decoder_output\n", + "\n", + "\n", + "\n", + "\"\"\"\n", + "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n", + "\"\"\"\n", + "tests.test_decoding_layer_train(decoding_layer_train)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Decoding - Inference\n", + "Create inference decoder:\n", + "* Create a [`tf.contrib.seq2seq.GreedyEmbeddingHelper`](https://www.tensorflow.org/api_docs/python/tf/contrib/seq2seq/GreedyEmbeddingHelper)\n", + "* Create a [`tf.contrib.seq2seq.BasicDecoder`](https://www.tensorflow.org/api_docs/python/tf/contrib/seq2seq/BasicDecoder)\n", + "* Obtain the decoder outputs from [`tf.contrib.seq2seq.dynamic_decode`](https://www.tensorflow.org/api_docs/python/tf/contrib/seq2seq/dynamic_decode)" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Tests Passed\n" + ] + } + ], + "source": [ + "def decoding_layer_infer(encoder_state, dec_cell, dec_embeddings, start_of_sequence_id,\n", + " end_of_sequence_id, max_target_sequence_length,\n", + " vocab_size, output_layer, batch_size, keep_prob):\n", + " \"\"\"\n", + " Create a decoding layer for inference\n", + " :param encoder_state: Encoder state\n", + " :param dec_cell: Decoder RNN Cell\n", + " :param dec_embeddings: Decoder embeddings\n", + " :param start_of_sequence_id: GO ID\n", + " :param end_of_sequence_id: EOS Id\n", + " :param max_target_sequence_length: Maximum length of target sequences\n", + " :param vocab_size: Size of decoder/target vocabulary\n", + " :param decoding_scope: TenorFlow Variable Scope for decoding\n", + " :param output_layer: Function to apply the output layer\n", + " :param batch_size: Batch size\n", + " :param keep_prob: Dropout keep probability\n", + " :return: BasicDecoderOutput containing inference logits and sample_id\n", + " \"\"\"\n", + " # Start from GO\n", + " start_tokens = tf.tile(tf.constant([start_of_sequence_id], dtype=tf.int32), [batch_size], name='start_tokens')\n", + "\n", + " # Helper for the inference process.\n", + " inference_helper = tf.contrib.seq2seq.GreedyEmbeddingHelper(dec_embeddings,\n", + " start_tokens,\n", + " end_of_sequence_id)\n", + "\n", + " # Basic decoder\n", + " inference_decoder = tf.contrib.seq2seq.BasicDecoder(dec_cell,\n", + " inference_helper,\n", + " encoder_state,\n", + " output_layer)\n", + "\n", + " # Perform dynamic decoding using the decoder\n", + " inference_decoder_output = tf.contrib.seq2seq.dynamic_decode(inference_decoder,\n", + " impute_finished=True,\n", + " maximum_iterations=max_target_sequence_length)[0]\n", + "\n", + " return inference_decoder_output\n", + "\n", + "\n", + "\n", + "\"\"\"\n", + "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n", + "\"\"\"\n", + "tests.test_decoding_layer_infer(decoding_layer_infer)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Build the Decoding Layer\n", + "Implement `decoding_layer()` to create a Decoder RNN layer.\n", + "\n", + "* Embed the target sequences\n", + "* Construct the decoder LSTM cell (just like you constructed the encoder cell above)\n", + "* Create an output layer to map the outputs of the decoder to the elements of our vocabulary\n", + "* Use the your `decoding_layer_train(encoder_state, dec_cell, dec_embed_input, target_sequence_length, max_target_sequence_length, output_layer, keep_prob)` function to get the training logits.\n", + "* Use your `decoding_layer_infer(encoder_state, dec_cell, dec_embeddings, start_of_sequence_id, end_of_sequence_id, max_target_sequence_length, vocab_size, output_layer, batch_size, keep_prob)` function to get the inference logits.\n", + "\n", + "Note: You'll need to use [tf.variable_scope](https://www.tensorflow.org/api_docs/python/tf/variable_scope) to share variables between training and inference." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Tests Passed\n" + ] + } + ], + "source": [ + "def decoding_layer(dec_input, encoder_state,\n", + " target_sequence_length, max_target_sequence_length,\n", + " rnn_size,\n", + " num_layers, target_vocab_to_int, target_vocab_size,\n", + " batch_size, keep_prob, decoding_embedding_size):\n", + " \"\"\"\n", + " Create decoding layer\n", + " :param dec_input: Decoder input\n", + " :param encoder_state: Encoder state\n", + " :param target_sequence_length: The lengths of each sequence in the target batch\n", + " :param max_target_sequence_length: Maximum length of target sequences\n", + " :param rnn_size: RNN Size\n", + " :param num_layers: Number of layers\n", + " :param target_vocab_to_int: Dictionary to go from the target words to an id\n", + " :param target_vocab_size: Size of target vocabulary\n", + " :param batch_size: The size of the batch\n", + " :param keep_prob: Dropout keep probability\n", + " :param decoding_embedding_size: Decoding embedding size\n", + " :return: Tuple of (Training BasicDecoderOutput, Inference BasicDecoderOutput)\n", + " \"\"\"\n", + " \n", + " # 1. Decoder Embedding\n", + " dec_embeddings = tf.Variable(tf.random_uniform([target_vocab_size, decoding_embedding_size]))\n", + " dec_embed_input = tf.nn.embedding_lookup(dec_embeddings, dec_input)\n", + " \n", + " # 2. Construct the decoder cell\n", + " def make_cell(rnn_size):\n", + " dec_cell = tf.contrib.rnn.LSTMCell(rnn_size,\n", + " initializer=tf.random_uniform_initializer(-0.1, 0.1, seed=2))\n", + " \n", + " # Add dropout layer\n", + " dec_cell = tf.contrib.rnn.DropoutWrapper(dec_cell, output_keep_prob=keep_prob)\n", + " \n", + " return dec_cell\n", + " \n", + " dec_cell = tf.contrib.rnn.MultiRNNCell([make_cell(rnn_size) for _ in range(num_layers)])\n", + " \n", + " # 3. Dense layer to translate the decoder's output at each time \n", + " # step into a choice from the target vocabulary\n", + " output_layer = Dense(target_vocab_size,\n", + " kernel_initializer = tf.truncated_normal_initializer(mean = 0.0, stddev=0.1))\n", + " \n", + " \n", + " # 4. Get training and inference outputs\n", + " \n", + " ## In training mode\n", + " with tf.variable_scope('decode'):\n", + " \n", + " training_decoder_output = decoding_layer_train(encoder_state, \n", + " dec_cell, \n", + " dec_embed_input, \n", + " target_sequence_length, \n", + " max_target_sequence_length, \n", + " output_layer, \n", + " keep_prob)\n", + " \n", + " ## In inference mode we reuse variables\n", + " with tf.variable_scope('decode', reuse=True):\n", + " \n", + " inference_decoder_output = decoding_layer_infer(encoder_state, \n", + " dec_cell, \n", + " dec_embeddings, \n", + " target_vocab_to_int[''], #start of seq ID\n", + " target_vocab_to_int[''], # end of seq ID\n", + " max_target_sequence_length, \n", + " target_vocab_size,\n", + " output_layer,\n", + " batch_size,\n", + " keep_prob)\n", + " \n", + " \n", + " \n", + " return training_decoder_output, inference_decoder_output\n", + "\n", + "\n", + "\n", + "\"\"\"\n", + "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n", + "\"\"\"\n", + "tests.test_decoding_layer(decoding_layer)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Build the Neural Network\n", + "Apply the functions you implemented above to:\n", + "\n", + "- Encode the input using your `encoding_layer(rnn_inputs, rnn_size, num_layers, keep_prob, source_sequence_length, source_vocab_size, encoding_embedding_size)`.\n", + "- Process target data using your `process_decoder_input(target_data, target_vocab_to_int, batch_size)` function.\n", + "- Decode the encoded input using your `decoding_layer(dec_input, enc_state, target_sequence_length, max_target_sentence_length, rnn_size, num_layers, target_vocab_to_int, target_vocab_size, batch_size, keep_prob, dec_embedding_size)` function." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Tests Passed\n" + ] + } + ], + "source": [ + "def seq2seq_model(input_data, target_data, keep_prob, batch_size,\n", + " source_sequence_length, target_sequence_length,\n", + " max_target_sentence_length,\n", + " source_vocab_size, target_vocab_size,\n", + " enc_embedding_size, dec_embedding_size,\n", + " rnn_size, num_layers, target_vocab_to_int):\n", + " \"\"\"\n", + " Build the Sequence-to-Sequence part of the neural network\n", + " :param input_data: Input placeholder\n", + " :param target_data: Target placeholder\n", + " :param keep_prob: Dropout keep probability placeholder\n", + " :param batch_size: Batch Size\n", + " :param source_sequence_length: Sequence Lengths of source sequences in the batch\n", + " :param target_sequence_length: Sequence Lengths of target sequences in the batch\n", + " :param source_vocab_size: Source vocabulary size\n", + " :param target_vocab_size: Target vocabulary size\n", + " :param enc_embedding_size: Decoder embedding size\n", + " :param dec_embedding_size: Encoder embedding size\n", + " :param rnn_size: RNN Size\n", + " :param num_layers: Number of layers\n", + " :param target_vocab_to_int: Dictionary to go from the target words to an id\n", + " :return: Tuple of (Training BasicDecoderOutput, Inference BasicDecoderOutput)\n", + " \"\"\"\n", + " # TODO: Implement Function\n", + " \n", + " \n", + " # Pass the input data through the encoder. We'll ignore the encoder output, but use the state\n", + " _, enc_state = encoding_layer(input_data, \n", + " rnn_size, \n", + " num_layers,\n", + " keep_prob,\n", + " source_sequence_length,\n", + " source_vocab_size, \n", + " enc_embedding_size)\n", + " \n", + " # Prepare the target sequences we'll feed to the decoder in training mode\n", + " dec_input = process_decoder_input(target_data, target_vocab_to_int, batch_size)\n", + " \n", + " \n", + " # Pass encoder state and decoder inputs to the decoders\n", + " training_decoder_output, inference_decoder_output = decoding_layer(dec_input, \n", + " enc_state, \n", + " target_sequence_length, \n", + " max_target_sentence_length,\n", + " rnn_size,\n", + " num_layers,\n", + " target_vocab_to_int, \n", + " target_vocab_size,\n", + " batch_size,\n", + " keep_prob,\n", + " dec_embedding_size) \n", + " \n", + " \n", + " return training_decoder_output, inference_decoder_output\n", + "\n", + "\n", + "\"\"\"\n", + "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n", + "\"\"\"\n", + "tests.test_seq2seq_model(seq2seq_model)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Neural Network Training\n", + "### Hyperparameters\n", + "Tune the following parameters:\n", + "\n", + "- Set `epochs` to the number of epochs.\n", + "- Set `batch_size` to the batch size.\n", + "- Set `rnn_size` to the size of the RNNs.\n", + "- Set `num_layers` to the number of layers.\n", + "- Set `encoding_embedding_size` to the size of the embedding for the encoder.\n", + "- Set `decoding_embedding_size` to the size of the embedding for the decoder.\n", + "- Set `learning_rate` to the learning rate.\n", + "- Set `keep_probability` to the Dropout keep probability\n", + "- Set `display_step` to state how many steps between each debug output statement" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "# Number of Epochs\n", + "epochs = 30\n", + "# Batch Size\n", + "batch_size = 128\n", + "# RNN Size\n", + "rnn_size = 100\n", + "# Number of Layers\n", + "num_layers = 3\n", + "# Embedding Size\n", + "encoding_embedding_size = 100\n", + "decoding_embedding_size = 100\n", + "# Learning Rate\n", + "learning_rate = 0.001\n", + "# Dropout Keep Probability\n", + "keep_probability = 0.5\n", + "display_step = 10" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "RUN_NUMBER = 1\n", + "LOG_DIR = '/output/run_{}/logs/'\n", + "CHECKPOINT_DIR = '/output/run_{}/checkpoints/'\n", + "CHECKPOINT_PATH = CHECKPOINT_DIR.format(RUN_NUMBER)\n", + "LOG_PATH = LOG_DIR.format(RUN_NUMBER)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Build the Graph\n", + "Build the graph using the neural network you implemented." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "\"\"\"\n", + "DON'T MODIFY ANYTHING IN THIS CELL\n", + "\"\"\"\n", + "save_path = CHECKPOINT_PATH\n", + "(source_int_text, target_int_text), (source_vocab_to_int, target_vocab_to_int), _ = helper.load_preprocess()\n", + "max_target_sentence_length = max([len(sentence) for sentence in source_int_text])\n", + "\n", + "train_graph = tf.Graph()\n", + "with train_graph.as_default():\n", + " input_data, targets, lr, keep_prob, target_sequence_length, max_target_sequence_length, source_sequence_length = model_inputs()\n", + "\n", + " #sequence_length = tf.placeholder_with_default(max_target_sentence_length, None, name='sequence_length')\n", + " input_shape = tf.shape(input_data)\n", + "\n", + " train_logits, inference_logits = seq2seq_model(tf.reverse(input_data, [-1]),\n", + " targets,\n", + " keep_prob,\n", + " batch_size,\n", + " source_sequence_length,\n", + " target_sequence_length,\n", + " max_target_sequence_length,\n", + " len(source_vocab_to_int),\n", + " len(target_vocab_to_int),\n", + " encoding_embedding_size,\n", + " decoding_embedding_size,\n", + " rnn_size,\n", + " num_layers,\n", + " target_vocab_to_int)\n", + "\n", + "\n", + " training_logits = tf.identity(train_logits.rnn_output, name='logits')\n", + " inference_logits = tf.identity(inference_logits.sample_id, name='predictions')\n", + "\n", + " masks = tf.sequence_mask(target_sequence_length, max_target_sequence_length, dtype=tf.float32, name='masks')\n", + "\n", + " with tf.name_scope(\"optimization\"):\n", + " # Loss function\n", + " cost = tf.contrib.seq2seq.sequence_loss(\n", + " training_logits,\n", + " targets,\n", + " masks)\n", + " tf.summary.scalar('cost', cost)\n", + "\n", + " # Optimizer\n", + " optimizer = tf.train.AdamOptimizer(lr)\n", + "\n", + " # Gradient Clipping\n", + " gradients = optimizer.compute_gradients(cost)\n", + " capped_gradients = [(tf.clip_by_value(grad, -1., 1.), var) for grad, var in gradients if grad is not None]\n", + " train_op = optimizer.apply_gradients(capped_gradients)\n", + "\n", + " merged = tf.summary.merge_all()\n", + " " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Batch and pad the source and target sequences" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "\"\"\"\n", + "DON'T MODIFY ANYTHING IN THIS CELL\n", + "\"\"\"\n", + "def pad_sentence_batch(sentence_batch, pad_int):\n", + " \"\"\"Pad sentences with so that each sentence of a batch has the same length\"\"\"\n", + " max_sentence = max([len(sentence) for sentence in sentence_batch])\n", + " return [sentence + [pad_int] * (max_sentence - len(sentence)) for sentence in sentence_batch]\n", + "\n", + "\n", + "def get_batches(sources, targets, batch_size, source_pad_int, target_pad_int):\n", + " \"\"\"Batch targets, sources, and the lengths of their sentences together\"\"\"\n", + " for batch_i in range(0, len(sources)//batch_size):\n", + " start_i = batch_i * batch_size\n", + "\n", + " # Slice the right amount for the batch\n", + " sources_batch = sources[start_i:start_i + batch_size]\n", + " targets_batch = targets[start_i:start_i + batch_size]\n", + "\n", + " # Pad\n", + " pad_sources_batch = np.array(pad_sentence_batch(sources_batch, source_pad_int))\n", + " pad_targets_batch = np.array(pad_sentence_batch(targets_batch, target_pad_int))\n", + "\n", + " # Need the lengths for the _lengths parameters\n", + " pad_targets_lengths = []\n", + " for target in pad_targets_batch:\n", + " pad_targets_lengths.append(len(target))\n", + "\n", + " pad_source_lengths = []\n", + " for source in pad_sources_batch:\n", + " pad_source_lengths.append(len(source))\n", + "\n", + " yield pad_sources_batch, pad_targets_batch, pad_source_lengths, pad_targets_lengths\n" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "# write out the graph for tensorboard\n", + "\n", + "with tf.Session(graph=train_graph) as sess:\n", + " train_writer = tf.summary.FileWriter(LOG_PATH + '/train', sess.graph)\n", + " test_writer = tf.summary.FileWriter(LOG_PATH + '/test')\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Train\n", + "Train the neural network on the preprocessed data. If you have a hard time getting a good loss, check the forms to see if anyone is having the same problem." + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 0 Batch 10/1077 - Train Accuracy: 0.2068, Validation Accuracy: 0.3153, Loss: 4.8054\n", + "Epoch 0 Batch 20/1077 - Train Accuracy: 0.2707, Validation Accuracy: 0.3366, Loss: 3.9734\n", + "Epoch 0 Batch 30/1077 - Train Accuracy: 0.2781, Validation Accuracy: 0.3512, Loss: 3.6470\n", + "Epoch 0 Batch 40/1077 - Train Accuracy: 0.3199, Validation Accuracy: 0.3796, Loss: 3.4564\n", + "Epoch 0 Batch 50/1077 - Train Accuracy: 0.3254, Validation Accuracy: 0.3920, Loss: 3.3164\n", + "Epoch 0 Batch 60/1077 - Train Accuracy: 0.3839, Validation Accuracy: 0.4176, Loss: 3.0631\n", + "Epoch 0 Batch 70/1077 - Train Accuracy: 0.3326, Validation Accuracy: 0.4208, Loss: 3.1856\n", + "Epoch 0 Batch 80/1077 - Train Accuracy: 0.3852, Validation Accuracy: 0.4343, Loss: 2.9757\n", + "Epoch 0 Batch 90/1077 - Train Accuracy: 0.3703, Validation Accuracy: 0.4293, Loss: 2.9309\n", + "Epoch 0 Batch 100/1077 - Train Accuracy: 0.3984, Validation Accuracy: 0.4624, Loss: 2.8666\n", + "Epoch 0 Batch 110/1077 - Train Accuracy: 0.4301, Validation Accuracy: 0.4691, Loss: 2.7108\n", + "Epoch 0 Batch 120/1077 - Train Accuracy: 0.4000, Validation Accuracy: 0.4659, Loss: 2.7147\n", + "Epoch 0 Batch 130/1077 - Train Accuracy: 0.4565, Validation Accuracy: 0.4851, Loss: 2.5051\n", + "Epoch 0 Batch 140/1077 - Train Accuracy: 0.4050, Validation Accuracy: 0.4862, Loss: 2.7329\n", + "Epoch 0 Batch 150/1077 - Train Accuracy: 0.4769, Validation Accuracy: 0.4908, Loss: 2.3903\n", + "Epoch 0 Batch 160/1077 - Train Accuracy: 0.4363, Validation Accuracy: 0.4901, Loss: 2.4526\n", + "Epoch 0 Batch 170/1077 - Train Accuracy: 0.4066, Validation Accuracy: 0.4844, Loss: 2.5281\n", + "Epoch 0 Batch 180/1077 - Train Accuracy: 0.4434, Validation Accuracy: 0.4893, Loss: 2.3616\n", + "Epoch 0 Batch 190/1077 - Train Accuracy: 0.4480, Validation Accuracy: 0.4897, Loss: 2.3697\n", + "Epoch 0 Batch 200/1077 - Train Accuracy: 0.4352, Validation Accuracy: 0.5014, Loss: 2.3168\n", + "Epoch 0 Batch 210/1077 - Train Accuracy: 0.4598, Validation Accuracy: 0.4915, Loss: 2.3032\n", + "Epoch 0 Batch 220/1077 - Train Accuracy: 0.4268, Validation Accuracy: 0.5004, Loss: 2.2940\n", + "Epoch 0 Batch 230/1077 - Train Accuracy: 0.4792, Validation Accuracy: 0.5000, Loss: 2.0715\n", + "Epoch 0 Batch 240/1077 - Train Accuracy: 0.4566, Validation Accuracy: 0.4918, Loss: 2.1614\n", + "Epoch 0 Batch 250/1077 - Train Accuracy: 0.4631, Validation Accuracy: 0.5014, Loss: 2.0283\n", + "Epoch 0 Batch 260/1077 - Train Accuracy: 0.4732, Validation Accuracy: 0.4964, Loss: 2.0801\n", + "Epoch 0 Batch 270/1077 - Train Accuracy: 0.3953, Validation Accuracy: 0.4886, Loss: 2.2588\n", + "Epoch 0 Batch 280/1077 - Train Accuracy: 0.4664, Validation Accuracy: 0.4879, Loss: 2.0910\n", + "Epoch 0 Batch 290/1077 - Train Accuracy: 0.4477, Validation Accuracy: 0.5025, Loss: 2.0713\n", + "Epoch 0 Batch 300/1077 - Train Accuracy: 0.4108, Validation Accuracy: 0.4989, Loss: 2.1779\n", + "Epoch 0 Batch 310/1077 - Train Accuracy: 0.4324, Validation Accuracy: 0.5053, Loss: 1.9994\n", + "Epoch 0 Batch 320/1077 - Train Accuracy: 0.4668, Validation Accuracy: 0.4982, Loss: 2.0184\n", + "Epoch 0 Batch 330/1077 - Train Accuracy: 0.4809, Validation Accuracy: 0.5057, Loss: 1.9595\n", + "Epoch 0 Batch 340/1077 - Train Accuracy: 0.4239, Validation Accuracy: 0.4957, Loss: 1.9988\n", + "Epoch 0 Batch 350/1077 - Train Accuracy: 0.4230, Validation Accuracy: 0.5075, Loss: 1.9789\n", + "Epoch 0 Batch 360/1077 - Train Accuracy: 0.4395, Validation Accuracy: 0.5050, Loss: 1.8374\n", + "Epoch 0 Batch 370/1077 - Train Accuracy: 0.4342, Validation Accuracy: 0.4805, Loss: 1.7781\n", + "Epoch 0 Batch 380/1077 - Train Accuracy: 0.4309, Validation Accuracy: 0.4901, Loss: 1.7284\n", + "Epoch 0 Batch 390/1077 - Train Accuracy: 0.4254, Validation Accuracy: 0.5007, Loss: 1.8143\n", + "Epoch 0 Batch 400/1077 - Train Accuracy: 0.4359, Validation Accuracy: 0.4996, Loss: 1.7578\n", + "Epoch 0 Batch 410/1077 - Train Accuracy: 0.4219, Validation Accuracy: 0.5178, Loss: 1.7815\n", + "Epoch 0 Batch 420/1077 - Train Accuracy: 0.4449, Validation Accuracy: 0.5107, Loss: 1.6739\n", + "Epoch 0 Batch 430/1077 - Train Accuracy: 0.4277, Validation Accuracy: 0.4982, Loss: 1.6522\n", + "Epoch 0 Batch 440/1077 - Train Accuracy: 0.4383, Validation Accuracy: 0.4851, Loss: 1.6351\n", + "Epoch 0 Batch 450/1077 - Train Accuracy: 0.4238, Validation Accuracy: 0.4790, Loss: 1.6293\n", + "Epoch 0 Batch 460/1077 - Train Accuracy: 0.4602, Validation Accuracy: 0.5181, Loss: 1.6125\n", + "Epoch 0 Batch 470/1077 - Train Accuracy: 0.4285, Validation Accuracy: 0.5096, Loss: 1.6410\n", + "Epoch 0 Batch 480/1077 - Train Accuracy: 0.4412, Validation Accuracy: 0.4901, Loss: 1.6211\n", + "Epoch 0 Batch 490/1077 - Train Accuracy: 0.4113, Validation Accuracy: 0.4830, Loss: 1.5815\n", + "Epoch 0 Batch 500/1077 - Train Accuracy: 0.4535, Validation Accuracy: 0.4883, Loss: 1.5060\n", + "Epoch 0 Batch 510/1077 - Train Accuracy: 0.4680, Validation Accuracy: 0.5124, Loss: 1.5071\n", + "Epoch 0 Batch 520/1077 - Train Accuracy: 0.4978, Validation Accuracy: 0.5188, Loss: 1.4464\n", + "Epoch 0 Batch 530/1077 - Train Accuracy: 0.4285, Validation Accuracy: 0.5028, Loss: 1.4873\n", + "Epoch 0 Batch 540/1077 - Train Accuracy: 0.4484, Validation Accuracy: 0.4993, Loss: 1.4363\n", + "Epoch 0 Batch 550/1077 - Train Accuracy: 0.4164, Validation Accuracy: 0.5110, Loss: 1.4702\n", + "Epoch 0 Batch 560/1077 - Train Accuracy: 0.4617, Validation Accuracy: 0.5227, Loss: 1.4247\n", + "Epoch 0 Batch 570/1077 - Train Accuracy: 0.4457, Validation Accuracy: 0.5174, Loss: 1.4704\n", + "Epoch 0 Batch 580/1077 - Train Accuracy: 0.4754, Validation Accuracy: 0.5043, Loss: 1.3448\n", + "Epoch 0 Batch 590/1077 - Train Accuracy: 0.4354, Validation Accuracy: 0.5004, Loss: 1.4540\n", + "Epoch 0 Batch 600/1077 - Train Accuracy: 0.5041, Validation Accuracy: 0.5043, Loss: 1.3436\n", + "Epoch 0 Batch 610/1077 - Train Accuracy: 0.4663, Validation Accuracy: 0.5217, Loss: 1.4437\n", + "Epoch 0 Batch 620/1077 - Train Accuracy: 0.4863, Validation Accuracy: 0.5241, Loss: 1.3460\n", + "Epoch 0 Batch 630/1077 - Train Accuracy: 0.4906, Validation Accuracy: 0.5206, Loss: 1.3157\n", + "Epoch 0 Batch 640/1077 - Train Accuracy: 0.4758, Validation Accuracy: 0.5263, Loss: 1.2873\n", + "Epoch 0 Batch 650/1077 - Train Accuracy: 0.4500, Validation Accuracy: 0.5163, Loss: 1.3186\n", + "Epoch 0 Batch 660/1077 - Train Accuracy: 0.4922, Validation Accuracy: 0.5273, Loss: 1.3382\n", + "Epoch 0 Batch 670/1077 - Train Accuracy: 0.5394, Validation Accuracy: 0.5234, Loss: 1.1838\n", + "Epoch 0 Batch 680/1077 - Train Accuracy: 0.4963, Validation Accuracy: 0.5344, Loss: 1.2411\n", + "Epoch 0 Batch 690/1077 - Train Accuracy: 0.4973, Validation Accuracy: 0.5252, Loss: 1.2440\n", + "Epoch 0 Batch 700/1077 - Train Accuracy: 0.4621, Validation Accuracy: 0.5359, Loss: 1.2279\n", + "Epoch 0 Batch 710/1077 - Train Accuracy: 0.4648, Validation Accuracy: 0.5522, Loss: 1.2376\n", + "Epoch 0 Batch 720/1077 - Train Accuracy: 0.4576, Validation Accuracy: 0.5060, Loss: 1.3337\n", + "Epoch 0 Batch 730/1077 - Train Accuracy: 0.4984, Validation Accuracy: 0.5330, Loss: 1.2081\n", + "Epoch 0 Batch 740/1077 - Train Accuracy: 0.4961, Validation Accuracy: 0.5344, Loss: 1.1684\n", + "Epoch 0 Batch 750/1077 - Train Accuracy: 0.4848, Validation Accuracy: 0.5369, Loss: 1.1756\n", + "Epoch 0 Batch 760/1077 - Train Accuracy: 0.4992, Validation Accuracy: 0.5320, Loss: 1.2197\n", + "Epoch 0 Batch 770/1077 - Train Accuracy: 0.4996, Validation Accuracy: 0.5273, Loss: 1.1501\n", + "Epoch 0 Batch 780/1077 - Train Accuracy: 0.4898, Validation Accuracy: 0.5408, Loss: 1.2084\n", + "Epoch 0 Batch 790/1077 - Train Accuracy: 0.4277, Validation Accuracy: 0.5376, Loss: 1.2224\n", + "Epoch 0 Batch 800/1077 - Train Accuracy: 0.4680, Validation Accuracy: 0.5529, Loss: 1.1739\n", + "Epoch 0 Batch 810/1077 - Train Accuracy: 0.5301, Validation Accuracy: 0.5497, Loss: 1.0756\n", + "Epoch 0 Batch 820/1077 - Train Accuracy: 0.4832, Validation Accuracy: 0.5508, Loss: 1.1588\n", + "Epoch 0 Batch 830/1077 - Train Accuracy: 0.4992, Validation Accuracy: 0.5419, Loss: 1.0921\n", + "Epoch 0 Batch 840/1077 - Train Accuracy: 0.4840, Validation Accuracy: 0.5604, Loss: 1.1070\n", + "Epoch 0 Batch 850/1077 - Train Accuracy: 0.5115, Validation Accuracy: 0.5639, Loss: 1.1292\n", + "Epoch 0 Batch 860/1077 - Train Accuracy: 0.5060, Validation Accuracy: 0.5607, Loss: 1.0795\n", + "Epoch 0 Batch 870/1077 - Train Accuracy: 0.4799, Validation Accuracy: 0.5497, Loss: 1.2124\n", + "Epoch 0 Batch 880/1077 - Train Accuracy: 0.5363, Validation Accuracy: 0.5579, Loss: 1.0369\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 0 Batch 890/1077 - Train Accuracy: 0.5800, Validation Accuracy: 0.5611, Loss: 0.9741\n", + "Epoch 0 Batch 900/1077 - Train Accuracy: 0.4941, Validation Accuracy: 0.5629, Loss: 1.1257\n", + "Epoch 0 Batch 910/1077 - Train Accuracy: 0.5052, Validation Accuracy: 0.5572, Loss: 1.0554\n", + "Epoch 0 Batch 920/1077 - Train Accuracy: 0.4973, Validation Accuracy: 0.5529, Loss: 1.0764\n", + "Epoch 0 Batch 930/1077 - Train Accuracy: 0.5285, Validation Accuracy: 0.5625, Loss: 1.0595\n", + "Epoch 0 Batch 940/1077 - Train Accuracy: 0.5051, Validation Accuracy: 0.5579, Loss: 1.0643\n", + "Epoch 0 Batch 950/1077 - Train Accuracy: 0.4929, Validation Accuracy: 0.5572, Loss: 1.0296\n", + "Epoch 0 Batch 960/1077 - Train Accuracy: 0.5361, Validation Accuracy: 0.5643, Loss: 0.9874\n", + "Epoch 0 Batch 970/1077 - Train Accuracy: 0.5238, Validation Accuracy: 0.5760, Loss: 1.0584\n", + "Epoch 0 Batch 980/1077 - Train Accuracy: 0.5281, Validation Accuracy: 0.5675, Loss: 1.0047\n", + "Epoch 0 Batch 990/1077 - Train Accuracy: 0.5193, Validation Accuracy: 0.5646, Loss: 1.1023\n", + "Epoch 0 Batch 1000/1077 - Train Accuracy: 0.5495, Validation Accuracy: 0.5540, Loss: 0.9711\n", + "Epoch 0 Batch 1010/1077 - Train Accuracy: 0.5152, Validation Accuracy: 0.5568, Loss: 1.0354\n", + "Epoch 0 Batch 1020/1077 - Train Accuracy: 0.5117, Validation Accuracy: 0.5593, Loss: 0.9774\n", + "Epoch 0 Batch 1030/1077 - Train Accuracy: 0.5020, Validation Accuracy: 0.5565, Loss: 1.0170\n", + "Epoch 0 Batch 1040/1077 - Train Accuracy: 0.5214, Validation Accuracy: 0.5774, Loss: 1.0326\n", + "Epoch 0 Batch 1050/1077 - Train Accuracy: 0.4715, Validation Accuracy: 0.5813, Loss: 1.0371\n", + "Epoch 0 Batch 1060/1077 - Train Accuracy: 0.5176, Validation Accuracy: 0.5781, Loss: 0.9651\n", + "Epoch 0 Batch 1070/1077 - Train Accuracy: 0.5121, Validation Accuracy: 0.5753, Loss: 1.0146\n", + "Epoch 1 Batch 10/1077 - Train Accuracy: 0.5201, Validation Accuracy: 0.5806, Loss: 1.0158\n", + "Epoch 1 Batch 20/1077 - Train Accuracy: 0.5551, Validation Accuracy: 0.5884, Loss: 0.9220\n", + "Epoch 1 Batch 30/1077 - Train Accuracy: 0.5555, Validation Accuracy: 0.5938, Loss: 0.9371\n", + "Epoch 1 Batch 40/1077 - Train Accuracy: 0.5359, Validation Accuracy: 0.5859, Loss: 0.9603\n", + "Epoch 1 Batch 50/1077 - Train Accuracy: 0.5301, Validation Accuracy: 0.5817, Loss: 0.9874\n", + "Epoch 1 Batch 60/1077 - Train Accuracy: 0.5543, Validation Accuracy: 0.5913, Loss: 0.9223\n", + "Epoch 1 Batch 70/1077 - Train Accuracy: 0.5238, Validation Accuracy: 0.5906, Loss: 0.9876\n", + "Epoch 1 Batch 80/1077 - Train Accuracy: 0.5430, Validation Accuracy: 0.5977, Loss: 0.9424\n", + "Epoch 1 Batch 90/1077 - Train Accuracy: 0.5563, Validation Accuracy: 0.5948, Loss: 0.9796\n", + "Epoch 1 Batch 100/1077 - Train Accuracy: 0.5707, Validation Accuracy: 0.5991, Loss: 0.9486\n", + "Epoch 1 Batch 110/1077 - Train Accuracy: 0.5816, Validation Accuracy: 0.5948, Loss: 0.8814\n", + "Epoch 1 Batch 120/1077 - Train Accuracy: 0.5430, Validation Accuracy: 0.5895, Loss: 0.9393\n", + "Epoch 1 Batch 130/1077 - Train Accuracy: 0.5621, Validation Accuracy: 0.5906, Loss: 0.8740\n", + "Epoch 1 Batch 140/1077 - Train Accuracy: 0.5008, Validation Accuracy: 0.5845, Loss: 0.9523\n", + "Epoch 1 Batch 150/1077 - Train Accuracy: 0.5904, Validation Accuracy: 0.5863, Loss: 0.8786\n", + "Epoch 1 Batch 160/1077 - Train Accuracy: 0.5500, Validation Accuracy: 0.5898, Loss: 0.8778\n", + "Epoch 1 Batch 170/1077 - Train Accuracy: 0.5188, Validation Accuracy: 0.5902, Loss: 0.9434\n", + "Epoch 1 Batch 180/1077 - Train Accuracy: 0.5590, Validation Accuracy: 0.5859, Loss: 0.9147\n", + "Epoch 1 Batch 190/1077 - Train Accuracy: 0.5734, Validation Accuracy: 0.5838, Loss: 0.8906\n", + "Epoch 1 Batch 200/1077 - Train Accuracy: 0.5133, Validation Accuracy: 0.5952, Loss: 0.9241\n", + "Epoch 1 Batch 210/1077 - Train Accuracy: 0.5636, Validation Accuracy: 0.5778, Loss: 0.9116\n", + "Epoch 1 Batch 220/1077 - Train Accuracy: 0.5337, Validation Accuracy: 0.5895, Loss: 0.9174\n", + "Epoch 1 Batch 230/1077 - Train Accuracy: 0.5800, Validation Accuracy: 0.6005, Loss: 0.8552\n", + "Epoch 1 Batch 240/1077 - Train Accuracy: 0.5938, Validation Accuracy: 0.5909, Loss: 0.8971\n", + "Epoch 1 Batch 250/1077 - Train Accuracy: 0.5572, Validation Accuracy: 0.5881, Loss: 0.8257\n", + "Epoch 1 Batch 260/1077 - Train Accuracy: 0.5625, Validation Accuracy: 0.5845, Loss: 0.8276\n", + "Epoch 1 Batch 270/1077 - Train Accuracy: 0.5012, Validation Accuracy: 0.5909, Loss: 0.9309\n", + "Epoch 1 Batch 280/1077 - Train Accuracy: 0.5855, Validation Accuracy: 0.6019, Loss: 0.8798\n", + "Epoch 1 Batch 290/1077 - Train Accuracy: 0.5508, Validation Accuracy: 0.5831, Loss: 0.8977\n", + "Epoch 1 Batch 300/1077 - Train Accuracy: 0.5251, Validation Accuracy: 0.5909, Loss: 0.9215\n", + "Epoch 1 Batch 310/1077 - Train Accuracy: 0.5449, Validation Accuracy: 0.5856, Loss: 0.8728\n", + "Epoch 1 Batch 320/1077 - Train Accuracy: 0.5746, Validation Accuracy: 0.5984, Loss: 0.8715\n", + "Epoch 1 Batch 330/1077 - Train Accuracy: 0.5926, Validation Accuracy: 0.5987, Loss: 0.8628\n", + "Epoch 1 Batch 340/1077 - Train Accuracy: 0.4992, Validation Accuracy: 0.5984, Loss: 0.8922\n", + "Epoch 1 Batch 350/1077 - Train Accuracy: 0.5266, Validation Accuracy: 0.5966, Loss: 0.9021\n", + "Epoch 1 Batch 360/1077 - Train Accuracy: 0.5582, Validation Accuracy: 0.5859, Loss: 0.8790\n", + "Epoch 1 Batch 370/1077 - Train Accuracy: 0.5651, Validation Accuracy: 0.5934, Loss: 0.8215\n", + "Epoch 1 Batch 380/1077 - Train Accuracy: 0.5785, Validation Accuracy: 0.6051, Loss: 0.8272\n", + "Epoch 1 Batch 390/1077 - Train Accuracy: 0.5344, Validation Accuracy: 0.6048, Loss: 0.8952\n", + "Epoch 1 Batch 400/1077 - Train Accuracy: 0.5687, Validation Accuracy: 0.5973, Loss: 0.8480\n", + "Epoch 1 Batch 410/1077 - Train Accuracy: 0.5506, Validation Accuracy: 0.6065, Loss: 0.8688\n", + "Epoch 1 Batch 420/1077 - Train Accuracy: 0.5797, Validation Accuracy: 0.6037, Loss: 0.8266\n", + "Epoch 1 Batch 430/1077 - Train Accuracy: 0.5656, Validation Accuracy: 0.5909, Loss: 0.8365\n", + "Epoch 1 Batch 440/1077 - Train Accuracy: 0.5633, Validation Accuracy: 0.6083, Loss: 0.8696\n", + "Epoch 1 Batch 450/1077 - Train Accuracy: 0.5660, Validation Accuracy: 0.5980, Loss: 0.8449\n", + "Epoch 1 Batch 460/1077 - Train Accuracy: 0.5699, Validation Accuracy: 0.6005, Loss: 0.8436\n", + "Epoch 1 Batch 470/1077 - Train Accuracy: 0.5325, Validation Accuracy: 0.5852, Loss: 0.8719\n", + "Epoch 1 Batch 480/1077 - Train Accuracy: 0.5752, Validation Accuracy: 0.5945, Loss: 0.8617\n", + "Epoch 1 Batch 490/1077 - Train Accuracy: 0.5605, Validation Accuracy: 0.5977, Loss: 0.8487\n", + "Epoch 1 Batch 500/1077 - Train Accuracy: 0.5824, Validation Accuracy: 0.5973, Loss: 0.8165\n", + "Epoch 1 Batch 510/1077 - Train Accuracy: 0.5969, Validation Accuracy: 0.6072, Loss: 0.8120\n", + "Epoch 1 Batch 520/1077 - Train Accuracy: 0.6138, Validation Accuracy: 0.6069, Loss: 0.7912\n", + "Epoch 1 Batch 530/1077 - Train Accuracy: 0.5637, Validation Accuracy: 0.6069, Loss: 0.8342\n", + "Epoch 1 Batch 540/1077 - Train Accuracy: 0.5855, Validation Accuracy: 0.6030, Loss: 0.7929\n", + "Epoch 1 Batch 550/1077 - Train Accuracy: 0.5535, Validation Accuracy: 0.6112, Loss: 0.8427\n", + "Epoch 1 Batch 560/1077 - Train Accuracy: 0.5777, Validation Accuracy: 0.6072, Loss: 0.7943\n", + "Epoch 1 Batch 570/1077 - Train Accuracy: 0.5839, Validation Accuracy: 0.6175, Loss: 0.8286\n", + "Epoch 1 Batch 580/1077 - Train Accuracy: 0.6239, Validation Accuracy: 0.6076, Loss: 0.7470\n", + "Epoch 1 Batch 590/1077 - Train Accuracy: 0.5670, Validation Accuracy: 0.6101, Loss: 0.8526\n", + "Epoch 1 Batch 600/1077 - Train Accuracy: 0.6049, Validation Accuracy: 0.6161, Loss: 0.7728\n", + "Epoch 1 Batch 610/1077 - Train Accuracy: 0.5703, Validation Accuracy: 0.6080, Loss: 0.8427\n", + "Epoch 1 Batch 620/1077 - Train Accuracy: 0.5719, Validation Accuracy: 0.5998, Loss: 0.7809\n", + "Epoch 1 Batch 630/1077 - Train Accuracy: 0.5664, Validation Accuracy: 0.6083, Loss: 0.7734\n", + "Epoch 1 Batch 640/1077 - Train Accuracy: 0.5722, Validation Accuracy: 0.6115, Loss: 0.7488\n", + "Epoch 1 Batch 650/1077 - Train Accuracy: 0.5738, Validation Accuracy: 0.6112, Loss: 0.7811\n", + "Epoch 1 Batch 660/1077 - Train Accuracy: 0.5848, Validation Accuracy: 0.6016, Loss: 0.7984\n", + "Epoch 1 Batch 670/1077 - Train Accuracy: 0.6346, Validation Accuracy: 0.6037, Loss: 0.7211\n", + "Epoch 1 Batch 680/1077 - Train Accuracy: 0.6012, Validation Accuracy: 0.6115, Loss: 0.7470\n", + "Epoch 1 Batch 690/1077 - Train Accuracy: 0.6262, Validation Accuracy: 0.6058, Loss: 0.7671\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 1 Batch 700/1077 - Train Accuracy: 0.5559, Validation Accuracy: 0.5980, Loss: 0.7746\n", + "Epoch 1 Batch 710/1077 - Train Accuracy: 0.5445, Validation Accuracy: 0.6044, Loss: 0.7695\n", + "Epoch 1 Batch 720/1077 - Train Accuracy: 0.5970, Validation Accuracy: 0.6119, Loss: 0.8337\n", + "Epoch 1 Batch 730/1077 - Train Accuracy: 0.5734, Validation Accuracy: 0.6147, Loss: 0.7863\n", + "Epoch 1 Batch 740/1077 - Train Accuracy: 0.5746, Validation Accuracy: 0.5945, Loss: 0.7637\n", + "Epoch 1 Batch 750/1077 - Train Accuracy: 0.6023, Validation Accuracy: 0.6172, Loss: 0.7561\n", + "Epoch 1 Batch 760/1077 - Train Accuracy: 0.5945, Validation Accuracy: 0.6094, Loss: 0.7678\n", + "Epoch 1 Batch 770/1077 - Train Accuracy: 0.6287, Validation Accuracy: 0.6154, Loss: 0.7080\n", + "Epoch 1 Batch 780/1077 - Train Accuracy: 0.5938, Validation Accuracy: 0.6151, Loss: 0.7878\n", + "Epoch 1 Batch 790/1077 - Train Accuracy: 0.5383, Validation Accuracy: 0.6112, Loss: 0.8115\n", + "Epoch 1 Batch 800/1077 - Train Accuracy: 0.5711, Validation Accuracy: 0.6239, Loss: 0.7480\n", + "Epoch 1 Batch 810/1077 - Train Accuracy: 0.6179, Validation Accuracy: 0.6190, Loss: 0.7067\n", + "Epoch 1 Batch 820/1077 - Train Accuracy: 0.5844, Validation Accuracy: 0.6246, Loss: 0.7690\n", + "Epoch 1 Batch 830/1077 - Train Accuracy: 0.5840, Validation Accuracy: 0.6243, Loss: 0.7264\n", + "Epoch 1 Batch 840/1077 - Train Accuracy: 0.6203, Validation Accuracy: 0.6218, Loss: 0.7294\n", + "Epoch 1 Batch 850/1077 - Train Accuracy: 0.5885, Validation Accuracy: 0.6286, Loss: 0.7776\n", + "Epoch 1 Batch 860/1077 - Train Accuracy: 0.5807, Validation Accuracy: 0.6310, Loss: 0.7363\n", + "Epoch 1 Batch 870/1077 - Train Accuracy: 0.5707, Validation Accuracy: 0.6250, Loss: 0.7958\n", + "Epoch 1 Batch 880/1077 - Train Accuracy: 0.6258, Validation Accuracy: 0.6246, Loss: 0.7289\n", + "Epoch 1 Batch 890/1077 - Train Accuracy: 0.6685, Validation Accuracy: 0.6243, Loss: 0.6789\n", + "Epoch 1 Batch 900/1077 - Train Accuracy: 0.6117, Validation Accuracy: 0.6246, Loss: 0.7535\n", + "Epoch 1 Batch 910/1077 - Train Accuracy: 0.5841, Validation Accuracy: 0.6179, Loss: 0.7229\n", + "Epoch 1 Batch 920/1077 - Train Accuracy: 0.6008, Validation Accuracy: 0.6239, Loss: 0.7411\n", + "Epoch 1 Batch 930/1077 - Train Accuracy: 0.6188, Validation Accuracy: 0.6229, Loss: 0.7168\n", + "Epoch 1 Batch 940/1077 - Train Accuracy: 0.5918, Validation Accuracy: 0.6271, Loss: 0.7332\n", + "Epoch 1 Batch 950/1077 - Train Accuracy: 0.5893, Validation Accuracy: 0.6232, Loss: 0.6840\n", + "Epoch 1 Batch 960/1077 - Train Accuracy: 0.6310, Validation Accuracy: 0.6321, Loss: 0.6854\n", + "Epoch 1 Batch 970/1077 - Train Accuracy: 0.6293, Validation Accuracy: 0.6307, Loss: 0.7430\n", + "Epoch 1 Batch 980/1077 - Train Accuracy: 0.6238, Validation Accuracy: 0.6325, Loss: 0.7040\n", + "Epoch 1 Batch 990/1077 - Train Accuracy: 0.6102, Validation Accuracy: 0.6349, Loss: 0.7441\n", + "Epoch 1 Batch 1000/1077 - Train Accuracy: 0.6626, Validation Accuracy: 0.6282, Loss: 0.6564\n", + "Epoch 1 Batch 1010/1077 - Train Accuracy: 0.6129, Validation Accuracy: 0.6282, Loss: 0.7226\n", + "Epoch 1 Batch 1020/1077 - Train Accuracy: 0.6086, Validation Accuracy: 0.6293, Loss: 0.6749\n", + "Epoch 1 Batch 1030/1077 - Train Accuracy: 0.5945, Validation Accuracy: 0.6374, Loss: 0.7197\n", + "Epoch 1 Batch 1040/1077 - Train Accuracy: 0.5942, Validation Accuracy: 0.6353, Loss: 0.7343\n", + "Epoch 1 Batch 1050/1077 - Train Accuracy: 0.5492, Validation Accuracy: 0.6399, Loss: 0.7068\n", + "Epoch 1 Batch 1060/1077 - Train Accuracy: 0.6148, Validation Accuracy: 0.6346, Loss: 0.6741\n", + "Epoch 1 Batch 1070/1077 - Train Accuracy: 0.5762, Validation Accuracy: 0.6364, Loss: 0.7347\n", + "Epoch 2 Batch 10/1077 - Train Accuracy: 0.6032, Validation Accuracy: 0.6420, Loss: 0.7120\n", + "Epoch 2 Batch 20/1077 - Train Accuracy: 0.6000, Validation Accuracy: 0.6470, Loss: 0.6532\n", + "Epoch 2 Batch 30/1077 - Train Accuracy: 0.6113, Validation Accuracy: 0.6417, Loss: 0.6639\n", + "Epoch 2 Batch 40/1077 - Train Accuracy: 0.6270, Validation Accuracy: 0.6463, Loss: 0.6820\n", + "Epoch 2 Batch 50/1077 - Train Accuracy: 0.5813, Validation Accuracy: 0.6342, Loss: 0.6976\n", + "Epoch 2 Batch 60/1077 - Train Accuracy: 0.6161, Validation Accuracy: 0.6470, Loss: 0.6660\n", + "Epoch 2 Batch 70/1077 - Train Accuracy: 0.5938, Validation Accuracy: 0.6413, Loss: 0.6931\n", + "Epoch 2 Batch 80/1077 - Train Accuracy: 0.6277, Validation Accuracy: 0.6477, Loss: 0.6595\n", + "Epoch 2 Batch 90/1077 - Train Accuracy: 0.5984, Validation Accuracy: 0.6474, Loss: 0.7169\n", + "Epoch 2 Batch 100/1077 - Train Accuracy: 0.6402, Validation Accuracy: 0.6502, Loss: 0.6720\n", + "Epoch 2 Batch 110/1077 - Train Accuracy: 0.6520, Validation Accuracy: 0.6502, Loss: 0.6393\n", + "Epoch 2 Batch 120/1077 - Train Accuracy: 0.6406, Validation Accuracy: 0.6491, Loss: 0.6647\n", + "Epoch 2 Batch 130/1077 - Train Accuracy: 0.6332, Validation Accuracy: 0.6495, Loss: 0.6443\n", + "Epoch 2 Batch 140/1077 - Train Accuracy: 0.5979, Validation Accuracy: 0.6552, Loss: 0.6842\n", + "Epoch 2 Batch 150/1077 - Train Accuracy: 0.6551, Validation Accuracy: 0.6527, Loss: 0.6237\n", + "Epoch 2 Batch 160/1077 - Train Accuracy: 0.6441, Validation Accuracy: 0.6403, Loss: 0.6408\n", + "Epoch 2 Batch 170/1077 - Train Accuracy: 0.5852, Validation Accuracy: 0.6449, Loss: 0.6759\n", + "Epoch 2 Batch 180/1077 - Train Accuracy: 0.6188, Validation Accuracy: 0.6452, Loss: 0.6540\n", + "Epoch 2 Batch 190/1077 - Train Accuracy: 0.6559, Validation Accuracy: 0.6481, Loss: 0.6341\n", + "Epoch 2 Batch 200/1077 - Train Accuracy: 0.6219, Validation Accuracy: 0.6420, Loss: 0.6726\n", + "Epoch 2 Batch 210/1077 - Train Accuracy: 0.6484, Validation Accuracy: 0.6559, Loss: 0.6498\n", + "Epoch 2 Batch 220/1077 - Train Accuracy: 0.6160, Validation Accuracy: 0.6527, Loss: 0.6705\n", + "Epoch 2 Batch 230/1077 - Train Accuracy: 0.6577, Validation Accuracy: 0.6566, Loss: 0.6111\n", + "Epoch 2 Batch 240/1077 - Train Accuracy: 0.6781, Validation Accuracy: 0.6516, Loss: 0.6253\n", + "Epoch 2 Batch 250/1077 - Train Accuracy: 0.6112, Validation Accuracy: 0.6538, Loss: 0.6155\n", + "Epoch 2 Batch 260/1077 - Train Accuracy: 0.6585, Validation Accuracy: 0.6573, Loss: 0.5932\n", + "Epoch 2 Batch 270/1077 - Train Accuracy: 0.6062, Validation Accuracy: 0.6531, Loss: 0.6607\n", + "Epoch 2 Batch 280/1077 - Train Accuracy: 0.6586, Validation Accuracy: 0.6598, Loss: 0.6697\n", + "Epoch 2 Batch 290/1077 - Train Accuracy: 0.6367, Validation Accuracy: 0.6545, Loss: 0.6541\n", + "Epoch 2 Batch 300/1077 - Train Accuracy: 0.6242, Validation Accuracy: 0.6665, Loss: 0.6625\n", + "Epoch 2 Batch 310/1077 - Train Accuracy: 0.6348, Validation Accuracy: 0.6573, Loss: 0.6534\n", + "Epoch 2 Batch 320/1077 - Train Accuracy: 0.6449, Validation Accuracy: 0.6559, Loss: 0.6487\n", + "Epoch 2 Batch 330/1077 - Train Accuracy: 0.6676, Validation Accuracy: 0.6445, Loss: 0.6425\n", + "Epoch 2 Batch 340/1077 - Train Accuracy: 0.5962, Validation Accuracy: 0.6605, Loss: 0.6331\n", + "Epoch 2 Batch 350/1077 - Train Accuracy: 0.5965, Validation Accuracy: 0.6609, Loss: 0.6446\n", + "Epoch 2 Batch 360/1077 - Train Accuracy: 0.6293, Validation Accuracy: 0.6644, Loss: 0.6339\n", + "Epoch 2 Batch 370/1077 - Train Accuracy: 0.6391, Validation Accuracy: 0.6634, Loss: 0.5957\n", + "Epoch 2 Batch 380/1077 - Train Accuracy: 0.6414, Validation Accuracy: 0.6555, Loss: 0.6086\n", + "Epoch 2 Batch 390/1077 - Train Accuracy: 0.6055, Validation Accuracy: 0.6598, Loss: 0.6509\n", + "Epoch 2 Batch 400/1077 - Train Accuracy: 0.6512, Validation Accuracy: 0.6630, Loss: 0.6346\n", + "Epoch 2 Batch 410/1077 - Train Accuracy: 0.6110, Validation Accuracy: 0.6502, Loss: 0.6315\n", + "Epoch 2 Batch 420/1077 - Train Accuracy: 0.6332, Validation Accuracy: 0.6634, Loss: 0.5854\n", + "Epoch 2 Batch 430/1077 - Train Accuracy: 0.6320, Validation Accuracy: 0.6637, Loss: 0.6120\n", + "Epoch 2 Batch 440/1077 - Train Accuracy: 0.6273, Validation Accuracy: 0.6648, Loss: 0.6486\n", + "Epoch 2 Batch 450/1077 - Train Accuracy: 0.6305, Validation Accuracy: 0.6623, Loss: 0.6008\n", + "Epoch 2 Batch 460/1077 - Train Accuracy: 0.6301, Validation Accuracy: 0.6690, Loss: 0.6139\n", + "Epoch 2 Batch 470/1077 - Train Accuracy: 0.6164, Validation Accuracy: 0.6694, Loss: 0.6300\n", + "Epoch 2 Batch 480/1077 - Train Accuracy: 0.6612, Validation Accuracy: 0.6676, Loss: 0.6043\n", + "Epoch 2 Batch 490/1077 - Train Accuracy: 0.6387, Validation Accuracy: 0.6719, Loss: 0.6190\n", + "Epoch 2 Batch 500/1077 - Train Accuracy: 0.6461, Validation Accuracy: 0.6680, Loss: 0.5966\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 2 Batch 510/1077 - Train Accuracy: 0.6617, Validation Accuracy: 0.6705, Loss: 0.5906\n", + "Epoch 2 Batch 520/1077 - Train Accuracy: 0.6726, Validation Accuracy: 0.6644, Loss: 0.5635\n", + "Epoch 2 Batch 530/1077 - Train Accuracy: 0.6379, Validation Accuracy: 0.6694, Loss: 0.6121\n", + "Epoch 2 Batch 540/1077 - Train Accuracy: 0.6656, Validation Accuracy: 0.6658, Loss: 0.5683\n", + "Epoch 2 Batch 550/1077 - Train Accuracy: 0.6160, Validation Accuracy: 0.6648, Loss: 0.6148\n", + "Epoch 2 Batch 560/1077 - Train Accuracy: 0.6305, Validation Accuracy: 0.6662, Loss: 0.5748\n", + "Epoch 2 Batch 570/1077 - Train Accuracy: 0.6521, Validation Accuracy: 0.6616, Loss: 0.6042\n", + "Epoch 2 Batch 580/1077 - Train Accuracy: 0.6871, Validation Accuracy: 0.6690, Loss: 0.5451\n", + "Epoch 2 Batch 590/1077 - Train Accuracy: 0.6357, Validation Accuracy: 0.6715, Loss: 0.6056\n", + "Epoch 2 Batch 600/1077 - Train Accuracy: 0.6815, Validation Accuracy: 0.6715, Loss: 0.5687\n", + "Epoch 2 Batch 610/1077 - Train Accuracy: 0.6517, Validation Accuracy: 0.6761, Loss: 0.6149\n", + "Epoch 2 Batch 620/1077 - Train Accuracy: 0.6484, Validation Accuracy: 0.6669, Loss: 0.5752\n", + "Epoch 2 Batch 630/1077 - Train Accuracy: 0.6578, Validation Accuracy: 0.6740, Loss: 0.5821\n", + "Epoch 2 Batch 640/1077 - Train Accuracy: 0.6399, Validation Accuracy: 0.6726, Loss: 0.5539\n", + "Epoch 2 Batch 650/1077 - Train Accuracy: 0.6328, Validation Accuracy: 0.6637, Loss: 0.5880\n", + "Epoch 2 Batch 660/1077 - Train Accuracy: 0.6535, Validation Accuracy: 0.6630, Loss: 0.5929\n", + "Epoch 2 Batch 670/1077 - Train Accuracy: 0.6935, Validation Accuracy: 0.6765, Loss: 0.5347\n", + "Epoch 2 Batch 680/1077 - Train Accuracy: 0.6652, Validation Accuracy: 0.6662, Loss: 0.5694\n", + "Epoch 2 Batch 690/1077 - Train Accuracy: 0.6633, Validation Accuracy: 0.6626, Loss: 0.5641\n", + "Epoch 2 Batch 700/1077 - Train Accuracy: 0.6426, Validation Accuracy: 0.6747, Loss: 0.5530\n", + "Epoch 2 Batch 710/1077 - Train Accuracy: 0.6258, Validation Accuracy: 0.6715, Loss: 0.5719\n", + "Epoch 2 Batch 720/1077 - Train Accuracy: 0.6612, Validation Accuracy: 0.6779, Loss: 0.6240\n", + "Epoch 2 Batch 730/1077 - Train Accuracy: 0.6656, Validation Accuracy: 0.6740, Loss: 0.5907\n", + "Epoch 2 Batch 740/1077 - Train Accuracy: 0.6605, Validation Accuracy: 0.6669, Loss: 0.5496\n", + "Epoch 2 Batch 750/1077 - Train Accuracy: 0.6961, Validation Accuracy: 0.6772, Loss: 0.5499\n", + "Epoch 2 Batch 760/1077 - Train Accuracy: 0.6734, Validation Accuracy: 0.6729, Loss: 0.5567\n", + "Epoch 2 Batch 770/1077 - Train Accuracy: 0.6793, Validation Accuracy: 0.6740, Loss: 0.5409\n", + "Epoch 2 Batch 780/1077 - Train Accuracy: 0.6590, Validation Accuracy: 0.6744, Loss: 0.5741\n", + "Epoch 2 Batch 790/1077 - Train Accuracy: 0.5996, Validation Accuracy: 0.6712, Loss: 0.6006\n", + "Epoch 2 Batch 800/1077 - Train Accuracy: 0.6379, Validation Accuracy: 0.6751, Loss: 0.5495\n", + "Epoch 2 Batch 810/1077 - Train Accuracy: 0.6667, Validation Accuracy: 0.6719, Loss: 0.5228\n", + "Epoch 2 Batch 820/1077 - Train Accuracy: 0.6449, Validation Accuracy: 0.6822, Loss: 0.5628\n", + "Epoch 2 Batch 830/1077 - Train Accuracy: 0.6609, Validation Accuracy: 0.6793, Loss: 0.5333\n", + "Epoch 2 Batch 840/1077 - Train Accuracy: 0.6789, Validation Accuracy: 0.6822, Loss: 0.5261\n", + "Epoch 2 Batch 850/1077 - Train Accuracy: 0.6462, Validation Accuracy: 0.6854, Loss: 0.5702\n", + "Epoch 2 Batch 860/1077 - Train Accuracy: 0.6644, Validation Accuracy: 0.6790, Loss: 0.5391\n", + "Epoch 2 Batch 870/1077 - Train Accuracy: 0.6361, Validation Accuracy: 0.6800, Loss: 0.5783\n", + "Epoch 2 Batch 880/1077 - Train Accuracy: 0.7227, Validation Accuracy: 0.6857, Loss: 0.5084\n", + "Epoch 2 Batch 890/1077 - Train Accuracy: 0.7292, Validation Accuracy: 0.6868, Loss: 0.5106\n", + "Epoch 2 Batch 900/1077 - Train Accuracy: 0.6805, Validation Accuracy: 0.6850, Loss: 0.5479\n", + "Epoch 2 Batch 910/1077 - Train Accuracy: 0.6533, Validation Accuracy: 0.6832, Loss: 0.5296\n", + "Epoch 2 Batch 920/1077 - Train Accuracy: 0.6789, Validation Accuracy: 0.6783, Loss: 0.5519\n", + "Epoch 2 Batch 930/1077 - Train Accuracy: 0.6844, Validation Accuracy: 0.6744, Loss: 0.5244\n", + "Epoch 2 Batch 940/1077 - Train Accuracy: 0.6551, Validation Accuracy: 0.6839, Loss: 0.5318\n", + "Epoch 2 Batch 950/1077 - Train Accuracy: 0.6440, Validation Accuracy: 0.6850, Loss: 0.5118\n", + "Epoch 2 Batch 960/1077 - Train Accuracy: 0.6990, Validation Accuracy: 0.6886, Loss: 0.5013\n", + "Epoch 2 Batch 970/1077 - Train Accuracy: 0.6816, Validation Accuracy: 0.6861, Loss: 0.5589\n", + "Epoch 2 Batch 980/1077 - Train Accuracy: 0.6828, Validation Accuracy: 0.6843, Loss: 0.5212\n", + "Epoch 2 Batch 990/1077 - Train Accuracy: 0.6772, Validation Accuracy: 0.6815, Loss: 0.5464\n", + "Epoch 2 Batch 1000/1077 - Train Accuracy: 0.6994, Validation Accuracy: 0.6868, Loss: 0.4816\n", + "Epoch 2 Batch 1010/1077 - Train Accuracy: 0.6793, Validation Accuracy: 0.6871, Loss: 0.5466\n", + "Epoch 2 Batch 1020/1077 - Train Accuracy: 0.6926, Validation Accuracy: 0.6797, Loss: 0.4869\n", + "Epoch 2 Batch 1030/1077 - Train Accuracy: 0.6699, Validation Accuracy: 0.6847, Loss: 0.5303\n", + "Epoch 2 Batch 1040/1077 - Train Accuracy: 0.6678, Validation Accuracy: 0.6857, Loss: 0.5584\n", + "Epoch 2 Batch 1050/1077 - Train Accuracy: 0.6113, Validation Accuracy: 0.6879, Loss: 0.5344\n", + "Epoch 2 Batch 1060/1077 - Train Accuracy: 0.6824, Validation Accuracy: 0.6836, Loss: 0.5030\n", + "Epoch 2 Batch 1070/1077 - Train Accuracy: 0.6332, Validation Accuracy: 0.6879, Loss: 0.5497\n", + "Epoch 3 Batch 10/1077 - Train Accuracy: 0.6694, Validation Accuracy: 0.6900, Loss: 0.5489\n", + "Epoch 3 Batch 20/1077 - Train Accuracy: 0.6637, Validation Accuracy: 0.6953, Loss: 0.4860\n", + "Epoch 3 Batch 30/1077 - Train Accuracy: 0.6781, Validation Accuracy: 0.6957, Loss: 0.4922\n", + "Epoch 3 Batch 40/1077 - Train Accuracy: 0.6879, Validation Accuracy: 0.6928, Loss: 0.5025\n", + "Epoch 3 Batch 50/1077 - Train Accuracy: 0.6551, Validation Accuracy: 0.6925, Loss: 0.5074\n", + "Epoch 3 Batch 60/1077 - Train Accuracy: 0.6648, Validation Accuracy: 0.6879, Loss: 0.4969\n", + "Epoch 3 Batch 70/1077 - Train Accuracy: 0.6768, Validation Accuracy: 0.6914, Loss: 0.5215\n", + "Epoch 3 Batch 80/1077 - Train Accuracy: 0.6871, Validation Accuracy: 0.6914, Loss: 0.5023\n", + "Epoch 3 Batch 90/1077 - Train Accuracy: 0.6551, Validation Accuracy: 0.6925, Loss: 0.5421\n", + "Epoch 3 Batch 100/1077 - Train Accuracy: 0.6879, Validation Accuracy: 0.6957, Loss: 0.5095\n", + "Epoch 3 Batch 110/1077 - Train Accuracy: 0.7105, Validation Accuracy: 0.6946, Loss: 0.4791\n", + "Epoch 3 Batch 120/1077 - Train Accuracy: 0.6945, Validation Accuracy: 0.6893, Loss: 0.5263\n", + "Epoch 3 Batch 130/1077 - Train Accuracy: 0.6615, Validation Accuracy: 0.6907, Loss: 0.4713\n", + "Epoch 3 Batch 140/1077 - Train Accuracy: 0.6731, Validation Accuracy: 0.6871, Loss: 0.4991\n", + "Epoch 3 Batch 150/1077 - Train Accuracy: 0.6990, Validation Accuracy: 0.6921, Loss: 0.4679\n", + "Epoch 3 Batch 160/1077 - Train Accuracy: 0.6992, Validation Accuracy: 0.6967, Loss: 0.4845\n", + "Epoch 3 Batch 170/1077 - Train Accuracy: 0.6391, Validation Accuracy: 0.6964, Loss: 0.5325\n", + "Epoch 3 Batch 180/1077 - Train Accuracy: 0.6840, Validation Accuracy: 0.6953, Loss: 0.5022\n", + "Epoch 3 Batch 190/1077 - Train Accuracy: 0.7168, Validation Accuracy: 0.6907, Loss: 0.4874\n", + "Epoch 3 Batch 200/1077 - Train Accuracy: 0.6629, Validation Accuracy: 0.6967, Loss: 0.5079\n", + "Epoch 3 Batch 210/1077 - Train Accuracy: 0.6994, Validation Accuracy: 0.7042, Loss: 0.4900\n", + "Epoch 3 Batch 220/1077 - Train Accuracy: 0.6846, Validation Accuracy: 0.6907, Loss: 0.5008\n", + "Epoch 3 Batch 230/1077 - Train Accuracy: 0.7124, Validation Accuracy: 0.6879, Loss: 0.4787\n", + "Epoch 3 Batch 240/1077 - Train Accuracy: 0.7180, Validation Accuracy: 0.6879, Loss: 0.4700\n", + "Epoch 3 Batch 250/1077 - Train Accuracy: 0.6651, Validation Accuracy: 0.6918, Loss: 0.4671\n", + "Epoch 3 Batch 260/1077 - Train Accuracy: 0.7065, Validation Accuracy: 0.6932, Loss: 0.4676\n", + "Epoch 3 Batch 270/1077 - Train Accuracy: 0.6594, Validation Accuracy: 0.6857, Loss: 0.5070\n", + "Epoch 3 Batch 280/1077 - Train Accuracy: 0.7098, Validation Accuracy: 0.6911, Loss: 0.5017\n", + "Epoch 3 Batch 290/1077 - Train Accuracy: 0.6754, Validation Accuracy: 0.6957, Loss: 0.5175\n", + "Epoch 3 Batch 300/1077 - Train Accuracy: 0.6665, Validation Accuracy: 0.6971, Loss: 0.5092\n", + "Epoch 3 Batch 310/1077 - Train Accuracy: 0.6805, Validation Accuracy: 0.6939, Loss: 0.5053\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 3 Batch 320/1077 - Train Accuracy: 0.7027, Validation Accuracy: 0.6967, Loss: 0.4980\n", + "Epoch 3 Batch 330/1077 - Train Accuracy: 0.7164, Validation Accuracy: 0.7003, Loss: 0.4836\n", + "Epoch 3 Batch 340/1077 - Train Accuracy: 0.6546, Validation Accuracy: 0.6974, Loss: 0.4959\n", + "Epoch 3 Batch 350/1077 - Train Accuracy: 0.6684, Validation Accuracy: 0.6928, Loss: 0.4912\n", + "Epoch 3 Batch 360/1077 - Train Accuracy: 0.6840, Validation Accuracy: 0.6942, Loss: 0.4989\n", + "Epoch 3 Batch 370/1077 - Train Accuracy: 0.6842, Validation Accuracy: 0.6953, Loss: 0.4690\n", + "Epoch 3 Batch 380/1077 - Train Accuracy: 0.6797, Validation Accuracy: 0.6960, Loss: 0.4569\n", + "Epoch 3 Batch 390/1077 - Train Accuracy: 0.6637, Validation Accuracy: 0.6989, Loss: 0.4978\n", + "Epoch 3 Batch 400/1077 - Train Accuracy: 0.6879, Validation Accuracy: 0.7003, Loss: 0.4820\n", + "Epoch 3 Batch 410/1077 - Train Accuracy: 0.6645, Validation Accuracy: 0.6978, Loss: 0.4945\n", + "Epoch 3 Batch 420/1077 - Train Accuracy: 0.6918, Validation Accuracy: 0.6960, Loss: 0.4651\n", + "Epoch 3 Batch 430/1077 - Train Accuracy: 0.6758, Validation Accuracy: 0.6985, Loss: 0.4717\n", + "Epoch 3 Batch 440/1077 - Train Accuracy: 0.6844, Validation Accuracy: 0.7013, Loss: 0.4886\n", + "Epoch 3 Batch 450/1077 - Train Accuracy: 0.6883, Validation Accuracy: 0.6982, Loss: 0.4662\n", + "Epoch 3 Batch 460/1077 - Train Accuracy: 0.6918, Validation Accuracy: 0.7060, Loss: 0.4938\n", + "Epoch 3 Batch 470/1077 - Train Accuracy: 0.6488, Validation Accuracy: 0.7056, Loss: 0.4880\n", + "Epoch 3 Batch 480/1077 - Train Accuracy: 0.7229, Validation Accuracy: 0.7067, Loss: 0.4797\n", + "Epoch 3 Batch 490/1077 - Train Accuracy: 0.6957, Validation Accuracy: 0.7045, Loss: 0.4873\n", + "Epoch 3 Batch 500/1077 - Train Accuracy: 0.6996, Validation Accuracy: 0.7006, Loss: 0.4613\n", + "Epoch 3 Batch 510/1077 - Train Accuracy: 0.6883, Validation Accuracy: 0.6960, Loss: 0.4585\n", + "Epoch 3 Batch 520/1077 - Train Accuracy: 0.7236, Validation Accuracy: 0.7010, Loss: 0.4435\n", + "Epoch 3 Batch 530/1077 - Train Accuracy: 0.6770, Validation Accuracy: 0.7081, Loss: 0.4809\n", + "Epoch 3 Batch 540/1077 - Train Accuracy: 0.7031, Validation Accuracy: 0.7021, Loss: 0.4581\n", + "Epoch 3 Batch 550/1077 - Train Accuracy: 0.6750, Validation Accuracy: 0.7028, Loss: 0.4825\n", + "Epoch 3 Batch 560/1077 - Train Accuracy: 0.7008, Validation Accuracy: 0.6992, Loss: 0.4444\n", + "Epoch 3 Batch 570/1077 - Train Accuracy: 0.6920, Validation Accuracy: 0.7042, Loss: 0.4875\n", + "Epoch 3 Batch 580/1077 - Train Accuracy: 0.7318, Validation Accuracy: 0.6996, Loss: 0.4333\n", + "Epoch 3 Batch 590/1077 - Train Accuracy: 0.6686, Validation Accuracy: 0.6992, Loss: 0.4811\n", + "Epoch 3 Batch 600/1077 - Train Accuracy: 0.7303, Validation Accuracy: 0.7035, Loss: 0.4546\n", + "Epoch 3 Batch 610/1077 - Train Accuracy: 0.7052, Validation Accuracy: 0.7021, Loss: 0.4913\n", + "Epoch 3 Batch 620/1077 - Train Accuracy: 0.6926, Validation Accuracy: 0.7067, Loss: 0.4550\n", + "Epoch 3 Batch 630/1077 - Train Accuracy: 0.7152, Validation Accuracy: 0.7049, Loss: 0.4589\n", + "Epoch 3 Batch 640/1077 - Train Accuracy: 0.6704, Validation Accuracy: 0.6992, Loss: 0.4448\n", + "Epoch 3 Batch 650/1077 - Train Accuracy: 0.6945, Validation Accuracy: 0.7067, Loss: 0.4800\n", + "Epoch 3 Batch 660/1077 - Train Accuracy: 0.6930, Validation Accuracy: 0.7063, Loss: 0.4744\n", + "Epoch 3 Batch 670/1077 - Train Accuracy: 0.7273, Validation Accuracy: 0.7081, Loss: 0.4477\n", + "Epoch 3 Batch 680/1077 - Train Accuracy: 0.6968, Validation Accuracy: 0.7017, Loss: 0.4419\n", + "Epoch 3 Batch 690/1077 - Train Accuracy: 0.7316, Validation Accuracy: 0.6999, Loss: 0.4472\n", + "Epoch 3 Batch 700/1077 - Train Accuracy: 0.6793, Validation Accuracy: 0.7031, Loss: 0.4491\n", + "Epoch 3 Batch 710/1077 - Train Accuracy: 0.6527, Validation Accuracy: 0.7017, Loss: 0.4527\n", + "Epoch 3 Batch 720/1077 - Train Accuracy: 0.6817, Validation Accuracy: 0.7077, Loss: 0.4775\n", + "Epoch 3 Batch 730/1077 - Train Accuracy: 0.6898, Validation Accuracy: 0.7038, Loss: 0.4726\n", + "Epoch 3 Batch 740/1077 - Train Accuracy: 0.7066, Validation Accuracy: 0.7003, Loss: 0.4498\n", + "Epoch 3 Batch 750/1077 - Train Accuracy: 0.7105, Validation Accuracy: 0.7092, Loss: 0.4527\n", + "Epoch 3 Batch 760/1077 - Train Accuracy: 0.7039, Validation Accuracy: 0.7028, Loss: 0.4494\n", + "Epoch 3 Batch 770/1077 - Train Accuracy: 0.7202, Validation Accuracy: 0.7038, Loss: 0.4205\n", + "Epoch 3 Batch 780/1077 - Train Accuracy: 0.7145, Validation Accuracy: 0.7053, Loss: 0.4737\n", + "Epoch 3 Batch 790/1077 - Train Accuracy: 0.6496, Validation Accuracy: 0.7042, Loss: 0.4918\n", + "Epoch 3 Batch 800/1077 - Train Accuracy: 0.6703, Validation Accuracy: 0.7060, Loss: 0.4572\n", + "Epoch 3 Batch 810/1077 - Train Accuracy: 0.6983, Validation Accuracy: 0.7031, Loss: 0.4161\n", + "Epoch 3 Batch 820/1077 - Train Accuracy: 0.6855, Validation Accuracy: 0.6992, Loss: 0.4723\n", + "Epoch 3 Batch 830/1077 - Train Accuracy: 0.6945, Validation Accuracy: 0.7049, Loss: 0.4462\n", + "Epoch 3 Batch 840/1077 - Train Accuracy: 0.7164, Validation Accuracy: 0.7013, Loss: 0.4374\n", + "Epoch 3 Batch 850/1077 - Train Accuracy: 0.6856, Validation Accuracy: 0.7095, Loss: 0.4641\n", + "Epoch 3 Batch 860/1077 - Train Accuracy: 0.7154, Validation Accuracy: 0.7031, Loss: 0.4394\n", + "Epoch 3 Batch 870/1077 - Train Accuracy: 0.6583, Validation Accuracy: 0.7021, Loss: 0.4721\n", + "Epoch 3 Batch 880/1077 - Train Accuracy: 0.7578, Validation Accuracy: 0.7067, Loss: 0.4295\n", + "Epoch 3 Batch 890/1077 - Train Accuracy: 0.7831, Validation Accuracy: 0.7077, Loss: 0.4001\n", + "Epoch 3 Batch 900/1077 - Train Accuracy: 0.7094, Validation Accuracy: 0.7077, Loss: 0.4629\n", + "Epoch 3 Batch 910/1077 - Train Accuracy: 0.6864, Validation Accuracy: 0.7074, Loss: 0.4215\n", + "Epoch 3 Batch 920/1077 - Train Accuracy: 0.7102, Validation Accuracy: 0.7067, Loss: 0.4481\n", + "Epoch 3 Batch 930/1077 - Train Accuracy: 0.7129, Validation Accuracy: 0.7067, Loss: 0.4210\n", + "Epoch 3 Batch 940/1077 - Train Accuracy: 0.6887, Validation Accuracy: 0.7067, Loss: 0.4310\n", + "Epoch 3 Batch 950/1077 - Train Accuracy: 0.6830, Validation Accuracy: 0.7049, Loss: 0.4056\n", + "Epoch 3 Batch 960/1077 - Train Accuracy: 0.7273, Validation Accuracy: 0.7045, Loss: 0.4201\n", + "Epoch 3 Batch 970/1077 - Train Accuracy: 0.7246, Validation Accuracy: 0.7081, Loss: 0.4581\n", + "Epoch 3 Batch 980/1077 - Train Accuracy: 0.7270, Validation Accuracy: 0.7077, Loss: 0.4350\n", + "Epoch 3 Batch 990/1077 - Train Accuracy: 0.6982, Validation Accuracy: 0.7134, Loss: 0.4524\n", + "Epoch 3 Batch 1000/1077 - Train Accuracy: 0.7444, Validation Accuracy: 0.7081, Loss: 0.4119\n", + "Epoch 3 Batch 1010/1077 - Train Accuracy: 0.6879, Validation Accuracy: 0.7092, Loss: 0.4427\n", + "Epoch 3 Batch 1020/1077 - Train Accuracy: 0.7059, Validation Accuracy: 0.7092, Loss: 0.4221\n", + "Epoch 3 Batch 1030/1077 - Train Accuracy: 0.6855, Validation Accuracy: 0.7156, Loss: 0.4429\n", + "Epoch 3 Batch 1040/1077 - Train Accuracy: 0.7052, Validation Accuracy: 0.7099, Loss: 0.4486\n", + "Epoch 3 Batch 1050/1077 - Train Accuracy: 0.6516, Validation Accuracy: 0.7120, Loss: 0.4320\n", + "Epoch 3 Batch 1060/1077 - Train Accuracy: 0.7152, Validation Accuracy: 0.7113, Loss: 0.4113\n", + "Epoch 3 Batch 1070/1077 - Train Accuracy: 0.6691, Validation Accuracy: 0.7095, Loss: 0.4564\n", + "Epoch 4 Batch 10/1077 - Train Accuracy: 0.6982, Validation Accuracy: 0.7124, Loss: 0.4444\n", + "Epoch 4 Batch 20/1077 - Train Accuracy: 0.7258, Validation Accuracy: 0.7156, Loss: 0.4078\n", + "Epoch 4 Batch 30/1077 - Train Accuracy: 0.7031, Validation Accuracy: 0.7184, Loss: 0.4171\n", + "Epoch 4 Batch 40/1077 - Train Accuracy: 0.7250, Validation Accuracy: 0.7170, Loss: 0.4290\n", + "Epoch 4 Batch 50/1077 - Train Accuracy: 0.7094, Validation Accuracy: 0.7205, Loss: 0.4252\n", + "Epoch 4 Batch 60/1077 - Train Accuracy: 0.7028, Validation Accuracy: 0.7131, Loss: 0.4088\n", + "Epoch 4 Batch 70/1077 - Train Accuracy: 0.7089, Validation Accuracy: 0.7074, Loss: 0.4371\n", + "Epoch 4 Batch 80/1077 - Train Accuracy: 0.7152, Validation Accuracy: 0.7099, Loss: 0.4208\n", + "Epoch 4 Batch 90/1077 - Train Accuracy: 0.6887, Validation Accuracy: 0.7180, Loss: 0.4433\n", + "Epoch 4 Batch 100/1077 - Train Accuracy: 0.7043, Validation Accuracy: 0.7102, Loss: 0.4340\n", + "Epoch 4 Batch 110/1077 - Train Accuracy: 0.7395, Validation Accuracy: 0.7120, Loss: 0.3831\n", + "Epoch 4 Batch 120/1077 - Train Accuracy: 0.7328, Validation Accuracy: 0.7170, Loss: 0.4378\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 4 Batch 130/1077 - Train Accuracy: 0.7057, Validation Accuracy: 0.7152, Loss: 0.4009\n", + "Epoch 4 Batch 140/1077 - Train Accuracy: 0.6994, Validation Accuracy: 0.7024, Loss: 0.4411\n", + "Epoch 4 Batch 150/1077 - Train Accuracy: 0.7333, Validation Accuracy: 0.7113, Loss: 0.3941\n", + "Epoch 4 Batch 160/1077 - Train Accuracy: 0.7352, Validation Accuracy: 0.7205, Loss: 0.4088\n", + "Epoch 4 Batch 170/1077 - Train Accuracy: 0.6574, Validation Accuracy: 0.7127, Loss: 0.4415\n", + "Epoch 4 Batch 180/1077 - Train Accuracy: 0.7273, Validation Accuracy: 0.7152, Loss: 0.4060\n", + "Epoch 4 Batch 190/1077 - Train Accuracy: 0.7371, Validation Accuracy: 0.7148, Loss: 0.4034\n", + "Epoch 4 Batch 200/1077 - Train Accuracy: 0.7086, Validation Accuracy: 0.7188, Loss: 0.4332\n", + "Epoch 4 Batch 210/1077 - Train Accuracy: 0.7288, Validation Accuracy: 0.7163, Loss: 0.4265\n", + "Epoch 4 Batch 220/1077 - Train Accuracy: 0.7225, Validation Accuracy: 0.7138, Loss: 0.4185\n", + "Epoch 4 Batch 230/1077 - Train Accuracy: 0.7347, Validation Accuracy: 0.7141, Loss: 0.3936\n", + "Epoch 4 Batch 240/1077 - Train Accuracy: 0.7594, Validation Accuracy: 0.7077, Loss: 0.3972\n", + "Epoch 4 Batch 250/1077 - Train Accuracy: 0.6903, Validation Accuracy: 0.7159, Loss: 0.4091\n", + "Epoch 4 Batch 260/1077 - Train Accuracy: 0.7199, Validation Accuracy: 0.7170, Loss: 0.3707\n", + "Epoch 4 Batch 270/1077 - Train Accuracy: 0.6824, Validation Accuracy: 0.7163, Loss: 0.4271\n", + "Epoch 4 Batch 280/1077 - Train Accuracy: 0.7320, Validation Accuracy: 0.7131, Loss: 0.4258\n", + "Epoch 4 Batch 290/1077 - Train Accuracy: 0.7078, Validation Accuracy: 0.7237, Loss: 0.4272\n", + "Epoch 4 Batch 300/1077 - Train Accuracy: 0.7146, Validation Accuracy: 0.7227, Loss: 0.4194\n", + "Epoch 4 Batch 310/1077 - Train Accuracy: 0.6953, Validation Accuracy: 0.7244, Loss: 0.4269\n", + "Epoch 4 Batch 320/1077 - Train Accuracy: 0.7305, Validation Accuracy: 0.7237, Loss: 0.4239\n", + "Epoch 4 Batch 330/1077 - Train Accuracy: 0.7566, Validation Accuracy: 0.7166, Loss: 0.4001\n", + "Epoch 4 Batch 340/1077 - Train Accuracy: 0.7027, Validation Accuracy: 0.7188, Loss: 0.4230\n", + "Epoch 4 Batch 350/1077 - Train Accuracy: 0.6996, Validation Accuracy: 0.7092, Loss: 0.4113\n", + "Epoch 4 Batch 360/1077 - Train Accuracy: 0.7191, Validation Accuracy: 0.7202, Loss: 0.4114\n", + "Epoch 4 Batch 370/1077 - Train Accuracy: 0.7176, Validation Accuracy: 0.7184, Loss: 0.3994\n", + "Epoch 4 Batch 380/1077 - Train Accuracy: 0.7141, Validation Accuracy: 0.7156, Loss: 0.3933\n", + "Epoch 4 Batch 390/1077 - Train Accuracy: 0.6891, Validation Accuracy: 0.7248, Loss: 0.4169\n", + "Epoch 4 Batch 400/1077 - Train Accuracy: 0.7312, Validation Accuracy: 0.7259, Loss: 0.4060\n", + "Epoch 4 Batch 410/1077 - Train Accuracy: 0.6982, Validation Accuracy: 0.7255, Loss: 0.4199\n", + "Epoch 4 Batch 420/1077 - Train Accuracy: 0.7227, Validation Accuracy: 0.7173, Loss: 0.3779\n", + "Epoch 4 Batch 430/1077 - Train Accuracy: 0.6867, Validation Accuracy: 0.7177, Loss: 0.3999\n", + "Epoch 4 Batch 440/1077 - Train Accuracy: 0.7035, Validation Accuracy: 0.7219, Loss: 0.4273\n", + "Epoch 4 Batch 450/1077 - Train Accuracy: 0.7055, Validation Accuracy: 0.7180, Loss: 0.4088\n", + "Epoch 4 Batch 460/1077 - Train Accuracy: 0.7238, Validation Accuracy: 0.7173, Loss: 0.4170\n", + "Epoch 4 Batch 470/1077 - Train Accuracy: 0.7085, Validation Accuracy: 0.7205, Loss: 0.4144\n", + "Epoch 4 Batch 480/1077 - Train Accuracy: 0.7541, Validation Accuracy: 0.7173, Loss: 0.4025\n", + "Epoch 4 Batch 490/1077 - Train Accuracy: 0.7066, Validation Accuracy: 0.7159, Loss: 0.4196\n", + "Epoch 4 Batch 500/1077 - Train Accuracy: 0.7234, Validation Accuracy: 0.7202, Loss: 0.3770\n", + "Epoch 4 Batch 510/1077 - Train Accuracy: 0.7223, Validation Accuracy: 0.7244, Loss: 0.3913\n", + "Epoch 4 Batch 520/1077 - Train Accuracy: 0.7675, Validation Accuracy: 0.7340, Loss: 0.3696\n", + "Epoch 4 Batch 530/1077 - Train Accuracy: 0.7102, Validation Accuracy: 0.7301, Loss: 0.4068\n", + "Epoch 4 Batch 540/1077 - Train Accuracy: 0.7344, Validation Accuracy: 0.7276, Loss: 0.3666\n", + "Epoch 4 Batch 550/1077 - Train Accuracy: 0.7152, Validation Accuracy: 0.7326, Loss: 0.4128\n", + "Epoch 4 Batch 560/1077 - Train Accuracy: 0.7316, Validation Accuracy: 0.7212, Loss: 0.3703\n", + "Epoch 4 Batch 570/1077 - Train Accuracy: 0.7048, Validation Accuracy: 0.7188, Loss: 0.4120\n", + "Epoch 4 Batch 580/1077 - Train Accuracy: 0.7753, Validation Accuracy: 0.7290, Loss: 0.3774\n", + "Epoch 4 Batch 590/1077 - Train Accuracy: 0.7015, Validation Accuracy: 0.7223, Loss: 0.4087\n", + "Epoch 4 Batch 600/1077 - Train Accuracy: 0.7515, Validation Accuracy: 0.7287, Loss: 0.3824\n", + "Epoch 4 Batch 610/1077 - Train Accuracy: 0.7290, Validation Accuracy: 0.7237, Loss: 0.4193\n", + "Epoch 4 Batch 620/1077 - Train Accuracy: 0.7129, Validation Accuracy: 0.7241, Loss: 0.3849\n", + "Epoch 4 Batch 630/1077 - Train Accuracy: 0.7609, Validation Accuracy: 0.7354, Loss: 0.3944\n", + "Epoch 4 Batch 640/1077 - Train Accuracy: 0.7188, Validation Accuracy: 0.7315, Loss: 0.3909\n", + "Epoch 4 Batch 650/1077 - Train Accuracy: 0.7164, Validation Accuracy: 0.7287, Loss: 0.3943\n", + "Epoch 4 Batch 660/1077 - Train Accuracy: 0.7191, Validation Accuracy: 0.7269, Loss: 0.4038\n", + "Epoch 4 Batch 670/1077 - Train Accuracy: 0.7468, Validation Accuracy: 0.7330, Loss: 0.3753\n", + "Epoch 4 Batch 680/1077 - Train Accuracy: 0.7176, Validation Accuracy: 0.7035, Loss: 0.4064\n", + "Epoch 4 Batch 690/1077 - Train Accuracy: 0.7410, Validation Accuracy: 0.7319, Loss: 0.4047\n", + "Epoch 4 Batch 700/1077 - Train Accuracy: 0.7102, Validation Accuracy: 0.7131, Loss: 0.3876\n", + "Epoch 4 Batch 710/1077 - Train Accuracy: 0.6762, Validation Accuracy: 0.7166, Loss: 0.4178\n", + "Epoch 4 Batch 720/1077 - Train Accuracy: 0.7286, Validation Accuracy: 0.7173, Loss: 0.4202\n", + "Epoch 4 Batch 730/1077 - Train Accuracy: 0.6992, Validation Accuracy: 0.7223, Loss: 0.4053\n", + "Epoch 4 Batch 740/1077 - Train Accuracy: 0.7285, Validation Accuracy: 0.7188, Loss: 0.3816\n", + "Epoch 4 Batch 750/1077 - Train Accuracy: 0.7492, Validation Accuracy: 0.7248, Loss: 0.3781\n", + "Epoch 4 Batch 760/1077 - Train Accuracy: 0.7480, Validation Accuracy: 0.7333, Loss: 0.3964\n", + "Epoch 4 Batch 770/1077 - Train Accuracy: 0.7429, Validation Accuracy: 0.7322, Loss: 0.3637\n", + "Epoch 4 Batch 780/1077 - Train Accuracy: 0.7285, Validation Accuracy: 0.7134, Loss: 0.4083\n", + "Epoch 4 Batch 790/1077 - Train Accuracy: 0.6703, Validation Accuracy: 0.7138, Loss: 0.4123\n", + "Epoch 4 Batch 800/1077 - Train Accuracy: 0.7176, Validation Accuracy: 0.7340, Loss: 0.3768\n", + "Epoch 4 Batch 810/1077 - Train Accuracy: 0.7191, Validation Accuracy: 0.7376, Loss: 0.3585\n", + "Epoch 4 Batch 820/1077 - Train Accuracy: 0.7105, Validation Accuracy: 0.7418, Loss: 0.4121\n", + "Epoch 4 Batch 830/1077 - Train Accuracy: 0.7176, Validation Accuracy: 0.7358, Loss: 0.3746\n", + "Epoch 4 Batch 840/1077 - Train Accuracy: 0.7281, Validation Accuracy: 0.7347, Loss: 0.3688\n", + "Epoch 4 Batch 850/1077 - Train Accuracy: 0.7109, Validation Accuracy: 0.7301, Loss: 0.4002\n", + "Epoch 4 Batch 860/1077 - Train Accuracy: 0.7355, Validation Accuracy: 0.7415, Loss: 0.3942\n", + "Epoch 4 Batch 870/1077 - Train Accuracy: 0.7007, Validation Accuracy: 0.7148, Loss: 0.4137\n", + "Epoch 4 Batch 880/1077 - Train Accuracy: 0.7695, Validation Accuracy: 0.7369, Loss: 0.3590\n", + "Epoch 4 Batch 890/1077 - Train Accuracy: 0.7891, Validation Accuracy: 0.7354, Loss: 0.3495\n", + "Epoch 4 Batch 900/1077 - Train Accuracy: 0.7375, Validation Accuracy: 0.7308, Loss: 0.3926\n", + "Epoch 4 Batch 910/1077 - Train Accuracy: 0.7143, Validation Accuracy: 0.7362, Loss: 0.3677\n", + "Epoch 4 Batch 920/1077 - Train Accuracy: 0.7367, Validation Accuracy: 0.7422, Loss: 0.3858\n", + "Epoch 4 Batch 930/1077 - Train Accuracy: 0.7352, Validation Accuracy: 0.7397, Loss: 0.3717\n", + "Epoch 4 Batch 940/1077 - Train Accuracy: 0.7230, Validation Accuracy: 0.7354, Loss: 0.3739\n", + "Epoch 4 Batch 950/1077 - Train Accuracy: 0.7355, Validation Accuracy: 0.7337, Loss: 0.3460\n", + "Epoch 4 Batch 960/1077 - Train Accuracy: 0.7295, Validation Accuracy: 0.7404, Loss: 0.3695\n", + "Epoch 4 Batch 970/1077 - Train Accuracy: 0.7516, Validation Accuracy: 0.7422, Loss: 0.3959\n", + "Epoch 4 Batch 980/1077 - Train Accuracy: 0.7449, Validation Accuracy: 0.7415, Loss: 0.3719\n", + "Epoch 4 Batch 990/1077 - Train Accuracy: 0.7179, Validation Accuracy: 0.7411, Loss: 0.3868\n", + "Epoch 4 Batch 1000/1077 - Train Accuracy: 0.7798, Validation Accuracy: 0.7479, Loss: 0.3439\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 4 Batch 1010/1077 - Train Accuracy: 0.7102, Validation Accuracy: 0.7390, Loss: 0.3838\n", + "Epoch 4 Batch 1020/1077 - Train Accuracy: 0.7289, Validation Accuracy: 0.7390, Loss: 0.3580\n", + "Epoch 4 Batch 1030/1077 - Train Accuracy: 0.7055, Validation Accuracy: 0.7454, Loss: 0.3815\n", + "Epoch 4 Batch 1040/1077 - Train Accuracy: 0.7512, Validation Accuracy: 0.7418, Loss: 0.3794\n", + "Epoch 4 Batch 1050/1077 - Train Accuracy: 0.7047, Validation Accuracy: 0.7315, Loss: 0.3732\n", + "Epoch 4 Batch 1060/1077 - Train Accuracy: 0.7422, Validation Accuracy: 0.7393, Loss: 0.3528\n", + "Epoch 4 Batch 1070/1077 - Train Accuracy: 0.6973, Validation Accuracy: 0.7447, Loss: 0.3826\n", + "Epoch 5 Batch 10/1077 - Train Accuracy: 0.7282, Validation Accuracy: 0.7447, Loss: 0.3855\n", + "Epoch 5 Batch 20/1077 - Train Accuracy: 0.7301, Validation Accuracy: 0.7464, Loss: 0.3576\n", + "Epoch 5 Batch 30/1077 - Train Accuracy: 0.7438, Validation Accuracy: 0.7418, Loss: 0.3560\n", + "Epoch 5 Batch 40/1077 - Train Accuracy: 0.7441, Validation Accuracy: 0.7433, Loss: 0.3773\n", + "Epoch 5 Batch 50/1077 - Train Accuracy: 0.7328, Validation Accuracy: 0.7447, Loss: 0.3647\n", + "Epoch 5 Batch 60/1077 - Train Accuracy: 0.7165, Validation Accuracy: 0.7347, Loss: 0.3531\n", + "Epoch 5 Batch 70/1077 - Train Accuracy: 0.7455, Validation Accuracy: 0.7290, Loss: 0.3709\n", + "Epoch 5 Batch 80/1077 - Train Accuracy: 0.7359, Validation Accuracy: 0.7397, Loss: 0.3658\n", + "Epoch 5 Batch 90/1077 - Train Accuracy: 0.7184, Validation Accuracy: 0.7525, Loss: 0.3916\n", + "Epoch 5 Batch 100/1077 - Train Accuracy: 0.7363, Validation Accuracy: 0.7546, Loss: 0.3623\n", + "Epoch 5 Batch 110/1077 - Train Accuracy: 0.7715, Validation Accuracy: 0.7479, Loss: 0.3418\n", + "Epoch 5 Batch 120/1077 - Train Accuracy: 0.7449, Validation Accuracy: 0.7294, Loss: 0.3821\n", + "Epoch 5 Batch 130/1077 - Train Accuracy: 0.7325, Validation Accuracy: 0.7195, Loss: 0.3292\n", + "Epoch 5 Batch 140/1077 - Train Accuracy: 0.7163, Validation Accuracy: 0.7333, Loss: 0.3801\n", + "Epoch 5 Batch 150/1077 - Train Accuracy: 0.7552, Validation Accuracy: 0.7415, Loss: 0.3498\n", + "Epoch 5 Batch 160/1077 - Train Accuracy: 0.7547, Validation Accuracy: 0.7425, Loss: 0.3617\n", + "Epoch 5 Batch 170/1077 - Train Accuracy: 0.7125, Validation Accuracy: 0.7514, Loss: 0.3720\n", + "Epoch 5 Batch 180/1077 - Train Accuracy: 0.7465, Validation Accuracy: 0.7429, Loss: 0.3568\n", + "Epoch 5 Batch 190/1077 - Train Accuracy: 0.7707, Validation Accuracy: 0.7507, Loss: 0.3536\n", + "Epoch 5 Batch 200/1077 - Train Accuracy: 0.7242, Validation Accuracy: 0.7571, Loss: 0.3808\n", + "Epoch 5 Batch 210/1077 - Train Accuracy: 0.7418, Validation Accuracy: 0.7397, Loss: 0.3629\n", + "Epoch 5 Batch 220/1077 - Train Accuracy: 0.7500, Validation Accuracy: 0.7418, Loss: 0.3606\n", + "Epoch 5 Batch 230/1077 - Train Accuracy: 0.7556, Validation Accuracy: 0.7418, Loss: 0.3416\n", + "Epoch 5 Batch 240/1077 - Train Accuracy: 0.7785, Validation Accuracy: 0.7486, Loss: 0.3406\n", + "Epoch 5 Batch 250/1077 - Train Accuracy: 0.7212, Validation Accuracy: 0.7369, Loss: 0.3485\n", + "Epoch 5 Batch 260/1077 - Train Accuracy: 0.7414, Validation Accuracy: 0.7386, Loss: 0.3286\n", + "Epoch 5 Batch 270/1077 - Train Accuracy: 0.7023, Validation Accuracy: 0.7454, Loss: 0.3769\n", + "Epoch 5 Batch 280/1077 - Train Accuracy: 0.7672, Validation Accuracy: 0.7305, Loss: 0.3770\n", + "Epoch 5 Batch 290/1077 - Train Accuracy: 0.7320, Validation Accuracy: 0.7365, Loss: 0.3820\n", + "Epoch 5 Batch 300/1077 - Train Accuracy: 0.7303, Validation Accuracy: 0.7390, Loss: 0.3640\n", + "Epoch 5 Batch 310/1077 - Train Accuracy: 0.7121, Validation Accuracy: 0.7539, Loss: 0.3688\n", + "Epoch 5 Batch 320/1077 - Train Accuracy: 0.7746, Validation Accuracy: 0.7518, Loss: 0.3570\n", + "Epoch 5 Batch 330/1077 - Train Accuracy: 0.7824, Validation Accuracy: 0.7298, Loss: 0.3535\n", + "Epoch 5 Batch 340/1077 - Train Accuracy: 0.7253, Validation Accuracy: 0.7543, Loss: 0.3874\n", + "Epoch 5 Batch 350/1077 - Train Accuracy: 0.7238, Validation Accuracy: 0.7543, Loss: 0.3684\n", + "Epoch 5 Batch 360/1077 - Train Accuracy: 0.7441, Validation Accuracy: 0.7479, Loss: 0.3603\n", + "Epoch 5 Batch 370/1077 - Train Accuracy: 0.7433, Validation Accuracy: 0.7504, Loss: 0.3508\n", + "Epoch 5 Batch 380/1077 - Train Accuracy: 0.7301, Validation Accuracy: 0.7578, Loss: 0.3434\n", + "Epoch 5 Batch 390/1077 - Train Accuracy: 0.7152, Validation Accuracy: 0.7500, Loss: 0.3805\n", + "Epoch 5 Batch 400/1077 - Train Accuracy: 0.7797, Validation Accuracy: 0.7592, Loss: 0.3505\n", + "Epoch 5 Batch 410/1077 - Train Accuracy: 0.7278, Validation Accuracy: 0.7628, Loss: 0.3553\n", + "Epoch 5 Batch 420/1077 - Train Accuracy: 0.7613, Validation Accuracy: 0.7578, Loss: 0.3396\n", + "Epoch 5 Batch 430/1077 - Train Accuracy: 0.7254, Validation Accuracy: 0.7504, Loss: 0.3376\n", + "Epoch 5 Batch 440/1077 - Train Accuracy: 0.7105, Validation Accuracy: 0.7454, Loss: 0.3756\n", + "Epoch 5 Batch 450/1077 - Train Accuracy: 0.7391, Validation Accuracy: 0.7507, Loss: 0.3437\n", + "Epoch 5 Batch 460/1077 - Train Accuracy: 0.7516, Validation Accuracy: 0.7553, Loss: 0.3690\n", + "Epoch 5 Batch 470/1077 - Train Accuracy: 0.7331, Validation Accuracy: 0.7592, Loss: 0.3696\n", + "Epoch 5 Batch 480/1077 - Train Accuracy: 0.7718, Validation Accuracy: 0.7557, Loss: 0.3529\n", + "Epoch 5 Batch 490/1077 - Train Accuracy: 0.7230, Validation Accuracy: 0.7607, Loss: 0.3623\n", + "Epoch 5 Batch 500/1077 - Train Accuracy: 0.7621, Validation Accuracy: 0.7496, Loss: 0.3329\n", + "Epoch 5 Batch 510/1077 - Train Accuracy: 0.7520, Validation Accuracy: 0.7578, Loss: 0.3500\n", + "Epoch 5 Batch 520/1077 - Train Accuracy: 0.7779, Validation Accuracy: 0.7536, Loss: 0.3273\n", + "Epoch 5 Batch 530/1077 - Train Accuracy: 0.7332, Validation Accuracy: 0.7543, Loss: 0.3480\n", + "Epoch 5 Batch 540/1077 - Train Accuracy: 0.7484, Validation Accuracy: 0.7489, Loss: 0.3295\n", + "Epoch 5 Batch 550/1077 - Train Accuracy: 0.7266, Validation Accuracy: 0.7532, Loss: 0.3563\n", + "Epoch 5 Batch 560/1077 - Train Accuracy: 0.7621, Validation Accuracy: 0.7443, Loss: 0.3376\n", + "Epoch 5 Batch 570/1077 - Train Accuracy: 0.7463, Validation Accuracy: 0.7546, Loss: 0.3779\n", + "Epoch 5 Batch 580/1077 - Train Accuracy: 0.7999, Validation Accuracy: 0.7511, Loss: 0.3174\n", + "Epoch 5 Batch 590/1077 - Train Accuracy: 0.7200, Validation Accuracy: 0.7550, Loss: 0.3673\n", + "Epoch 5 Batch 600/1077 - Train Accuracy: 0.7712, Validation Accuracy: 0.7575, Loss: 0.3353\n", + "Epoch 5 Batch 610/1077 - Train Accuracy: 0.7562, Validation Accuracy: 0.7543, Loss: 0.3719\n", + "Epoch 5 Batch 620/1077 - Train Accuracy: 0.7445, Validation Accuracy: 0.7610, Loss: 0.3316\n", + "Epoch 5 Batch 630/1077 - Train Accuracy: 0.7727, Validation Accuracy: 0.7603, Loss: 0.3359\n", + "Epoch 5 Batch 640/1077 - Train Accuracy: 0.7277, Validation Accuracy: 0.7635, Loss: 0.3341\n", + "Epoch 5 Batch 650/1077 - Train Accuracy: 0.7391, Validation Accuracy: 0.7674, Loss: 0.3546\n", + "Epoch 5 Batch 660/1077 - Train Accuracy: 0.7402, Validation Accuracy: 0.7596, Loss: 0.3582\n", + "Epoch 5 Batch 670/1077 - Train Accuracy: 0.7564, Validation Accuracy: 0.7706, Loss: 0.3193\n", + "Epoch 5 Batch 680/1077 - Train Accuracy: 0.7593, Validation Accuracy: 0.7706, Loss: 0.3388\n", + "Epoch 5 Batch 690/1077 - Train Accuracy: 0.7719, Validation Accuracy: 0.7646, Loss: 0.3309\n", + "Epoch 5 Batch 700/1077 - Train Accuracy: 0.7551, Validation Accuracy: 0.7582, Loss: 0.3206\n", + "Epoch 5 Batch 710/1077 - Train Accuracy: 0.6895, Validation Accuracy: 0.7521, Loss: 0.3380\n", + "Epoch 5 Batch 720/1077 - Train Accuracy: 0.7615, Validation Accuracy: 0.7638, Loss: 0.3701\n", + "Epoch 5 Batch 730/1077 - Train Accuracy: 0.7125, Validation Accuracy: 0.7734, Loss: 0.3605\n", + "Epoch 5 Batch 740/1077 - Train Accuracy: 0.7488, Validation Accuracy: 0.7607, Loss: 0.3281\n", + "Epoch 5 Batch 750/1077 - Train Accuracy: 0.7543, Validation Accuracy: 0.7567, Loss: 0.3422\n", + "Epoch 5 Batch 760/1077 - Train Accuracy: 0.7852, Validation Accuracy: 0.7649, Loss: 0.3405\n", + "Epoch 5 Batch 770/1077 - Train Accuracy: 0.7686, Validation Accuracy: 0.7525, Loss: 0.3192\n", + "Epoch 5 Batch 780/1077 - Train Accuracy: 0.7477, Validation Accuracy: 0.7610, Loss: 0.3513\n", + "Epoch 5 Batch 790/1077 - Train Accuracy: 0.6887, Validation Accuracy: 0.7617, Loss: 0.3610\n", + "Epoch 5 Batch 800/1077 - Train Accuracy: 0.7289, Validation Accuracy: 0.7649, Loss: 0.3269\n", + "Epoch 5 Batch 810/1077 - Train Accuracy: 0.7414, Validation Accuracy: 0.7667, Loss: 0.3260\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 5 Batch 820/1077 - Train Accuracy: 0.7246, Validation Accuracy: 0.7607, Loss: 0.3643\n", + "Epoch 5 Batch 830/1077 - Train Accuracy: 0.7336, Validation Accuracy: 0.7514, Loss: 0.3344\n", + "Epoch 5 Batch 840/1077 - Train Accuracy: 0.7488, Validation Accuracy: 0.7617, Loss: 0.3190\n", + "Epoch 5 Batch 850/1077 - Train Accuracy: 0.7128, Validation Accuracy: 0.7642, Loss: 0.3708\n", + "Epoch 5 Batch 860/1077 - Train Accuracy: 0.7467, Validation Accuracy: 0.7667, Loss: 0.3401\n", + "Epoch 5 Batch 870/1077 - Train Accuracy: 0.7430, Validation Accuracy: 0.7564, Loss: 0.3572\n", + "Epoch 5 Batch 880/1077 - Train Accuracy: 0.7969, Validation Accuracy: 0.7614, Loss: 0.3307\n", + "Epoch 5 Batch 890/1077 - Train Accuracy: 0.8114, Validation Accuracy: 0.7607, Loss: 0.3237\n", + "Epoch 5 Batch 900/1077 - Train Accuracy: 0.7566, Validation Accuracy: 0.7578, Loss: 0.3418\n", + "Epoch 5 Batch 910/1077 - Train Accuracy: 0.7314, Validation Accuracy: 0.7631, Loss: 0.3299\n", + "Epoch 5 Batch 920/1077 - Train Accuracy: 0.7461, Validation Accuracy: 0.7649, Loss: 0.3335\n", + "Epoch 5 Batch 930/1077 - Train Accuracy: 0.7531, Validation Accuracy: 0.7713, Loss: 0.3345\n", + "Epoch 5 Batch 940/1077 - Train Accuracy: 0.7391, Validation Accuracy: 0.7610, Loss: 0.3203\n", + "Epoch 5 Batch 950/1077 - Train Accuracy: 0.7407, Validation Accuracy: 0.7710, Loss: 0.3118\n", + "Epoch 5 Batch 960/1077 - Train Accuracy: 0.7541, Validation Accuracy: 0.7571, Loss: 0.3304\n", + "Epoch 5 Batch 970/1077 - Train Accuracy: 0.7652, Validation Accuracy: 0.7532, Loss: 0.3402\n", + "Epoch 5 Batch 980/1077 - Train Accuracy: 0.7574, Validation Accuracy: 0.7635, Loss: 0.3386\n", + "Epoch 5 Batch 990/1077 - Train Accuracy: 0.7582, Validation Accuracy: 0.7678, Loss: 0.3435\n", + "Epoch 5 Batch 1000/1077 - Train Accuracy: 0.7798, Validation Accuracy: 0.7692, Loss: 0.3053\n", + "Epoch 5 Batch 1010/1077 - Train Accuracy: 0.7266, Validation Accuracy: 0.7717, Loss: 0.3341\n", + "Epoch 5 Batch 1020/1077 - Train Accuracy: 0.7480, Validation Accuracy: 0.7724, Loss: 0.3171\n", + "Epoch 5 Batch 1030/1077 - Train Accuracy: 0.7418, Validation Accuracy: 0.7692, Loss: 0.3314\n", + "Epoch 5 Batch 1040/1077 - Train Accuracy: 0.7796, Validation Accuracy: 0.7681, Loss: 0.3248\n", + "Epoch 5 Batch 1050/1077 - Train Accuracy: 0.7063, Validation Accuracy: 0.7603, Loss: 0.3448\n", + "Epoch 5 Batch 1060/1077 - Train Accuracy: 0.7688, Validation Accuracy: 0.7702, Loss: 0.2968\n", + "Epoch 5 Batch 1070/1077 - Train Accuracy: 0.7164, Validation Accuracy: 0.7710, Loss: 0.3417\n", + "Epoch 6 Batch 10/1077 - Train Accuracy: 0.7410, Validation Accuracy: 0.7575, Loss: 0.3446\n", + "Epoch 6 Batch 20/1077 - Train Accuracy: 0.7410, Validation Accuracy: 0.7681, Loss: 0.3138\n", + "Epoch 6 Batch 30/1077 - Train Accuracy: 0.7402, Validation Accuracy: 0.7589, Loss: 0.3187\n", + "Epoch 6 Batch 40/1077 - Train Accuracy: 0.7668, Validation Accuracy: 0.7617, Loss: 0.3369\n", + "Epoch 6 Batch 50/1077 - Train Accuracy: 0.7379, Validation Accuracy: 0.7607, Loss: 0.3188\n", + "Epoch 6 Batch 60/1077 - Train Accuracy: 0.7377, Validation Accuracy: 0.7518, Loss: 0.3142\n", + "Epoch 6 Batch 70/1077 - Train Accuracy: 0.7595, Validation Accuracy: 0.7447, Loss: 0.3496\n", + "Epoch 6 Batch 80/1077 - Train Accuracy: 0.7684, Validation Accuracy: 0.7518, Loss: 0.3262\n", + "Epoch 6 Batch 90/1077 - Train Accuracy: 0.7305, Validation Accuracy: 0.7553, Loss: 0.3404\n", + "Epoch 6 Batch 100/1077 - Train Accuracy: 0.7750, Validation Accuracy: 0.7670, Loss: 0.3293\n", + "Epoch 6 Batch 110/1077 - Train Accuracy: 0.7758, Validation Accuracy: 0.7681, Loss: 0.2918\n", + "Epoch 6 Batch 120/1077 - Train Accuracy: 0.7551, Validation Accuracy: 0.7571, Loss: 0.3477\n", + "Epoch 6 Batch 130/1077 - Train Accuracy: 0.7318, Validation Accuracy: 0.7436, Loss: 0.3111\n", + "Epoch 6 Batch 140/1077 - Train Accuracy: 0.7410, Validation Accuracy: 0.7553, Loss: 0.3268\n", + "Epoch 6 Batch 150/1077 - Train Accuracy: 0.7716, Validation Accuracy: 0.7610, Loss: 0.3116\n", + "Epoch 6 Batch 160/1077 - Train Accuracy: 0.7738, Validation Accuracy: 0.7585, Loss: 0.3298\n", + "Epoch 6 Batch 170/1077 - Train Accuracy: 0.7332, Validation Accuracy: 0.7557, Loss: 0.3385\n", + "Epoch 6 Batch 180/1077 - Train Accuracy: 0.7641, Validation Accuracy: 0.7667, Loss: 0.3181\n", + "Epoch 6 Batch 190/1077 - Train Accuracy: 0.7887, Validation Accuracy: 0.7631, Loss: 0.3168\n", + "Epoch 6 Batch 200/1077 - Train Accuracy: 0.7375, Validation Accuracy: 0.7553, Loss: 0.3363\n", + "Epoch 6 Batch 210/1077 - Train Accuracy: 0.7656, Validation Accuracy: 0.7610, Loss: 0.3194\n", + "Epoch 6 Batch 220/1077 - Train Accuracy: 0.7681, Validation Accuracy: 0.7571, Loss: 0.3205\n", + "Epoch 6 Batch 230/1077 - Train Accuracy: 0.7667, Validation Accuracy: 0.7717, Loss: 0.3198\n", + "Epoch 6 Batch 240/1077 - Train Accuracy: 0.7961, Validation Accuracy: 0.7649, Loss: 0.3133\n", + "Epoch 6 Batch 250/1077 - Train Accuracy: 0.7280, Validation Accuracy: 0.7699, Loss: 0.3105\n", + "Epoch 6 Batch 260/1077 - Train Accuracy: 0.7690, Validation Accuracy: 0.7607, Loss: 0.3055\n", + "Epoch 6 Batch 270/1077 - Train Accuracy: 0.7172, Validation Accuracy: 0.7610, Loss: 0.3525\n", + "Epoch 6 Batch 280/1077 - Train Accuracy: 0.7852, Validation Accuracy: 0.7571, Loss: 0.3288\n", + "Epoch 6 Batch 290/1077 - Train Accuracy: 0.7477, Validation Accuracy: 0.7717, Loss: 0.3378\n", + "Epoch 6 Batch 300/1077 - Train Accuracy: 0.7508, Validation Accuracy: 0.7756, Loss: 0.3133\n", + "Epoch 6 Batch 310/1077 - Train Accuracy: 0.7258, Validation Accuracy: 0.7713, Loss: 0.3380\n", + "Epoch 6 Batch 320/1077 - Train Accuracy: 0.7930, Validation Accuracy: 0.7702, Loss: 0.3232\n", + "Epoch 6 Batch 330/1077 - Train Accuracy: 0.7965, Validation Accuracy: 0.7738, Loss: 0.3226\n", + "Epoch 6 Batch 340/1077 - Train Accuracy: 0.7393, Validation Accuracy: 0.7685, Loss: 0.3338\n", + "Epoch 6 Batch 350/1077 - Train Accuracy: 0.7508, Validation Accuracy: 0.7642, Loss: 0.3177\n", + "Epoch 6 Batch 360/1077 - Train Accuracy: 0.7566, Validation Accuracy: 0.7550, Loss: 0.3221\n", + "Epoch 6 Batch 370/1077 - Train Accuracy: 0.7567, Validation Accuracy: 0.7631, Loss: 0.3138\n", + "Epoch 6 Batch 380/1077 - Train Accuracy: 0.7387, Validation Accuracy: 0.7607, Loss: 0.3084\n", + "Epoch 6 Batch 390/1077 - Train Accuracy: 0.7199, Validation Accuracy: 0.7685, Loss: 0.3372\n", + "Epoch 6 Batch 400/1077 - Train Accuracy: 0.7988, Validation Accuracy: 0.7731, Loss: 0.3225\n", + "Epoch 6 Batch 410/1077 - Train Accuracy: 0.7373, Validation Accuracy: 0.7727, Loss: 0.3313\n", + "Epoch 6 Batch 420/1077 - Train Accuracy: 0.7660, Validation Accuracy: 0.7628, Loss: 0.2944\n", + "Epoch 6 Batch 430/1077 - Train Accuracy: 0.7309, Validation Accuracy: 0.7628, Loss: 0.3191\n", + "Epoch 6 Batch 440/1077 - Train Accuracy: 0.7102, Validation Accuracy: 0.7638, Loss: 0.3421\n", + "Epoch 6 Batch 450/1077 - Train Accuracy: 0.7473, Validation Accuracy: 0.7678, Loss: 0.3075\n", + "Epoch 6 Batch 460/1077 - Train Accuracy: 0.7621, Validation Accuracy: 0.7553, Loss: 0.3382\n", + "Epoch 6 Batch 470/1077 - Train Accuracy: 0.7438, Validation Accuracy: 0.7550, Loss: 0.3384\n", + "Epoch 6 Batch 480/1077 - Train Accuracy: 0.7911, Validation Accuracy: 0.7500, Loss: 0.3157\n", + "Epoch 6 Batch 490/1077 - Train Accuracy: 0.7426, Validation Accuracy: 0.7546, Loss: 0.3307\n", + "Epoch 6 Batch 500/1077 - Train Accuracy: 0.7699, Validation Accuracy: 0.7638, Loss: 0.3146\n", + "Epoch 6 Batch 510/1077 - Train Accuracy: 0.7438, Validation Accuracy: 0.7543, Loss: 0.3057\n", + "Epoch 6 Batch 520/1077 - Train Accuracy: 0.7872, Validation Accuracy: 0.7670, Loss: 0.2850\n", + "Epoch 6 Batch 530/1077 - Train Accuracy: 0.7488, Validation Accuracy: 0.7564, Loss: 0.3248\n", + "Epoch 6 Batch 540/1077 - Train Accuracy: 0.7703, Validation Accuracy: 0.7681, Loss: 0.2893\n", + "Epoch 6 Batch 550/1077 - Train Accuracy: 0.7516, Validation Accuracy: 0.7681, Loss: 0.3299\n", + "Epoch 6 Batch 560/1077 - Train Accuracy: 0.7676, Validation Accuracy: 0.7589, Loss: 0.2975\n", + "Epoch 6 Batch 570/1077 - Train Accuracy: 0.7430, Validation Accuracy: 0.7539, Loss: 0.3324\n", + "Epoch 6 Batch 580/1077 - Train Accuracy: 0.8043, Validation Accuracy: 0.7631, Loss: 0.2860\n", + "Epoch 6 Batch 590/1077 - Train Accuracy: 0.7368, Validation Accuracy: 0.7610, Loss: 0.3384\n", + "Epoch 6 Batch 600/1077 - Train Accuracy: 0.7879, Validation Accuracy: 0.7624, Loss: 0.2955\n", + "Epoch 6 Batch 610/1077 - Train Accuracy: 0.7586, Validation Accuracy: 0.7674, Loss: 0.3253\n", + "Epoch 6 Batch 620/1077 - Train Accuracy: 0.7605, Validation Accuracy: 0.7660, Loss: 0.3078\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 6 Batch 630/1077 - Train Accuracy: 0.7941, Validation Accuracy: 0.7731, Loss: 0.3083\n", + "Epoch 6 Batch 640/1077 - Train Accuracy: 0.7392, Validation Accuracy: 0.7667, Loss: 0.3079\n", + "Epoch 6 Batch 650/1077 - Train Accuracy: 0.7527, Validation Accuracy: 0.7674, Loss: 0.3134\n", + "Epoch 6 Batch 660/1077 - Train Accuracy: 0.7578, Validation Accuracy: 0.7710, Loss: 0.3195\n", + "Epoch 6 Batch 670/1077 - Train Accuracy: 0.7656, Validation Accuracy: 0.7756, Loss: 0.2962\n", + "Epoch 6 Batch 680/1077 - Train Accuracy: 0.7626, Validation Accuracy: 0.7770, Loss: 0.3178\n", + "Epoch 6 Batch 690/1077 - Train Accuracy: 0.7816, Validation Accuracy: 0.7692, Loss: 0.3028\n", + "Epoch 6 Batch 700/1077 - Train Accuracy: 0.7422, Validation Accuracy: 0.7688, Loss: 0.3124\n", + "Epoch 6 Batch 710/1077 - Train Accuracy: 0.6965, Validation Accuracy: 0.7624, Loss: 0.3221\n", + "Epoch 6 Batch 720/1077 - Train Accuracy: 0.7854, Validation Accuracy: 0.7621, Loss: 0.3356\n", + "Epoch 6 Batch 730/1077 - Train Accuracy: 0.7230, Validation Accuracy: 0.7713, Loss: 0.3194\n", + "Epoch 6 Batch 740/1077 - Train Accuracy: 0.7734, Validation Accuracy: 0.7695, Loss: 0.3029\n", + "Epoch 6 Batch 750/1077 - Train Accuracy: 0.7703, Validation Accuracy: 0.7717, Loss: 0.3207\n", + "Epoch 6 Batch 760/1077 - Train Accuracy: 0.7969, Validation Accuracy: 0.7621, Loss: 0.3132\n", + "Epoch 6 Batch 770/1077 - Train Accuracy: 0.7786, Validation Accuracy: 0.7685, Loss: 0.2854\n", + "Epoch 6 Batch 780/1077 - Train Accuracy: 0.7609, Validation Accuracy: 0.7560, Loss: 0.3217\n", + "Epoch 6 Batch 790/1077 - Train Accuracy: 0.7051, Validation Accuracy: 0.7667, Loss: 0.3373\n", + "Epoch 6 Batch 800/1077 - Train Accuracy: 0.7371, Validation Accuracy: 0.7667, Loss: 0.2977\n", + "Epoch 6 Batch 810/1077 - Train Accuracy: 0.7548, Validation Accuracy: 0.7663, Loss: 0.2953\n", + "Epoch 6 Batch 820/1077 - Train Accuracy: 0.7453, Validation Accuracy: 0.7695, Loss: 0.3506\n", + "Epoch 6 Batch 830/1077 - Train Accuracy: 0.7367, Validation Accuracy: 0.7731, Loss: 0.3131\n", + "Epoch 6 Batch 840/1077 - Train Accuracy: 0.7641, Validation Accuracy: 0.7713, Loss: 0.3060\n", + "Epoch 6 Batch 850/1077 - Train Accuracy: 0.7303, Validation Accuracy: 0.7777, Loss: 0.3393\n", + "Epoch 6 Batch 860/1077 - Train Accuracy: 0.7727, Validation Accuracy: 0.7692, Loss: 0.3104\n", + "Epoch 6 Batch 870/1077 - Train Accuracy: 0.7623, Validation Accuracy: 0.7649, Loss: 0.3233\n", + "Epoch 6 Batch 880/1077 - Train Accuracy: 0.7941, Validation Accuracy: 0.7706, Loss: 0.2955\n", + "Epoch 6 Batch 890/1077 - Train Accuracy: 0.8211, Validation Accuracy: 0.7759, Loss: 0.2821\n", + "Epoch 6 Batch 900/1077 - Train Accuracy: 0.7668, Validation Accuracy: 0.7667, Loss: 0.3137\n", + "Epoch 6 Batch 910/1077 - Train Accuracy: 0.7400, Validation Accuracy: 0.7731, Loss: 0.2996\n", + "Epoch 6 Batch 920/1077 - Train Accuracy: 0.7738, Validation Accuracy: 0.7759, Loss: 0.3022\n", + "Epoch 6 Batch 930/1077 - Train Accuracy: 0.7551, Validation Accuracy: 0.7607, Loss: 0.3031\n", + "Epoch 6 Batch 940/1077 - Train Accuracy: 0.7492, Validation Accuracy: 0.7724, Loss: 0.2966\n", + "Epoch 6 Batch 950/1077 - Train Accuracy: 0.7582, Validation Accuracy: 0.7731, Loss: 0.2837\n", + "Epoch 6 Batch 960/1077 - Train Accuracy: 0.7615, Validation Accuracy: 0.7720, Loss: 0.2962\n", + "Epoch 6 Batch 970/1077 - Train Accuracy: 0.7688, Validation Accuracy: 0.7738, Loss: 0.3156\n", + "Epoch 6 Batch 980/1077 - Train Accuracy: 0.7684, Validation Accuracy: 0.7653, Loss: 0.3156\n", + "Epoch 6 Batch 990/1077 - Train Accuracy: 0.7775, Validation Accuracy: 0.7702, Loss: 0.3151\n", + "Epoch 6 Batch 1000/1077 - Train Accuracy: 0.7865, Validation Accuracy: 0.7773, Loss: 0.2850\n", + "Epoch 6 Batch 1010/1077 - Train Accuracy: 0.7555, Validation Accuracy: 0.7731, Loss: 0.3093\n", + "Epoch 6 Batch 1020/1077 - Train Accuracy: 0.7504, Validation Accuracy: 0.7763, Loss: 0.2996\n", + "Epoch 6 Batch 1030/1077 - Train Accuracy: 0.7535, Validation Accuracy: 0.7731, Loss: 0.3047\n", + "Epoch 6 Batch 1040/1077 - Train Accuracy: 0.7800, Validation Accuracy: 0.7759, Loss: 0.3155\n", + "Epoch 6 Batch 1050/1077 - Train Accuracy: 0.7355, Validation Accuracy: 0.7670, Loss: 0.3112\n", + "Epoch 6 Batch 1060/1077 - Train Accuracy: 0.7773, Validation Accuracy: 0.7713, Loss: 0.2685\n", + "Epoch 6 Batch 1070/1077 - Train Accuracy: 0.7336, Validation Accuracy: 0.7724, Loss: 0.3250\n", + "Epoch 7 Batch 10/1077 - Train Accuracy: 0.7438, Validation Accuracy: 0.7795, Loss: 0.3330\n", + "Epoch 7 Batch 20/1077 - Train Accuracy: 0.7457, Validation Accuracy: 0.7731, Loss: 0.3002\n", + "Epoch 7 Batch 30/1077 - Train Accuracy: 0.7754, Validation Accuracy: 0.7738, Loss: 0.2967\n", + "Epoch 7 Batch 40/1077 - Train Accuracy: 0.7781, Validation Accuracy: 0.7646, Loss: 0.2995\n", + "Epoch 7 Batch 50/1077 - Train Accuracy: 0.7609, Validation Accuracy: 0.7670, Loss: 0.3048\n", + "Epoch 7 Batch 60/1077 - Train Accuracy: 0.7504, Validation Accuracy: 0.7678, Loss: 0.2967\n", + "Epoch 7 Batch 70/1077 - Train Accuracy: 0.7619, Validation Accuracy: 0.7699, Loss: 0.3187\n", + "Epoch 7 Batch 80/1077 - Train Accuracy: 0.7816, Validation Accuracy: 0.7745, Loss: 0.3120\n", + "Epoch 7 Batch 90/1077 - Train Accuracy: 0.7430, Validation Accuracy: 0.7717, Loss: 0.3165\n", + "Epoch 7 Batch 100/1077 - Train Accuracy: 0.7793, Validation Accuracy: 0.7773, Loss: 0.3004\n", + "Epoch 7 Batch 110/1077 - Train Accuracy: 0.7953, Validation Accuracy: 0.7791, Loss: 0.2606\n", + "Epoch 7 Batch 120/1077 - Train Accuracy: 0.7766, Validation Accuracy: 0.7713, Loss: 0.3147\n", + "Epoch 7 Batch 130/1077 - Train Accuracy: 0.7422, Validation Accuracy: 0.7514, Loss: 0.2787\n", + "Epoch 7 Batch 140/1077 - Train Accuracy: 0.7599, Validation Accuracy: 0.7685, Loss: 0.3047\n", + "Epoch 7 Batch 150/1077 - Train Accuracy: 0.7772, Validation Accuracy: 0.7788, Loss: 0.2895\n", + "Epoch 7 Batch 160/1077 - Train Accuracy: 0.7660, Validation Accuracy: 0.7784, Loss: 0.3069\n", + "Epoch 7 Batch 170/1077 - Train Accuracy: 0.7305, Validation Accuracy: 0.7702, Loss: 0.3159\n", + "Epoch 7 Batch 180/1077 - Train Accuracy: 0.7773, Validation Accuracy: 0.7752, Loss: 0.2876\n", + "Epoch 7 Batch 190/1077 - Train Accuracy: 0.8066, Validation Accuracy: 0.7702, Loss: 0.2899\n", + "Epoch 7 Batch 200/1077 - Train Accuracy: 0.7465, Validation Accuracy: 0.7734, Loss: 0.3216\n", + "Epoch 7 Batch 210/1077 - Train Accuracy: 0.7690, Validation Accuracy: 0.7699, Loss: 0.2967\n", + "Epoch 7 Batch 220/1077 - Train Accuracy: 0.7714, Validation Accuracy: 0.7692, Loss: 0.2799\n", + "Epoch 7 Batch 230/1077 - Train Accuracy: 0.7779, Validation Accuracy: 0.7699, Loss: 0.2805\n", + "Epoch 7 Batch 240/1077 - Train Accuracy: 0.8121, Validation Accuracy: 0.7745, Loss: 0.2831\n", + "Epoch 7 Batch 250/1077 - Train Accuracy: 0.7330, Validation Accuracy: 0.7749, Loss: 0.2926\n", + "Epoch 7 Batch 260/1077 - Train Accuracy: 0.7705, Validation Accuracy: 0.7749, Loss: 0.2651\n", + "Epoch 7 Batch 270/1077 - Train Accuracy: 0.7293, Validation Accuracy: 0.7766, Loss: 0.3168\n", + "Epoch 7 Batch 280/1077 - Train Accuracy: 0.7867, Validation Accuracy: 0.7685, Loss: 0.2932\n", + "Epoch 7 Batch 290/1077 - Train Accuracy: 0.7645, Validation Accuracy: 0.7688, Loss: 0.3070\n", + "Epoch 7 Batch 300/1077 - Train Accuracy: 0.7862, Validation Accuracy: 0.7738, Loss: 0.2964\n", + "Epoch 7 Batch 310/1077 - Train Accuracy: 0.7336, Validation Accuracy: 0.7670, Loss: 0.3141\n", + "Epoch 7 Batch 320/1077 - Train Accuracy: 0.8121, Validation Accuracy: 0.7695, Loss: 0.2967\n", + "Epoch 7 Batch 330/1077 - Train Accuracy: 0.8066, Validation Accuracy: 0.7663, Loss: 0.2813\n", + "Epoch 7 Batch 340/1077 - Train Accuracy: 0.7364, Validation Accuracy: 0.7660, Loss: 0.2961\n", + "Epoch 7 Batch 350/1077 - Train Accuracy: 0.7781, Validation Accuracy: 0.7727, Loss: 0.2916\n", + "Epoch 7 Batch 360/1077 - Train Accuracy: 0.7676, Validation Accuracy: 0.7592, Loss: 0.3015\n", + "Epoch 7 Batch 370/1077 - Train Accuracy: 0.7664, Validation Accuracy: 0.7603, Loss: 0.2911\n", + "Epoch 7 Batch 380/1077 - Train Accuracy: 0.7750, Validation Accuracy: 0.7553, Loss: 0.2830\n", + "Epoch 7 Batch 390/1077 - Train Accuracy: 0.7246, Validation Accuracy: 0.7702, Loss: 0.3176\n", + "Epoch 7 Batch 400/1077 - Train Accuracy: 0.8027, Validation Accuracy: 0.7724, Loss: 0.2874\n", + "Epoch 7 Batch 410/1077 - Train Accuracy: 0.7685, Validation Accuracy: 0.7727, Loss: 0.3114\n", + "Epoch 7 Batch 420/1077 - Train Accuracy: 0.8176, Validation Accuracy: 0.7763, Loss: 0.2606\n", + "Epoch 7 Batch 430/1077 - Train Accuracy: 0.7305, Validation Accuracy: 0.7685, Loss: 0.2738\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 7 Batch 440/1077 - Train Accuracy: 0.7535, Validation Accuracy: 0.7642, Loss: 0.3306\n", + "Epoch 7 Batch 450/1077 - Train Accuracy: 0.7840, Validation Accuracy: 0.7695, Loss: 0.2810\n", + "Epoch 7 Batch 460/1077 - Train Accuracy: 0.7672, Validation Accuracy: 0.7685, Loss: 0.3011\n", + "Epoch 7 Batch 470/1077 - Train Accuracy: 0.7895, Validation Accuracy: 0.7756, Loss: 0.2980\n", + "Epoch 7 Batch 480/1077 - Train Accuracy: 0.8174, Validation Accuracy: 0.7667, Loss: 0.2830\n", + "Epoch 7 Batch 490/1077 - Train Accuracy: 0.7641, Validation Accuracy: 0.7795, Loss: 0.3020\n", + "Epoch 7 Batch 500/1077 - Train Accuracy: 0.7953, Validation Accuracy: 0.7685, Loss: 0.2666\n", + "Epoch 7 Batch 510/1077 - Train Accuracy: 0.7652, Validation Accuracy: 0.7777, Loss: 0.2670\n", + "Epoch 7 Batch 520/1077 - Train Accuracy: 0.7995, Validation Accuracy: 0.7734, Loss: 0.2561\n", + "Epoch 7 Batch 530/1077 - Train Accuracy: 0.7270, Validation Accuracy: 0.7820, Loss: 0.2895\n", + "Epoch 7 Batch 540/1077 - Train Accuracy: 0.7836, Validation Accuracy: 0.7575, Loss: 0.2641\n", + "Epoch 7 Batch 550/1077 - Train Accuracy: 0.7441, Validation Accuracy: 0.7575, Loss: 0.2937\n", + "Epoch 7 Batch 560/1077 - Train Accuracy: 0.7984, Validation Accuracy: 0.7667, Loss: 0.2609\n", + "Epoch 7 Batch 570/1077 - Train Accuracy: 0.7558, Validation Accuracy: 0.7855, Loss: 0.3016\n", + "Epoch 7 Batch 580/1077 - Train Accuracy: 0.8326, Validation Accuracy: 0.7866, Loss: 0.2520\n", + "Epoch 7 Batch 590/1077 - Train Accuracy: 0.7553, Validation Accuracy: 0.7812, Loss: 0.2960\n", + "Epoch 7 Batch 600/1077 - Train Accuracy: 0.8047, Validation Accuracy: 0.7855, Loss: 0.2500\n", + "Epoch 7 Batch 610/1077 - Train Accuracy: 0.7484, Validation Accuracy: 0.7670, Loss: 0.2995\n", + "Epoch 7 Batch 620/1077 - Train Accuracy: 0.7492, Validation Accuracy: 0.7827, Loss: 0.2746\n", + "Epoch 7 Batch 630/1077 - Train Accuracy: 0.7992, Validation Accuracy: 0.7930, Loss: 0.2629\n", + "Epoch 7 Batch 640/1077 - Train Accuracy: 0.7768, Validation Accuracy: 0.7923, Loss: 0.2628\n", + "Epoch 7 Batch 650/1077 - Train Accuracy: 0.7418, Validation Accuracy: 0.7994, Loss: 0.2719\n", + "Epoch 7 Batch 660/1077 - Train Accuracy: 0.7937, Validation Accuracy: 0.7887, Loss: 0.2838\n", + "Epoch 7 Batch 670/1077 - Train Accuracy: 0.7994, Validation Accuracy: 0.7770, Loss: 0.2534\n", + "Epoch 7 Batch 680/1077 - Train Accuracy: 0.7805, Validation Accuracy: 0.7862, Loss: 0.2696\n", + "Epoch 7 Batch 690/1077 - Train Accuracy: 0.7863, Validation Accuracy: 0.7887, Loss: 0.2600\n", + "Epoch 7 Batch 700/1077 - Train Accuracy: 0.7695, Validation Accuracy: 0.7884, Loss: 0.2553\n", + "Epoch 7 Batch 710/1077 - Train Accuracy: 0.7582, Validation Accuracy: 0.7962, Loss: 0.2722\n", + "Epoch 7 Batch 720/1077 - Train Accuracy: 0.7714, Validation Accuracy: 0.7624, Loss: 0.2834\n", + "Epoch 7 Batch 730/1077 - Train Accuracy: 0.7711, Validation Accuracy: 0.7795, Loss: 0.2961\n", + "Epoch 7 Batch 740/1077 - Train Accuracy: 0.7582, Validation Accuracy: 0.7699, Loss: 0.2726\n", + "Epoch 7 Batch 750/1077 - Train Accuracy: 0.8285, Validation Accuracy: 0.8033, Loss: 0.2575\n", + "Epoch 7 Batch 760/1077 - Train Accuracy: 0.8020, Validation Accuracy: 0.7908, Loss: 0.2722\n", + "Epoch 7 Batch 770/1077 - Train Accuracy: 0.7798, Validation Accuracy: 0.7866, Loss: 0.2429\n", + "Epoch 7 Batch 780/1077 - Train Accuracy: 0.7605, Validation Accuracy: 0.7702, Loss: 0.2810\n", + "Epoch 7 Batch 790/1077 - Train Accuracy: 0.7523, Validation Accuracy: 0.7905, Loss: 0.2939\n", + "Epoch 7 Batch 800/1077 - Train Accuracy: 0.7789, Validation Accuracy: 0.7937, Loss: 0.2425\n", + "Epoch 7 Batch 810/1077 - Train Accuracy: 0.7868, Validation Accuracy: 0.8075, Loss: 0.2460\n", + "Epoch 7 Batch 820/1077 - Train Accuracy: 0.7395, Validation Accuracy: 0.7994, Loss: 0.2886\n", + "Epoch 7 Batch 830/1077 - Train Accuracy: 0.7723, Validation Accuracy: 0.8058, Loss: 0.2531\n", + "Epoch 7 Batch 840/1077 - Train Accuracy: 0.7984, Validation Accuracy: 0.8026, Loss: 0.2505\n", + "Epoch 7 Batch 850/1077 - Train Accuracy: 0.7318, Validation Accuracy: 0.7901, Loss: 0.2832\n", + "Epoch 7 Batch 860/1077 - Train Accuracy: 0.7786, Validation Accuracy: 0.7912, Loss: 0.2723\n", + "Epoch 7 Batch 870/1077 - Train Accuracy: 0.7870, Validation Accuracy: 0.7884, Loss: 0.2883\n", + "Epoch 7 Batch 880/1077 - Train Accuracy: 0.8035, Validation Accuracy: 0.7876, Loss: 0.2569\n", + "Epoch 7 Batch 890/1077 - Train Accuracy: 0.8181, Validation Accuracy: 0.7773, Loss: 0.2434\n", + "Epoch 7 Batch 900/1077 - Train Accuracy: 0.7918, Validation Accuracy: 0.7923, Loss: 0.2553\n", + "Epoch 7 Batch 910/1077 - Train Accuracy: 0.8136, Validation Accuracy: 0.7830, Loss: 0.2387\n", + "Epoch 7 Batch 920/1077 - Train Accuracy: 0.7918, Validation Accuracy: 0.7809, Loss: 0.2429\n", + "Epoch 7 Batch 930/1077 - Train Accuracy: 0.7957, Validation Accuracy: 0.8132, Loss: 0.2580\n", + "Epoch 7 Batch 940/1077 - Train Accuracy: 0.8074, Validation Accuracy: 0.8125, Loss: 0.2458\n", + "Epoch 7 Batch 950/1077 - Train Accuracy: 0.8207, Validation Accuracy: 0.7969, Loss: 0.2339\n", + "Epoch 7 Batch 960/1077 - Train Accuracy: 0.7783, Validation Accuracy: 0.7805, Loss: 0.2500\n", + "Epoch 7 Batch 970/1077 - Train Accuracy: 0.8230, Validation Accuracy: 0.7823, Loss: 0.2574\n", + "Epoch 7 Batch 980/1077 - Train Accuracy: 0.8039, Validation Accuracy: 0.7915, Loss: 0.2509\n", + "Epoch 7 Batch 990/1077 - Train Accuracy: 0.8018, Validation Accuracy: 0.8107, Loss: 0.2807\n", + "Epoch 7 Batch 1000/1077 - Train Accuracy: 0.8475, Validation Accuracy: 0.8072, Loss: 0.2353\n", + "Epoch 7 Batch 1010/1077 - Train Accuracy: 0.7945, Validation Accuracy: 0.8022, Loss: 0.2597\n", + "Epoch 7 Batch 1020/1077 - Train Accuracy: 0.8301, Validation Accuracy: 0.7926, Loss: 0.2436\n", + "Epoch 7 Batch 1030/1077 - Train Accuracy: 0.8254, Validation Accuracy: 0.8082, Loss: 0.2476\n", + "Epoch 7 Batch 1040/1077 - Train Accuracy: 0.8405, Validation Accuracy: 0.7805, Loss: 0.2458\n", + "Epoch 7 Batch 1050/1077 - Train Accuracy: 0.7781, Validation Accuracy: 0.7915, Loss: 0.2535\n", + "Epoch 7 Batch 1060/1077 - Train Accuracy: 0.8184, Validation Accuracy: 0.7915, Loss: 0.2176\n", + "Epoch 7 Batch 1070/1077 - Train Accuracy: 0.7918, Validation Accuracy: 0.8022, Loss: 0.2516\n", + "Epoch 8 Batch 10/1077 - Train Accuracy: 0.7998, Validation Accuracy: 0.8008, Loss: 0.2625\n", + "Epoch 8 Batch 20/1077 - Train Accuracy: 0.7781, Validation Accuracy: 0.8036, Loss: 0.2324\n", + "Epoch 8 Batch 30/1077 - Train Accuracy: 0.8027, Validation Accuracy: 0.8061, Loss: 0.2386\n", + "Epoch 8 Batch 40/1077 - Train Accuracy: 0.8313, Validation Accuracy: 0.7979, Loss: 0.2516\n", + "Epoch 8 Batch 50/1077 - Train Accuracy: 0.8035, Validation Accuracy: 0.8139, Loss: 0.2463\n", + "Epoch 8 Batch 60/1077 - Train Accuracy: 0.7824, Validation Accuracy: 0.8143, Loss: 0.2352\n", + "Epoch 8 Batch 70/1077 - Train Accuracy: 0.7858, Validation Accuracy: 0.8150, Loss: 0.2413\n", + "Epoch 8 Batch 80/1077 - Train Accuracy: 0.7957, Validation Accuracy: 0.8036, Loss: 0.2352\n", + "Epoch 8 Batch 90/1077 - Train Accuracy: 0.8016, Validation Accuracy: 0.8033, Loss: 0.2428\n", + "Epoch 8 Batch 100/1077 - Train Accuracy: 0.8340, Validation Accuracy: 0.8217, Loss: 0.2323\n", + "Epoch 8 Batch 110/1077 - Train Accuracy: 0.8398, Validation Accuracy: 0.8448, Loss: 0.2263\n", + "Epoch 8 Batch 120/1077 - Train Accuracy: 0.8230, Validation Accuracy: 0.8228, Loss: 0.2617\n", + "Epoch 8 Batch 130/1077 - Train Accuracy: 0.7883, Validation Accuracy: 0.8221, Loss: 0.2157\n", + "Epoch 8 Batch 140/1077 - Train Accuracy: 0.7948, Validation Accuracy: 0.8079, Loss: 0.2484\n", + "Epoch 8 Batch 150/1077 - Train Accuracy: 0.8304, Validation Accuracy: 0.8175, Loss: 0.2291\n", + "Epoch 8 Batch 160/1077 - Train Accuracy: 0.8176, Validation Accuracy: 0.7951, Loss: 0.2434\n", + "Epoch 8 Batch 170/1077 - Train Accuracy: 0.7910, Validation Accuracy: 0.8072, Loss: 0.2499\n", + "Epoch 8 Batch 180/1077 - Train Accuracy: 0.7801, Validation Accuracy: 0.8143, Loss: 0.2188\n", + "Epoch 8 Batch 190/1077 - Train Accuracy: 0.8617, Validation Accuracy: 0.8143, Loss: 0.2316\n", + "Epoch 8 Batch 200/1077 - Train Accuracy: 0.7777, Validation Accuracy: 0.8278, Loss: 0.2596\n", + "Epoch 8 Batch 210/1077 - Train Accuracy: 0.8233, Validation Accuracy: 0.8107, Loss: 0.2309\n", + "Epoch 8 Batch 220/1077 - Train Accuracy: 0.8277, Validation Accuracy: 0.8150, Loss: 0.2206\n", + "Epoch 8 Batch 230/1077 - Train Accuracy: 0.8237, Validation Accuracy: 0.8232, Loss: 0.2176\n", + "Epoch 8 Batch 240/1077 - Train Accuracy: 0.8754, Validation Accuracy: 0.8207, Loss: 0.2235\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 8 Batch 250/1077 - Train Accuracy: 0.8292, Validation Accuracy: 0.8445, Loss: 0.2162\n", + "Epoch 8 Batch 260/1077 - Train Accuracy: 0.8315, Validation Accuracy: 0.8430, Loss: 0.2156\n", + "Epoch 8 Batch 270/1077 - Train Accuracy: 0.7719, Validation Accuracy: 0.8086, Loss: 0.2543\n", + "Epoch 8 Batch 280/1077 - Train Accuracy: 0.8266, Validation Accuracy: 0.8391, Loss: 0.2328\n", + "Epoch 8 Batch 290/1077 - Train Accuracy: 0.8133, Validation Accuracy: 0.8310, Loss: 0.2703\n", + "Epoch 8 Batch 300/1077 - Train Accuracy: 0.8701, Validation Accuracy: 0.8125, Loss: 0.2301\n", + "Epoch 8 Batch 310/1077 - Train Accuracy: 0.8102, Validation Accuracy: 0.8359, Loss: 0.2479\n", + "Epoch 8 Batch 320/1077 - Train Accuracy: 0.8535, Validation Accuracy: 0.8303, Loss: 0.2404\n", + "Epoch 8 Batch 330/1077 - Train Accuracy: 0.8328, Validation Accuracy: 0.8384, Loss: 0.2217\n", + "Epoch 8 Batch 340/1077 - Train Accuracy: 0.8351, Validation Accuracy: 0.8260, Loss: 0.2478\n", + "Epoch 8 Batch 350/1077 - Train Accuracy: 0.8609, Validation Accuracy: 0.8388, Loss: 0.2316\n", + "Epoch 8 Batch 360/1077 - Train Accuracy: 0.8262, Validation Accuracy: 0.8196, Loss: 0.2343\n", + "Epoch 8 Batch 370/1077 - Train Accuracy: 0.8709, Validation Accuracy: 0.8505, Loss: 0.2104\n", + "Epoch 8 Batch 380/1077 - Train Accuracy: 0.8438, Validation Accuracy: 0.8409, Loss: 0.2205\n", + "Epoch 8 Batch 390/1077 - Train Accuracy: 0.8227, Validation Accuracy: 0.8349, Loss: 0.2384\n", + "Epoch 8 Batch 400/1077 - Train Accuracy: 0.8762, Validation Accuracy: 0.8480, Loss: 0.2408\n", + "Epoch 8 Batch 410/1077 - Train Accuracy: 0.8544, Validation Accuracy: 0.8434, Loss: 0.2294\n", + "Epoch 8 Batch 420/1077 - Train Accuracy: 0.8684, Validation Accuracy: 0.8214, Loss: 0.2067\n", + "Epoch 8 Batch 430/1077 - Train Accuracy: 0.8008, Validation Accuracy: 0.8281, Loss: 0.2165\n", + "Epoch 8 Batch 440/1077 - Train Accuracy: 0.8246, Validation Accuracy: 0.8427, Loss: 0.2555\n", + "Epoch 8 Batch 450/1077 - Train Accuracy: 0.8816, Validation Accuracy: 0.8455, Loss: 0.2178\n", + "Epoch 8 Batch 460/1077 - Train Accuracy: 0.8492, Validation Accuracy: 0.8338, Loss: 0.2317\n", + "Epoch 8 Batch 470/1077 - Train Accuracy: 0.8610, Validation Accuracy: 0.8402, Loss: 0.2253\n", + "Epoch 8 Batch 480/1077 - Train Accuracy: 0.8503, Validation Accuracy: 0.8374, Loss: 0.2239\n", + "Epoch 8 Batch 490/1077 - Train Accuracy: 0.8570, Validation Accuracy: 0.8384, Loss: 0.2281\n", + "Epoch 8 Batch 500/1077 - Train Accuracy: 0.8652, Validation Accuracy: 0.8381, Loss: 0.2028\n", + "Epoch 8 Batch 510/1077 - Train Accuracy: 0.8246, Validation Accuracy: 0.8388, Loss: 0.2000\n", + "Epoch 8 Batch 520/1077 - Train Accuracy: 0.8411, Validation Accuracy: 0.8391, Loss: 0.2004\n", + "Epoch 8 Batch 530/1077 - Train Accuracy: 0.8223, Validation Accuracy: 0.8331, Loss: 0.2320\n", + "Epoch 8 Batch 540/1077 - Train Accuracy: 0.8777, Validation Accuracy: 0.8317, Loss: 0.2021\n", + "Epoch 8 Batch 550/1077 - Train Accuracy: 0.8316, Validation Accuracy: 0.8494, Loss: 0.2205\n", + "Epoch 8 Batch 560/1077 - Train Accuracy: 0.8520, Validation Accuracy: 0.8501, Loss: 0.2156\n", + "Epoch 8 Batch 570/1077 - Train Accuracy: 0.8261, Validation Accuracy: 0.8420, Loss: 0.2472\n", + "Epoch 8 Batch 580/1077 - Train Accuracy: 0.8873, Validation Accuracy: 0.8292, Loss: 0.2006\n", + "Epoch 8 Batch 590/1077 - Train Accuracy: 0.8039, Validation Accuracy: 0.8505, Loss: 0.2310\n", + "Epoch 8 Batch 600/1077 - Train Accuracy: 0.8739, Validation Accuracy: 0.8523, Loss: 0.2066\n", + "Epoch 8 Batch 610/1077 - Train Accuracy: 0.8355, Validation Accuracy: 0.8462, Loss: 0.2173\n", + "Epoch 8 Batch 620/1077 - Train Accuracy: 0.8641, Validation Accuracy: 0.8509, Loss: 0.2018\n", + "Epoch 8 Batch 630/1077 - Train Accuracy: 0.8816, Validation Accuracy: 0.8480, Loss: 0.2163\n", + "Epoch 8 Batch 640/1077 - Train Accuracy: 0.8233, Validation Accuracy: 0.8509, Loss: 0.2102\n", + "Epoch 8 Batch 650/1077 - Train Accuracy: 0.8527, Validation Accuracy: 0.8580, Loss: 0.2219\n", + "Epoch 8 Batch 660/1077 - Train Accuracy: 0.8695, Validation Accuracy: 0.8636, Loss: 0.2103\n", + "Epoch 8 Batch 670/1077 - Train Accuracy: 0.8711, Validation Accuracy: 0.8601, Loss: 0.2046\n", + "Epoch 8 Batch 680/1077 - Train Accuracy: 0.8542, Validation Accuracy: 0.8604, Loss: 0.2022\n", + "Epoch 8 Batch 690/1077 - Train Accuracy: 0.8875, Validation Accuracy: 0.8658, Loss: 0.2031\n", + "Epoch 8 Batch 700/1077 - Train Accuracy: 0.8680, Validation Accuracy: 0.8803, Loss: 0.1975\n", + "Epoch 8 Batch 710/1077 - Train Accuracy: 0.8391, Validation Accuracy: 0.8590, Loss: 0.2062\n", + "Epoch 8 Batch 720/1077 - Train Accuracy: 0.8573, Validation Accuracy: 0.8626, Loss: 0.2335\n", + "Epoch 8 Batch 730/1077 - Train Accuracy: 0.8488, Validation Accuracy: 0.8530, Loss: 0.2256\n", + "Epoch 8 Batch 740/1077 - Train Accuracy: 0.8621, Validation Accuracy: 0.8572, Loss: 0.1935\n", + "Epoch 8 Batch 750/1077 - Train Accuracy: 0.8938, Validation Accuracy: 0.8406, Loss: 0.1974\n", + "Epoch 8 Batch 760/1077 - Train Accuracy: 0.8461, Validation Accuracy: 0.8430, Loss: 0.2160\n", + "Epoch 8 Batch 770/1077 - Train Accuracy: 0.8478, Validation Accuracy: 0.8569, Loss: 0.2056\n", + "Epoch 8 Batch 780/1077 - Train Accuracy: 0.8316, Validation Accuracy: 0.8512, Loss: 0.2313\n", + "Epoch 8 Batch 790/1077 - Train Accuracy: 0.7918, Validation Accuracy: 0.8565, Loss: 0.2287\n", + "Epoch 8 Batch 800/1077 - Train Accuracy: 0.8711, Validation Accuracy: 0.8473, Loss: 0.1926\n", + "Epoch 8 Batch 810/1077 - Train Accuracy: 0.8571, Validation Accuracy: 0.8519, Loss: 0.1891\n", + "Epoch 8 Batch 820/1077 - Train Accuracy: 0.8465, Validation Accuracy: 0.8516, Loss: 0.2143\n", + "Epoch 8 Batch 830/1077 - Train Accuracy: 0.8594, Validation Accuracy: 0.8619, Loss: 0.2045\n", + "Epoch 8 Batch 840/1077 - Train Accuracy: 0.8758, Validation Accuracy: 0.8626, Loss: 0.2002\n", + "Epoch 8 Batch 850/1077 - Train Accuracy: 0.8248, Validation Accuracy: 0.8647, Loss: 0.2242\n", + "Epoch 8 Batch 860/1077 - Train Accuracy: 0.8571, Validation Accuracy: 0.8647, Loss: 0.2157\n", + "Epoch 8 Batch 870/1077 - Train Accuracy: 0.8183, Validation Accuracy: 0.8523, Loss: 0.2142\n", + "Epoch 8 Batch 880/1077 - Train Accuracy: 0.8676, Validation Accuracy: 0.8587, Loss: 0.1964\n", + "Epoch 8 Batch 890/1077 - Train Accuracy: 0.8936, Validation Accuracy: 0.8572, Loss: 0.1920\n", + "Epoch 8 Batch 900/1077 - Train Accuracy: 0.8574, Validation Accuracy: 0.8544, Loss: 0.1990\n", + "Epoch 8 Batch 910/1077 - Train Accuracy: 0.8850, Validation Accuracy: 0.8459, Loss: 0.1943\n", + "Epoch 8 Batch 920/1077 - Train Accuracy: 0.8375, Validation Accuracy: 0.8569, Loss: 0.2117\n", + "Epoch 8 Batch 930/1077 - Train Accuracy: 0.8617, Validation Accuracy: 0.8445, Loss: 0.1943\n", + "Epoch 8 Batch 940/1077 - Train Accuracy: 0.8625, Validation Accuracy: 0.8477, Loss: 0.1992\n", + "Epoch 8 Batch 950/1077 - Train Accuracy: 0.8549, Validation Accuracy: 0.8562, Loss: 0.1801\n", + "Epoch 8 Batch 960/1077 - Train Accuracy: 0.8393, Validation Accuracy: 0.8551, Loss: 0.1883\n", + "Epoch 8 Batch 970/1077 - Train Accuracy: 0.8906, Validation Accuracy: 0.8469, Loss: 0.2009\n", + "Epoch 8 Batch 980/1077 - Train Accuracy: 0.8426, Validation Accuracy: 0.8505, Loss: 0.2123\n", + "Epoch 8 Batch 990/1077 - Train Accuracy: 0.8466, Validation Accuracy: 0.8572, Loss: 0.2048\n", + "Epoch 8 Batch 1000/1077 - Train Accuracy: 0.8865, Validation Accuracy: 0.8643, Loss: 0.1928\n", + "Epoch 8 Batch 1010/1077 - Train Accuracy: 0.8770, Validation Accuracy: 0.8580, Loss: 0.1984\n", + "Epoch 8 Batch 1020/1077 - Train Accuracy: 0.8785, Validation Accuracy: 0.8597, Loss: 0.1824\n", + "Epoch 8 Batch 1030/1077 - Train Accuracy: 0.8652, Validation Accuracy: 0.8679, Loss: 0.2075\n", + "Epoch 8 Batch 1040/1077 - Train Accuracy: 0.9021, Validation Accuracy: 0.8512, Loss: 0.2085\n", + "Epoch 8 Batch 1050/1077 - Train Accuracy: 0.8508, Validation Accuracy: 0.8540, Loss: 0.1999\n", + "Epoch 8 Batch 1060/1077 - Train Accuracy: 0.8578, Validation Accuracy: 0.8633, Loss: 0.1902\n", + "Epoch 8 Batch 1070/1077 - Train Accuracy: 0.8648, Validation Accuracy: 0.8658, Loss: 0.1957\n", + "Epoch 9 Batch 10/1077 - Train Accuracy: 0.8964, Validation Accuracy: 0.8661, Loss: 0.2049\n", + "Epoch 9 Batch 20/1077 - Train Accuracy: 0.8562, Validation Accuracy: 0.8551, Loss: 0.1826\n", + "Epoch 9 Batch 30/1077 - Train Accuracy: 0.8781, Validation Accuracy: 0.8576, Loss: 0.1831\n", + "Epoch 9 Batch 40/1077 - Train Accuracy: 0.8758, Validation Accuracy: 0.8626, Loss: 0.1917\n", + "Epoch 9 Batch 50/1077 - Train Accuracy: 0.8898, Validation Accuracy: 0.8622, Loss: 0.1757\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 9 Batch 60/1077 - Train Accuracy: 0.8594, Validation Accuracy: 0.8374, Loss: 0.1717\n", + "Epoch 9 Batch 70/1077 - Train Accuracy: 0.8799, Validation Accuracy: 0.8615, Loss: 0.1842\n", + "Epoch 9 Batch 80/1077 - Train Accuracy: 0.8680, Validation Accuracy: 0.8530, Loss: 0.1970\n", + "Epoch 9 Batch 90/1077 - Train Accuracy: 0.8617, Validation Accuracy: 0.8729, Loss: 0.1937\n", + "Epoch 9 Batch 100/1077 - Train Accuracy: 0.8703, Validation Accuracy: 0.8697, Loss: 0.1754\n", + "Epoch 9 Batch 110/1077 - Train Accuracy: 0.8922, Validation Accuracy: 0.8569, Loss: 0.1624\n", + "Epoch 9 Batch 120/1077 - Train Accuracy: 0.8707, Validation Accuracy: 0.8530, Loss: 0.2073\n", + "Epoch 9 Batch 130/1077 - Train Accuracy: 0.8523, Validation Accuracy: 0.8509, Loss: 0.1772\n", + "Epoch 9 Batch 140/1077 - Train Accuracy: 0.8857, Validation Accuracy: 0.8548, Loss: 0.2015\n", + "Epoch 9 Batch 150/1077 - Train Accuracy: 0.8720, Validation Accuracy: 0.8580, Loss: 0.1851\n", + "Epoch 9 Batch 160/1077 - Train Accuracy: 0.8734, Validation Accuracy: 0.8675, Loss: 0.1741\n", + "Epoch 9 Batch 170/1077 - Train Accuracy: 0.8676, Validation Accuracy: 0.8651, Loss: 0.2062\n", + "Epoch 9 Batch 180/1077 - Train Accuracy: 0.8555, Validation Accuracy: 0.8679, Loss: 0.1797\n", + "Epoch 9 Batch 190/1077 - Train Accuracy: 0.8816, Validation Accuracy: 0.8668, Loss: 0.1833\n", + "Epoch 9 Batch 200/1077 - Train Accuracy: 0.8383, Validation Accuracy: 0.8746, Loss: 0.2005\n", + "Epoch 9 Batch 210/1077 - Train Accuracy: 0.8787, Validation Accuracy: 0.8612, Loss: 0.1830\n", + "Epoch 9 Batch 220/1077 - Train Accuracy: 0.8746, Validation Accuracy: 0.8548, Loss: 0.1928\n", + "Epoch 9 Batch 230/1077 - Train Accuracy: 0.8668, Validation Accuracy: 0.8597, Loss: 0.1753\n", + "Epoch 9 Batch 240/1077 - Train Accuracy: 0.9129, Validation Accuracy: 0.8594, Loss: 0.1731\n", + "Epoch 9 Batch 250/1077 - Train Accuracy: 0.8842, Validation Accuracy: 0.8697, Loss: 0.1754\n", + "Epoch 9 Batch 260/1077 - Train Accuracy: 0.9118, Validation Accuracy: 0.8736, Loss: 0.1573\n", + "Epoch 9 Batch 270/1077 - Train Accuracy: 0.8477, Validation Accuracy: 0.8768, Loss: 0.2083\n", + "Epoch 9 Batch 280/1077 - Train Accuracy: 0.8613, Validation Accuracy: 0.8832, Loss: 0.2085\n", + "Epoch 9 Batch 290/1077 - Train Accuracy: 0.8523, Validation Accuracy: 0.8768, Loss: 0.2161\n", + "Epoch 9 Batch 300/1077 - Train Accuracy: 0.9145, Validation Accuracy: 0.8803, Loss: 0.1717\n", + "Epoch 9 Batch 310/1077 - Train Accuracy: 0.8535, Validation Accuracy: 0.8647, Loss: 0.1927\n", + "Epoch 9 Batch 320/1077 - Train Accuracy: 0.8793, Validation Accuracy: 0.8516, Loss: 0.2041\n", + "Epoch 9 Batch 330/1077 - Train Accuracy: 0.8629, Validation Accuracy: 0.8786, Loss: 0.1877\n", + "Epoch 9 Batch 340/1077 - Train Accuracy: 0.8857, Validation Accuracy: 0.8778, Loss: 0.1915\n", + "Epoch 9 Batch 350/1077 - Train Accuracy: 0.8957, Validation Accuracy: 0.8622, Loss: 0.1857\n", + "Epoch 9 Batch 360/1077 - Train Accuracy: 0.8844, Validation Accuracy: 0.8622, Loss: 0.1875\n", + "Epoch 9 Batch 370/1077 - Train Accuracy: 0.8921, Validation Accuracy: 0.8743, Loss: 0.1695\n", + "Epoch 9 Batch 380/1077 - Train Accuracy: 0.8934, Validation Accuracy: 0.8690, Loss: 0.1713\n", + "Epoch 9 Batch 390/1077 - Train Accuracy: 0.8445, Validation Accuracy: 0.8629, Loss: 0.1963\n", + "Epoch 9 Batch 400/1077 - Train Accuracy: 0.8758, Validation Accuracy: 0.8668, Loss: 0.1964\n", + "Epoch 9 Batch 410/1077 - Train Accuracy: 0.8692, Validation Accuracy: 0.8683, Loss: 0.1957\n", + "Epoch 9 Batch 420/1077 - Train Accuracy: 0.9082, Validation Accuracy: 0.8707, Loss: 0.1591\n", + "Epoch 9 Batch 430/1077 - Train Accuracy: 0.8910, Validation Accuracy: 0.8743, Loss: 0.1766\n", + "Epoch 9 Batch 440/1077 - Train Accuracy: 0.8562, Validation Accuracy: 0.8707, Loss: 0.2085\n", + "Epoch 9 Batch 450/1077 - Train Accuracy: 0.8957, Validation Accuracy: 0.8690, Loss: 0.1892\n", + "Epoch 9 Batch 460/1077 - Train Accuracy: 0.8637, Validation Accuracy: 0.8683, Loss: 0.1872\n", + "Epoch 9 Batch 470/1077 - Train Accuracy: 0.8766, Validation Accuracy: 0.8750, Loss: 0.1926\n", + "Epoch 9 Batch 480/1077 - Train Accuracy: 0.8803, Validation Accuracy: 0.8700, Loss: 0.1834\n", + "Epoch 9 Batch 490/1077 - Train Accuracy: 0.8668, Validation Accuracy: 0.8675, Loss: 0.1814\n", + "Epoch 9 Batch 500/1077 - Train Accuracy: 0.8961, Validation Accuracy: 0.8658, Loss: 0.1506\n", + "Epoch 9 Batch 510/1077 - Train Accuracy: 0.8664, Validation Accuracy: 0.8509, Loss: 0.1754\n", + "Epoch 9 Batch 520/1077 - Train Accuracy: 0.8717, Validation Accuracy: 0.8619, Loss: 0.1664\n", + "Epoch 9 Batch 530/1077 - Train Accuracy: 0.8562, Validation Accuracy: 0.8690, Loss: 0.1739\n", + "Epoch 9 Batch 540/1077 - Train Accuracy: 0.8957, Validation Accuracy: 0.8722, Loss: 0.1554\n", + "Epoch 9 Batch 550/1077 - Train Accuracy: 0.8383, Validation Accuracy: 0.8750, Loss: 0.1869\n", + "Epoch 9 Batch 560/1077 - Train Accuracy: 0.8871, Validation Accuracy: 0.8839, Loss: 0.1668\n", + "Epoch 9 Batch 570/1077 - Train Accuracy: 0.8569, Validation Accuracy: 0.8679, Loss: 0.1937\n", + "Epoch 9 Batch 580/1077 - Train Accuracy: 0.9070, Validation Accuracy: 0.8832, Loss: 0.1606\n", + "Epoch 9 Batch 590/1077 - Train Accuracy: 0.8339, Validation Accuracy: 0.8714, Loss: 0.1936\n", + "Epoch 9 Batch 600/1077 - Train Accuracy: 0.8839, Validation Accuracy: 0.8732, Loss: 0.1608\n", + "Epoch 9 Batch 610/1077 - Train Accuracy: 0.8598, Validation Accuracy: 0.8764, Loss: 0.1942\n", + "Epoch 9 Batch 620/1077 - Train Accuracy: 0.8953, Validation Accuracy: 0.8633, Loss: 0.1594\n", + "Epoch 9 Batch 630/1077 - Train Accuracy: 0.9141, Validation Accuracy: 0.8817, Loss: 0.1654\n", + "Epoch 9 Batch 640/1077 - Train Accuracy: 0.8616, Validation Accuracy: 0.8704, Loss: 0.1655\n", + "Epoch 9 Batch 650/1077 - Train Accuracy: 0.8770, Validation Accuracy: 0.8693, Loss: 0.1718\n", + "Epoch 9 Batch 660/1077 - Train Accuracy: 0.8777, Validation Accuracy: 0.8746, Loss: 0.1850\n", + "Epoch 9 Batch 670/1077 - Train Accuracy: 0.8629, Validation Accuracy: 0.8736, Loss: 0.1761\n", + "Epoch 9 Batch 680/1077 - Train Accuracy: 0.8761, Validation Accuracy: 0.8636, Loss: 0.1675\n", + "Epoch 9 Batch 690/1077 - Train Accuracy: 0.9082, Validation Accuracy: 0.8672, Loss: 0.1699\n", + "Epoch 9 Batch 700/1077 - Train Accuracy: 0.8922, Validation Accuracy: 0.8707, Loss: 0.1665\n", + "Epoch 9 Batch 710/1077 - Train Accuracy: 0.8645, Validation Accuracy: 0.8743, Loss: 0.1630\n", + "Epoch 9 Batch 720/1077 - Train Accuracy: 0.8725, Validation Accuracy: 0.8754, Loss: 0.1810\n", + "Epoch 9 Batch 730/1077 - Train Accuracy: 0.8770, Validation Accuracy: 0.8825, Loss: 0.1860\n", + "Epoch 9 Batch 740/1077 - Train Accuracy: 0.8875, Validation Accuracy: 0.8864, Loss: 0.1603\n", + "Epoch 9 Batch 750/1077 - Train Accuracy: 0.9266, Validation Accuracy: 0.8920, Loss: 0.1543\n", + "Epoch 9 Batch 760/1077 - Train Accuracy: 0.8695, Validation Accuracy: 0.8761, Loss: 0.1809\n", + "Epoch 9 Batch 770/1077 - Train Accuracy: 0.8731, Validation Accuracy: 0.8963, Loss: 0.1645\n", + "Epoch 9 Batch 780/1077 - Train Accuracy: 0.8457, Validation Accuracy: 0.8910, Loss: 0.1906\n", + "Epoch 9 Batch 790/1077 - Train Accuracy: 0.8375, Validation Accuracy: 0.8810, Loss: 0.1881\n", + "Epoch 9 Batch 800/1077 - Train Accuracy: 0.8867, Validation Accuracy: 0.8768, Loss: 0.1581\n", + "Epoch 9 Batch 810/1077 - Train Accuracy: 0.8929, Validation Accuracy: 0.8931, Loss: 0.1685\n", + "Epoch 9 Batch 820/1077 - Train Accuracy: 0.8531, Validation Accuracy: 0.8612, Loss: 0.1834\n", + "Epoch 9 Batch 830/1077 - Train Accuracy: 0.8738, Validation Accuracy: 0.8775, Loss: 0.1796\n", + "Epoch 9 Batch 840/1077 - Train Accuracy: 0.8898, Validation Accuracy: 0.8754, Loss: 0.1596\n", + "Epoch 9 Batch 850/1077 - Train Accuracy: 0.8519, Validation Accuracy: 0.8853, Loss: 0.1961\n", + "Epoch 9 Batch 860/1077 - Train Accuracy: 0.8899, Validation Accuracy: 0.8899, Loss: 0.1753\n", + "Epoch 9 Batch 870/1077 - Train Accuracy: 0.8433, Validation Accuracy: 0.8817, Loss: 0.1779\n", + "Epoch 9 Batch 880/1077 - Train Accuracy: 0.8984, Validation Accuracy: 0.8757, Loss: 0.1588\n", + "Epoch 9 Batch 890/1077 - Train Accuracy: 0.9096, Validation Accuracy: 0.8881, Loss: 0.1647\n", + "Epoch 9 Batch 900/1077 - Train Accuracy: 0.8910, Validation Accuracy: 0.8810, Loss: 0.1772\n", + "Epoch 9 Batch 910/1077 - Train Accuracy: 0.8966, Validation Accuracy: 0.8718, Loss: 0.1652\n", + "Epoch 9 Batch 920/1077 - Train Accuracy: 0.9066, Validation Accuracy: 0.8700, Loss: 0.1624\n", + "Epoch 9 Batch 930/1077 - Train Accuracy: 0.8910, Validation Accuracy: 0.8690, Loss: 0.1530\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 9 Batch 940/1077 - Train Accuracy: 0.9012, Validation Accuracy: 0.8754, Loss: 0.1521\n", + "Epoch 9 Batch 950/1077 - Train Accuracy: 0.8873, Validation Accuracy: 0.8825, Loss: 0.1392\n", + "Epoch 9 Batch 960/1077 - Train Accuracy: 0.8642, Validation Accuracy: 0.8828, Loss: 0.1581\n", + "Epoch 9 Batch 970/1077 - Train Accuracy: 0.9113, Validation Accuracy: 0.8750, Loss: 0.1741\n", + "Epoch 9 Batch 980/1077 - Train Accuracy: 0.8730, Validation Accuracy: 0.8711, Loss: 0.1556\n", + "Epoch 9 Batch 990/1077 - Train Accuracy: 0.8919, Validation Accuracy: 0.8825, Loss: 0.1779\n", + "Epoch 9 Batch 1000/1077 - Train Accuracy: 0.8865, Validation Accuracy: 0.8768, Loss: 0.1587\n", + "Epoch 9 Batch 1010/1077 - Train Accuracy: 0.9187, Validation Accuracy: 0.8761, Loss: 0.1586\n", + "Epoch 9 Batch 1020/1077 - Train Accuracy: 0.9105, Validation Accuracy: 0.8828, Loss: 0.1456\n", + "Epoch 9 Batch 1030/1077 - Train Accuracy: 0.8988, Validation Accuracy: 0.8828, Loss: 0.1641\n", + "Epoch 9 Batch 1040/1077 - Train Accuracy: 0.9046, Validation Accuracy: 0.8683, Loss: 0.1554\n", + "Epoch 9 Batch 1050/1077 - Train Accuracy: 0.8809, Validation Accuracy: 0.8672, Loss: 0.1650\n", + "Epoch 9 Batch 1060/1077 - Train Accuracy: 0.9051, Validation Accuracy: 0.8974, Loss: 0.1340\n", + "Epoch 9 Batch 1070/1077 - Train Accuracy: 0.9074, Validation Accuracy: 0.8924, Loss: 0.1640\n", + "Epoch 10 Batch 10/1077 - Train Accuracy: 0.9161, Validation Accuracy: 0.8690, Loss: 0.1588\n", + "Epoch 10 Batch 20/1077 - Train Accuracy: 0.9035, Validation Accuracy: 0.8931, Loss: 0.1438\n", + "Epoch 10 Batch 30/1077 - Train Accuracy: 0.8918, Validation Accuracy: 0.8874, Loss: 0.1455\n", + "Epoch 10 Batch 40/1077 - Train Accuracy: 0.9043, Validation Accuracy: 0.8857, Loss: 0.1605\n", + "Epoch 10 Batch 50/1077 - Train Accuracy: 0.9035, Validation Accuracy: 0.8835, Loss: 0.1459\n", + "Epoch 10 Batch 60/1077 - Train Accuracy: 0.8858, Validation Accuracy: 0.8857, Loss: 0.1462\n", + "Epoch 10 Batch 70/1077 - Train Accuracy: 0.8758, Validation Accuracy: 0.8796, Loss: 0.1454\n", + "Epoch 10 Batch 80/1077 - Train Accuracy: 0.8820, Validation Accuracy: 0.8988, Loss: 0.1602\n", + "Epoch 10 Batch 90/1077 - Train Accuracy: 0.8777, Validation Accuracy: 0.8903, Loss: 0.1748\n", + "Epoch 10 Batch 100/1077 - Train Accuracy: 0.8887, Validation Accuracy: 0.8949, Loss: 0.1622\n", + "Epoch 10 Batch 110/1077 - Train Accuracy: 0.9223, Validation Accuracy: 0.8931, Loss: 0.1197\n", + "Epoch 10 Batch 120/1077 - Train Accuracy: 0.9090, Validation Accuracy: 0.8974, Loss: 0.1737\n", + "Epoch 10 Batch 130/1077 - Train Accuracy: 0.8735, Validation Accuracy: 0.9052, Loss: 0.1479\n", + "Epoch 10 Batch 140/1077 - Train Accuracy: 0.9013, Validation Accuracy: 0.8913, Loss: 0.1525\n", + "Epoch 10 Batch 150/1077 - Train Accuracy: 0.8936, Validation Accuracy: 0.8906, Loss: 0.1561\n", + "Epoch 10 Batch 160/1077 - Train Accuracy: 0.8820, Validation Accuracy: 0.8864, Loss: 0.1470\n", + "Epoch 10 Batch 170/1077 - Train Accuracy: 0.9223, Validation Accuracy: 0.8878, Loss: 0.1633\n", + "Epoch 10 Batch 180/1077 - Train Accuracy: 0.8781, Validation Accuracy: 0.8942, Loss: 0.1494\n", + "Epoch 10 Batch 190/1077 - Train Accuracy: 0.9086, Validation Accuracy: 0.8718, Loss: 0.1481\n", + "Epoch 10 Batch 200/1077 - Train Accuracy: 0.8805, Validation Accuracy: 0.8732, Loss: 0.1661\n", + "Epoch 10 Batch 210/1077 - Train Accuracy: 0.9018, Validation Accuracy: 0.8828, Loss: 0.1582\n", + "Epoch 10 Batch 220/1077 - Train Accuracy: 0.8988, Validation Accuracy: 0.8999, Loss: 0.1430\n", + "Epoch 10 Batch 230/1077 - Train Accuracy: 0.9107, Validation Accuracy: 0.8977, Loss: 0.1415\n", + "Epoch 10 Batch 240/1077 - Train Accuracy: 0.9445, Validation Accuracy: 0.8995, Loss: 0.1403\n", + "Epoch 10 Batch 250/1077 - Train Accuracy: 0.9240, Validation Accuracy: 0.9016, Loss: 0.1240\n", + "Epoch 10 Batch 260/1077 - Train Accuracy: 0.9044, Validation Accuracy: 0.8647, Loss: 0.1326\n", + "Epoch 10 Batch 270/1077 - Train Accuracy: 0.8781, Validation Accuracy: 0.8835, Loss: 0.1657\n", + "Epoch 10 Batch 280/1077 - Train Accuracy: 0.8992, Validation Accuracy: 0.8771, Loss: 0.1496\n", + "Epoch 10 Batch 290/1077 - Train Accuracy: 0.8867, Validation Accuracy: 0.8970, Loss: 0.1783\n", + "Epoch 10 Batch 300/1077 - Train Accuracy: 0.9544, Validation Accuracy: 0.8935, Loss: 0.1443\n", + "Epoch 10 Batch 310/1077 - Train Accuracy: 0.8918, Validation Accuracy: 0.8903, Loss: 0.1454\n", + "Epoch 10 Batch 320/1077 - Train Accuracy: 0.9105, Validation Accuracy: 0.8885, Loss: 0.1593\n", + "Epoch 10 Batch 330/1077 - Train Accuracy: 0.8895, Validation Accuracy: 0.8874, Loss: 0.1623\n", + "Epoch 10 Batch 340/1077 - Train Accuracy: 0.9276, Validation Accuracy: 0.8956, Loss: 0.1538\n", + "Epoch 10 Batch 350/1077 - Train Accuracy: 0.8996, Validation Accuracy: 0.8928, Loss: 0.1429\n", + "Epoch 10 Batch 360/1077 - Train Accuracy: 0.9027, Validation Accuracy: 0.8896, Loss: 0.1559\n", + "Epoch 10 Batch 370/1077 - Train Accuracy: 0.9263, Validation Accuracy: 0.8977, Loss: 0.1461\n", + "Epoch 10 Batch 380/1077 - Train Accuracy: 0.9152, Validation Accuracy: 0.9002, Loss: 0.1386\n", + "Epoch 10 Batch 390/1077 - Train Accuracy: 0.8695, Validation Accuracy: 0.9038, Loss: 0.1604\n", + "Epoch 10 Batch 400/1077 - Train Accuracy: 0.9184, Validation Accuracy: 0.8935, Loss: 0.1493\n", + "Epoch 10 Batch 410/1077 - Train Accuracy: 0.8972, Validation Accuracy: 0.9091, Loss: 0.1590\n", + "Epoch 10 Batch 420/1077 - Train Accuracy: 0.9434, Validation Accuracy: 0.8878, Loss: 0.1318\n", + "Epoch 10 Batch 430/1077 - Train Accuracy: 0.9023, Validation Accuracy: 0.8917, Loss: 0.1352\n", + "Epoch 10 Batch 440/1077 - Train Accuracy: 0.8949, Validation Accuracy: 0.8956, Loss: 0.1628\n", + "Epoch 10 Batch 450/1077 - Train Accuracy: 0.9199, Validation Accuracy: 0.8803, Loss: 0.1574\n", + "Epoch 10 Batch 460/1077 - Train Accuracy: 0.8945, Validation Accuracy: 0.8711, Loss: 0.1523\n", + "Epoch 10 Batch 470/1077 - Train Accuracy: 0.8997, Validation Accuracy: 0.8920, Loss: 0.1591\n", + "Epoch 10 Batch 480/1077 - Train Accuracy: 0.9219, Validation Accuracy: 0.8942, Loss: 0.1335\n", + "Epoch 10 Batch 490/1077 - Train Accuracy: 0.9152, Validation Accuracy: 0.9013, Loss: 0.1380\n", + "Epoch 10 Batch 500/1077 - Train Accuracy: 0.9234, Validation Accuracy: 0.8981, Loss: 0.1236\n", + "Epoch 10 Batch 510/1077 - Train Accuracy: 0.8766, Validation Accuracy: 0.8991, Loss: 0.1322\n", + "Epoch 10 Batch 520/1077 - Train Accuracy: 0.8984, Validation Accuracy: 0.8778, Loss: 0.1390\n", + "Epoch 10 Batch 530/1077 - Train Accuracy: 0.9055, Validation Accuracy: 0.8814, Loss: 0.1477\n", + "Epoch 10 Batch 540/1077 - Train Accuracy: 0.9184, Validation Accuracy: 0.8839, Loss: 0.1374\n", + "Epoch 10 Batch 550/1077 - Train Accuracy: 0.8785, Validation Accuracy: 0.8832, Loss: 0.1522\n", + "Epoch 10 Batch 560/1077 - Train Accuracy: 0.8949, Validation Accuracy: 0.8913, Loss: 0.1298\n", + "Epoch 10 Batch 570/1077 - Train Accuracy: 0.8840, Validation Accuracy: 0.8906, Loss: 0.1460\n", + "Epoch 10 Batch 580/1077 - Train Accuracy: 0.9263, Validation Accuracy: 0.9031, Loss: 0.1284\n", + "Epoch 10 Batch 590/1077 - Train Accuracy: 0.8618, Validation Accuracy: 0.8828, Loss: 0.1568\n", + "Epoch 10 Batch 600/1077 - Train Accuracy: 0.9126, Validation Accuracy: 0.9002, Loss: 0.1356\n", + "Epoch 10 Batch 610/1077 - Train Accuracy: 0.8812, Validation Accuracy: 0.9062, Loss: 0.1695\n", + "Epoch 10 Batch 620/1077 - Train Accuracy: 0.9098, Validation Accuracy: 0.9020, Loss: 0.1318\n", + "Epoch 10 Batch 630/1077 - Train Accuracy: 0.9461, Validation Accuracy: 0.8896, Loss: 0.1318\n", + "Epoch 10 Batch 640/1077 - Train Accuracy: 0.8761, Validation Accuracy: 0.8981, Loss: 0.1358\n", + "Epoch 10 Batch 650/1077 - Train Accuracy: 0.9223, Validation Accuracy: 0.9094, Loss: 0.1343\n", + "Epoch 10 Batch 660/1077 - Train Accuracy: 0.8859, Validation Accuracy: 0.8956, Loss: 0.1501\n", + "Epoch 10 Batch 670/1077 - Train Accuracy: 0.8871, Validation Accuracy: 0.8903, Loss: 0.1506\n", + "Epoch 10 Batch 680/1077 - Train Accuracy: 0.9025, Validation Accuracy: 0.8974, Loss: 0.1251\n", + "Epoch 10 Batch 690/1077 - Train Accuracy: 0.9129, Validation Accuracy: 0.9048, Loss: 0.1387\n", + "Epoch 10 Batch 700/1077 - Train Accuracy: 0.9254, Validation Accuracy: 0.9023, Loss: 0.1373\n", + "Epoch 10 Batch 710/1077 - Train Accuracy: 0.9070, Validation Accuracy: 0.9070, Loss: 0.1362\n", + "Epoch 10 Batch 720/1077 - Train Accuracy: 0.9132, Validation Accuracy: 0.9098, Loss: 0.1438\n", + "Epoch 10 Batch 730/1077 - Train Accuracy: 0.8988, Validation Accuracy: 0.8963, Loss: 0.1683\n", + "Epoch 10 Batch 740/1077 - Train Accuracy: 0.9113, Validation Accuracy: 0.9016, Loss: 0.1279\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 10 Batch 750/1077 - Train Accuracy: 0.9219, Validation Accuracy: 0.8988, Loss: 0.1209\n", + "Epoch 10 Batch 760/1077 - Train Accuracy: 0.9035, Validation Accuracy: 0.8960, Loss: 0.1502\n", + "Epoch 10 Batch 770/1077 - Train Accuracy: 0.8903, Validation Accuracy: 0.9098, Loss: 0.1291\n", + "Epoch 10 Batch 780/1077 - Train Accuracy: 0.8836, Validation Accuracy: 0.8984, Loss: 0.1669\n", + "Epoch 10 Batch 790/1077 - Train Accuracy: 0.8426, Validation Accuracy: 0.9023, Loss: 0.1450\n", + "Epoch 10 Batch 800/1077 - Train Accuracy: 0.9242, Validation Accuracy: 0.9084, Loss: 0.1278\n", + "Epoch 10 Batch 810/1077 - Train Accuracy: 0.9074, Validation Accuracy: 0.9102, Loss: 0.1299\n", + "Epoch 10 Batch 820/1077 - Train Accuracy: 0.8891, Validation Accuracy: 0.9048, Loss: 0.1447\n", + "Epoch 10 Batch 830/1077 - Train Accuracy: 0.8941, Validation Accuracy: 0.8896, Loss: 0.1462\n", + "Epoch 10 Batch 840/1077 - Train Accuracy: 0.9148, Validation Accuracy: 0.8981, Loss: 0.1177\n", + "Epoch 10 Batch 850/1077 - Train Accuracy: 0.8925, Validation Accuracy: 0.9087, Loss: 0.1529\n", + "Epoch 10 Batch 860/1077 - Train Accuracy: 0.9297, Validation Accuracy: 0.9084, Loss: 0.1375\n", + "Epoch 10 Batch 870/1077 - Train Accuracy: 0.8968, Validation Accuracy: 0.8956, Loss: 0.1431\n", + "Epoch 10 Batch 880/1077 - Train Accuracy: 0.9352, Validation Accuracy: 0.8924, Loss: 0.1345\n", + "Epoch 10 Batch 890/1077 - Train Accuracy: 0.9353, Validation Accuracy: 0.8995, Loss: 0.1407\n", + "Epoch 10 Batch 900/1077 - Train Accuracy: 0.9055, Validation Accuracy: 0.9148, Loss: 0.1406\n", + "Epoch 10 Batch 910/1077 - Train Accuracy: 0.9141, Validation Accuracy: 0.8974, Loss: 0.1256\n", + "Epoch 10 Batch 920/1077 - Train Accuracy: 0.9219, Validation Accuracy: 0.8928, Loss: 0.1432\n", + "Epoch 10 Batch 930/1077 - Train Accuracy: 0.9023, Validation Accuracy: 0.9023, Loss: 0.1217\n", + "Epoch 10 Batch 940/1077 - Train Accuracy: 0.9098, Validation Accuracy: 0.9045, Loss: 0.1223\n", + "Epoch 10 Batch 950/1077 - Train Accuracy: 0.9301, Validation Accuracy: 0.9187, Loss: 0.1120\n", + "Epoch 10 Batch 960/1077 - Train Accuracy: 0.9100, Validation Accuracy: 0.8849, Loss: 0.1284\n", + "Epoch 10 Batch 970/1077 - Train Accuracy: 0.9266, Validation Accuracy: 0.8963, Loss: 0.1552\n", + "Epoch 10 Batch 980/1077 - Train Accuracy: 0.8930, Validation Accuracy: 0.9027, Loss: 0.1487\n", + "Epoch 10 Batch 990/1077 - Train Accuracy: 0.9182, Validation Accuracy: 0.8935, Loss: 0.1457\n", + "Epoch 10 Batch 1000/1077 - Train Accuracy: 0.9036, Validation Accuracy: 0.8952, Loss: 0.1252\n", + "Epoch 10 Batch 1010/1077 - Train Accuracy: 0.9320, Validation Accuracy: 0.8920, Loss: 0.1388\n", + "Epoch 10 Batch 1020/1077 - Train Accuracy: 0.9031, Validation Accuracy: 0.9048, Loss: 0.1201\n", + "Epoch 10 Batch 1030/1077 - Train Accuracy: 0.9141, Validation Accuracy: 0.8981, Loss: 0.1585\n", + "Epoch 10 Batch 1040/1077 - Train Accuracy: 0.9174, Validation Accuracy: 0.8910, Loss: 0.1424\n", + "Epoch 10 Batch 1050/1077 - Train Accuracy: 0.9258, Validation Accuracy: 0.8935, Loss: 0.1178\n", + "Epoch 10 Batch 1060/1077 - Train Accuracy: 0.9160, Validation Accuracy: 0.9002, Loss: 0.1064\n", + "Epoch 10 Batch 1070/1077 - Train Accuracy: 0.9102, Validation Accuracy: 0.9062, Loss: 0.1244\n", + "Epoch 11 Batch 10/1077 - Train Accuracy: 0.9235, Validation Accuracy: 0.9055, Loss: 0.1254\n", + "Epoch 11 Batch 20/1077 - Train Accuracy: 0.9098, Validation Accuracy: 0.9087, Loss: 0.1194\n", + "Epoch 11 Batch 30/1077 - Train Accuracy: 0.9297, Validation Accuracy: 0.9109, Loss: 0.1133\n", + "Epoch 11 Batch 40/1077 - Train Accuracy: 0.9234, Validation Accuracy: 0.9176, Loss: 0.1254\n", + "Epoch 11 Batch 50/1077 - Train Accuracy: 0.9176, Validation Accuracy: 0.9016, Loss: 0.1271\n", + "Epoch 11 Batch 60/1077 - Train Accuracy: 0.9256, Validation Accuracy: 0.9073, Loss: 0.1132\n", + "Epoch 11 Batch 70/1077 - Train Accuracy: 0.8931, Validation Accuracy: 0.8952, Loss: 0.1257\n", + "Epoch 11 Batch 80/1077 - Train Accuracy: 0.9090, Validation Accuracy: 0.9016, Loss: 0.1177\n", + "Epoch 11 Batch 90/1077 - Train Accuracy: 0.8832, Validation Accuracy: 0.9006, Loss: 0.1256\n", + "Epoch 11 Batch 100/1077 - Train Accuracy: 0.9109, Validation Accuracy: 0.9055, Loss: 0.1196\n", + "Epoch 11 Batch 110/1077 - Train Accuracy: 0.9371, Validation Accuracy: 0.9038, Loss: 0.1060\n", + "Epoch 11 Batch 120/1077 - Train Accuracy: 0.9367, Validation Accuracy: 0.9077, Loss: 0.1424\n", + "Epoch 11 Batch 130/1077 - Train Accuracy: 0.8999, Validation Accuracy: 0.8942, Loss: 0.1161\n", + "Epoch 11 Batch 140/1077 - Train Accuracy: 0.9141, Validation Accuracy: 0.8995, Loss: 0.1353\n", + "Epoch 11 Batch 150/1077 - Train Accuracy: 0.9111, Validation Accuracy: 0.9084, Loss: 0.1320\n", + "Epoch 11 Batch 160/1077 - Train Accuracy: 0.9027, Validation Accuracy: 0.9119, Loss: 0.1167\n", + "Epoch 11 Batch 170/1077 - Train Accuracy: 0.9066, Validation Accuracy: 0.9165, Loss: 0.1296\n", + "Epoch 11 Batch 180/1077 - Train Accuracy: 0.9156, Validation Accuracy: 0.9059, Loss: 0.1165\n", + "Epoch 11 Batch 190/1077 - Train Accuracy: 0.9270, Validation Accuracy: 0.9055, Loss: 0.1266\n", + "Epoch 11 Batch 200/1077 - Train Accuracy: 0.8844, Validation Accuracy: 0.8995, Loss: 0.1404\n", + "Epoch 11 Batch 210/1077 - Train Accuracy: 0.9096, Validation Accuracy: 0.9059, Loss: 0.1424\n", + "Epoch 11 Batch 220/1077 - Train Accuracy: 0.9305, Validation Accuracy: 0.8857, Loss: 0.1194\n", + "Epoch 11 Batch 230/1077 - Train Accuracy: 0.9040, Validation Accuracy: 0.8967, Loss: 0.1286\n", + "Epoch 11 Batch 240/1077 - Train Accuracy: 0.9410, Validation Accuracy: 0.9169, Loss: 0.1145\n", + "Epoch 11 Batch 250/1077 - Train Accuracy: 0.9347, Validation Accuracy: 0.9134, Loss: 0.1149\n", + "Epoch 11 Batch 260/1077 - Train Accuracy: 0.9163, Validation Accuracy: 0.8974, Loss: 0.1128\n", + "Epoch 11 Batch 270/1077 - Train Accuracy: 0.9012, Validation Accuracy: 0.9055, Loss: 0.1261\n", + "Epoch 11 Batch 280/1077 - Train Accuracy: 0.9059, Validation Accuracy: 0.8970, Loss: 0.1389\n", + "Epoch 11 Batch 290/1077 - Train Accuracy: 0.9105, Validation Accuracy: 0.9077, Loss: 0.1463\n", + "Epoch 11 Batch 300/1077 - Train Accuracy: 0.9293, Validation Accuracy: 0.9062, Loss: 0.1004\n", + "Epoch 11 Batch 310/1077 - Train Accuracy: 0.9008, Validation Accuracy: 0.9070, Loss: 0.1263\n", + "Epoch 11 Batch 320/1077 - Train Accuracy: 0.9344, Validation Accuracy: 0.9055, Loss: 0.1553\n", + "Epoch 11 Batch 330/1077 - Train Accuracy: 0.8969, Validation Accuracy: 0.8885, Loss: 0.1356\n", + "Epoch 11 Batch 340/1077 - Train Accuracy: 0.9305, Validation Accuracy: 0.9151, Loss: 0.1252\n", + "Epoch 11 Batch 350/1077 - Train Accuracy: 0.9141, Validation Accuracy: 0.9087, Loss: 0.1158\n", + "Epoch 11 Batch 360/1077 - Train Accuracy: 0.9031, Validation Accuracy: 0.8995, Loss: 0.1191\n", + "Epoch 11 Batch 370/1077 - Train Accuracy: 0.9323, Validation Accuracy: 0.9084, Loss: 0.1121\n", + "Epoch 11 Batch 380/1077 - Train Accuracy: 0.9148, Validation Accuracy: 0.9006, Loss: 0.1156\n", + "Epoch 11 Batch 390/1077 - Train Accuracy: 0.8859, Validation Accuracy: 0.8899, Loss: 0.1353\n", + "Epoch 11 Batch 400/1077 - Train Accuracy: 0.9168, Validation Accuracy: 0.8885, Loss: 0.1230\n", + "Epoch 11 Batch 410/1077 - Train Accuracy: 0.8886, Validation Accuracy: 0.9062, Loss: 0.1411\n", + "Epoch 11 Batch 420/1077 - Train Accuracy: 0.9535, Validation Accuracy: 0.8960, Loss: 0.0992\n", + "Epoch 11 Batch 430/1077 - Train Accuracy: 0.9441, Validation Accuracy: 0.9094, Loss: 0.1071\n", + "Epoch 11 Batch 440/1077 - Train Accuracy: 0.9043, Validation Accuracy: 0.9180, Loss: 0.1434\n", + "Epoch 11 Batch 450/1077 - Train Accuracy: 0.9363, Validation Accuracy: 0.9084, Loss: 0.1335\n", + "Epoch 11 Batch 460/1077 - Train Accuracy: 0.9176, Validation Accuracy: 0.8899, Loss: 0.1304\n", + "Epoch 11 Batch 470/1077 - Train Accuracy: 0.9149, Validation Accuracy: 0.8942, Loss: 0.1318\n", + "Epoch 11 Batch 480/1077 - Train Accuracy: 0.9091, Validation Accuracy: 0.8984, Loss: 0.1131\n", + "Epoch 11 Batch 490/1077 - Train Accuracy: 0.9172, Validation Accuracy: 0.8867, Loss: 0.1239\n", + "Epoch 11 Batch 500/1077 - Train Accuracy: 0.9164, Validation Accuracy: 0.9002, Loss: 0.0983\n", + "Epoch 11 Batch 510/1077 - Train Accuracy: 0.8910, Validation Accuracy: 0.8977, Loss: 0.1211\n", + "Epoch 11 Batch 520/1077 - Train Accuracy: 0.9368, Validation Accuracy: 0.8832, Loss: 0.1098\n", + "Epoch 11 Batch 530/1077 - Train Accuracy: 0.9328, Validation Accuracy: 0.9062, Loss: 0.1292\n", + "Epoch 11 Batch 540/1077 - Train Accuracy: 0.9418, Validation Accuracy: 0.8910, Loss: 0.1064\n", + "Epoch 11 Batch 550/1077 - Train Accuracy: 0.8969, Validation Accuracy: 0.8991, Loss: 0.1268\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 11 Batch 560/1077 - Train Accuracy: 0.9238, Validation Accuracy: 0.8977, Loss: 0.1209\n", + "Epoch 11 Batch 570/1077 - Train Accuracy: 0.8869, Validation Accuracy: 0.9059, Loss: 0.1350\n", + "Epoch 11 Batch 580/1077 - Train Accuracy: 0.9267, Validation Accuracy: 0.9055, Loss: 0.1094\n", + "Epoch 11 Batch 590/1077 - Train Accuracy: 0.8808, Validation Accuracy: 0.8871, Loss: 0.1622\n", + "Epoch 11 Batch 600/1077 - Train Accuracy: 0.9211, Validation Accuracy: 0.8928, Loss: 0.1200\n", + "Epoch 11 Batch 610/1077 - Train Accuracy: 0.9038, Validation Accuracy: 0.8910, Loss: 0.1308\n", + "Epoch 11 Batch 620/1077 - Train Accuracy: 0.9328, Validation Accuracy: 0.8789, Loss: 0.1097\n", + "Epoch 11 Batch 630/1077 - Train Accuracy: 0.9461, Validation Accuracy: 0.8864, Loss: 0.1166\n", + "Epoch 11 Batch 640/1077 - Train Accuracy: 0.9096, Validation Accuracy: 0.8860, Loss: 0.1111\n", + "Epoch 11 Batch 650/1077 - Train Accuracy: 0.9348, Validation Accuracy: 0.9031, Loss: 0.1211\n", + "Epoch 11 Batch 660/1077 - Train Accuracy: 0.8930, Validation Accuracy: 0.9048, Loss: 0.1198\n", + "Epoch 11 Batch 670/1077 - Train Accuracy: 0.9094, Validation Accuracy: 0.9226, Loss: 0.1268\n", + "Epoch 11 Batch 680/1077 - Train Accuracy: 0.9159, Validation Accuracy: 0.9226, Loss: 0.1025\n", + "Epoch 11 Batch 690/1077 - Train Accuracy: 0.9301, Validation Accuracy: 0.9066, Loss: 0.1124\n", + "Epoch 11 Batch 700/1077 - Train Accuracy: 0.9301, Validation Accuracy: 0.8999, Loss: 0.1080\n", + "Epoch 11 Batch 710/1077 - Train Accuracy: 0.9105, Validation Accuracy: 0.9048, Loss: 0.1008\n", + "Epoch 11 Batch 720/1077 - Train Accuracy: 0.9211, Validation Accuracy: 0.8967, Loss: 0.1284\n", + "Epoch 11 Batch 730/1077 - Train Accuracy: 0.9215, Validation Accuracy: 0.8924, Loss: 0.1296\n", + "Epoch 11 Batch 740/1077 - Train Accuracy: 0.9359, Validation Accuracy: 0.8906, Loss: 0.1101\n", + "Epoch 11 Batch 750/1077 - Train Accuracy: 0.9297, Validation Accuracy: 0.8970, Loss: 0.1052\n", + "Epoch 11 Batch 760/1077 - Train Accuracy: 0.9152, Validation Accuracy: 0.8938, Loss: 0.1298\n", + "Epoch 11 Batch 770/1077 - Train Accuracy: 0.8906, Validation Accuracy: 0.9144, Loss: 0.1143\n", + "Epoch 11 Batch 780/1077 - Train Accuracy: 0.9137, Validation Accuracy: 0.9134, Loss: 0.1376\n", + "Epoch 11 Batch 790/1077 - Train Accuracy: 0.8633, Validation Accuracy: 0.9073, Loss: 0.1297\n", + "Epoch 11 Batch 800/1077 - Train Accuracy: 0.9277, Validation Accuracy: 0.9148, Loss: 0.1149\n", + "Epoch 11 Batch 810/1077 - Train Accuracy: 0.9200, Validation Accuracy: 0.9126, Loss: 0.0918\n", + "Epoch 11 Batch 820/1077 - Train Accuracy: 0.9004, Validation Accuracy: 0.9155, Loss: 0.1174\n", + "Epoch 11 Batch 830/1077 - Train Accuracy: 0.9180, Validation Accuracy: 0.9119, Loss: 0.1305\n", + "Epoch 11 Batch 840/1077 - Train Accuracy: 0.9488, Validation Accuracy: 0.9062, Loss: 0.1064\n", + "Epoch 11 Batch 850/1077 - Train Accuracy: 0.9122, Validation Accuracy: 0.9077, Loss: 0.1553\n", + "Epoch 11 Batch 860/1077 - Train Accuracy: 0.9111, Validation Accuracy: 0.9134, Loss: 0.1272\n", + "Epoch 11 Batch 870/1077 - Train Accuracy: 0.9161, Validation Accuracy: 0.9020, Loss: 0.1312\n", + "Epoch 11 Batch 880/1077 - Train Accuracy: 0.9367, Validation Accuracy: 0.9080, Loss: 0.1281\n", + "Epoch 11 Batch 890/1077 - Train Accuracy: 0.9234, Validation Accuracy: 0.8974, Loss: 0.1145\n", + "Epoch 11 Batch 900/1077 - Train Accuracy: 0.9375, Validation Accuracy: 0.9151, Loss: 0.1187\n", + "Epoch 11 Batch 910/1077 - Train Accuracy: 0.9263, Validation Accuracy: 0.9165, Loss: 0.1191\n", + "Epoch 11 Batch 920/1077 - Train Accuracy: 0.9145, Validation Accuracy: 0.9009, Loss: 0.1155\n", + "Epoch 11 Batch 930/1077 - Train Accuracy: 0.9109, Validation Accuracy: 0.9123, Loss: 0.1101\n", + "Epoch 11 Batch 940/1077 - Train Accuracy: 0.9348, Validation Accuracy: 0.9002, Loss: 0.0923\n", + "Epoch 11 Batch 950/1077 - Train Accuracy: 0.9137, Validation Accuracy: 0.9073, Loss: 0.1090\n", + "Epoch 11 Batch 960/1077 - Train Accuracy: 0.9107, Validation Accuracy: 0.9091, Loss: 0.1070\n", + "Epoch 11 Batch 970/1077 - Train Accuracy: 0.9277, Validation Accuracy: 0.8991, Loss: 0.1270\n", + "Epoch 11 Batch 980/1077 - Train Accuracy: 0.8984, Validation Accuracy: 0.8988, Loss: 0.1311\n", + "Epoch 11 Batch 990/1077 - Train Accuracy: 0.9396, Validation Accuracy: 0.9105, Loss: 0.1255\n", + "Epoch 11 Batch 1000/1077 - Train Accuracy: 0.9007, Validation Accuracy: 0.8988, Loss: 0.1138\n", + "Epoch 11 Batch 1010/1077 - Train Accuracy: 0.9395, Validation Accuracy: 0.9134, Loss: 0.1111\n", + "Epoch 11 Batch 1020/1077 - Train Accuracy: 0.9516, Validation Accuracy: 0.9119, Loss: 0.0999\n", + "Epoch 11 Batch 1030/1077 - Train Accuracy: 0.9465, Validation Accuracy: 0.9105, Loss: 0.1081\n", + "Epoch 11 Batch 1040/1077 - Train Accuracy: 0.9120, Validation Accuracy: 0.8960, Loss: 0.1279\n", + "Epoch 11 Batch 1050/1077 - Train Accuracy: 0.9266, Validation Accuracy: 0.9062, Loss: 0.1094\n", + "Epoch 11 Batch 1060/1077 - Train Accuracy: 0.9246, Validation Accuracy: 0.9009, Loss: 0.0986\n", + "Epoch 11 Batch 1070/1077 - Train Accuracy: 0.9094, Validation Accuracy: 0.9112, Loss: 0.1313\n", + "Epoch 12 Batch 10/1077 - Train Accuracy: 0.9289, Validation Accuracy: 0.9045, Loss: 0.1250\n", + "Epoch 12 Batch 20/1077 - Train Accuracy: 0.9066, Validation Accuracy: 0.8977, Loss: 0.1233\n", + "Epoch 12 Batch 30/1077 - Train Accuracy: 0.9309, Validation Accuracy: 0.9041, Loss: 0.1137\n", + "Epoch 12 Batch 40/1077 - Train Accuracy: 0.9324, Validation Accuracy: 0.8999, Loss: 0.1137\n", + "Epoch 12 Batch 50/1077 - Train Accuracy: 0.9230, Validation Accuracy: 0.9013, Loss: 0.1162\n", + "Epoch 12 Batch 60/1077 - Train Accuracy: 0.9368, Validation Accuracy: 0.9038, Loss: 0.1049\n", + "Epoch 12 Batch 70/1077 - Train Accuracy: 0.8988, Validation Accuracy: 0.9180, Loss: 0.1110\n", + "Epoch 12 Batch 80/1077 - Train Accuracy: 0.9148, Validation Accuracy: 0.9055, Loss: 0.0998\n", + "Epoch 12 Batch 90/1077 - Train Accuracy: 0.9062, Validation Accuracy: 0.9077, Loss: 0.1133\n", + "Epoch 12 Batch 100/1077 - Train Accuracy: 0.9199, Validation Accuracy: 0.9148, Loss: 0.1009\n", + "Epoch 12 Batch 110/1077 - Train Accuracy: 0.9371, Validation Accuracy: 0.9162, Loss: 0.0897\n", + "Epoch 12 Batch 120/1077 - Train Accuracy: 0.9285, Validation Accuracy: 0.9116, Loss: 0.1331\n", + "Epoch 12 Batch 130/1077 - Train Accuracy: 0.9178, Validation Accuracy: 0.9190, Loss: 0.1022\n", + "Epoch 12 Batch 140/1077 - Train Accuracy: 0.9367, Validation Accuracy: 0.9151, Loss: 0.1210\n", + "Epoch 12 Batch 150/1077 - Train Accuracy: 0.9129, Validation Accuracy: 0.9098, Loss: 0.1107\n", + "Epoch 12 Batch 160/1077 - Train Accuracy: 0.9281, Validation Accuracy: 0.9098, Loss: 0.1034\n", + "Epoch 12 Batch 170/1077 - Train Accuracy: 0.9371, Validation Accuracy: 0.9031, Loss: 0.1119\n", + "Epoch 12 Batch 180/1077 - Train Accuracy: 0.9289, Validation Accuracy: 0.9013, Loss: 0.1070\n", + "Epoch 12 Batch 190/1077 - Train Accuracy: 0.9449, Validation Accuracy: 0.9045, Loss: 0.0932\n", + "Epoch 12 Batch 200/1077 - Train Accuracy: 0.9020, Validation Accuracy: 0.9059, Loss: 0.1147\n", + "Epoch 12 Batch 210/1077 - Train Accuracy: 0.9107, Validation Accuracy: 0.9098, Loss: 0.1195\n", + "Epoch 12 Batch 220/1077 - Train Accuracy: 0.9383, Validation Accuracy: 0.9077, Loss: 0.1066\n", + "Epoch 12 Batch 230/1077 - Train Accuracy: 0.9539, Validation Accuracy: 0.9073, Loss: 0.1091\n", + "Epoch 12 Batch 240/1077 - Train Accuracy: 0.9652, Validation Accuracy: 0.9027, Loss: 0.0956\n", + "Epoch 12 Batch 250/1077 - Train Accuracy: 0.9471, Validation Accuracy: 0.8977, Loss: 0.1011\n", + "Epoch 12 Batch 260/1077 - Train Accuracy: 0.9111, Validation Accuracy: 0.9016, Loss: 0.0902\n", + "Epoch 12 Batch 270/1077 - Train Accuracy: 0.8922, Validation Accuracy: 0.9041, Loss: 0.1098\n", + "Epoch 12 Batch 280/1077 - Train Accuracy: 0.9145, Validation Accuracy: 0.8988, Loss: 0.1196\n", + "Epoch 12 Batch 290/1077 - Train Accuracy: 0.9168, Validation Accuracy: 0.8956, Loss: 0.1377\n", + "Epoch 12 Batch 300/1077 - Train Accuracy: 0.9523, Validation Accuracy: 0.8995, Loss: 0.1108\n", + "Epoch 12 Batch 310/1077 - Train Accuracy: 0.9238, Validation Accuracy: 0.8981, Loss: 0.1132\n", + "Epoch 12 Batch 320/1077 - Train Accuracy: 0.9254, Validation Accuracy: 0.9002, Loss: 0.1088\n", + "Epoch 12 Batch 330/1077 - Train Accuracy: 0.9074, Validation Accuracy: 0.9215, Loss: 0.1106\n", + "Epoch 12 Batch 340/1077 - Train Accuracy: 0.9256, Validation Accuracy: 0.9070, Loss: 0.1224\n", + "Epoch 12 Batch 350/1077 - Train Accuracy: 0.9461, Validation Accuracy: 0.9087, Loss: 0.1104\n", + "Epoch 12 Batch 360/1077 - Train Accuracy: 0.9270, Validation Accuracy: 0.9031, Loss: 0.1094\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 12 Batch 370/1077 - Train Accuracy: 0.9427, Validation Accuracy: 0.9201, Loss: 0.0999\n", + "Epoch 12 Batch 380/1077 - Train Accuracy: 0.9383, Validation Accuracy: 0.9038, Loss: 0.1027\n", + "Epoch 12 Batch 390/1077 - Train Accuracy: 0.8773, Validation Accuracy: 0.9155, Loss: 0.1103\n", + "Epoch 12 Batch 400/1077 - Train Accuracy: 0.9270, Validation Accuracy: 0.9233, Loss: 0.1086\n", + "Epoch 12 Batch 410/1077 - Train Accuracy: 0.9009, Validation Accuracy: 0.9226, Loss: 0.1229\n", + "Epoch 12 Batch 420/1077 - Train Accuracy: 0.9570, Validation Accuracy: 0.9165, Loss: 0.0826\n", + "Epoch 12 Batch 430/1077 - Train Accuracy: 0.9387, Validation Accuracy: 0.9066, Loss: 0.0914\n", + "Epoch 12 Batch 440/1077 - Train Accuracy: 0.9055, Validation Accuracy: 0.9034, Loss: 0.1306\n", + "Epoch 12 Batch 450/1077 - Train Accuracy: 0.9391, Validation Accuracy: 0.9105, Loss: 0.1103\n", + "Epoch 12 Batch 460/1077 - Train Accuracy: 0.9133, Validation Accuracy: 0.9034, Loss: 0.1177\n", + "Epoch 12 Batch 470/1077 - Train Accuracy: 0.9268, Validation Accuracy: 0.9084, Loss: 0.1064\n", + "Epoch 12 Batch 480/1077 - Train Accuracy: 0.9042, Validation Accuracy: 0.9055, Loss: 0.0918\n", + "Epoch 12 Batch 490/1077 - Train Accuracy: 0.9355, Validation Accuracy: 0.9077, Loss: 0.1161\n", + "Epoch 12 Batch 500/1077 - Train Accuracy: 0.9305, Validation Accuracy: 0.9073, Loss: 0.0811\n", + "Epoch 12 Batch 510/1077 - Train Accuracy: 0.9289, Validation Accuracy: 0.9009, Loss: 0.1049\n", + "Epoch 12 Batch 520/1077 - Train Accuracy: 0.9475, Validation Accuracy: 0.8956, Loss: 0.1049\n", + "Epoch 12 Batch 530/1077 - Train Accuracy: 0.9445, Validation Accuracy: 0.9144, Loss: 0.1013\n", + "Epoch 12 Batch 540/1077 - Train Accuracy: 0.9383, Validation Accuracy: 0.9219, Loss: 0.0927\n", + "Epoch 12 Batch 550/1077 - Train Accuracy: 0.9340, Validation Accuracy: 0.9187, Loss: 0.1071\n", + "Epoch 12 Batch 560/1077 - Train Accuracy: 0.9250, Validation Accuracy: 0.8988, Loss: 0.0979\n", + "Epoch 12 Batch 570/1077 - Train Accuracy: 0.9071, Validation Accuracy: 0.9009, Loss: 0.1208\n", + "Epoch 12 Batch 580/1077 - Train Accuracy: 0.9416, Validation Accuracy: 0.9055, Loss: 0.0924\n", + "Epoch 12 Batch 590/1077 - Train Accuracy: 0.9124, Validation Accuracy: 0.8960, Loss: 0.1125\n", + "Epoch 12 Batch 600/1077 - Train Accuracy: 0.9222, Validation Accuracy: 0.9038, Loss: 0.1004\n", + "Epoch 12 Batch 610/1077 - Train Accuracy: 0.9075, Validation Accuracy: 0.9034, Loss: 0.1222\n", + "Epoch 12 Batch 620/1077 - Train Accuracy: 0.9465, Validation Accuracy: 0.9119, Loss: 0.1002\n", + "Epoch 12 Batch 630/1077 - Train Accuracy: 0.9598, Validation Accuracy: 0.9158, Loss: 0.1095\n", + "Epoch 12 Batch 640/1077 - Train Accuracy: 0.9215, Validation Accuracy: 0.9268, Loss: 0.0977\n", + "Epoch 12 Batch 650/1077 - Train Accuracy: 0.9328, Validation Accuracy: 0.9379, Loss: 0.1094\n", + "Epoch 12 Batch 660/1077 - Train Accuracy: 0.9328, Validation Accuracy: 0.9134, Loss: 0.0952\n", + "Epoch 12 Batch 670/1077 - Train Accuracy: 0.9094, Validation Accuracy: 0.9169, Loss: 0.1057\n", + "Epoch 12 Batch 680/1077 - Train Accuracy: 0.9029, Validation Accuracy: 0.9169, Loss: 0.1048\n", + "Epoch 12 Batch 690/1077 - Train Accuracy: 0.9434, Validation Accuracy: 0.9165, Loss: 0.0921\n", + "Epoch 12 Batch 700/1077 - Train Accuracy: 0.9375, Validation Accuracy: 0.9084, Loss: 0.0916\n", + "Epoch 12 Batch 710/1077 - Train Accuracy: 0.9398, Validation Accuracy: 0.9023, Loss: 0.0899\n", + "Epoch 12 Batch 720/1077 - Train Accuracy: 0.9317, Validation Accuracy: 0.9087, Loss: 0.1198\n", + "Epoch 12 Batch 730/1077 - Train Accuracy: 0.9398, Validation Accuracy: 0.9073, Loss: 0.1357\n", + "Epoch 12 Batch 740/1077 - Train Accuracy: 0.9391, Validation Accuracy: 0.9094, Loss: 0.0912\n", + "Epoch 12 Batch 750/1077 - Train Accuracy: 0.9242, Validation Accuracy: 0.9105, Loss: 0.0983\n", + "Epoch 12 Batch 760/1077 - Train Accuracy: 0.9273, Validation Accuracy: 0.8960, Loss: 0.1125\n", + "Epoch 12 Batch 770/1077 - Train Accuracy: 0.9040, Validation Accuracy: 0.9130, Loss: 0.1102\n", + "Epoch 12 Batch 780/1077 - Train Accuracy: 0.9172, Validation Accuracy: 0.9240, Loss: 0.1270\n", + "Epoch 12 Batch 790/1077 - Train Accuracy: 0.8613, Validation Accuracy: 0.9183, Loss: 0.0940\n", + "Epoch 12 Batch 800/1077 - Train Accuracy: 0.9344, Validation Accuracy: 0.9226, Loss: 0.0898\n", + "Epoch 12 Batch 810/1077 - Train Accuracy: 0.9282, Validation Accuracy: 0.9212, Loss: 0.0895\n", + "Epoch 12 Batch 820/1077 - Train Accuracy: 0.9332, Validation Accuracy: 0.9119, Loss: 0.1078\n", + "Epoch 12 Batch 830/1077 - Train Accuracy: 0.9113, Validation Accuracy: 0.9073, Loss: 0.1049\n", + "Epoch 12 Batch 840/1077 - Train Accuracy: 0.9566, Validation Accuracy: 0.9062, Loss: 0.0849\n", + "Epoch 12 Batch 850/1077 - Train Accuracy: 0.9156, Validation Accuracy: 0.9254, Loss: 0.1245\n", + "Epoch 12 Batch 860/1077 - Train Accuracy: 0.9334, Validation Accuracy: 0.9229, Loss: 0.1089\n", + "Epoch 12 Batch 870/1077 - Train Accuracy: 0.9256, Validation Accuracy: 0.8970, Loss: 0.1052\n", + "Epoch 12 Batch 880/1077 - Train Accuracy: 0.9402, Validation Accuracy: 0.9102, Loss: 0.1024\n", + "Epoch 12 Batch 890/1077 - Train Accuracy: 0.9457, Validation Accuracy: 0.9130, Loss: 0.1105\n", + "Epoch 12 Batch 900/1077 - Train Accuracy: 0.9441, Validation Accuracy: 0.9251, Loss: 0.1062\n", + "Epoch 12 Batch 910/1077 - Train Accuracy: 0.9319, Validation Accuracy: 0.9151, Loss: 0.0983\n", + "Epoch 12 Batch 920/1077 - Train Accuracy: 0.9250, Validation Accuracy: 0.9055, Loss: 0.1002\n", + "Epoch 12 Batch 930/1077 - Train Accuracy: 0.9328, Validation Accuracy: 0.9183, Loss: 0.0923\n", + "Epoch 12 Batch 940/1077 - Train Accuracy: 0.9273, Validation Accuracy: 0.9279, Loss: 0.0853\n", + "Epoch 12 Batch 950/1077 - Train Accuracy: 0.9356, Validation Accuracy: 0.9091, Loss: 0.0823\n", + "Epoch 12 Batch 960/1077 - Train Accuracy: 0.9349, Validation Accuracy: 0.9134, Loss: 0.0893\n", + "Epoch 12 Batch 970/1077 - Train Accuracy: 0.9305, Validation Accuracy: 0.9091, Loss: 0.1074\n", + "Epoch 12 Batch 980/1077 - Train Accuracy: 0.9195, Validation Accuracy: 0.8995, Loss: 0.1072\n", + "Epoch 12 Batch 990/1077 - Train Accuracy: 0.9597, Validation Accuracy: 0.9084, Loss: 0.0951\n", + "Epoch 12 Batch 1000/1077 - Train Accuracy: 0.9174, Validation Accuracy: 0.9130, Loss: 0.1023\n", + "Epoch 12 Batch 1010/1077 - Train Accuracy: 0.9555, Validation Accuracy: 0.9219, Loss: 0.0974\n", + "Epoch 12 Batch 1020/1077 - Train Accuracy: 0.9543, Validation Accuracy: 0.9123, Loss: 0.0807\n", + "Epoch 12 Batch 1030/1077 - Train Accuracy: 0.9594, Validation Accuracy: 0.9109, Loss: 0.0824\n", + "Epoch 12 Batch 1040/1077 - Train Accuracy: 0.9494, Validation Accuracy: 0.9087, Loss: 0.1069\n", + "Epoch 12 Batch 1050/1077 - Train Accuracy: 0.9488, Validation Accuracy: 0.9109, Loss: 0.0876\n", + "Epoch 12 Batch 1060/1077 - Train Accuracy: 0.9215, Validation Accuracy: 0.9229, Loss: 0.0821\n", + "Epoch 12 Batch 1070/1077 - Train Accuracy: 0.9402, Validation Accuracy: 0.9187, Loss: 0.0929\n", + "Epoch 13 Batch 10/1077 - Train Accuracy: 0.9441, Validation Accuracy: 0.8885, Loss: 0.1078\n", + "Epoch 13 Batch 20/1077 - Train Accuracy: 0.9402, Validation Accuracy: 0.9034, Loss: 0.0833\n", + "Epoch 13 Batch 30/1077 - Train Accuracy: 0.9410, Validation Accuracy: 0.9094, Loss: 0.0879\n", + "Epoch 13 Batch 40/1077 - Train Accuracy: 0.9563, Validation Accuracy: 0.9087, Loss: 0.1104\n", + "Epoch 13 Batch 50/1077 - Train Accuracy: 0.9496, Validation Accuracy: 0.9112, Loss: 0.0821\n", + "Epoch 13 Batch 60/1077 - Train Accuracy: 0.9408, Validation Accuracy: 0.9112, Loss: 0.0888\n", + "Epoch 13 Batch 70/1077 - Train Accuracy: 0.9013, Validation Accuracy: 0.9038, Loss: 0.0987\n", + "Epoch 13 Batch 80/1077 - Train Accuracy: 0.9293, Validation Accuracy: 0.9020, Loss: 0.0867\n", + "Epoch 13 Batch 90/1077 - Train Accuracy: 0.9121, Validation Accuracy: 0.9222, Loss: 0.0906\n", + "Epoch 13 Batch 100/1077 - Train Accuracy: 0.9387, Validation Accuracy: 0.9293, Loss: 0.0954\n", + "Epoch 13 Batch 110/1077 - Train Accuracy: 0.9598, Validation Accuracy: 0.9190, Loss: 0.0803\n", + "Epoch 13 Batch 120/1077 - Train Accuracy: 0.9250, Validation Accuracy: 0.9304, Loss: 0.1079\n", + "Epoch 13 Batch 130/1077 - Train Accuracy: 0.9286, Validation Accuracy: 0.9183, Loss: 0.0907\n", + "Epoch 13 Batch 140/1077 - Train Accuracy: 0.9408, Validation Accuracy: 0.9130, Loss: 0.0839\n", + "Epoch 13 Batch 150/1077 - Train Accuracy: 0.9438, Validation Accuracy: 0.9144, Loss: 0.1026\n", + "Epoch 13 Batch 160/1077 - Train Accuracy: 0.9352, Validation Accuracy: 0.9116, Loss: 0.0901\n", + "Epoch 13 Batch 170/1077 - Train Accuracy: 0.9301, Validation Accuracy: 0.9268, Loss: 0.0944\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 13 Batch 180/1077 - Train Accuracy: 0.9469, Validation Accuracy: 0.9055, Loss: 0.0951\n", + "Epoch 13 Batch 190/1077 - Train Accuracy: 0.9480, Validation Accuracy: 0.9073, Loss: 0.0969\n", + "Epoch 13 Batch 200/1077 - Train Accuracy: 0.9082, Validation Accuracy: 0.9268, Loss: 0.1037\n", + "Epoch 13 Batch 210/1077 - Train Accuracy: 0.9252, Validation Accuracy: 0.9268, Loss: 0.1092\n", + "Epoch 13 Batch 220/1077 - Train Accuracy: 0.9408, Validation Accuracy: 0.9222, Loss: 0.0951\n", + "Epoch 13 Batch 230/1077 - Train Accuracy: 0.9565, Validation Accuracy: 0.9197, Loss: 0.0999\n", + "Epoch 13 Batch 240/1077 - Train Accuracy: 0.9598, Validation Accuracy: 0.9276, Loss: 0.0972\n", + "Epoch 13 Batch 250/1077 - Train Accuracy: 0.9474, Validation Accuracy: 0.9169, Loss: 0.0778\n", + "Epoch 13 Batch 260/1077 - Train Accuracy: 0.9182, Validation Accuracy: 0.9126, Loss: 0.0862\n", + "Epoch 13 Batch 270/1077 - Train Accuracy: 0.9016, Validation Accuracy: 0.9126, Loss: 0.1014\n", + "Epoch 13 Batch 280/1077 - Train Accuracy: 0.9250, Validation Accuracy: 0.9013, Loss: 0.0989\n", + "Epoch 13 Batch 290/1077 - Train Accuracy: 0.9215, Validation Accuracy: 0.9144, Loss: 0.1260\n", + "Epoch 13 Batch 300/1077 - Train Accuracy: 0.9572, Validation Accuracy: 0.9084, Loss: 0.0875\n", + "Epoch 13 Batch 310/1077 - Train Accuracy: 0.9336, Validation Accuracy: 0.9197, Loss: 0.0859\n", + "Epoch 13 Batch 320/1077 - Train Accuracy: 0.9535, Validation Accuracy: 0.9148, Loss: 0.0965\n", + "Epoch 13 Batch 330/1077 - Train Accuracy: 0.9246, Validation Accuracy: 0.9187, Loss: 0.0910\n", + "Epoch 13 Batch 340/1077 - Train Accuracy: 0.9609, Validation Accuracy: 0.9247, Loss: 0.0826\n", + "Epoch 13 Batch 350/1077 - Train Accuracy: 0.9340, Validation Accuracy: 0.9176, Loss: 0.0903\n", + "Epoch 13 Batch 360/1077 - Train Accuracy: 0.9324, Validation Accuracy: 0.9197, Loss: 0.0838\n", + "Epoch 13 Batch 370/1077 - Train Accuracy: 0.9423, Validation Accuracy: 0.9258, Loss: 0.0811\n", + "Epoch 13 Batch 380/1077 - Train Accuracy: 0.9305, Validation Accuracy: 0.9066, Loss: 0.0790\n", + "Epoch 13 Batch 390/1077 - Train Accuracy: 0.8898, Validation Accuracy: 0.9155, Loss: 0.1012\n", + "Epoch 13 Batch 400/1077 - Train Accuracy: 0.9387, Validation Accuracy: 0.9279, Loss: 0.1016\n", + "Epoch 13 Batch 410/1077 - Train Accuracy: 0.9157, Validation Accuracy: 0.9055, Loss: 0.1046\n", + "Epoch 13 Batch 420/1077 - Train Accuracy: 0.9691, Validation Accuracy: 0.9148, Loss: 0.0789\n", + "Epoch 13 Batch 430/1077 - Train Accuracy: 0.9461, Validation Accuracy: 0.9151, Loss: 0.0903\n", + "Epoch 13 Batch 440/1077 - Train Accuracy: 0.9078, Validation Accuracy: 0.9215, Loss: 0.1117\n", + "Epoch 13 Batch 450/1077 - Train Accuracy: 0.9449, Validation Accuracy: 0.8991, Loss: 0.0954\n", + "Epoch 13 Batch 460/1077 - Train Accuracy: 0.9328, Validation Accuracy: 0.9091, Loss: 0.1015\n", + "Epoch 13 Batch 470/1077 - Train Accuracy: 0.9342, Validation Accuracy: 0.8999, Loss: 0.1018\n", + "Epoch 13 Batch 480/1077 - Train Accuracy: 0.9445, Validation Accuracy: 0.9144, Loss: 0.0951\n", + "Epoch 13 Batch 490/1077 - Train Accuracy: 0.9379, Validation Accuracy: 0.9226, Loss: 0.0929\n", + "Epoch 13 Batch 500/1077 - Train Accuracy: 0.9301, Validation Accuracy: 0.9176, Loss: 0.0806\n", + "Epoch 13 Batch 510/1077 - Train Accuracy: 0.9359, Validation Accuracy: 0.9237, Loss: 0.0891\n", + "Epoch 13 Batch 520/1077 - Train Accuracy: 0.9479, Validation Accuracy: 0.9020, Loss: 0.0832\n", + "Epoch 13 Batch 530/1077 - Train Accuracy: 0.9418, Validation Accuracy: 0.9276, Loss: 0.0957\n", + "Epoch 13 Batch 540/1077 - Train Accuracy: 0.9352, Validation Accuracy: 0.9155, Loss: 0.0799\n", + "Epoch 13 Batch 550/1077 - Train Accuracy: 0.9410, Validation Accuracy: 0.9190, Loss: 0.0952\n", + "Epoch 13 Batch 560/1077 - Train Accuracy: 0.9215, Validation Accuracy: 0.9201, Loss: 0.0855\n", + "Epoch 13 Batch 570/1077 - Train Accuracy: 0.9062, Validation Accuracy: 0.9144, Loss: 0.1080\n", + "Epoch 13 Batch 580/1077 - Train Accuracy: 0.9438, Validation Accuracy: 0.9102, Loss: 0.0769\n", + "Epoch 13 Batch 590/1077 - Train Accuracy: 0.9013, Validation Accuracy: 0.9020, Loss: 0.1018\n", + "Epoch 13 Batch 600/1077 - Train Accuracy: 0.9483, Validation Accuracy: 0.9169, Loss: 0.0913\n", + "Epoch 13 Batch 610/1077 - Train Accuracy: 0.9132, Validation Accuracy: 0.9180, Loss: 0.1060\n", + "Epoch 13 Batch 620/1077 - Train Accuracy: 0.9441, Validation Accuracy: 0.9130, Loss: 0.0966\n", + "Epoch 13 Batch 630/1077 - Train Accuracy: 0.9645, Validation Accuracy: 0.9229, Loss: 0.0888\n", + "Epoch 13 Batch 640/1077 - Train Accuracy: 0.9461, Validation Accuracy: 0.9201, Loss: 0.0823\n", + "Epoch 13 Batch 650/1077 - Train Accuracy: 0.9473, Validation Accuracy: 0.9173, Loss: 0.0867\n", + "Epoch 13 Batch 660/1077 - Train Accuracy: 0.9211, Validation Accuracy: 0.9222, Loss: 0.0975\n", + "Epoch 13 Batch 670/1077 - Train Accuracy: 0.9247, Validation Accuracy: 0.9105, Loss: 0.0959\n", + "Epoch 13 Batch 680/1077 - Train Accuracy: 0.9330, Validation Accuracy: 0.9020, Loss: 0.0912\n", + "Epoch 13 Batch 690/1077 - Train Accuracy: 0.9445, Validation Accuracy: 0.9293, Loss: 0.0885\n", + "Epoch 13 Batch 700/1077 - Train Accuracy: 0.9371, Validation Accuracy: 0.9165, Loss: 0.0821\n", + "Epoch 13 Batch 710/1077 - Train Accuracy: 0.9313, Validation Accuracy: 0.9158, Loss: 0.0840\n", + "Epoch 13 Batch 720/1077 - Train Accuracy: 0.9256, Validation Accuracy: 0.9229, Loss: 0.0998\n", + "Epoch 13 Batch 730/1077 - Train Accuracy: 0.9469, Validation Accuracy: 0.9247, Loss: 0.1161\n", + "Epoch 13 Batch 740/1077 - Train Accuracy: 0.9434, Validation Accuracy: 0.9272, Loss: 0.0782\n", + "Epoch 13 Batch 750/1077 - Train Accuracy: 0.9398, Validation Accuracy: 0.8984, Loss: 0.0892\n", + "Epoch 13 Batch 760/1077 - Train Accuracy: 0.9434, Validation Accuracy: 0.9066, Loss: 0.0998\n", + "Epoch 13 Batch 770/1077 - Train Accuracy: 0.9036, Validation Accuracy: 0.9265, Loss: 0.1021\n", + "Epoch 13 Batch 780/1077 - Train Accuracy: 0.9113, Validation Accuracy: 0.9233, Loss: 0.1105\n", + "Epoch 13 Batch 790/1077 - Train Accuracy: 0.8902, Validation Accuracy: 0.9304, Loss: 0.1001\n", + "Epoch 13 Batch 800/1077 - Train Accuracy: 0.9574, Validation Accuracy: 0.9322, Loss: 0.0815\n", + "Epoch 13 Batch 810/1077 - Train Accuracy: 0.9289, Validation Accuracy: 0.9190, Loss: 0.0705\n", + "Epoch 13 Batch 820/1077 - Train Accuracy: 0.9332, Validation Accuracy: 0.9183, Loss: 0.0967\n", + "Epoch 13 Batch 830/1077 - Train Accuracy: 0.9191, Validation Accuracy: 0.9201, Loss: 0.1044\n", + "Epoch 13 Batch 840/1077 - Train Accuracy: 0.9484, Validation Accuracy: 0.9244, Loss: 0.0898\n", + "Epoch 13 Batch 850/1077 - Train Accuracy: 0.9260, Validation Accuracy: 0.9308, Loss: 0.1078\n", + "Epoch 13 Batch 860/1077 - Train Accuracy: 0.9360, Validation Accuracy: 0.9205, Loss: 0.0897\n", + "Epoch 13 Batch 870/1077 - Train Accuracy: 0.9227, Validation Accuracy: 0.9187, Loss: 0.0933\n", + "Epoch 13 Batch 880/1077 - Train Accuracy: 0.9449, Validation Accuracy: 0.9169, Loss: 0.0914\n", + "Epoch 13 Batch 890/1077 - Train Accuracy: 0.9461, Validation Accuracy: 0.9144, Loss: 0.0987\n", + "Epoch 13 Batch 900/1077 - Train Accuracy: 0.9449, Validation Accuracy: 0.9240, Loss: 0.1066\n", + "Epoch 13 Batch 910/1077 - Train Accuracy: 0.9260, Validation Accuracy: 0.9158, Loss: 0.0857\n", + "Epoch 13 Batch 920/1077 - Train Accuracy: 0.9316, Validation Accuracy: 0.9173, Loss: 0.0969\n", + "Epoch 13 Batch 930/1077 - Train Accuracy: 0.9320, Validation Accuracy: 0.9162, Loss: 0.0947\n", + "Epoch 13 Batch 940/1077 - Train Accuracy: 0.9367, Validation Accuracy: 0.9180, Loss: 0.0902\n", + "Epoch 13 Batch 950/1077 - Train Accuracy: 0.9583, Validation Accuracy: 0.9080, Loss: 0.0719\n", + "Epoch 13 Batch 960/1077 - Train Accuracy: 0.9371, Validation Accuracy: 0.9254, Loss: 0.0873\n", + "Epoch 13 Batch 970/1077 - Train Accuracy: 0.9375, Validation Accuracy: 0.9350, Loss: 0.1034\n", + "Epoch 13 Batch 980/1077 - Train Accuracy: 0.9191, Validation Accuracy: 0.9180, Loss: 0.1000\n", + "Epoch 13 Batch 990/1077 - Train Accuracy: 0.9363, Validation Accuracy: 0.9109, Loss: 0.0888\n", + "Epoch 13 Batch 1000/1077 - Train Accuracy: 0.9178, Validation Accuracy: 0.9176, Loss: 0.0990\n", + "Epoch 13 Batch 1010/1077 - Train Accuracy: 0.9508, Validation Accuracy: 0.9173, Loss: 0.0865\n", + "Epoch 13 Batch 1020/1077 - Train Accuracy: 0.9582, Validation Accuracy: 0.9222, Loss: 0.0712\n", + "Epoch 13 Batch 1030/1077 - Train Accuracy: 0.9516, Validation Accuracy: 0.9350, Loss: 0.0963\n", + "Epoch 13 Batch 1040/1077 - Train Accuracy: 0.9424, Validation Accuracy: 0.9194, Loss: 0.0954\n", + "Epoch 13 Batch 1050/1077 - Train Accuracy: 0.9418, Validation Accuracy: 0.9123, Loss: 0.0757\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 13 Batch 1060/1077 - Train Accuracy: 0.9301, Validation Accuracy: 0.9173, Loss: 0.0844\n", + "Epoch 13 Batch 1070/1077 - Train Accuracy: 0.9367, Validation Accuracy: 0.9258, Loss: 0.0787\n", + "Epoch 14 Batch 10/1077 - Train Accuracy: 0.9437, Validation Accuracy: 0.9272, Loss: 0.0885\n", + "Epoch 14 Batch 20/1077 - Train Accuracy: 0.9254, Validation Accuracy: 0.9432, Loss: 0.0903\n", + "Epoch 14 Batch 30/1077 - Train Accuracy: 0.9465, Validation Accuracy: 0.9322, Loss: 0.0782\n", + "Epoch 14 Batch 40/1077 - Train Accuracy: 0.9570, Validation Accuracy: 0.9240, Loss: 0.0827\n", + "Epoch 14 Batch 50/1077 - Train Accuracy: 0.9516, Validation Accuracy: 0.9258, Loss: 0.0826\n", + "Epoch 14 Batch 60/1077 - Train Accuracy: 0.9509, Validation Accuracy: 0.9237, Loss: 0.0886\n", + "Epoch 14 Batch 70/1077 - Train Accuracy: 0.9116, Validation Accuracy: 0.9300, Loss: 0.0828\n", + "Epoch 14 Batch 80/1077 - Train Accuracy: 0.9352, Validation Accuracy: 0.9194, Loss: 0.0721\n", + "Epoch 14 Batch 90/1077 - Train Accuracy: 0.9160, Validation Accuracy: 0.9272, Loss: 0.0863\n", + "Epoch 14 Batch 100/1077 - Train Accuracy: 0.9375, Validation Accuracy: 0.9229, Loss: 0.0995\n", + "Epoch 14 Batch 110/1077 - Train Accuracy: 0.9730, Validation Accuracy: 0.9237, Loss: 0.0731\n", + "Epoch 14 Batch 120/1077 - Train Accuracy: 0.9527, Validation Accuracy: 0.9343, Loss: 0.1183\n", + "Epoch 14 Batch 130/1077 - Train Accuracy: 0.9256, Validation Accuracy: 0.9215, Loss: 0.0794\n", + "Epoch 14 Batch 140/1077 - Train Accuracy: 0.9330, Validation Accuracy: 0.9130, Loss: 0.0894\n", + "Epoch 14 Batch 150/1077 - Train Accuracy: 0.9498, Validation Accuracy: 0.9109, Loss: 0.0816\n", + "Epoch 14 Batch 160/1077 - Train Accuracy: 0.9324, Validation Accuracy: 0.9041, Loss: 0.0722\n", + "Epoch 14 Batch 170/1077 - Train Accuracy: 0.9301, Validation Accuracy: 0.9322, Loss: 0.0814\n", + "Epoch 14 Batch 180/1077 - Train Accuracy: 0.9465, Validation Accuracy: 0.9283, Loss: 0.0820\n", + "Epoch 14 Batch 190/1077 - Train Accuracy: 0.9613, Validation Accuracy: 0.9290, Loss: 0.0833\n", + "Epoch 14 Batch 200/1077 - Train Accuracy: 0.9105, Validation Accuracy: 0.9254, Loss: 0.0924\n", + "Epoch 14 Batch 210/1077 - Train Accuracy: 0.9435, Validation Accuracy: 0.9062, Loss: 0.0919\n", + "Epoch 14 Batch 220/1077 - Train Accuracy: 0.9371, Validation Accuracy: 0.9347, Loss: 0.0997\n", + "Epoch 14 Batch 230/1077 - Train Accuracy: 0.9423, Validation Accuracy: 0.9297, Loss: 0.0835\n", + "Epoch 14 Batch 240/1077 - Train Accuracy: 0.9660, Validation Accuracy: 0.9244, Loss: 0.0777\n", + "Epoch 14 Batch 250/1077 - Train Accuracy: 0.9453, Validation Accuracy: 0.9286, Loss: 0.0821\n", + "Epoch 14 Batch 260/1077 - Train Accuracy: 0.9260, Validation Accuracy: 0.9215, Loss: 0.0675\n", + "Epoch 14 Batch 270/1077 - Train Accuracy: 0.9273, Validation Accuracy: 0.9105, Loss: 0.1076\n", + "Epoch 14 Batch 280/1077 - Train Accuracy: 0.9148, Validation Accuracy: 0.9311, Loss: 0.0845\n", + "Epoch 14 Batch 290/1077 - Train Accuracy: 0.9125, Validation Accuracy: 0.9208, Loss: 0.1250\n", + "Epoch 14 Batch 300/1077 - Train Accuracy: 0.9552, Validation Accuracy: 0.9254, Loss: 0.0749\n", + "Epoch 14 Batch 310/1077 - Train Accuracy: 0.9379, Validation Accuracy: 0.9254, Loss: 0.0941\n", + "Epoch 14 Batch 320/1077 - Train Accuracy: 0.9410, Validation Accuracy: 0.9329, Loss: 0.0767\n", + "Epoch 14 Batch 330/1077 - Train Accuracy: 0.9109, Validation Accuracy: 0.9201, Loss: 0.0886\n", + "Epoch 14 Batch 340/1077 - Train Accuracy: 0.9634, Validation Accuracy: 0.9297, Loss: 0.0800\n", + "Epoch 14 Batch 350/1077 - Train Accuracy: 0.9281, Validation Accuracy: 0.9300, Loss: 0.0780\n", + "Epoch 14 Batch 360/1077 - Train Accuracy: 0.9297, Validation Accuracy: 0.9244, Loss: 0.0799\n", + "Epoch 14 Batch 370/1077 - Train Accuracy: 0.9468, Validation Accuracy: 0.9279, Loss: 0.0823\n", + "Epoch 14 Batch 380/1077 - Train Accuracy: 0.9359, Validation Accuracy: 0.9311, Loss: 0.0901\n", + "Epoch 14 Batch 390/1077 - Train Accuracy: 0.8871, Validation Accuracy: 0.9247, Loss: 0.0964\n", + "Epoch 14 Batch 400/1077 - Train Accuracy: 0.9395, Validation Accuracy: 0.9329, Loss: 0.0953\n", + "Epoch 14 Batch 410/1077 - Train Accuracy: 0.9383, Validation Accuracy: 0.9339, Loss: 0.1032\n", + "Epoch 14 Batch 420/1077 - Train Accuracy: 0.9645, Validation Accuracy: 0.9261, Loss: 0.0781\n", + "Epoch 14 Batch 430/1077 - Train Accuracy: 0.9484, Validation Accuracy: 0.9290, Loss: 0.0686\n", + "Epoch 14 Batch 440/1077 - Train Accuracy: 0.9176, Validation Accuracy: 0.9105, Loss: 0.1065\n", + "Epoch 14 Batch 450/1077 - Train Accuracy: 0.9520, Validation Accuracy: 0.9155, Loss: 0.0894\n", + "Epoch 14 Batch 460/1077 - Train Accuracy: 0.9309, Validation Accuracy: 0.9091, Loss: 0.0970\n", + "Epoch 14 Batch 470/1077 - Train Accuracy: 0.9371, Validation Accuracy: 0.9173, Loss: 0.0817\n", + "Epoch 14 Batch 480/1077 - Train Accuracy: 0.9260, Validation Accuracy: 0.8999, Loss: 0.0930\n", + "Epoch 14 Batch 490/1077 - Train Accuracy: 0.9371, Validation Accuracy: 0.9347, Loss: 0.0900\n", + "Epoch 14 Batch 500/1077 - Train Accuracy: 0.9488, Validation Accuracy: 0.9290, Loss: 0.0824\n", + "Epoch 14 Batch 510/1077 - Train Accuracy: 0.9324, Validation Accuracy: 0.9283, Loss: 0.0839\n", + "Epoch 14 Batch 520/1077 - Train Accuracy: 0.9561, Validation Accuracy: 0.9194, Loss: 0.0921\n", + "Epoch 14 Batch 530/1077 - Train Accuracy: 0.9363, Validation Accuracy: 0.9226, Loss: 0.0946\n", + "Epoch 14 Batch 540/1077 - Train Accuracy: 0.9559, Validation Accuracy: 0.9332, Loss: 0.0838\n", + "Epoch 14 Batch 550/1077 - Train Accuracy: 0.9434, Validation Accuracy: 0.9205, Loss: 0.0826\n", + "Epoch 14 Batch 560/1077 - Train Accuracy: 0.9430, Validation Accuracy: 0.9194, Loss: 0.0723\n", + "Epoch 14 Batch 570/1077 - Train Accuracy: 0.9067, Validation Accuracy: 0.9212, Loss: 0.1070\n", + "Epoch 14 Batch 580/1077 - Train Accuracy: 0.9464, Validation Accuracy: 0.9293, Loss: 0.0712\n", + "Epoch 14 Batch 590/1077 - Train Accuracy: 0.9145, Validation Accuracy: 0.9134, Loss: 0.1038\n", + "Epoch 14 Batch 600/1077 - Train Accuracy: 0.9453, Validation Accuracy: 0.9240, Loss: 0.0897\n", + "Epoch 14 Batch 610/1077 - Train Accuracy: 0.9326, Validation Accuracy: 0.9293, Loss: 0.0928\n", + "Epoch 14 Batch 620/1077 - Train Accuracy: 0.9570, Validation Accuracy: 0.9240, Loss: 0.0785\n", + "Epoch 14 Batch 630/1077 - Train Accuracy: 0.9543, Validation Accuracy: 0.9418, Loss: 0.0813\n", + "Epoch 14 Batch 640/1077 - Train Accuracy: 0.9368, Validation Accuracy: 0.9425, Loss: 0.0740\n", + "Epoch 14 Batch 650/1077 - Train Accuracy: 0.9418, Validation Accuracy: 0.9368, Loss: 0.0821\n", + "Epoch 14 Batch 660/1077 - Train Accuracy: 0.9410, Validation Accuracy: 0.9286, Loss: 0.0826\n", + "Epoch 14 Batch 670/1077 - Train Accuracy: 0.9453, Validation Accuracy: 0.9165, Loss: 0.0905\n", + "Epoch 14 Batch 680/1077 - Train Accuracy: 0.9308, Validation Accuracy: 0.9244, Loss: 0.0799\n", + "Epoch 14 Batch 690/1077 - Train Accuracy: 0.9582, Validation Accuracy: 0.9435, Loss: 0.0783\n", + "Epoch 14 Batch 700/1077 - Train Accuracy: 0.9457, Validation Accuracy: 0.9425, Loss: 0.0832\n", + "Epoch 14 Batch 710/1077 - Train Accuracy: 0.9445, Validation Accuracy: 0.9215, Loss: 0.0693\n", + "Epoch 14 Batch 720/1077 - Train Accuracy: 0.9371, Validation Accuracy: 0.9435, Loss: 0.0939\n", + "Epoch 14 Batch 730/1077 - Train Accuracy: 0.9461, Validation Accuracy: 0.9393, Loss: 0.1097\n", + "Epoch 14 Batch 740/1077 - Train Accuracy: 0.9410, Validation Accuracy: 0.9389, Loss: 0.0652\n", + "Epoch 14 Batch 750/1077 - Train Accuracy: 0.9488, Validation Accuracy: 0.9407, Loss: 0.0776\n", + "Epoch 14 Batch 760/1077 - Train Accuracy: 0.9434, Validation Accuracy: 0.9315, Loss: 0.0869\n", + "Epoch 14 Batch 770/1077 - Train Accuracy: 0.9092, Validation Accuracy: 0.9389, Loss: 0.0920\n", + "Epoch 14 Batch 780/1077 - Train Accuracy: 0.9203, Validation Accuracy: 0.9339, Loss: 0.1242\n", + "Epoch 14 Batch 790/1077 - Train Accuracy: 0.8887, Validation Accuracy: 0.9361, Loss: 0.0945\n", + "Epoch 14 Batch 800/1077 - Train Accuracy: 0.9531, Validation Accuracy: 0.9450, Loss: 0.0831\n", + "Epoch 14 Batch 810/1077 - Train Accuracy: 0.9468, Validation Accuracy: 0.9446, Loss: 0.0668\n", + "Epoch 14 Batch 820/1077 - Train Accuracy: 0.9273, Validation Accuracy: 0.9343, Loss: 0.0776\n", + "Epoch 14 Batch 830/1077 - Train Accuracy: 0.9187, Validation Accuracy: 0.9432, Loss: 0.0944\n", + "Epoch 14 Batch 840/1077 - Train Accuracy: 0.9488, Validation Accuracy: 0.9311, Loss: 0.0723\n", + "Epoch 14 Batch 850/1077 - Train Accuracy: 0.9263, Validation Accuracy: 0.9276, Loss: 0.0981\n", + "Epoch 14 Batch 860/1077 - Train Accuracy: 0.9319, Validation Accuracy: 0.9403, Loss: 0.0907\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 14 Batch 870/1077 - Train Accuracy: 0.9293, Validation Accuracy: 0.9297, Loss: 0.1088\n", + "Epoch 14 Batch 880/1077 - Train Accuracy: 0.9480, Validation Accuracy: 0.9403, Loss: 0.0901\n", + "Epoch 14 Batch 890/1077 - Train Accuracy: 0.9572, Validation Accuracy: 0.9315, Loss: 0.0745\n", + "Epoch 14 Batch 900/1077 - Train Accuracy: 0.9375, Validation Accuracy: 0.9272, Loss: 0.0905\n", + "Epoch 14 Batch 910/1077 - Train Accuracy: 0.9286, Validation Accuracy: 0.9357, Loss: 0.0860\n", + "Epoch 14 Batch 920/1077 - Train Accuracy: 0.9398, Validation Accuracy: 0.9272, Loss: 0.0750\n", + "Epoch 14 Batch 930/1077 - Train Accuracy: 0.9340, Validation Accuracy: 0.9215, Loss: 0.0916\n", + "Epoch 14 Batch 940/1077 - Train Accuracy: 0.9441, Validation Accuracy: 0.9439, Loss: 0.0817\n", + "Epoch 14 Batch 950/1077 - Train Accuracy: 0.9542, Validation Accuracy: 0.9482, Loss: 0.0671\n", + "Epoch 14 Batch 960/1077 - Train Accuracy: 0.9416, Validation Accuracy: 0.9432, Loss: 0.0760\n", + "Epoch 14 Batch 970/1077 - Train Accuracy: 0.9469, Validation Accuracy: 0.9396, Loss: 0.0853\n", + "Epoch 14 Batch 980/1077 - Train Accuracy: 0.9277, Validation Accuracy: 0.9265, Loss: 0.0947\n", + "Epoch 14 Batch 990/1077 - Train Accuracy: 0.9400, Validation Accuracy: 0.9364, Loss: 0.0885\n", + "Epoch 14 Batch 1000/1077 - Train Accuracy: 0.9360, Validation Accuracy: 0.9368, Loss: 0.0857\n", + "Epoch 14 Batch 1010/1077 - Train Accuracy: 0.9547, Validation Accuracy: 0.9375, Loss: 0.0713\n", + "Epoch 14 Batch 1020/1077 - Train Accuracy: 0.9523, Validation Accuracy: 0.9496, Loss: 0.0724\n", + "Epoch 14 Batch 1030/1077 - Train Accuracy: 0.9563, Validation Accuracy: 0.9403, Loss: 0.0903\n", + "Epoch 14 Batch 1040/1077 - Train Accuracy: 0.9548, Validation Accuracy: 0.9300, Loss: 0.0912\n", + "Epoch 14 Batch 1050/1077 - Train Accuracy: 0.9469, Validation Accuracy: 0.9329, Loss: 0.0723\n", + "Epoch 14 Batch 1060/1077 - Train Accuracy: 0.9367, Validation Accuracy: 0.9379, Loss: 0.0704\n", + "Epoch 14 Batch 1070/1077 - Train Accuracy: 0.9391, Validation Accuracy: 0.9485, Loss: 0.0674\n", + "Epoch 15 Batch 10/1077 - Train Accuracy: 0.9420, Validation Accuracy: 0.9205, Loss: 0.0813\n", + "Epoch 15 Batch 20/1077 - Train Accuracy: 0.9535, Validation Accuracy: 0.9300, Loss: 0.0766\n", + "Epoch 15 Batch 30/1077 - Train Accuracy: 0.9359, Validation Accuracy: 0.9350, Loss: 0.0695\n", + "Epoch 15 Batch 40/1077 - Train Accuracy: 0.9551, Validation Accuracy: 0.9439, Loss: 0.0779\n", + "Epoch 15 Batch 50/1077 - Train Accuracy: 0.9664, Validation Accuracy: 0.9386, Loss: 0.0799\n", + "Epoch 15 Batch 60/1077 - Train Accuracy: 0.9494, Validation Accuracy: 0.9308, Loss: 0.0772\n", + "Epoch 15 Batch 70/1077 - Train Accuracy: 0.9046, Validation Accuracy: 0.9272, Loss: 0.0846\n", + "Epoch 15 Batch 80/1077 - Train Accuracy: 0.9477, Validation Accuracy: 0.9464, Loss: 0.0742\n", + "Epoch 15 Batch 90/1077 - Train Accuracy: 0.9301, Validation Accuracy: 0.9318, Loss: 0.0818\n", + "Epoch 15 Batch 100/1077 - Train Accuracy: 0.9320, Validation Accuracy: 0.9286, Loss: 0.0848\n", + "Epoch 15 Batch 110/1077 - Train Accuracy: 0.9691, Validation Accuracy: 0.9371, Loss: 0.0634\n", + "Epoch 15 Batch 120/1077 - Train Accuracy: 0.9441, Validation Accuracy: 0.9450, Loss: 0.0915\n", + "Epoch 15 Batch 130/1077 - Train Accuracy: 0.9353, Validation Accuracy: 0.9350, Loss: 0.0759\n", + "Epoch 15 Batch 140/1077 - Train Accuracy: 0.9383, Validation Accuracy: 0.9421, Loss: 0.0869\n", + "Epoch 15 Batch 150/1077 - Train Accuracy: 0.9446, Validation Accuracy: 0.9261, Loss: 0.0780\n", + "Epoch 15 Batch 160/1077 - Train Accuracy: 0.9344, Validation Accuracy: 0.9272, Loss: 0.0794\n", + "Epoch 15 Batch 170/1077 - Train Accuracy: 0.9418, Validation Accuracy: 0.9258, Loss: 0.0817\n", + "Epoch 15 Batch 180/1077 - Train Accuracy: 0.9492, Validation Accuracy: 0.9386, Loss: 0.0836\n", + "Epoch 15 Batch 190/1077 - Train Accuracy: 0.9586, Validation Accuracy: 0.9411, Loss: 0.0757\n", + "Epoch 15 Batch 200/1077 - Train Accuracy: 0.9129, Validation Accuracy: 0.9375, Loss: 0.0890\n", + "Epoch 15 Batch 210/1077 - Train Accuracy: 0.9368, Validation Accuracy: 0.9382, Loss: 0.0866\n", + "Epoch 15 Batch 220/1077 - Train Accuracy: 0.9482, Validation Accuracy: 0.9205, Loss: 0.0823\n", + "Epoch 15 Batch 230/1077 - Train Accuracy: 0.9542, Validation Accuracy: 0.9361, Loss: 0.0701\n", + "Epoch 15 Batch 240/1077 - Train Accuracy: 0.9688, Validation Accuracy: 0.9318, Loss: 0.0823\n", + "Epoch 15 Batch 250/1077 - Train Accuracy: 0.9379, Validation Accuracy: 0.9489, Loss: 0.0741\n", + "Epoch 15 Batch 260/1077 - Train Accuracy: 0.9408, Validation Accuracy: 0.9311, Loss: 0.0643\n", + "Epoch 15 Batch 270/1077 - Train Accuracy: 0.9145, Validation Accuracy: 0.9237, Loss: 0.0866\n", + "Epoch 15 Batch 280/1077 - Train Accuracy: 0.9273, Validation Accuracy: 0.9283, Loss: 0.0916\n", + "Epoch 15 Batch 290/1077 - Train Accuracy: 0.9133, Validation Accuracy: 0.9300, Loss: 0.1161\n", + "Epoch 15 Batch 300/1077 - Train Accuracy: 0.9568, Validation Accuracy: 0.9286, Loss: 0.0783\n", + "Epoch 15 Batch 310/1077 - Train Accuracy: 0.9453, Validation Accuracy: 0.9446, Loss: 0.0798\n", + "Epoch 15 Batch 320/1077 - Train Accuracy: 0.9496, Validation Accuracy: 0.9247, Loss: 0.0828\n", + "Epoch 15 Batch 330/1077 - Train Accuracy: 0.9187, Validation Accuracy: 0.9379, Loss: 0.0941\n", + "Epoch 15 Batch 340/1077 - Train Accuracy: 0.9564, Validation Accuracy: 0.9421, Loss: 0.0845\n", + "Epoch 15 Batch 350/1077 - Train Accuracy: 0.9340, Validation Accuracy: 0.9474, Loss: 0.0740\n", + "Epoch 15 Batch 360/1077 - Train Accuracy: 0.9336, Validation Accuracy: 0.9418, Loss: 0.0770\n", + "Epoch 15 Batch 370/1077 - Train Accuracy: 0.9457, Validation Accuracy: 0.9240, Loss: 0.0688\n", + "Epoch 15 Batch 380/1077 - Train Accuracy: 0.9453, Validation Accuracy: 0.9361, Loss: 0.0815\n", + "Epoch 15 Batch 390/1077 - Train Accuracy: 0.9016, Validation Accuracy: 0.9286, Loss: 0.0997\n", + "Epoch 15 Batch 400/1077 - Train Accuracy: 0.9352, Validation Accuracy: 0.9347, Loss: 0.0893\n", + "Epoch 15 Batch 410/1077 - Train Accuracy: 0.9153, Validation Accuracy: 0.9446, Loss: 0.0935\n", + "Epoch 15 Batch 420/1077 - Train Accuracy: 0.9641, Validation Accuracy: 0.9361, Loss: 0.0721\n", + "Epoch 15 Batch 430/1077 - Train Accuracy: 0.9523, Validation Accuracy: 0.9371, Loss: 0.0687\n", + "Epoch 15 Batch 440/1077 - Train Accuracy: 0.9266, Validation Accuracy: 0.9364, Loss: 0.0887\n", + "Epoch 15 Batch 450/1077 - Train Accuracy: 0.9426, Validation Accuracy: 0.9332, Loss: 0.0853\n", + "Epoch 15 Batch 460/1077 - Train Accuracy: 0.9379, Validation Accuracy: 0.9265, Loss: 0.0832\n", + "Epoch 15 Batch 470/1077 - Train Accuracy: 0.9498, Validation Accuracy: 0.9261, Loss: 0.0918\n", + "Epoch 15 Batch 480/1077 - Train Accuracy: 0.9375, Validation Accuracy: 0.9265, Loss: 0.0870\n", + "Epoch 15 Batch 490/1077 - Train Accuracy: 0.9328, Validation Accuracy: 0.9300, Loss: 0.0769\n", + "Epoch 15 Batch 500/1077 - Train Accuracy: 0.9422, Validation Accuracy: 0.9361, Loss: 0.0685\n", + "Epoch 15 Batch 510/1077 - Train Accuracy: 0.9363, Validation Accuracy: 0.9350, Loss: 0.0787\n", + "Epoch 15 Batch 520/1077 - Train Accuracy: 0.9654, Validation Accuracy: 0.9212, Loss: 0.0855\n", + "Epoch 15 Batch 530/1077 - Train Accuracy: 0.9656, Validation Accuracy: 0.9389, Loss: 0.0827\n", + "Epoch 15 Batch 540/1077 - Train Accuracy: 0.9621, Validation Accuracy: 0.9396, Loss: 0.0660\n", + "Epoch 15 Batch 550/1077 - Train Accuracy: 0.9480, Validation Accuracy: 0.9343, Loss: 0.0755\n", + "Epoch 15 Batch 560/1077 - Train Accuracy: 0.9246, Validation Accuracy: 0.9393, Loss: 0.0832\n", + "Epoch 15 Batch 570/1077 - Train Accuracy: 0.9412, Validation Accuracy: 0.9322, Loss: 0.0939\n", + "Epoch 15 Batch 580/1077 - Train Accuracy: 0.9382, Validation Accuracy: 0.9290, Loss: 0.0629\n", + "Epoch 15 Batch 590/1077 - Train Accuracy: 0.9239, Validation Accuracy: 0.9418, Loss: 0.0970\n", + "Epoch 15 Batch 600/1077 - Train Accuracy: 0.9278, Validation Accuracy: 0.9251, Loss: 0.0874\n", + "Epoch 15 Batch 610/1077 - Train Accuracy: 0.9272, Validation Accuracy: 0.9361, Loss: 0.0958\n", + "Epoch 15 Batch 620/1077 - Train Accuracy: 0.9480, Validation Accuracy: 0.9411, Loss: 0.0791\n", + "Epoch 15 Batch 630/1077 - Train Accuracy: 0.9699, Validation Accuracy: 0.9432, Loss: 0.0701\n", + "Epoch 15 Batch 640/1077 - Train Accuracy: 0.9319, Validation Accuracy: 0.9556, Loss: 0.0858\n", + "Epoch 15 Batch 650/1077 - Train Accuracy: 0.9441, Validation Accuracy: 0.9336, Loss: 0.0719\n", + "Epoch 15 Batch 660/1077 - Train Accuracy: 0.9207, Validation Accuracy: 0.9339, Loss: 0.0930\n", + "Epoch 15 Batch 670/1077 - Train Accuracy: 0.9325, Validation Accuracy: 0.9339, Loss: 0.0816\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 15 Batch 680/1077 - Train Accuracy: 0.9379, Validation Accuracy: 0.9453, Loss: 0.0720\n", + "Epoch 15 Batch 690/1077 - Train Accuracy: 0.9465, Validation Accuracy: 0.9478, Loss: 0.0802\n", + "Epoch 15 Batch 700/1077 - Train Accuracy: 0.9527, Validation Accuracy: 0.9521, Loss: 0.0687\n", + "Epoch 15 Batch 710/1077 - Train Accuracy: 0.9297, Validation Accuracy: 0.9379, Loss: 0.0671\n", + "Epoch 15 Batch 720/1077 - Train Accuracy: 0.9206, Validation Accuracy: 0.9442, Loss: 0.0898\n", + "Epoch 15 Batch 730/1077 - Train Accuracy: 0.9551, Validation Accuracy: 0.9513, Loss: 0.0978\n", + "Epoch 15 Batch 740/1077 - Train Accuracy: 0.9563, Validation Accuracy: 0.9538, Loss: 0.0615\n", + "Epoch 15 Batch 750/1077 - Train Accuracy: 0.9504, Validation Accuracy: 0.9482, Loss: 0.0748\n", + "Epoch 15 Batch 760/1077 - Train Accuracy: 0.9465, Validation Accuracy: 0.9315, Loss: 0.0923\n", + "Epoch 15 Batch 770/1077 - Train Accuracy: 0.9163, Validation Accuracy: 0.9343, Loss: 0.0721\n", + "Epoch 15 Batch 780/1077 - Train Accuracy: 0.9262, Validation Accuracy: 0.9350, Loss: 0.0997\n", + "Epoch 15 Batch 790/1077 - Train Accuracy: 0.9148, Validation Accuracy: 0.9187, Loss: 0.0741\n", + "Epoch 15 Batch 800/1077 - Train Accuracy: 0.9574, Validation Accuracy: 0.9428, Loss: 0.0718\n", + "Epoch 15 Batch 810/1077 - Train Accuracy: 0.9509, Validation Accuracy: 0.9535, Loss: 0.0642\n", + "Epoch 15 Batch 820/1077 - Train Accuracy: 0.9418, Validation Accuracy: 0.9460, Loss: 0.0802\n", + "Epoch 15 Batch 830/1077 - Train Accuracy: 0.9254, Validation Accuracy: 0.9396, Loss: 0.0949\n", + "Epoch 15 Batch 840/1077 - Train Accuracy: 0.9695, Validation Accuracy: 0.9428, Loss: 0.0733\n", + "Epoch 15 Batch 850/1077 - Train Accuracy: 0.9349, Validation Accuracy: 0.9283, Loss: 0.1000\n", + "Epoch 15 Batch 860/1077 - Train Accuracy: 0.9427, Validation Accuracy: 0.9379, Loss: 0.0794\n", + "Epoch 15 Batch 870/1077 - Train Accuracy: 0.9305, Validation Accuracy: 0.9368, Loss: 0.0778\n", + "Epoch 15 Batch 880/1077 - Train Accuracy: 0.9641, Validation Accuracy: 0.9336, Loss: 0.0913\n", + "Epoch 15 Batch 890/1077 - Train Accuracy: 0.9453, Validation Accuracy: 0.9389, Loss: 0.0780\n", + "Epoch 15 Batch 900/1077 - Train Accuracy: 0.9480, Validation Accuracy: 0.9439, Loss: 0.0868\n", + "Epoch 15 Batch 910/1077 - Train Accuracy: 0.9420, Validation Accuracy: 0.9322, Loss: 0.0836\n", + "Epoch 15 Batch 920/1077 - Train Accuracy: 0.9320, Validation Accuracy: 0.9478, Loss: 0.0663\n", + "Epoch 15 Batch 930/1077 - Train Accuracy: 0.9383, Validation Accuracy: 0.9361, Loss: 0.0734\n", + "Epoch 15 Batch 940/1077 - Train Accuracy: 0.9414, Validation Accuracy: 0.9421, Loss: 0.0754\n", + "Epoch 15 Batch 950/1077 - Train Accuracy: 0.9483, Validation Accuracy: 0.9442, Loss: 0.0679\n", + "Epoch 15 Batch 960/1077 - Train Accuracy: 0.9516, Validation Accuracy: 0.9478, Loss: 0.0697\n", + "Epoch 15 Batch 970/1077 - Train Accuracy: 0.9508, Validation Accuracy: 0.9375, Loss: 0.0806\n", + "Epoch 15 Batch 980/1077 - Train Accuracy: 0.9418, Validation Accuracy: 0.9482, Loss: 0.0849\n", + "Epoch 15 Batch 990/1077 - Train Accuracy: 0.9474, Validation Accuracy: 0.9411, Loss: 0.0954\n", + "Epoch 15 Batch 1000/1077 - Train Accuracy: 0.9178, Validation Accuracy: 0.9478, Loss: 0.0872\n", + "Epoch 15 Batch 1010/1077 - Train Accuracy: 0.9656, Validation Accuracy: 0.9560, Loss: 0.0847\n", + "Epoch 15 Batch 1020/1077 - Train Accuracy: 0.9508, Validation Accuracy: 0.9506, Loss: 0.0693\n", + "Epoch 15 Batch 1030/1077 - Train Accuracy: 0.9574, Validation Accuracy: 0.9517, Loss: 0.0808\n", + "Epoch 15 Batch 1040/1077 - Train Accuracy: 0.9597, Validation Accuracy: 0.9407, Loss: 0.0824\n", + "Epoch 15 Batch 1050/1077 - Train Accuracy: 0.9555, Validation Accuracy: 0.9386, Loss: 0.0654\n", + "Epoch 15 Batch 1060/1077 - Train Accuracy: 0.9609, Validation Accuracy: 0.9517, Loss: 0.0634\n", + "Epoch 15 Batch 1070/1077 - Train Accuracy: 0.9437, Validation Accuracy: 0.9403, Loss: 0.0662\n", + "Epoch 16 Batch 10/1077 - Train Accuracy: 0.9371, Validation Accuracy: 0.9276, Loss: 0.0760\n", + "Epoch 16 Batch 20/1077 - Train Accuracy: 0.9469, Validation Accuracy: 0.9343, Loss: 0.0751\n", + "Epoch 16 Batch 30/1077 - Train Accuracy: 0.9637, Validation Accuracy: 0.9396, Loss: 0.0710\n", + "Epoch 16 Batch 40/1077 - Train Accuracy: 0.9555, Validation Accuracy: 0.9450, Loss: 0.0663\n", + "Epoch 16 Batch 50/1077 - Train Accuracy: 0.9480, Validation Accuracy: 0.9457, Loss: 0.0693\n", + "Epoch 16 Batch 60/1077 - Train Accuracy: 0.9624, Validation Accuracy: 0.9393, Loss: 0.0620\n", + "Epoch 16 Batch 70/1077 - Train Accuracy: 0.9120, Validation Accuracy: 0.9464, Loss: 0.0872\n", + "Epoch 16 Batch 80/1077 - Train Accuracy: 0.9336, Validation Accuracy: 0.9368, Loss: 0.0662\n", + "Epoch 16 Batch 90/1077 - Train Accuracy: 0.9309, Validation Accuracy: 0.9414, Loss: 0.0823\n", + "Epoch 16 Batch 100/1077 - Train Accuracy: 0.9602, Validation Accuracy: 0.9361, Loss: 0.0742\n", + "Epoch 16 Batch 110/1077 - Train Accuracy: 0.9660, Validation Accuracy: 0.9556, Loss: 0.0611\n", + "Epoch 16 Batch 120/1077 - Train Accuracy: 0.9551, Validation Accuracy: 0.9446, Loss: 0.0834\n", + "Epoch 16 Batch 130/1077 - Train Accuracy: 0.9435, Validation Accuracy: 0.9411, Loss: 0.0728\n", + "Epoch 16 Batch 140/1077 - Train Accuracy: 0.9498, Validation Accuracy: 0.9396, Loss: 0.0793\n", + "Epoch 16 Batch 150/1077 - Train Accuracy: 0.9453, Validation Accuracy: 0.9389, Loss: 0.0693\n", + "Epoch 16 Batch 160/1077 - Train Accuracy: 0.9598, Validation Accuracy: 0.9510, Loss: 0.0764\n", + "Epoch 16 Batch 170/1077 - Train Accuracy: 0.9352, Validation Accuracy: 0.9418, Loss: 0.0735\n", + "Epoch 16 Batch 180/1077 - Train Accuracy: 0.9508, Validation Accuracy: 0.9311, Loss: 0.0683\n", + "Epoch 16 Batch 190/1077 - Train Accuracy: 0.9453, Validation Accuracy: 0.9336, Loss: 0.0754\n", + "Epoch 16 Batch 200/1077 - Train Accuracy: 0.9313, Validation Accuracy: 0.9361, Loss: 0.0976\n", + "Epoch 16 Batch 210/1077 - Train Accuracy: 0.9349, Validation Accuracy: 0.9347, Loss: 0.0937\n", + "Epoch 16 Batch 220/1077 - Train Accuracy: 0.9494, Validation Accuracy: 0.9482, Loss: 0.0783\n", + "Epoch 16 Batch 230/1077 - Train Accuracy: 0.9684, Validation Accuracy: 0.9379, Loss: 0.0696\n", + "Epoch 16 Batch 240/1077 - Train Accuracy: 0.9742, Validation Accuracy: 0.9521, Loss: 0.0643\n", + "Epoch 16 Batch 250/1077 - Train Accuracy: 0.9482, Validation Accuracy: 0.9428, Loss: 0.0678\n", + "Epoch 16 Batch 260/1077 - Train Accuracy: 0.9606, Validation Accuracy: 0.9418, Loss: 0.0684\n", + "Epoch 16 Batch 270/1077 - Train Accuracy: 0.9344, Validation Accuracy: 0.9329, Loss: 0.0882\n", + "Epoch 16 Batch 280/1077 - Train Accuracy: 0.9168, Validation Accuracy: 0.9304, Loss: 0.0811\n", + "Epoch 16 Batch 290/1077 - Train Accuracy: 0.9324, Validation Accuracy: 0.9421, Loss: 0.1069\n", + "Epoch 16 Batch 300/1077 - Train Accuracy: 0.9552, Validation Accuracy: 0.9467, Loss: 0.0694\n", + "Epoch 16 Batch 310/1077 - Train Accuracy: 0.9563, Validation Accuracy: 0.9457, Loss: 0.0769\n", + "Epoch 16 Batch 320/1077 - Train Accuracy: 0.9602, Validation Accuracy: 0.9375, Loss: 0.0790\n", + "Epoch 16 Batch 330/1077 - Train Accuracy: 0.9445, Validation Accuracy: 0.9418, Loss: 0.0853\n", + "Epoch 16 Batch 340/1077 - Train Accuracy: 0.9663, Validation Accuracy: 0.9439, Loss: 0.0636\n", + "Epoch 16 Batch 350/1077 - Train Accuracy: 0.9418, Validation Accuracy: 0.9446, Loss: 0.0775\n", + "Epoch 16 Batch 360/1077 - Train Accuracy: 0.9391, Validation Accuracy: 0.9428, Loss: 0.0715\n", + "Epoch 16 Batch 370/1077 - Train Accuracy: 0.9386, Validation Accuracy: 0.9403, Loss: 0.0741\n", + "Epoch 16 Batch 380/1077 - Train Accuracy: 0.9441, Validation Accuracy: 0.9407, Loss: 0.0676\n", + "Epoch 16 Batch 390/1077 - Train Accuracy: 0.9051, Validation Accuracy: 0.9386, Loss: 0.0984\n", + "Epoch 16 Batch 400/1077 - Train Accuracy: 0.9473, Validation Accuracy: 0.9329, Loss: 0.1073\n", + "Epoch 16 Batch 410/1077 - Train Accuracy: 0.9153, Validation Accuracy: 0.9304, Loss: 0.0971\n", + "Epoch 16 Batch 420/1077 - Train Accuracy: 0.9570, Validation Accuracy: 0.9343, Loss: 0.0694\n", + "Epoch 16 Batch 430/1077 - Train Accuracy: 0.9563, Validation Accuracy: 0.9450, Loss: 0.0687\n", + "Epoch 16 Batch 440/1077 - Train Accuracy: 0.9191, Validation Accuracy: 0.9496, Loss: 0.0821\n", + "Epoch 16 Batch 450/1077 - Train Accuracy: 0.9457, Validation Accuracy: 0.9279, Loss: 0.0882\n", + "Epoch 16 Batch 460/1077 - Train Accuracy: 0.9363, Validation Accuracy: 0.9489, Loss: 0.0791\n", + "Epoch 16 Batch 470/1077 - Train Accuracy: 0.9211, Validation Accuracy: 0.9343, Loss: 0.0758\n", + "Epoch 16 Batch 480/1077 - Train Accuracy: 0.9309, Validation Accuracy: 0.9357, Loss: 0.0859\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 16 Batch 490/1077 - Train Accuracy: 0.9441, Validation Accuracy: 0.9357, Loss: 0.0922\n", + "Epoch 16 Batch 500/1077 - Train Accuracy: 0.9367, Validation Accuracy: 0.9400, Loss: 0.0713\n", + "Epoch 16 Batch 510/1077 - Train Accuracy: 0.9355, Validation Accuracy: 0.9325, Loss: 0.0809\n", + "Epoch 16 Batch 520/1077 - Train Accuracy: 0.9535, Validation Accuracy: 0.9375, Loss: 0.0744\n", + "Epoch 16 Batch 530/1077 - Train Accuracy: 0.9566, Validation Accuracy: 0.9347, Loss: 0.0785\n", + "Epoch 16 Batch 540/1077 - Train Accuracy: 0.9602, Validation Accuracy: 0.9542, Loss: 0.0657\n", + "Epoch 16 Batch 550/1077 - Train Accuracy: 0.9367, Validation Accuracy: 0.9364, Loss: 0.0708\n", + "Epoch 16 Batch 560/1077 - Train Accuracy: 0.9355, Validation Accuracy: 0.9180, Loss: 0.0538\n", + "Epoch 16 Batch 570/1077 - Train Accuracy: 0.9128, Validation Accuracy: 0.9219, Loss: 0.0914\n", + "Epoch 16 Batch 580/1077 - Train Accuracy: 0.9457, Validation Accuracy: 0.9233, Loss: 0.0524\n", + "Epoch 16 Batch 590/1077 - Train Accuracy: 0.9095, Validation Accuracy: 0.9297, Loss: 0.0851\n", + "Epoch 16 Batch 600/1077 - Train Accuracy: 0.9613, Validation Accuracy: 0.9347, Loss: 0.0786\n", + "Epoch 16 Batch 610/1077 - Train Accuracy: 0.9202, Validation Accuracy: 0.9439, Loss: 0.0957\n", + "Epoch 16 Batch 620/1077 - Train Accuracy: 0.9703, Validation Accuracy: 0.9325, Loss: 0.0679\n", + "Epoch 16 Batch 630/1077 - Train Accuracy: 0.9695, Validation Accuracy: 0.9332, Loss: 0.0674\n", + "Epoch 16 Batch 640/1077 - Train Accuracy: 0.9479, Validation Accuracy: 0.9297, Loss: 0.0686\n", + "Epoch 16 Batch 650/1077 - Train Accuracy: 0.9512, Validation Accuracy: 0.9382, Loss: 0.0664\n", + "Epoch 16 Batch 660/1077 - Train Accuracy: 0.9391, Validation Accuracy: 0.9531, Loss: 0.0727\n", + "Epoch 16 Batch 670/1077 - Train Accuracy: 0.9389, Validation Accuracy: 0.9411, Loss: 0.0860\n", + "Epoch 16 Batch 680/1077 - Train Accuracy: 0.9479, Validation Accuracy: 0.9471, Loss: 0.0594\n", + "Epoch 16 Batch 690/1077 - Train Accuracy: 0.9590, Validation Accuracy: 0.9531, Loss: 0.0708\n", + "Epoch 16 Batch 700/1077 - Train Accuracy: 0.9523, Validation Accuracy: 0.9389, Loss: 0.0647\n", + "Epoch 16 Batch 710/1077 - Train Accuracy: 0.9449, Validation Accuracy: 0.9485, Loss: 0.0556\n", + "Epoch 16 Batch 720/1077 - Train Accuracy: 0.9478, Validation Accuracy: 0.9439, Loss: 0.0821\n", + "Epoch 16 Batch 730/1077 - Train Accuracy: 0.9512, Validation Accuracy: 0.9450, Loss: 0.0934\n", + "Epoch 16 Batch 740/1077 - Train Accuracy: 0.9637, Validation Accuracy: 0.9400, Loss: 0.0491\n", + "Epoch 16 Batch 750/1077 - Train Accuracy: 0.9512, Validation Accuracy: 0.9304, Loss: 0.0595\n", + "Epoch 16 Batch 760/1077 - Train Accuracy: 0.9484, Validation Accuracy: 0.9339, Loss: 0.0878\n", + "Epoch 16 Batch 770/1077 - Train Accuracy: 0.9297, Validation Accuracy: 0.9403, Loss: 0.0703\n", + "Epoch 16 Batch 780/1077 - Train Accuracy: 0.9367, Validation Accuracy: 0.9304, Loss: 0.0935\n", + "Epoch 16 Batch 790/1077 - Train Accuracy: 0.9168, Validation Accuracy: 0.9457, Loss: 0.0743\n", + "Epoch 16 Batch 800/1077 - Train Accuracy: 0.9539, Validation Accuracy: 0.9499, Loss: 0.0697\n", + "Epoch 16 Batch 810/1077 - Train Accuracy: 0.9591, Validation Accuracy: 0.9442, Loss: 0.0628\n", + "Epoch 16 Batch 820/1077 - Train Accuracy: 0.9418, Validation Accuracy: 0.9432, Loss: 0.0782\n", + "Epoch 16 Batch 830/1077 - Train Accuracy: 0.9137, Validation Accuracy: 0.9457, Loss: 0.0774\n", + "Epoch 16 Batch 840/1077 - Train Accuracy: 0.9660, Validation Accuracy: 0.9396, Loss: 0.0647\n", + "Epoch 16 Batch 850/1077 - Train Accuracy: 0.9435, Validation Accuracy: 0.9403, Loss: 0.0922\n", + "Epoch 16 Batch 860/1077 - Train Accuracy: 0.9405, Validation Accuracy: 0.9375, Loss: 0.0601\n", + "Epoch 16 Batch 870/1077 - Train Accuracy: 0.9367, Validation Accuracy: 0.9350, Loss: 0.0653\n", + "Epoch 16 Batch 880/1077 - Train Accuracy: 0.9625, Validation Accuracy: 0.9457, Loss: 0.0729\n", + "Epoch 16 Batch 890/1077 - Train Accuracy: 0.9442, Validation Accuracy: 0.9496, Loss: 0.0647\n", + "Epoch 16 Batch 900/1077 - Train Accuracy: 0.9453, Validation Accuracy: 0.9453, Loss: 0.0734\n", + "Epoch 16 Batch 910/1077 - Train Accuracy: 0.9360, Validation Accuracy: 0.9460, Loss: 0.0748\n", + "Epoch 16 Batch 920/1077 - Train Accuracy: 0.9473, Validation Accuracy: 0.9513, Loss: 0.0637\n", + "Epoch 16 Batch 930/1077 - Train Accuracy: 0.9488, Validation Accuracy: 0.9574, Loss: 0.0752\n", + "Epoch 16 Batch 940/1077 - Train Accuracy: 0.9574, Validation Accuracy: 0.9389, Loss: 0.0619\n", + "Epoch 16 Batch 950/1077 - Train Accuracy: 0.9613, Validation Accuracy: 0.9322, Loss: 0.0561\n", + "Epoch 16 Batch 960/1077 - Train Accuracy: 0.9472, Validation Accuracy: 0.9318, Loss: 0.0614\n", + "Epoch 16 Batch 970/1077 - Train Accuracy: 0.9449, Validation Accuracy: 0.9524, Loss: 0.0757\n", + "Epoch 16 Batch 980/1077 - Train Accuracy: 0.9242, Validation Accuracy: 0.9634, Loss: 0.0955\n", + "Epoch 16 Batch 990/1077 - Train Accuracy: 0.9618, Validation Accuracy: 0.9467, Loss: 0.0740\n", + "Epoch 16 Batch 1000/1077 - Train Accuracy: 0.9386, Validation Accuracy: 0.9343, Loss: 0.0808\n", + "Epoch 16 Batch 1010/1077 - Train Accuracy: 0.9691, Validation Accuracy: 0.9400, Loss: 0.0685\n", + "Epoch 16 Batch 1020/1077 - Train Accuracy: 0.9730, Validation Accuracy: 0.9439, Loss: 0.0623\n", + "Epoch 16 Batch 1030/1077 - Train Accuracy: 0.9602, Validation Accuracy: 0.9261, Loss: 0.0877\n", + "Epoch 16 Batch 1040/1077 - Train Accuracy: 0.9609, Validation Accuracy: 0.9336, Loss: 0.0669\n", + "Epoch 16 Batch 1050/1077 - Train Accuracy: 0.9500, Validation Accuracy: 0.9460, Loss: 0.0683\n", + "Epoch 16 Batch 1060/1077 - Train Accuracy: 0.9402, Validation Accuracy: 0.9407, Loss: 0.0568\n", + "Epoch 16 Batch 1070/1077 - Train Accuracy: 0.9387, Validation Accuracy: 0.9382, Loss: 0.0552\n", + "Epoch 17 Batch 10/1077 - Train Accuracy: 0.9379, Validation Accuracy: 0.9411, Loss: 0.0705\n", + "Epoch 17 Batch 20/1077 - Train Accuracy: 0.9668, Validation Accuracy: 0.9272, Loss: 0.0589\n", + "Epoch 17 Batch 30/1077 - Train Accuracy: 0.9539, Validation Accuracy: 0.9474, Loss: 0.0616\n", + "Epoch 17 Batch 40/1077 - Train Accuracy: 0.9547, Validation Accuracy: 0.9471, Loss: 0.0692\n", + "Epoch 17 Batch 50/1077 - Train Accuracy: 0.9539, Validation Accuracy: 0.9439, Loss: 0.0607\n", + "Epoch 17 Batch 60/1077 - Train Accuracy: 0.9576, Validation Accuracy: 0.9368, Loss: 0.0545\n", + "Epoch 17 Batch 70/1077 - Train Accuracy: 0.9289, Validation Accuracy: 0.9570, Loss: 0.0735\n", + "Epoch 17 Batch 80/1077 - Train Accuracy: 0.9492, Validation Accuracy: 0.9435, Loss: 0.0628\n", + "Epoch 17 Batch 90/1077 - Train Accuracy: 0.9391, Validation Accuracy: 0.9524, Loss: 0.0662\n", + "Epoch 17 Batch 100/1077 - Train Accuracy: 0.9418, Validation Accuracy: 0.9471, Loss: 0.0769\n", + "Epoch 17 Batch 110/1077 - Train Accuracy: 0.9766, Validation Accuracy: 0.9506, Loss: 0.0660\n", + "Epoch 17 Batch 120/1077 - Train Accuracy: 0.9543, Validation Accuracy: 0.9581, Loss: 0.0869\n", + "Epoch 17 Batch 130/1077 - Train Accuracy: 0.9397, Validation Accuracy: 0.9556, Loss: 0.0537\n", + "Epoch 17 Batch 140/1077 - Train Accuracy: 0.9461, Validation Accuracy: 0.9403, Loss: 0.0700\n", + "Epoch 17 Batch 150/1077 - Train Accuracy: 0.9572, Validation Accuracy: 0.9414, Loss: 0.0729\n", + "Epoch 17 Batch 160/1077 - Train Accuracy: 0.9617, Validation Accuracy: 0.9411, Loss: 0.0638\n", + "Epoch 17 Batch 170/1077 - Train Accuracy: 0.9340, Validation Accuracy: 0.9478, Loss: 0.0748\n", + "Epoch 17 Batch 180/1077 - Train Accuracy: 0.9602, Validation Accuracy: 0.9538, Loss: 0.0590\n", + "Epoch 17 Batch 190/1077 - Train Accuracy: 0.9523, Validation Accuracy: 0.9421, Loss: 0.0666\n", + "Epoch 17 Batch 200/1077 - Train Accuracy: 0.9164, Validation Accuracy: 0.9279, Loss: 0.0804\n", + "Epoch 17 Batch 210/1077 - Train Accuracy: 0.9446, Validation Accuracy: 0.9453, Loss: 0.0936\n", + "Epoch 17 Batch 220/1077 - Train Accuracy: 0.9428, Validation Accuracy: 0.9396, Loss: 0.0713\n", + "Epoch 17 Batch 230/1077 - Train Accuracy: 0.9736, Validation Accuracy: 0.9364, Loss: 0.0667\n", + "Epoch 17 Batch 240/1077 - Train Accuracy: 0.9695, Validation Accuracy: 0.9435, Loss: 0.0709\n", + "Epoch 17 Batch 250/1077 - Train Accuracy: 0.9450, Validation Accuracy: 0.9556, Loss: 0.0626\n", + "Epoch 17 Batch 260/1077 - Train Accuracy: 0.9353, Validation Accuracy: 0.9464, Loss: 0.0477\n", + "Epoch 17 Batch 270/1077 - Train Accuracy: 0.9207, Validation Accuracy: 0.9428, Loss: 0.0823\n", + "Epoch 17 Batch 280/1077 - Train Accuracy: 0.9305, Validation Accuracy: 0.9336, Loss: 0.0766\n", + "Epoch 17 Batch 290/1077 - Train Accuracy: 0.9289, Validation Accuracy: 0.9513, Loss: 0.0902\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 17 Batch 300/1077 - Train Accuracy: 0.9531, Validation Accuracy: 0.9606, Loss: 0.0624\n", + "Epoch 17 Batch 310/1077 - Train Accuracy: 0.9582, Validation Accuracy: 0.9478, Loss: 0.0671\n", + "Epoch 17 Batch 320/1077 - Train Accuracy: 0.9535, Validation Accuracy: 0.9315, Loss: 0.0682\n", + "Epoch 17 Batch 330/1077 - Train Accuracy: 0.9285, Validation Accuracy: 0.9215, Loss: 0.0740\n", + "Epoch 17 Batch 340/1077 - Train Accuracy: 0.9737, Validation Accuracy: 0.9322, Loss: 0.0699\n", + "Epoch 17 Batch 350/1077 - Train Accuracy: 0.9469, Validation Accuracy: 0.9457, Loss: 0.0672\n", + "Epoch 17 Batch 360/1077 - Train Accuracy: 0.9422, Validation Accuracy: 0.9407, Loss: 0.0708\n", + "Epoch 17 Batch 370/1077 - Train Accuracy: 0.9550, Validation Accuracy: 0.9432, Loss: 0.0689\n", + "Epoch 17 Batch 380/1077 - Train Accuracy: 0.9359, Validation Accuracy: 0.9308, Loss: 0.0684\n", + "Epoch 17 Batch 390/1077 - Train Accuracy: 0.9047, Validation Accuracy: 0.9375, Loss: 0.0805\n", + "Epoch 17 Batch 400/1077 - Train Accuracy: 0.9512, Validation Accuracy: 0.9432, Loss: 0.0741\n", + "Epoch 17 Batch 410/1077 - Train Accuracy: 0.9219, Validation Accuracy: 0.9457, Loss: 0.0908\n", + "Epoch 17 Batch 420/1077 - Train Accuracy: 0.9738, Validation Accuracy: 0.9339, Loss: 0.0656\n", + "Epoch 17 Batch 430/1077 - Train Accuracy: 0.9559, Validation Accuracy: 0.9332, Loss: 0.0586\n", + "Epoch 17 Batch 440/1077 - Train Accuracy: 0.9328, Validation Accuracy: 0.9361, Loss: 0.0806\n", + "Epoch 17 Batch 450/1077 - Train Accuracy: 0.9461, Validation Accuracy: 0.9290, Loss: 0.0789\n", + "Epoch 17 Batch 460/1077 - Train Accuracy: 0.9418, Validation Accuracy: 0.9364, Loss: 0.0734\n", + "Epoch 17 Batch 470/1077 - Train Accuracy: 0.9428, Validation Accuracy: 0.9439, Loss: 0.0738\n", + "Epoch 17 Batch 480/1077 - Train Accuracy: 0.9371, Validation Accuracy: 0.9599, Loss: 0.0765\n", + "Epoch 17 Batch 490/1077 - Train Accuracy: 0.9395, Validation Accuracy: 0.9460, Loss: 0.0690\n", + "Epoch 17 Batch 500/1077 - Train Accuracy: 0.9414, Validation Accuracy: 0.9457, Loss: 0.0739\n", + "Epoch 17 Batch 510/1077 - Train Accuracy: 0.9441, Validation Accuracy: 0.9528, Loss: 0.0737\n", + "Epoch 17 Batch 520/1077 - Train Accuracy: 0.9784, Validation Accuracy: 0.9421, Loss: 0.0595\n", + "Epoch 17 Batch 530/1077 - Train Accuracy: 0.9512, Validation Accuracy: 0.9435, Loss: 0.0656\n", + "Epoch 17 Batch 540/1077 - Train Accuracy: 0.9734, Validation Accuracy: 0.9489, Loss: 0.0481\n", + "Epoch 17 Batch 550/1077 - Train Accuracy: 0.9391, Validation Accuracy: 0.9279, Loss: 0.0749\n", + "Epoch 17 Batch 560/1077 - Train Accuracy: 0.9445, Validation Accuracy: 0.9460, Loss: 0.0844\n", + "Epoch 17 Batch 570/1077 - Train Accuracy: 0.9465, Validation Accuracy: 0.9389, Loss: 0.1033\n", + "Epoch 17 Batch 580/1077 - Train Accuracy: 0.9490, Validation Accuracy: 0.9283, Loss: 0.0593\n", + "Epoch 17 Batch 590/1077 - Train Accuracy: 0.9256, Validation Accuracy: 0.9464, Loss: 0.0771\n", + "Epoch 17 Batch 600/1077 - Train Accuracy: 0.9706, Validation Accuracy: 0.9425, Loss: 0.0591\n", + "Epoch 17 Batch 610/1077 - Train Accuracy: 0.9470, Validation Accuracy: 0.9513, Loss: 0.0884\n", + "Epoch 17 Batch 620/1077 - Train Accuracy: 0.9711, Validation Accuracy: 0.9474, Loss: 0.0699\n", + "Epoch 17 Batch 630/1077 - Train Accuracy: 0.9691, Validation Accuracy: 0.9382, Loss: 0.0717\n", + "Epoch 17 Batch 640/1077 - Train Accuracy: 0.9416, Validation Accuracy: 0.9482, Loss: 0.0870\n", + "Epoch 17 Batch 650/1077 - Train Accuracy: 0.9355, Validation Accuracy: 0.9464, Loss: 0.0805\n", + "Epoch 17 Batch 660/1077 - Train Accuracy: 0.9551, Validation Accuracy: 0.9471, Loss: 0.0664\n", + "Epoch 17 Batch 670/1077 - Train Accuracy: 0.9513, Validation Accuracy: 0.9467, Loss: 0.0749\n", + "Epoch 17 Batch 680/1077 - Train Accuracy: 0.9394, Validation Accuracy: 0.9329, Loss: 0.0628\n", + "Epoch 17 Batch 690/1077 - Train Accuracy: 0.9563, Validation Accuracy: 0.9425, Loss: 0.0699\n", + "Epoch 17 Batch 700/1077 - Train Accuracy: 0.9551, Validation Accuracy: 0.9325, Loss: 0.0726\n", + "Epoch 17 Batch 710/1077 - Train Accuracy: 0.9582, Validation Accuracy: 0.9325, Loss: 0.0644\n", + "Epoch 17 Batch 720/1077 - Train Accuracy: 0.9428, Validation Accuracy: 0.9361, Loss: 0.0883\n", + "Epoch 17 Batch 730/1077 - Train Accuracy: 0.9605, Validation Accuracy: 0.9467, Loss: 0.0843\n", + "Epoch 17 Batch 740/1077 - Train Accuracy: 0.9523, Validation Accuracy: 0.9453, Loss: 0.0567\n", + "Epoch 17 Batch 750/1077 - Train Accuracy: 0.9570, Validation Accuracy: 0.9460, Loss: 0.0660\n", + "Epoch 17 Batch 760/1077 - Train Accuracy: 0.9602, Validation Accuracy: 0.9435, Loss: 0.0795\n", + "Epoch 17 Batch 770/1077 - Train Accuracy: 0.9275, Validation Accuracy: 0.9570, Loss: 0.0640\n", + "Epoch 17 Batch 780/1077 - Train Accuracy: 0.9352, Validation Accuracy: 0.9453, Loss: 0.0942\n", + "Epoch 17 Batch 790/1077 - Train Accuracy: 0.9242, Validation Accuracy: 0.9482, Loss: 0.0836\n", + "Epoch 17 Batch 800/1077 - Train Accuracy: 0.9539, Validation Accuracy: 0.9592, Loss: 0.0602\n", + "Epoch 17 Batch 810/1077 - Train Accuracy: 0.9691, Validation Accuracy: 0.9592, Loss: 0.0569\n", + "Epoch 17 Batch 820/1077 - Train Accuracy: 0.9527, Validation Accuracy: 0.9648, Loss: 0.0676\n", + "Epoch 17 Batch 830/1077 - Train Accuracy: 0.9258, Validation Accuracy: 0.9510, Loss: 0.0851\n", + "Epoch 17 Batch 840/1077 - Train Accuracy: 0.9629, Validation Accuracy: 0.9609, Loss: 0.0618\n", + "Epoch 17 Batch 850/1077 - Train Accuracy: 0.9554, Validation Accuracy: 0.9592, Loss: 0.0916\n", + "Epoch 17 Batch 860/1077 - Train Accuracy: 0.9550, Validation Accuracy: 0.9538, Loss: 0.0656\n", + "Epoch 17 Batch 870/1077 - Train Accuracy: 0.9457, Validation Accuracy: 0.9613, Loss: 0.0625\n", + "Epoch 17 Batch 880/1077 - Train Accuracy: 0.9641, Validation Accuracy: 0.9379, Loss: 0.0772\n", + "Epoch 17 Batch 890/1077 - Train Accuracy: 0.9591, Validation Accuracy: 0.9460, Loss: 0.0712\n", + "Epoch 17 Batch 900/1077 - Train Accuracy: 0.9539, Validation Accuracy: 0.9535, Loss: 0.0794\n", + "Epoch 17 Batch 910/1077 - Train Accuracy: 0.9435, Validation Accuracy: 0.9609, Loss: 0.0781\n", + "Epoch 17 Batch 920/1077 - Train Accuracy: 0.9527, Validation Accuracy: 0.9471, Loss: 0.0716\n", + "Epoch 17 Batch 930/1077 - Train Accuracy: 0.9500, Validation Accuracy: 0.9460, Loss: 0.0595\n", + "Epoch 17 Batch 940/1077 - Train Accuracy: 0.9656, Validation Accuracy: 0.9428, Loss: 0.0589\n", + "Epoch 17 Batch 950/1077 - Train Accuracy: 0.9654, Validation Accuracy: 0.9442, Loss: 0.0568\n", + "Epoch 17 Batch 960/1077 - Train Accuracy: 0.9505, Validation Accuracy: 0.9524, Loss: 0.0721\n", + "Epoch 17 Batch 970/1077 - Train Accuracy: 0.9559, Validation Accuracy: 0.9375, Loss: 0.0737\n", + "Epoch 17 Batch 980/1077 - Train Accuracy: 0.9352, Validation Accuracy: 0.9588, Loss: 0.0794\n", + "Epoch 17 Batch 990/1077 - Train Accuracy: 0.9556, Validation Accuracy: 0.9549, Loss: 0.0743\n", + "Epoch 17 Batch 1000/1077 - Train Accuracy: 0.9263, Validation Accuracy: 0.9688, Loss: 0.0714\n", + "Epoch 17 Batch 1010/1077 - Train Accuracy: 0.9672, Validation Accuracy: 0.9567, Loss: 0.0574\n", + "Epoch 17 Batch 1020/1077 - Train Accuracy: 0.9621, Validation Accuracy: 0.9421, Loss: 0.0584\n", + "Epoch 17 Batch 1030/1077 - Train Accuracy: 0.9738, Validation Accuracy: 0.9567, Loss: 0.0592\n", + "Epoch 17 Batch 1040/1077 - Train Accuracy: 0.9585, Validation Accuracy: 0.9499, Loss: 0.0733\n", + "Epoch 17 Batch 1050/1077 - Train Accuracy: 0.9656, Validation Accuracy: 0.9446, Loss: 0.0601\n", + "Epoch 17 Batch 1060/1077 - Train Accuracy: 0.9437, Validation Accuracy: 0.9506, Loss: 0.0551\n", + "Epoch 17 Batch 1070/1077 - Train Accuracy: 0.9430, Validation Accuracy: 0.9474, Loss: 0.0683\n", + "Epoch 18 Batch 10/1077 - Train Accuracy: 0.9363, Validation Accuracy: 0.9482, Loss: 0.0688\n", + "Epoch 18 Batch 20/1077 - Train Accuracy: 0.9703, Validation Accuracy: 0.9503, Loss: 0.0618\n", + "Epoch 18 Batch 30/1077 - Train Accuracy: 0.9621, Validation Accuracy: 0.9545, Loss: 0.0589\n", + "Epoch 18 Batch 40/1077 - Train Accuracy: 0.9660, Validation Accuracy: 0.9467, Loss: 0.0779\n", + "Epoch 18 Batch 50/1077 - Train Accuracy: 0.9535, Validation Accuracy: 0.9499, Loss: 0.0766\n", + "Epoch 18 Batch 60/1077 - Train Accuracy: 0.9557, Validation Accuracy: 0.9371, Loss: 0.0693\n", + "Epoch 18 Batch 70/1077 - Train Accuracy: 0.9215, Validation Accuracy: 0.9421, Loss: 0.0745\n", + "Epoch 18 Batch 80/1077 - Train Accuracy: 0.9602, Validation Accuracy: 0.9616, Loss: 0.0612\n", + "Epoch 18 Batch 90/1077 - Train Accuracy: 0.9348, Validation Accuracy: 0.9521, Loss: 0.0609\n", + "Epoch 18 Batch 100/1077 - Train Accuracy: 0.9668, Validation Accuracy: 0.9499, Loss: 0.0668\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 18 Batch 110/1077 - Train Accuracy: 0.9668, Validation Accuracy: 0.9556, Loss: 0.0567\n", + "Epoch 18 Batch 120/1077 - Train Accuracy: 0.9664, Validation Accuracy: 0.9425, Loss: 0.0772\n", + "Epoch 18 Batch 130/1077 - Train Accuracy: 0.9427, Validation Accuracy: 0.9400, Loss: 0.0610\n", + "Epoch 18 Batch 140/1077 - Train Accuracy: 0.9527, Validation Accuracy: 0.9403, Loss: 0.0648\n", + "Epoch 18 Batch 150/1077 - Train Accuracy: 0.9572, Validation Accuracy: 0.9421, Loss: 0.0704\n", + "Epoch 18 Batch 160/1077 - Train Accuracy: 0.9535, Validation Accuracy: 0.9371, Loss: 0.0614\n", + "Epoch 18 Batch 170/1077 - Train Accuracy: 0.9324, Validation Accuracy: 0.9400, Loss: 0.0724\n", + "Epoch 18 Batch 180/1077 - Train Accuracy: 0.9652, Validation Accuracy: 0.9492, Loss: 0.0643\n", + "Epoch 18 Batch 190/1077 - Train Accuracy: 0.9680, Validation Accuracy: 0.9453, Loss: 0.0563\n", + "Epoch 18 Batch 200/1077 - Train Accuracy: 0.9504, Validation Accuracy: 0.9435, Loss: 0.0690\n", + "Epoch 18 Batch 210/1077 - Train Accuracy: 0.9494, Validation Accuracy: 0.9425, Loss: 0.0796\n", + "Epoch 18 Batch 220/1077 - Train Accuracy: 0.9597, Validation Accuracy: 0.9592, Loss: 0.0764\n", + "Epoch 18 Batch 230/1077 - Train Accuracy: 0.9650, Validation Accuracy: 0.9499, Loss: 0.0658\n", + "Epoch 18 Batch 240/1077 - Train Accuracy: 0.9734, Validation Accuracy: 0.9414, Loss: 0.0605\n", + "Epoch 18 Batch 250/1077 - Train Accuracy: 0.9496, Validation Accuracy: 0.9553, Loss: 0.0560\n", + "Epoch 18 Batch 260/1077 - Train Accuracy: 0.9490, Validation Accuracy: 0.9531, Loss: 0.0575\n", + "Epoch 18 Batch 270/1077 - Train Accuracy: 0.9355, Validation Accuracy: 0.9609, Loss: 0.0680\n", + "Epoch 18 Batch 280/1077 - Train Accuracy: 0.9359, Validation Accuracy: 0.9517, Loss: 0.0730\n", + "Epoch 18 Batch 290/1077 - Train Accuracy: 0.9434, Validation Accuracy: 0.9446, Loss: 0.0973\n", + "Epoch 18 Batch 300/1077 - Train Accuracy: 0.9564, Validation Accuracy: 0.9517, Loss: 0.0621\n", + "Epoch 18 Batch 310/1077 - Train Accuracy: 0.9457, Validation Accuracy: 0.9382, Loss: 0.0699\n", + "Epoch 18 Batch 320/1077 - Train Accuracy: 0.9609, Validation Accuracy: 0.9499, Loss: 0.0821\n", + "Epoch 18 Batch 330/1077 - Train Accuracy: 0.9324, Validation Accuracy: 0.9442, Loss: 0.0692\n", + "Epoch 18 Batch 340/1077 - Train Accuracy: 0.9498, Validation Accuracy: 0.9411, Loss: 0.0628\n", + "Epoch 18 Batch 350/1077 - Train Accuracy: 0.9457, Validation Accuracy: 0.9474, Loss: 0.0669\n", + "Epoch 18 Batch 360/1077 - Train Accuracy: 0.9617, Validation Accuracy: 0.9528, Loss: 0.0663\n", + "Epoch 18 Batch 370/1077 - Train Accuracy: 0.9516, Validation Accuracy: 0.9428, Loss: 0.0674\n", + "Epoch 18 Batch 380/1077 - Train Accuracy: 0.9504, Validation Accuracy: 0.9496, Loss: 0.0546\n", + "Epoch 18 Batch 390/1077 - Train Accuracy: 0.9187, Validation Accuracy: 0.9453, Loss: 0.0710\n", + "Epoch 18 Batch 400/1077 - Train Accuracy: 0.9535, Validation Accuracy: 0.9489, Loss: 0.0803\n", + "Epoch 18 Batch 410/1077 - Train Accuracy: 0.9313, Validation Accuracy: 0.9489, Loss: 0.0788\n", + "Epoch 18 Batch 420/1077 - Train Accuracy: 0.9645, Validation Accuracy: 0.9396, Loss: 0.0580\n", + "Epoch 18 Batch 430/1077 - Train Accuracy: 0.9449, Validation Accuracy: 0.9517, Loss: 0.0623\n", + "Epoch 18 Batch 440/1077 - Train Accuracy: 0.9238, Validation Accuracy: 0.9435, Loss: 0.0759\n", + "Epoch 18 Batch 450/1077 - Train Accuracy: 0.9582, Validation Accuracy: 0.9421, Loss: 0.0810\n", + "Epoch 18 Batch 460/1077 - Train Accuracy: 0.9430, Validation Accuracy: 0.9535, Loss: 0.0715\n", + "Epoch 18 Batch 470/1077 - Train Accuracy: 0.9556, Validation Accuracy: 0.9457, Loss: 0.0531\n", + "Epoch 18 Batch 480/1077 - Train Accuracy: 0.9490, Validation Accuracy: 0.9496, Loss: 0.0625\n", + "Epoch 18 Batch 490/1077 - Train Accuracy: 0.9449, Validation Accuracy: 0.9450, Loss: 0.0599\n", + "Epoch 18 Batch 500/1077 - Train Accuracy: 0.9336, Validation Accuracy: 0.9379, Loss: 0.0699\n", + "Epoch 18 Batch 510/1077 - Train Accuracy: 0.9484, Validation Accuracy: 0.9496, Loss: 0.0674\n", + "Epoch 18 Batch 520/1077 - Train Accuracy: 0.9769, Validation Accuracy: 0.9446, Loss: 0.0664\n", + "Epoch 18 Batch 530/1077 - Train Accuracy: 0.9605, Validation Accuracy: 0.9432, Loss: 0.0733\n", + "Epoch 18 Batch 540/1077 - Train Accuracy: 0.9641, Validation Accuracy: 0.9542, Loss: 0.0472\n", + "Epoch 18 Batch 550/1077 - Train Accuracy: 0.9531, Validation Accuracy: 0.9499, Loss: 0.0567\n", + "Epoch 18 Batch 560/1077 - Train Accuracy: 0.9348, Validation Accuracy: 0.9513, Loss: 0.0691\n", + "Epoch 18 Batch 570/1077 - Train Accuracy: 0.9178, Validation Accuracy: 0.9588, Loss: 0.0844\n", + "Epoch 18 Batch 580/1077 - Train Accuracy: 0.9565, Validation Accuracy: 0.9535, Loss: 0.0502\n", + "Epoch 18 Batch 590/1077 - Train Accuracy: 0.9326, Validation Accuracy: 0.9393, Loss: 0.0680\n", + "Epoch 18 Batch 600/1077 - Train Accuracy: 0.9632, Validation Accuracy: 0.9499, Loss: 0.0616\n", + "Epoch 18 Batch 610/1077 - Train Accuracy: 0.9502, Validation Accuracy: 0.9499, Loss: 0.0759\n", + "Epoch 18 Batch 620/1077 - Train Accuracy: 0.9563, Validation Accuracy: 0.9464, Loss: 0.0717\n", + "Epoch 18 Batch 630/1077 - Train Accuracy: 0.9637, Validation Accuracy: 0.9457, Loss: 0.0610\n", + "Epoch 18 Batch 640/1077 - Train Accuracy: 0.9464, Validation Accuracy: 0.9556, Loss: 0.0695\n", + "Epoch 18 Batch 650/1077 - Train Accuracy: 0.9539, Validation Accuracy: 0.9585, Loss: 0.0652\n", + "Epoch 18 Batch 660/1077 - Train Accuracy: 0.9516, Validation Accuracy: 0.9521, Loss: 0.0585\n", + "Epoch 18 Batch 670/1077 - Train Accuracy: 0.9666, Validation Accuracy: 0.9563, Loss: 0.0648\n", + "Epoch 18 Batch 680/1077 - Train Accuracy: 0.9442, Validation Accuracy: 0.9531, Loss: 0.0648\n", + "Epoch 18 Batch 690/1077 - Train Accuracy: 0.9535, Validation Accuracy: 0.9474, Loss: 0.0665\n", + "Epoch 18 Batch 700/1077 - Train Accuracy: 0.9641, Validation Accuracy: 0.9485, Loss: 0.0564\n", + "Epoch 18 Batch 710/1077 - Train Accuracy: 0.9598, Validation Accuracy: 0.9432, Loss: 0.0552\n", + "Epoch 18 Batch 720/1077 - Train Accuracy: 0.9511, Validation Accuracy: 0.9521, Loss: 0.0726\n", + "Epoch 18 Batch 730/1077 - Train Accuracy: 0.9535, Validation Accuracy: 0.9517, Loss: 0.0849\n", + "Epoch 18 Batch 740/1077 - Train Accuracy: 0.9793, Validation Accuracy: 0.9418, Loss: 0.0508\n", + "Epoch 18 Batch 750/1077 - Train Accuracy: 0.9578, Validation Accuracy: 0.9577, Loss: 0.0541\n", + "Epoch 18 Batch 760/1077 - Train Accuracy: 0.9578, Validation Accuracy: 0.9560, Loss: 0.0719\n", + "Epoch 18 Batch 770/1077 - Train Accuracy: 0.9297, Validation Accuracy: 0.9524, Loss: 0.0650\n", + "Epoch 18 Batch 780/1077 - Train Accuracy: 0.9395, Validation Accuracy: 0.9453, Loss: 0.0863\n", + "Epoch 18 Batch 790/1077 - Train Accuracy: 0.9375, Validation Accuracy: 0.9570, Loss: 0.0847\n", + "Epoch 18 Batch 800/1077 - Train Accuracy: 0.9695, Validation Accuracy: 0.9467, Loss: 0.0644\n", + "Epoch 18 Batch 810/1077 - Train Accuracy: 0.9624, Validation Accuracy: 0.9467, Loss: 0.0607\n", + "Epoch 18 Batch 820/1077 - Train Accuracy: 0.9449, Validation Accuracy: 0.9361, Loss: 0.0768\n", + "Epoch 18 Batch 830/1077 - Train Accuracy: 0.9246, Validation Accuracy: 0.9517, Loss: 0.0806\n", + "Epoch 18 Batch 840/1077 - Train Accuracy: 0.9613, Validation Accuracy: 0.9350, Loss: 0.0529\n", + "Epoch 18 Batch 850/1077 - Train Accuracy: 0.9568, Validation Accuracy: 0.9549, Loss: 0.0782\n", + "Epoch 18 Batch 860/1077 - Train Accuracy: 0.9676, Validation Accuracy: 0.9663, Loss: 0.0665\n", + "Epoch 18 Batch 870/1077 - Train Accuracy: 0.9408, Validation Accuracy: 0.9577, Loss: 0.0657\n", + "Epoch 18 Batch 880/1077 - Train Accuracy: 0.9578, Validation Accuracy: 0.9620, Loss: 0.0727\n", + "Epoch 18 Batch 890/1077 - Train Accuracy: 0.9453, Validation Accuracy: 0.9524, Loss: 0.0627\n", + "Epoch 18 Batch 900/1077 - Train Accuracy: 0.9480, Validation Accuracy: 0.9528, Loss: 0.0775\n", + "Epoch 18 Batch 910/1077 - Train Accuracy: 0.9591, Validation Accuracy: 0.9627, Loss: 0.0681\n", + "Epoch 18 Batch 920/1077 - Train Accuracy: 0.9504, Validation Accuracy: 0.9528, Loss: 0.0560\n", + "Epoch 18 Batch 930/1077 - Train Accuracy: 0.9535, Validation Accuracy: 0.9435, Loss: 0.0799\n", + "Epoch 18 Batch 940/1077 - Train Accuracy: 0.9660, Validation Accuracy: 0.9460, Loss: 0.0528\n", + "Epoch 18 Batch 950/1077 - Train Accuracy: 0.9706, Validation Accuracy: 0.9467, Loss: 0.0474\n", + "Epoch 18 Batch 960/1077 - Train Accuracy: 0.9498, Validation Accuracy: 0.9439, Loss: 0.0607\n", + "Epoch 18 Batch 970/1077 - Train Accuracy: 0.9516, Validation Accuracy: 0.9553, Loss: 0.0675\n", + "Epoch 18 Batch 980/1077 - Train Accuracy: 0.9367, Validation Accuracy: 0.9556, Loss: 0.0675\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 18 Batch 990/1077 - Train Accuracy: 0.9585, Validation Accuracy: 0.9592, Loss: 0.0736\n", + "Epoch 18 Batch 1000/1077 - Train Accuracy: 0.9237, Validation Accuracy: 0.9577, Loss: 0.0747\n", + "Epoch 18 Batch 1010/1077 - Train Accuracy: 0.9664, Validation Accuracy: 0.9446, Loss: 0.0585\n", + "Epoch 18 Batch 1020/1077 - Train Accuracy: 0.9672, Validation Accuracy: 0.9492, Loss: 0.0480\n", + "Epoch 18 Batch 1030/1077 - Train Accuracy: 0.9570, Validation Accuracy: 0.9556, Loss: 0.0567\n", + "Epoch 18 Batch 1040/1077 - Train Accuracy: 0.9552, Validation Accuracy: 0.9439, Loss: 0.0837\n", + "Epoch 18 Batch 1050/1077 - Train Accuracy: 0.9543, Validation Accuracy: 0.9457, Loss: 0.0633\n", + "Epoch 18 Batch 1060/1077 - Train Accuracy: 0.9531, Validation Accuracy: 0.9450, Loss: 0.0449\n", + "Epoch 18 Batch 1070/1077 - Train Accuracy: 0.9500, Validation Accuracy: 0.9524, Loss: 0.0464\n", + "Epoch 19 Batch 10/1077 - Train Accuracy: 0.9412, Validation Accuracy: 0.9531, Loss: 0.0702\n", + "Epoch 19 Batch 20/1077 - Train Accuracy: 0.9543, Validation Accuracy: 0.9577, Loss: 0.0646\n", + "Epoch 19 Batch 30/1077 - Train Accuracy: 0.9578, Validation Accuracy: 0.9531, Loss: 0.0591\n", + "Epoch 19 Batch 40/1077 - Train Accuracy: 0.9508, Validation Accuracy: 0.9528, Loss: 0.0572\n", + "Epoch 19 Batch 50/1077 - Train Accuracy: 0.9641, Validation Accuracy: 0.9478, Loss: 0.0598\n", + "Epoch 19 Batch 60/1077 - Train Accuracy: 0.9676, Validation Accuracy: 0.9453, Loss: 0.0630\n", + "Epoch 19 Batch 70/1077 - Train Accuracy: 0.9305, Validation Accuracy: 0.9513, Loss: 0.0724\n", + "Epoch 19 Batch 80/1077 - Train Accuracy: 0.9668, Validation Accuracy: 0.9556, Loss: 0.0531\n", + "Epoch 19 Batch 90/1077 - Train Accuracy: 0.9367, Validation Accuracy: 0.9560, Loss: 0.0707\n", + "Epoch 19 Batch 100/1077 - Train Accuracy: 0.9582, Validation Accuracy: 0.9482, Loss: 0.0549\n", + "Epoch 19 Batch 110/1077 - Train Accuracy: 0.9754, Validation Accuracy: 0.9492, Loss: 0.0564\n", + "Epoch 19 Batch 120/1077 - Train Accuracy: 0.9613, Validation Accuracy: 0.9545, Loss: 0.0612\n", + "Epoch 19 Batch 130/1077 - Train Accuracy: 0.9535, Validation Accuracy: 0.9556, Loss: 0.0419\n", + "Epoch 19 Batch 140/1077 - Train Accuracy: 0.9556, Validation Accuracy: 0.9407, Loss: 0.0608\n", + "Epoch 19 Batch 150/1077 - Train Accuracy: 0.9416, Validation Accuracy: 0.9556, Loss: 0.0720\n", + "Epoch 19 Batch 160/1077 - Train Accuracy: 0.9598, Validation Accuracy: 0.9467, Loss: 0.0521\n", + "Epoch 19 Batch 170/1077 - Train Accuracy: 0.9355, Validation Accuracy: 0.9258, Loss: 0.0686\n", + "Epoch 19 Batch 180/1077 - Train Accuracy: 0.9621, Validation Accuracy: 0.9329, Loss: 0.0576\n", + "Epoch 19 Batch 190/1077 - Train Accuracy: 0.9711, Validation Accuracy: 0.9375, Loss: 0.0605\n", + "Epoch 19 Batch 200/1077 - Train Accuracy: 0.9676, Validation Accuracy: 0.9524, Loss: 0.0774\n", + "Epoch 19 Batch 210/1077 - Train Accuracy: 0.9431, Validation Accuracy: 0.9414, Loss: 0.0779\n", + "Epoch 19 Batch 220/1077 - Train Accuracy: 0.9539, Validation Accuracy: 0.9400, Loss: 0.0669\n", + "Epoch 19 Batch 230/1077 - Train Accuracy: 0.9717, Validation Accuracy: 0.9627, Loss: 0.0627\n", + "Epoch 19 Batch 240/1077 - Train Accuracy: 0.9855, Validation Accuracy: 0.9535, Loss: 0.0612\n", + "Epoch 19 Batch 250/1077 - Train Accuracy: 0.9411, Validation Accuracy: 0.9560, Loss: 0.0686\n", + "Epoch 19 Batch 260/1077 - Train Accuracy: 0.9621, Validation Accuracy: 0.9560, Loss: 0.0481\n", + "Epoch 19 Batch 270/1077 - Train Accuracy: 0.9324, Validation Accuracy: 0.9556, Loss: 0.0717\n", + "Epoch 19 Batch 280/1077 - Train Accuracy: 0.9297, Validation Accuracy: 0.9517, Loss: 0.0840\n", + "Epoch 19 Batch 290/1077 - Train Accuracy: 0.9313, Validation Accuracy: 0.9368, Loss: 0.0868\n", + "Epoch 19 Batch 300/1077 - Train Accuracy: 0.9507, Validation Accuracy: 0.9513, Loss: 0.0546\n", + "Epoch 19 Batch 310/1077 - Train Accuracy: 0.9488, Validation Accuracy: 0.9499, Loss: 0.0566\n", + "Epoch 19 Batch 320/1077 - Train Accuracy: 0.9617, Validation Accuracy: 0.9442, Loss: 0.0633\n", + "Epoch 19 Batch 330/1077 - Train Accuracy: 0.9422, Validation Accuracy: 0.9524, Loss: 0.0627\n", + "Epoch 19 Batch 340/1077 - Train Accuracy: 0.9803, Validation Accuracy: 0.9478, Loss: 0.0610\n", + "Epoch 19 Batch 350/1077 - Train Accuracy: 0.9422, Validation Accuracy: 0.9482, Loss: 0.0621\n", + "Epoch 19 Batch 360/1077 - Train Accuracy: 0.9504, Validation Accuracy: 0.9446, Loss: 0.0615\n", + "Epoch 19 Batch 370/1077 - Train Accuracy: 0.9624, Validation Accuracy: 0.9574, Loss: 0.0632\n", + "Epoch 19 Batch 380/1077 - Train Accuracy: 0.9398, Validation Accuracy: 0.9506, Loss: 0.0595\n", + "Epoch 19 Batch 390/1077 - Train Accuracy: 0.9320, Validation Accuracy: 0.9513, Loss: 0.0712\n", + "Epoch 19 Batch 400/1077 - Train Accuracy: 0.9520, Validation Accuracy: 0.9428, Loss: 0.0672\n", + "Epoch 19 Batch 410/1077 - Train Accuracy: 0.9190, Validation Accuracy: 0.9343, Loss: 0.0884\n", + "Epoch 19 Batch 420/1077 - Train Accuracy: 0.9742, Validation Accuracy: 0.9467, Loss: 0.0520\n", + "Epoch 19 Batch 430/1077 - Train Accuracy: 0.9703, Validation Accuracy: 0.9592, Loss: 0.0484\n", + "Epoch 19 Batch 440/1077 - Train Accuracy: 0.9387, Validation Accuracy: 0.9588, Loss: 0.0725\n", + "Epoch 19 Batch 450/1077 - Train Accuracy: 0.9648, Validation Accuracy: 0.9535, Loss: 0.0662\n", + "Epoch 19 Batch 460/1077 - Train Accuracy: 0.9496, Validation Accuracy: 0.9627, Loss: 0.0648\n", + "Epoch 19 Batch 470/1077 - Train Accuracy: 0.9560, Validation Accuracy: 0.9517, Loss: 0.0717\n", + "Epoch 19 Batch 480/1077 - Train Accuracy: 0.9461, Validation Accuracy: 0.9513, Loss: 0.0563\n", + "Epoch 19 Batch 490/1077 - Train Accuracy: 0.9473, Validation Accuracy: 0.9567, Loss: 0.0649\n", + "Epoch 19 Batch 500/1077 - Train Accuracy: 0.9512, Validation Accuracy: 0.9560, Loss: 0.0555\n", + "Epoch 19 Batch 510/1077 - Train Accuracy: 0.9629, Validation Accuracy: 0.9545, Loss: 0.0504\n", + "Epoch 19 Batch 520/1077 - Train Accuracy: 0.9740, Validation Accuracy: 0.9492, Loss: 0.0594\n", + "Epoch 19 Batch 530/1077 - Train Accuracy: 0.9680, Validation Accuracy: 0.9620, Loss: 0.0658\n", + "Epoch 19 Batch 540/1077 - Train Accuracy: 0.9637, Validation Accuracy: 0.9616, Loss: 0.0715\n", + "Epoch 19 Batch 550/1077 - Train Accuracy: 0.9492, Validation Accuracy: 0.9567, Loss: 0.0659\n", + "Epoch 19 Batch 560/1077 - Train Accuracy: 0.9461, Validation Accuracy: 0.9414, Loss: 0.0703\n", + "Epoch 19 Batch 570/1077 - Train Accuracy: 0.9437, Validation Accuracy: 0.9499, Loss: 0.0884\n", + "Epoch 19 Batch 580/1077 - Train Accuracy: 0.9613, Validation Accuracy: 0.9354, Loss: 0.0492\n", + "Epoch 19 Batch 590/1077 - Train Accuracy: 0.9248, Validation Accuracy: 0.9403, Loss: 0.0748\n", + "Epoch 19 Batch 600/1077 - Train Accuracy: 0.9535, Validation Accuracy: 0.9478, Loss: 0.0762\n", + "Epoch 19 Batch 610/1077 - Train Accuracy: 0.9498, Validation Accuracy: 0.9432, Loss: 0.0724\n", + "Epoch 19 Batch 620/1077 - Train Accuracy: 0.9652, Validation Accuracy: 0.9407, Loss: 0.0545\n", + "Epoch 19 Batch 630/1077 - Train Accuracy: 0.9742, Validation Accuracy: 0.9442, Loss: 0.0561\n", + "Epoch 19 Batch 640/1077 - Train Accuracy: 0.9457, Validation Accuracy: 0.9414, Loss: 0.0666\n", + "Epoch 19 Batch 650/1077 - Train Accuracy: 0.9582, Validation Accuracy: 0.9545, Loss: 0.0681\n", + "Epoch 19 Batch 660/1077 - Train Accuracy: 0.9629, Validation Accuracy: 0.9592, Loss: 0.0562\n", + "Epoch 19 Batch 670/1077 - Train Accuracy: 0.9748, Validation Accuracy: 0.9677, Loss: 0.0582\n", + "Epoch 19 Batch 680/1077 - Train Accuracy: 0.9505, Validation Accuracy: 0.9499, Loss: 0.0632\n", + "Epoch 19 Batch 690/1077 - Train Accuracy: 0.9574, Validation Accuracy: 0.9595, Loss: 0.0617\n", + "Epoch 19 Batch 700/1077 - Train Accuracy: 0.9742, Validation Accuracy: 0.9588, Loss: 0.0478\n", + "Epoch 19 Batch 710/1077 - Train Accuracy: 0.9551, Validation Accuracy: 0.9570, Loss: 0.0508\n", + "Epoch 19 Batch 720/1077 - Train Accuracy: 0.9556, Validation Accuracy: 0.9478, Loss: 0.0867\n", + "Epoch 19 Batch 730/1077 - Train Accuracy: 0.9523, Validation Accuracy: 0.9592, Loss: 0.0827\n", + "Epoch 19 Batch 740/1077 - Train Accuracy: 0.9812, Validation Accuracy: 0.9563, Loss: 0.0502\n", + "Epoch 19 Batch 750/1077 - Train Accuracy: 0.9539, Validation Accuracy: 0.9645, Loss: 0.0729\n", + "Epoch 19 Batch 760/1077 - Train Accuracy: 0.9566, Validation Accuracy: 0.9599, Loss: 0.0654\n", + "Epoch 19 Batch 770/1077 - Train Accuracy: 0.9338, Validation Accuracy: 0.9549, Loss: 0.0669\n", + "Epoch 19 Batch 780/1077 - Train Accuracy: 0.9383, Validation Accuracy: 0.9595, Loss: 0.0796\n", + "Epoch 19 Batch 790/1077 - Train Accuracy: 0.9328, Validation Accuracy: 0.9560, Loss: 0.0613\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 19 Batch 800/1077 - Train Accuracy: 0.9559, Validation Accuracy: 0.9567, Loss: 0.0658\n", + "Epoch 19 Batch 810/1077 - Train Accuracy: 0.9591, Validation Accuracy: 0.9634, Loss: 0.0562\n", + "Epoch 19 Batch 820/1077 - Train Accuracy: 0.9633, Validation Accuracy: 0.9560, Loss: 0.0535\n", + "Epoch 19 Batch 830/1077 - Train Accuracy: 0.9387, Validation Accuracy: 0.9599, Loss: 0.0748\n", + "Epoch 19 Batch 840/1077 - Train Accuracy: 0.9723, Validation Accuracy: 0.9407, Loss: 0.0597\n", + "Epoch 19 Batch 850/1077 - Train Accuracy: 0.9643, Validation Accuracy: 0.9602, Loss: 0.0851\n", + "Epoch 19 Batch 860/1077 - Train Accuracy: 0.9546, Validation Accuracy: 0.9560, Loss: 0.0679\n", + "Epoch 19 Batch 870/1077 - Train Accuracy: 0.9523, Validation Accuracy: 0.9574, Loss: 0.0621\n", + "Epoch 19 Batch 880/1077 - Train Accuracy: 0.9602, Validation Accuracy: 0.9535, Loss: 0.0660\n", + "Epoch 19 Batch 890/1077 - Train Accuracy: 0.9684, Validation Accuracy: 0.9510, Loss: 0.0631\n", + "Epoch 19 Batch 900/1077 - Train Accuracy: 0.9531, Validation Accuracy: 0.9496, Loss: 0.0558\n", + "Epoch 19 Batch 910/1077 - Train Accuracy: 0.9513, Validation Accuracy: 0.9670, Loss: 0.0627\n", + "Epoch 19 Batch 920/1077 - Train Accuracy: 0.9461, Validation Accuracy: 0.9577, Loss: 0.0643\n", + "Epoch 19 Batch 930/1077 - Train Accuracy: 0.9559, Validation Accuracy: 0.9673, Loss: 0.0599\n", + "Epoch 19 Batch 940/1077 - Train Accuracy: 0.9547, Validation Accuracy: 0.9556, Loss: 0.0595\n", + "Epoch 19 Batch 950/1077 - Train Accuracy: 0.9762, Validation Accuracy: 0.9553, Loss: 0.0498\n", + "Epoch 19 Batch 960/1077 - Train Accuracy: 0.9643, Validation Accuracy: 0.9542, Loss: 0.0571\n", + "Epoch 19 Batch 970/1077 - Train Accuracy: 0.9484, Validation Accuracy: 0.9638, Loss: 0.0867\n", + "Epoch 19 Batch 980/1077 - Train Accuracy: 0.9543, Validation Accuracy: 0.9680, Loss: 0.0700\n", + "Epoch 19 Batch 990/1077 - Train Accuracy: 0.9663, Validation Accuracy: 0.9734, Loss: 0.0673\n", + "Epoch 19 Batch 1000/1077 - Train Accuracy: 0.9196, Validation Accuracy: 0.9581, Loss: 0.0735\n", + "Epoch 19 Batch 1010/1077 - Train Accuracy: 0.9750, Validation Accuracy: 0.9482, Loss: 0.0599\n", + "Epoch 19 Batch 1020/1077 - Train Accuracy: 0.9770, Validation Accuracy: 0.9442, Loss: 0.0457\n", + "Epoch 19 Batch 1030/1077 - Train Accuracy: 0.9707, Validation Accuracy: 0.9592, Loss: 0.0623\n", + "Epoch 19 Batch 1040/1077 - Train Accuracy: 0.9708, Validation Accuracy: 0.9272, Loss: 0.0534\n", + "Epoch 19 Batch 1050/1077 - Train Accuracy: 0.9609, Validation Accuracy: 0.9513, Loss: 0.0494\n", + "Epoch 19 Batch 1060/1077 - Train Accuracy: 0.9496, Validation Accuracy: 0.9602, Loss: 0.0451\n", + "Epoch 19 Batch 1070/1077 - Train Accuracy: 0.9582, Validation Accuracy: 0.9577, Loss: 0.0554\n", + "Epoch 20 Batch 10/1077 - Train Accuracy: 0.9507, Validation Accuracy: 0.9414, Loss: 0.0612\n", + "Epoch 20 Batch 20/1077 - Train Accuracy: 0.9629, Validation Accuracy: 0.9577, Loss: 0.0484\n", + "Epoch 20 Batch 30/1077 - Train Accuracy: 0.9586, Validation Accuracy: 0.9446, Loss: 0.0549\n", + "Epoch 20 Batch 40/1077 - Train Accuracy: 0.9543, Validation Accuracy: 0.9581, Loss: 0.0600\n", + "Epoch 20 Batch 50/1077 - Train Accuracy: 0.9426, Validation Accuracy: 0.9698, Loss: 0.0607\n", + "Epoch 20 Batch 60/1077 - Train Accuracy: 0.9498, Validation Accuracy: 0.9553, Loss: 0.0540\n", + "Epoch 20 Batch 70/1077 - Train Accuracy: 0.9350, Validation Accuracy: 0.9560, Loss: 0.0778\n", + "Epoch 20 Batch 80/1077 - Train Accuracy: 0.9672, Validation Accuracy: 0.9535, Loss: 0.0452\n", + "Epoch 20 Batch 90/1077 - Train Accuracy: 0.9484, Validation Accuracy: 0.9638, Loss: 0.0715\n", + "Epoch 20 Batch 100/1077 - Train Accuracy: 0.9637, Validation Accuracy: 0.9691, Loss: 0.0493\n", + "Epoch 20 Batch 110/1077 - Train Accuracy: 0.9730, Validation Accuracy: 0.9627, Loss: 0.0453\n", + "Epoch 20 Batch 120/1077 - Train Accuracy: 0.9672, Validation Accuracy: 0.9634, Loss: 0.0684\n", + "Epoch 20 Batch 130/1077 - Train Accuracy: 0.9550, Validation Accuracy: 0.9553, Loss: 0.0538\n", + "Epoch 20 Batch 140/1077 - Train Accuracy: 0.9511, Validation Accuracy: 0.9659, Loss: 0.0629\n", + "Epoch 20 Batch 150/1077 - Train Accuracy: 0.9609, Validation Accuracy: 0.9595, Loss: 0.0524\n", + "Epoch 20 Batch 160/1077 - Train Accuracy: 0.9688, Validation Accuracy: 0.9432, Loss: 0.0492\n", + "Epoch 20 Batch 170/1077 - Train Accuracy: 0.9512, Validation Accuracy: 0.9361, Loss: 0.0669\n", + "Epoch 20 Batch 180/1077 - Train Accuracy: 0.9754, Validation Accuracy: 0.9400, Loss: 0.0485\n", + "Epoch 20 Batch 190/1077 - Train Accuracy: 0.9586, Validation Accuracy: 0.9421, Loss: 0.0611\n", + "Epoch 20 Batch 200/1077 - Train Accuracy: 0.9496, Validation Accuracy: 0.9506, Loss: 0.0630\n", + "Epoch 20 Batch 210/1077 - Train Accuracy: 0.9475, Validation Accuracy: 0.9513, Loss: 0.0628\n", + "Epoch 20 Batch 220/1077 - Train Accuracy: 0.9359, Validation Accuracy: 0.9556, Loss: 0.0547\n", + "Epoch 20 Batch 230/1077 - Train Accuracy: 0.9736, Validation Accuracy: 0.9393, Loss: 0.0509\n", + "Epoch 20 Batch 240/1077 - Train Accuracy: 0.9820, Validation Accuracy: 0.9453, Loss: 0.0514\n", + "Epoch 20 Batch 250/1077 - Train Accuracy: 0.9517, Validation Accuracy: 0.9553, Loss: 0.0568\n", + "Epoch 20 Batch 260/1077 - Train Accuracy: 0.9624, Validation Accuracy: 0.9616, Loss: 0.0393\n", + "Epoch 20 Batch 270/1077 - Train Accuracy: 0.9520, Validation Accuracy: 0.9435, Loss: 0.0745\n", + "Epoch 20 Batch 280/1077 - Train Accuracy: 0.9383, Validation Accuracy: 0.9379, Loss: 0.0708\n", + "Epoch 20 Batch 290/1077 - Train Accuracy: 0.9336, Validation Accuracy: 0.9425, Loss: 0.0915\n", + "Epoch 20 Batch 300/1077 - Train Accuracy: 0.9548, Validation Accuracy: 0.9499, Loss: 0.0568\n", + "Epoch 20 Batch 310/1077 - Train Accuracy: 0.9602, Validation Accuracy: 0.9492, Loss: 0.0538\n", + "Epoch 20 Batch 320/1077 - Train Accuracy: 0.9633, Validation Accuracy: 0.9638, Loss: 0.0574\n", + "Epoch 20 Batch 330/1077 - Train Accuracy: 0.9516, Validation Accuracy: 0.9567, Loss: 0.0655\n", + "Epoch 20 Batch 340/1077 - Train Accuracy: 0.9704, Validation Accuracy: 0.9556, Loss: 0.0570\n", + "Epoch 20 Batch 350/1077 - Train Accuracy: 0.9484, Validation Accuracy: 0.9567, Loss: 0.0565\n", + "Epoch 20 Batch 360/1077 - Train Accuracy: 0.9477, Validation Accuracy: 0.9620, Loss: 0.0536\n", + "Epoch 20 Batch 370/1077 - Train Accuracy: 0.9632, Validation Accuracy: 0.9560, Loss: 0.0672\n", + "Epoch 20 Batch 380/1077 - Train Accuracy: 0.9578, Validation Accuracy: 0.9592, Loss: 0.0648\n", + "Epoch 20 Batch 390/1077 - Train Accuracy: 0.9492, Validation Accuracy: 0.9627, Loss: 0.0652\n", + "Epoch 20 Batch 400/1077 - Train Accuracy: 0.9492, Validation Accuracy: 0.9361, Loss: 0.0852\n", + "Epoch 20 Batch 410/1077 - Train Accuracy: 0.9178, Validation Accuracy: 0.9411, Loss: 0.0795\n", + "Epoch 20 Batch 420/1077 - Train Accuracy: 0.9730, Validation Accuracy: 0.9414, Loss: 0.0538\n", + "Epoch 20 Batch 430/1077 - Train Accuracy: 0.9496, Validation Accuracy: 0.9482, Loss: 0.0573\n", + "Epoch 20 Batch 440/1077 - Train Accuracy: 0.9375, Validation Accuracy: 0.9446, Loss: 0.0654\n", + "Epoch 20 Batch 450/1077 - Train Accuracy: 0.9563, Validation Accuracy: 0.9506, Loss: 0.0688\n", + "Epoch 20 Batch 460/1077 - Train Accuracy: 0.9461, Validation Accuracy: 0.9616, Loss: 0.0729\n", + "Epoch 20 Batch 470/1077 - Train Accuracy: 0.9762, Validation Accuracy: 0.9432, Loss: 0.0590\n", + "Epoch 20 Batch 480/1077 - Train Accuracy: 0.9556, Validation Accuracy: 0.9581, Loss: 0.0632\n", + "Epoch 20 Batch 490/1077 - Train Accuracy: 0.9527, Validation Accuracy: 0.9592, Loss: 0.0609\n", + "Epoch 20 Batch 500/1077 - Train Accuracy: 0.9531, Validation Accuracy: 0.9496, Loss: 0.0440\n", + "Epoch 20 Batch 510/1077 - Train Accuracy: 0.9437, Validation Accuracy: 0.9585, Loss: 0.0564\n", + "Epoch 20 Batch 520/1077 - Train Accuracy: 0.9665, Validation Accuracy: 0.9489, Loss: 0.0604\n", + "Epoch 20 Batch 530/1077 - Train Accuracy: 0.9621, Validation Accuracy: 0.9506, Loss: 0.0536\n", + "Epoch 20 Batch 540/1077 - Train Accuracy: 0.9680, Validation Accuracy: 0.9531, Loss: 0.0515\n", + "Epoch 20 Batch 550/1077 - Train Accuracy: 0.9324, Validation Accuracy: 0.9517, Loss: 0.0632\n", + "Epoch 20 Batch 560/1077 - Train Accuracy: 0.9504, Validation Accuracy: 0.9435, Loss: 0.0542\n", + "Epoch 20 Batch 570/1077 - Train Accuracy: 0.9305, Validation Accuracy: 0.9513, Loss: 0.0842\n", + "Epoch 20 Batch 580/1077 - Train Accuracy: 0.9583, Validation Accuracy: 0.9478, Loss: 0.0494\n", + "Epoch 20 Batch 590/1077 - Train Accuracy: 0.9523, Validation Accuracy: 0.9357, Loss: 0.0773\n", + "Epoch 20 Batch 600/1077 - Train Accuracy: 0.9609, Validation Accuracy: 0.9393, Loss: 0.0656\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 20 Batch 610/1077 - Train Accuracy: 0.9441, Validation Accuracy: 0.9471, Loss: 0.0729\n", + "Epoch 20 Batch 620/1077 - Train Accuracy: 0.9738, Validation Accuracy: 0.9567, Loss: 0.0635\n", + "Epoch 20 Batch 630/1077 - Train Accuracy: 0.9781, Validation Accuracy: 0.9592, Loss: 0.0615\n", + "Epoch 20 Batch 640/1077 - Train Accuracy: 0.9442, Validation Accuracy: 0.9503, Loss: 0.0661\n", + "Epoch 20 Batch 650/1077 - Train Accuracy: 0.9617, Validation Accuracy: 0.9521, Loss: 0.0632\n", + "Epoch 20 Batch 660/1077 - Train Accuracy: 0.9727, Validation Accuracy: 0.9531, Loss: 0.0568\n", + "Epoch 20 Batch 670/1077 - Train Accuracy: 0.9719, Validation Accuracy: 0.9602, Loss: 0.0733\n", + "Epoch 20 Batch 680/1077 - Train Accuracy: 0.9542, Validation Accuracy: 0.9741, Loss: 0.0572\n", + "Epoch 20 Batch 690/1077 - Train Accuracy: 0.9559, Validation Accuracy: 0.9599, Loss: 0.0552\n", + "Epoch 20 Batch 700/1077 - Train Accuracy: 0.9801, Validation Accuracy: 0.9606, Loss: 0.0478\n", + "Epoch 20 Batch 710/1077 - Train Accuracy: 0.9730, Validation Accuracy: 0.9570, Loss: 0.0515\n", + "Epoch 20 Batch 720/1077 - Train Accuracy: 0.9511, Validation Accuracy: 0.9510, Loss: 0.0713\n", + "Epoch 20 Batch 730/1077 - Train Accuracy: 0.9652, Validation Accuracy: 0.9489, Loss: 0.0648\n", + "Epoch 20 Batch 740/1077 - Train Accuracy: 0.9703, Validation Accuracy: 0.9542, Loss: 0.0499\n", + "Epoch 20 Batch 750/1077 - Train Accuracy: 0.9641, Validation Accuracy: 0.9563, Loss: 0.0608\n", + "Epoch 20 Batch 760/1077 - Train Accuracy: 0.9633, Validation Accuracy: 0.9577, Loss: 0.0673\n", + "Epoch 20 Batch 770/1077 - Train Accuracy: 0.9364, Validation Accuracy: 0.9524, Loss: 0.0615\n", + "Epoch 20 Batch 780/1077 - Train Accuracy: 0.9391, Validation Accuracy: 0.9474, Loss: 0.0892\n", + "Epoch 20 Batch 790/1077 - Train Accuracy: 0.9234, Validation Accuracy: 0.9432, Loss: 0.0694\n", + "Epoch 20 Batch 800/1077 - Train Accuracy: 0.9605, Validation Accuracy: 0.9460, Loss: 0.0488\n", + "Epoch 20 Batch 810/1077 - Train Accuracy: 0.9728, Validation Accuracy: 0.9712, Loss: 0.0513\n", + "Epoch 20 Batch 820/1077 - Train Accuracy: 0.9582, Validation Accuracy: 0.9506, Loss: 0.0623\n", + "Epoch 20 Batch 830/1077 - Train Accuracy: 0.9172, Validation Accuracy: 0.9602, Loss: 0.0774\n", + "Epoch 20 Batch 840/1077 - Train Accuracy: 0.9781, Validation Accuracy: 0.9489, Loss: 0.0507\n", + "Epoch 20 Batch 850/1077 - Train Accuracy: 0.9650, Validation Accuracy: 0.9585, Loss: 0.0747\n", + "Epoch 20 Batch 860/1077 - Train Accuracy: 0.9632, Validation Accuracy: 0.9627, Loss: 0.0686\n", + "Epoch 20 Batch 870/1077 - Train Accuracy: 0.9396, Validation Accuracy: 0.9638, Loss: 0.0715\n", + "Epoch 20 Batch 880/1077 - Train Accuracy: 0.9621, Validation Accuracy: 0.9393, Loss: 0.0639\n", + "Epoch 20 Batch 890/1077 - Train Accuracy: 0.9643, Validation Accuracy: 0.9354, Loss: 0.0617\n", + "Epoch 20 Batch 900/1077 - Train Accuracy: 0.9508, Validation Accuracy: 0.9482, Loss: 0.0661\n", + "Epoch 20 Batch 910/1077 - Train Accuracy: 0.9613, Validation Accuracy: 0.9489, Loss: 0.0650\n", + "Epoch 20 Batch 920/1077 - Train Accuracy: 0.9621, Validation Accuracy: 0.9386, Loss: 0.0472\n", + "Epoch 20 Batch 930/1077 - Train Accuracy: 0.9629, Validation Accuracy: 0.9510, Loss: 0.0535\n", + "Epoch 20 Batch 940/1077 - Train Accuracy: 0.9719, Validation Accuracy: 0.9510, Loss: 0.0506\n", + "Epoch 20 Batch 950/1077 - Train Accuracy: 0.9673, Validation Accuracy: 0.9421, Loss: 0.0420\n", + "Epoch 20 Batch 960/1077 - Train Accuracy: 0.9494, Validation Accuracy: 0.9570, Loss: 0.0522\n", + "Epoch 20 Batch 970/1077 - Train Accuracy: 0.9613, Validation Accuracy: 0.9574, Loss: 0.0698\n", + "Epoch 20 Batch 980/1077 - Train Accuracy: 0.9500, Validation Accuracy: 0.9588, Loss: 0.0692\n", + "Epoch 20 Batch 990/1077 - Train Accuracy: 0.9630, Validation Accuracy: 0.9627, Loss: 0.0596\n", + "Epoch 20 Batch 1000/1077 - Train Accuracy: 0.9360, Validation Accuracy: 0.9577, Loss: 0.0727\n", + "Epoch 20 Batch 1010/1077 - Train Accuracy: 0.9656, Validation Accuracy: 0.9549, Loss: 0.0621\n", + "Epoch 20 Batch 1020/1077 - Train Accuracy: 0.9820, Validation Accuracy: 0.9673, Loss: 0.0473\n", + "Epoch 20 Batch 1030/1077 - Train Accuracy: 0.9766, Validation Accuracy: 0.9613, Loss: 0.0590\n", + "Epoch 20 Batch 1040/1077 - Train Accuracy: 0.9655, Validation Accuracy: 0.9581, Loss: 0.0654\n", + "Epoch 20 Batch 1050/1077 - Train Accuracy: 0.9641, Validation Accuracy: 0.9524, Loss: 0.0513\n", + "Epoch 20 Batch 1060/1077 - Train Accuracy: 0.9777, Validation Accuracy: 0.9492, Loss: 0.0502\n", + "Epoch 20 Batch 1070/1077 - Train Accuracy: 0.9605, Validation Accuracy: 0.9705, Loss: 0.0436\n", + "Epoch 21 Batch 10/1077 - Train Accuracy: 0.9453, Validation Accuracy: 0.9599, Loss: 0.0590\n", + "Epoch 21 Batch 20/1077 - Train Accuracy: 0.9660, Validation Accuracy: 0.9613, Loss: 0.0508\n", + "Epoch 21 Batch 30/1077 - Train Accuracy: 0.9688, Validation Accuracy: 0.9599, Loss: 0.0520\n", + "Epoch 21 Batch 40/1077 - Train Accuracy: 0.9523, Validation Accuracy: 0.9599, Loss: 0.0487\n", + "Epoch 21 Batch 50/1077 - Train Accuracy: 0.9762, Validation Accuracy: 0.9645, Loss: 0.0612\n", + "Epoch 21 Batch 60/1077 - Train Accuracy: 0.9661, Validation Accuracy: 0.9634, Loss: 0.0481\n", + "Epoch 21 Batch 70/1077 - Train Accuracy: 0.9202, Validation Accuracy: 0.9393, Loss: 0.0556\n", + "Epoch 21 Batch 80/1077 - Train Accuracy: 0.9574, Validation Accuracy: 0.9691, Loss: 0.0464\n", + "Epoch 21 Batch 90/1077 - Train Accuracy: 0.9473, Validation Accuracy: 0.9492, Loss: 0.0666\n", + "Epoch 21 Batch 100/1077 - Train Accuracy: 0.9641, Validation Accuracy: 0.9521, Loss: 0.0458\n", + "Epoch 21 Batch 110/1077 - Train Accuracy: 0.9832, Validation Accuracy: 0.9549, Loss: 0.0446\n", + "Epoch 21 Batch 120/1077 - Train Accuracy: 0.9727, Validation Accuracy: 0.9609, Loss: 0.0583\n", + "Epoch 21 Batch 130/1077 - Train Accuracy: 0.9561, Validation Accuracy: 0.9496, Loss: 0.0474\n", + "Epoch 21 Batch 140/1077 - Train Accuracy: 0.9556, Validation Accuracy: 0.9432, Loss: 0.0455\n", + "Epoch 21 Batch 150/1077 - Train Accuracy: 0.9710, Validation Accuracy: 0.9595, Loss: 0.0553\n", + "Epoch 21 Batch 160/1077 - Train Accuracy: 0.9734, Validation Accuracy: 0.9542, Loss: 0.0480\n", + "Epoch 21 Batch 170/1077 - Train Accuracy: 0.9496, Validation Accuracy: 0.9570, Loss: 0.0640\n", + "Epoch 21 Batch 180/1077 - Train Accuracy: 0.9809, Validation Accuracy: 0.9521, Loss: 0.0526\n", + "Epoch 21 Batch 190/1077 - Train Accuracy: 0.9711, Validation Accuracy: 0.9588, Loss: 0.0526\n", + "Epoch 21 Batch 200/1077 - Train Accuracy: 0.9633, Validation Accuracy: 0.9581, Loss: 0.0576\n", + "Epoch 21 Batch 210/1077 - Train Accuracy: 0.9431, Validation Accuracy: 0.9624, Loss: 0.0602\n", + "Epoch 21 Batch 220/1077 - Train Accuracy: 0.9539, Validation Accuracy: 0.9567, Loss: 0.0452\n", + "Epoch 21 Batch 230/1077 - Train Accuracy: 0.9714, Validation Accuracy: 0.9613, Loss: 0.0461\n", + "Epoch 21 Batch 240/1077 - Train Accuracy: 0.9750, Validation Accuracy: 0.9691, Loss: 0.0526\n", + "Epoch 21 Batch 250/1077 - Train Accuracy: 0.9517, Validation Accuracy: 0.9695, Loss: 0.0484\n", + "Epoch 21 Batch 260/1077 - Train Accuracy: 0.9583, Validation Accuracy: 0.9627, Loss: 0.0418\n", + "Epoch 21 Batch 270/1077 - Train Accuracy: 0.9508, Validation Accuracy: 0.9620, Loss: 0.0516\n", + "Epoch 21 Batch 280/1077 - Train Accuracy: 0.9395, Validation Accuracy: 0.9542, Loss: 0.0650\n", + "Epoch 21 Batch 290/1077 - Train Accuracy: 0.9344, Validation Accuracy: 0.9659, Loss: 0.0815\n", + "Epoch 21 Batch 300/1077 - Train Accuracy: 0.9597, Validation Accuracy: 0.9641, Loss: 0.0522\n", + "Epoch 21 Batch 310/1077 - Train Accuracy: 0.9488, Validation Accuracy: 0.9567, Loss: 0.0588\n", + "Epoch 21 Batch 320/1077 - Train Accuracy: 0.9633, Validation Accuracy: 0.9567, Loss: 0.0571\n", + "Epoch 21 Batch 330/1077 - Train Accuracy: 0.9492, Validation Accuracy: 0.9638, Loss: 0.0685\n", + "Epoch 21 Batch 340/1077 - Train Accuracy: 0.9646, Validation Accuracy: 0.9595, Loss: 0.0558\n", + "Epoch 21 Batch 350/1077 - Train Accuracy: 0.9645, Validation Accuracy: 0.9503, Loss: 0.0485\n", + "Epoch 21 Batch 360/1077 - Train Accuracy: 0.9477, Validation Accuracy: 0.9535, Loss: 0.0493\n", + "Epoch 21 Batch 370/1077 - Train Accuracy: 0.9613, Validation Accuracy: 0.9595, Loss: 0.0599\n", + "Epoch 21 Batch 380/1077 - Train Accuracy: 0.9629, Validation Accuracy: 0.9670, Loss: 0.0585\n", + "Epoch 21 Batch 390/1077 - Train Accuracy: 0.9410, Validation Accuracy: 0.9535, Loss: 0.0747\n", + "Epoch 21 Batch 400/1077 - Train Accuracy: 0.9453, Validation Accuracy: 0.9553, Loss: 0.0731\n", + "Epoch 21 Batch 410/1077 - Train Accuracy: 0.9424, Validation Accuracy: 0.9524, Loss: 0.0708\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 21 Batch 420/1077 - Train Accuracy: 0.9750, Validation Accuracy: 0.9414, Loss: 0.0477\n", + "Epoch 21 Batch 430/1077 - Train Accuracy: 0.9750, Validation Accuracy: 0.9347, Loss: 0.0504\n", + "Epoch 21 Batch 440/1077 - Train Accuracy: 0.9488, Validation Accuracy: 0.9503, Loss: 0.0685\n", + "Epoch 21 Batch 450/1077 - Train Accuracy: 0.9672, Validation Accuracy: 0.9361, Loss: 0.0684\n", + "Epoch 21 Batch 460/1077 - Train Accuracy: 0.9500, Validation Accuracy: 0.9432, Loss: 0.0641\n", + "Epoch 21 Batch 470/1077 - Train Accuracy: 0.9671, Validation Accuracy: 0.9503, Loss: 0.0642\n", + "Epoch 21 Batch 480/1077 - Train Accuracy: 0.9297, Validation Accuracy: 0.9556, Loss: 0.0626\n", + "Epoch 21 Batch 490/1077 - Train Accuracy: 0.9539, Validation Accuracy: 0.9432, Loss: 0.0556\n", + "Epoch 21 Batch 500/1077 - Train Accuracy: 0.9469, Validation Accuracy: 0.9549, Loss: 0.0487\n", + "Epoch 21 Batch 510/1077 - Train Accuracy: 0.9602, Validation Accuracy: 0.9556, Loss: 0.0547\n", + "Epoch 21 Batch 520/1077 - Train Accuracy: 0.9740, Validation Accuracy: 0.9492, Loss: 0.0608\n", + "Epoch 21 Batch 530/1077 - Train Accuracy: 0.9496, Validation Accuracy: 0.9485, Loss: 0.0630\n", + "Epoch 21 Batch 540/1077 - Train Accuracy: 0.9672, Validation Accuracy: 0.9588, Loss: 0.0505\n", + "Epoch 21 Batch 550/1077 - Train Accuracy: 0.9488, Validation Accuracy: 0.9577, Loss: 0.0606\n", + "Epoch 21 Batch 560/1077 - Train Accuracy: 0.9402, Validation Accuracy: 0.9467, Loss: 0.0562\n", + "Epoch 21 Batch 570/1077 - Train Accuracy: 0.9404, Validation Accuracy: 0.9474, Loss: 0.0797\n", + "Epoch 21 Batch 580/1077 - Train Accuracy: 0.9688, Validation Accuracy: 0.9379, Loss: 0.0411\n", + "Epoch 21 Batch 590/1077 - Train Accuracy: 0.9486, Validation Accuracy: 0.9386, Loss: 0.0626\n", + "Epoch 21 Batch 600/1077 - Train Accuracy: 0.9557, Validation Accuracy: 0.9506, Loss: 0.0639\n", + "Epoch 21 Batch 610/1077 - Train Accuracy: 0.9498, Validation Accuracy: 0.9688, Loss: 0.0739\n", + "Epoch 21 Batch 620/1077 - Train Accuracy: 0.9793, Validation Accuracy: 0.9553, Loss: 0.0551\n", + "Epoch 21 Batch 630/1077 - Train Accuracy: 0.9805, Validation Accuracy: 0.9570, Loss: 0.0590\n", + "Epoch 21 Batch 640/1077 - Train Accuracy: 0.9520, Validation Accuracy: 0.9531, Loss: 0.0520\n", + "Epoch 21 Batch 650/1077 - Train Accuracy: 0.9621, Validation Accuracy: 0.9602, Loss: 0.0571\n", + "Epoch 21 Batch 660/1077 - Train Accuracy: 0.9793, Validation Accuracy: 0.9606, Loss: 0.0499\n", + "Epoch 21 Batch 670/1077 - Train Accuracy: 0.9602, Validation Accuracy: 0.9609, Loss: 0.0527\n", + "Epoch 21 Batch 680/1077 - Train Accuracy: 0.9490, Validation Accuracy: 0.9691, Loss: 0.0588\n", + "Epoch 21 Batch 690/1077 - Train Accuracy: 0.9496, Validation Accuracy: 0.9503, Loss: 0.0654\n", + "Epoch 21 Batch 700/1077 - Train Accuracy: 0.9820, Validation Accuracy: 0.9563, Loss: 0.0436\n", + "Epoch 21 Batch 710/1077 - Train Accuracy: 0.9637, Validation Accuracy: 0.9563, Loss: 0.0482\n", + "Epoch 21 Batch 720/1077 - Train Accuracy: 0.9704, Validation Accuracy: 0.9659, Loss: 0.0599\n", + "Epoch 21 Batch 730/1077 - Train Accuracy: 0.9594, Validation Accuracy: 0.9595, Loss: 0.0622\n", + "Epoch 21 Batch 740/1077 - Train Accuracy: 0.9629, Validation Accuracy: 0.9659, Loss: 0.0433\n", + "Epoch 21 Batch 750/1077 - Train Accuracy: 0.9566, Validation Accuracy: 0.9567, Loss: 0.0496\n", + "Epoch 21 Batch 760/1077 - Train Accuracy: 0.9551, Validation Accuracy: 0.9613, Loss: 0.0766\n", + "Epoch 21 Batch 770/1077 - Train Accuracy: 0.9453, Validation Accuracy: 0.9577, Loss: 0.0519\n", + "Epoch 21 Batch 780/1077 - Train Accuracy: 0.9469, Validation Accuracy: 0.9531, Loss: 0.0771\n", + "Epoch 21 Batch 790/1077 - Train Accuracy: 0.9516, Validation Accuracy: 0.9634, Loss: 0.0755\n", + "Epoch 21 Batch 800/1077 - Train Accuracy: 0.9578, Validation Accuracy: 0.9588, Loss: 0.0511\n", + "Epoch 21 Batch 810/1077 - Train Accuracy: 0.9728, Validation Accuracy: 0.9588, Loss: 0.0446\n", + "Epoch 21 Batch 820/1077 - Train Accuracy: 0.9602, Validation Accuracy: 0.9592, Loss: 0.0499\n", + "Epoch 21 Batch 830/1077 - Train Accuracy: 0.9359, Validation Accuracy: 0.9464, Loss: 0.0758\n", + "Epoch 21 Batch 840/1077 - Train Accuracy: 0.9742, Validation Accuracy: 0.9567, Loss: 0.0449\n", + "Epoch 21 Batch 850/1077 - Train Accuracy: 0.9487, Validation Accuracy: 0.9474, Loss: 0.0861\n", + "Epoch 21 Batch 860/1077 - Train Accuracy: 0.9628, Validation Accuracy: 0.9638, Loss: 0.0571\n", + "Epoch 21 Batch 870/1077 - Train Accuracy: 0.9523, Validation Accuracy: 0.9489, Loss: 0.0583\n", + "Epoch 21 Batch 880/1077 - Train Accuracy: 0.9625, Validation Accuracy: 0.9588, Loss: 0.0678\n", + "Epoch 21 Batch 890/1077 - Train Accuracy: 0.9743, Validation Accuracy: 0.9545, Loss: 0.0625\n", + "Epoch 21 Batch 900/1077 - Train Accuracy: 0.9574, Validation Accuracy: 0.9510, Loss: 0.0664\n", + "Epoch 21 Batch 910/1077 - Train Accuracy: 0.9628, Validation Accuracy: 0.9663, Loss: 0.0669\n", + "Epoch 21 Batch 920/1077 - Train Accuracy: 0.9492, Validation Accuracy: 0.9567, Loss: 0.0578\n", + "Epoch 21 Batch 930/1077 - Train Accuracy: 0.9480, Validation Accuracy: 0.9531, Loss: 0.0446\n", + "Epoch 21 Batch 940/1077 - Train Accuracy: 0.9668, Validation Accuracy: 0.9553, Loss: 0.0512\n", + "Epoch 21 Batch 950/1077 - Train Accuracy: 0.9635, Validation Accuracy: 0.9542, Loss: 0.0442\n", + "Epoch 21 Batch 960/1077 - Train Accuracy: 0.9568, Validation Accuracy: 0.9400, Loss: 0.0600\n", + "Epoch 21 Batch 970/1077 - Train Accuracy: 0.9668, Validation Accuracy: 0.9414, Loss: 0.0666\n", + "Epoch 21 Batch 980/1077 - Train Accuracy: 0.9496, Validation Accuracy: 0.9489, Loss: 0.0759\n", + "Epoch 21 Batch 990/1077 - Train Accuracy: 0.9609, Validation Accuracy: 0.9581, Loss: 0.0586\n", + "Epoch 21 Batch 1000/1077 - Train Accuracy: 0.9386, Validation Accuracy: 0.9634, Loss: 0.0718\n", + "Epoch 21 Batch 1010/1077 - Train Accuracy: 0.9738, Validation Accuracy: 0.9634, Loss: 0.0531\n", + "Epoch 21 Batch 1020/1077 - Train Accuracy: 0.9793, Validation Accuracy: 0.9553, Loss: 0.0492\n", + "Epoch 21 Batch 1030/1077 - Train Accuracy: 0.9758, Validation Accuracy: 0.9446, Loss: 0.0619\n", + "Epoch 21 Batch 1040/1077 - Train Accuracy: 0.9778, Validation Accuracy: 0.9474, Loss: 0.0500\n", + "Epoch 21 Batch 1050/1077 - Train Accuracy: 0.9816, Validation Accuracy: 0.9478, Loss: 0.0437\n", + "Epoch 21 Batch 1060/1077 - Train Accuracy: 0.9621, Validation Accuracy: 0.9524, Loss: 0.0493\n", + "Epoch 21 Batch 1070/1077 - Train Accuracy: 0.9547, Validation Accuracy: 0.9609, Loss: 0.0554\n", + "Epoch 22 Batch 10/1077 - Train Accuracy: 0.9305, Validation Accuracy: 0.9435, Loss: 0.0590\n", + "Epoch 22 Batch 20/1077 - Train Accuracy: 0.9594, Validation Accuracy: 0.9556, Loss: 0.0577\n", + "Epoch 22 Batch 30/1077 - Train Accuracy: 0.9605, Validation Accuracy: 0.9659, Loss: 0.0463\n", + "Epoch 22 Batch 40/1077 - Train Accuracy: 0.9754, Validation Accuracy: 0.9684, Loss: 0.0488\n", + "Epoch 22 Batch 50/1077 - Train Accuracy: 0.9648, Validation Accuracy: 0.9759, Loss: 0.0468\n", + "Epoch 22 Batch 60/1077 - Train Accuracy: 0.9699, Validation Accuracy: 0.9641, Loss: 0.0455\n", + "Epoch 22 Batch 70/1077 - Train Accuracy: 0.9572, Validation Accuracy: 0.9489, Loss: 0.0593\n", + "Epoch 22 Batch 80/1077 - Train Accuracy: 0.9684, Validation Accuracy: 0.9620, Loss: 0.0504\n", + "Epoch 22 Batch 90/1077 - Train Accuracy: 0.9594, Validation Accuracy: 0.9549, Loss: 0.0453\n", + "Epoch 22 Batch 100/1077 - Train Accuracy: 0.9613, Validation Accuracy: 0.9411, Loss: 0.0500\n", + "Epoch 22 Batch 110/1077 - Train Accuracy: 0.9734, Validation Accuracy: 0.9538, Loss: 0.0481\n", + "Epoch 22 Batch 120/1077 - Train Accuracy: 0.9633, Validation Accuracy: 0.9616, Loss: 0.0637\n", + "Epoch 22 Batch 130/1077 - Train Accuracy: 0.9535, Validation Accuracy: 0.9602, Loss: 0.0429\n", + "Epoch 22 Batch 140/1077 - Train Accuracy: 0.9482, Validation Accuracy: 0.9634, Loss: 0.0538\n", + "Epoch 22 Batch 150/1077 - Train Accuracy: 0.9654, Validation Accuracy: 0.9609, Loss: 0.0651\n", + "Epoch 22 Batch 160/1077 - Train Accuracy: 0.9629, Validation Accuracy: 0.9549, Loss: 0.0480\n", + "Epoch 22 Batch 170/1077 - Train Accuracy: 0.9426, Validation Accuracy: 0.9577, Loss: 0.0716\n", + "Epoch 22 Batch 180/1077 - Train Accuracy: 0.9816, Validation Accuracy: 0.9592, Loss: 0.0449\n", + "Epoch 22 Batch 190/1077 - Train Accuracy: 0.9707, Validation Accuracy: 0.9531, Loss: 0.0490\n", + "Epoch 22 Batch 200/1077 - Train Accuracy: 0.9676, Validation Accuracy: 0.9581, Loss: 0.0683\n", + "Epoch 22 Batch 210/1077 - Train Accuracy: 0.9475, Validation Accuracy: 0.9482, Loss: 0.0649\n", + "Epoch 22 Batch 220/1077 - Train Accuracy: 0.9659, Validation Accuracy: 0.9489, Loss: 0.0582\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 22 Batch 230/1077 - Train Accuracy: 0.9792, Validation Accuracy: 0.9560, Loss: 0.0614\n", + "Epoch 22 Batch 240/1077 - Train Accuracy: 0.9512, Validation Accuracy: 0.9521, Loss: 0.0624\n", + "Epoch 22 Batch 250/1077 - Train Accuracy: 0.9446, Validation Accuracy: 0.9545, Loss: 0.0511\n", + "Epoch 22 Batch 260/1077 - Train Accuracy: 0.9565, Validation Accuracy: 0.9588, Loss: 0.0452\n", + "Epoch 22 Batch 270/1077 - Train Accuracy: 0.9477, Validation Accuracy: 0.9577, Loss: 0.0606\n", + "Epoch 22 Batch 280/1077 - Train Accuracy: 0.9363, Validation Accuracy: 0.9609, Loss: 0.0627\n", + "Epoch 22 Batch 290/1077 - Train Accuracy: 0.9430, Validation Accuracy: 0.9570, Loss: 0.0765\n", + "Epoch 22 Batch 300/1077 - Train Accuracy: 0.9535, Validation Accuracy: 0.9570, Loss: 0.0485\n", + "Epoch 22 Batch 310/1077 - Train Accuracy: 0.9637, Validation Accuracy: 0.9553, Loss: 0.0506\n", + "Epoch 22 Batch 320/1077 - Train Accuracy: 0.9734, Validation Accuracy: 0.9602, Loss: 0.0501\n", + "Epoch 22 Batch 330/1077 - Train Accuracy: 0.9488, Validation Accuracy: 0.9627, Loss: 0.0632\n", + "Epoch 22 Batch 340/1077 - Train Accuracy: 0.9807, Validation Accuracy: 0.9634, Loss: 0.0503\n", + "Epoch 22 Batch 350/1077 - Train Accuracy: 0.9551, Validation Accuracy: 0.9688, Loss: 0.0561\n", + "Epoch 22 Batch 360/1077 - Train Accuracy: 0.9578, Validation Accuracy: 0.9542, Loss: 0.0434\n", + "Epoch 22 Batch 370/1077 - Train Accuracy: 0.9561, Validation Accuracy: 0.9574, Loss: 0.0519\n", + "Epoch 22 Batch 380/1077 - Train Accuracy: 0.9707, Validation Accuracy: 0.9585, Loss: 0.0555\n", + "Epoch 22 Batch 390/1077 - Train Accuracy: 0.9496, Validation Accuracy: 0.9513, Loss: 0.0615\n", + "Epoch 22 Batch 400/1077 - Train Accuracy: 0.9469, Validation Accuracy: 0.9638, Loss: 0.0619\n", + "Epoch 22 Batch 410/1077 - Train Accuracy: 0.9396, Validation Accuracy: 0.9656, Loss: 0.0666\n", + "Epoch 22 Batch 420/1077 - Train Accuracy: 0.9645, Validation Accuracy: 0.9556, Loss: 0.0461\n", + "Epoch 22 Batch 430/1077 - Train Accuracy: 0.9730, Validation Accuracy: 0.9599, Loss: 0.0583\n", + "Epoch 22 Batch 440/1077 - Train Accuracy: 0.9375, Validation Accuracy: 0.9631, Loss: 0.0642\n", + "Epoch 22 Batch 450/1077 - Train Accuracy: 0.9691, Validation Accuracy: 0.9567, Loss: 0.0646\n", + "Epoch 22 Batch 460/1077 - Train Accuracy: 0.9477, Validation Accuracy: 0.9442, Loss: 0.0592\n", + "Epoch 22 Batch 470/1077 - Train Accuracy: 0.9811, Validation Accuracy: 0.9521, Loss: 0.0541\n", + "Epoch 22 Batch 480/1077 - Train Accuracy: 0.9416, Validation Accuracy: 0.9521, Loss: 0.0561\n", + "Epoch 22 Batch 490/1077 - Train Accuracy: 0.9574, Validation Accuracy: 0.9538, Loss: 0.0551\n", + "Epoch 22 Batch 500/1077 - Train Accuracy: 0.9500, Validation Accuracy: 0.9474, Loss: 0.0410\n", + "Epoch 22 Batch 510/1077 - Train Accuracy: 0.9508, Validation Accuracy: 0.9450, Loss: 0.0525\n", + "Epoch 22 Batch 520/1077 - Train Accuracy: 0.9743, Validation Accuracy: 0.9506, Loss: 0.0485\n", + "Epoch 22 Batch 530/1077 - Train Accuracy: 0.9539, Validation Accuracy: 0.9542, Loss: 0.0506\n", + "Epoch 22 Batch 540/1077 - Train Accuracy: 0.9711, Validation Accuracy: 0.9499, Loss: 0.0460\n", + "Epoch 22 Batch 550/1077 - Train Accuracy: 0.9480, Validation Accuracy: 0.9560, Loss: 0.0545\n", + "Epoch 22 Batch 560/1077 - Train Accuracy: 0.9500, Validation Accuracy: 0.9620, Loss: 0.0849\n", + "Epoch 22 Batch 570/1077 - Train Accuracy: 0.9383, Validation Accuracy: 0.9485, Loss: 0.0817\n", + "Epoch 22 Batch 580/1077 - Train Accuracy: 0.9602, Validation Accuracy: 0.9528, Loss: 0.0444\n", + "Epoch 22 Batch 590/1077 - Train Accuracy: 0.9515, Validation Accuracy: 0.9599, Loss: 0.0661\n", + "Epoch 22 Batch 600/1077 - Train Accuracy: 0.9661, Validation Accuracy: 0.9631, Loss: 0.0512\n", + "Epoch 22 Batch 610/1077 - Train Accuracy: 0.9482, Validation Accuracy: 0.9652, Loss: 0.0621\n", + "Epoch 22 Batch 620/1077 - Train Accuracy: 0.9781, Validation Accuracy: 0.9656, Loss: 0.0477\n", + "Epoch 22 Batch 630/1077 - Train Accuracy: 0.9746, Validation Accuracy: 0.9656, Loss: 0.0527\n", + "Epoch 22 Batch 640/1077 - Train Accuracy: 0.9449, Validation Accuracy: 0.9680, Loss: 0.0552\n", + "Epoch 22 Batch 650/1077 - Train Accuracy: 0.9660, Validation Accuracy: 0.9709, Loss: 0.0559\n", + "Epoch 22 Batch 660/1077 - Train Accuracy: 0.9727, Validation Accuracy: 0.9656, Loss: 0.0504\n", + "Epoch 22 Batch 670/1077 - Train Accuracy: 0.9545, Validation Accuracy: 0.9691, Loss: 0.0610\n", + "Epoch 22 Batch 680/1077 - Train Accuracy: 0.9557, Validation Accuracy: 0.9521, Loss: 0.0480\n", + "Epoch 22 Batch 690/1077 - Train Accuracy: 0.9582, Validation Accuracy: 0.9588, Loss: 0.0561\n", + "Epoch 22 Batch 700/1077 - Train Accuracy: 0.9699, Validation Accuracy: 0.9538, Loss: 0.0436\n", + "Epoch 22 Batch 710/1077 - Train Accuracy: 0.9660, Validation Accuracy: 0.9638, Loss: 0.0563\n", + "Epoch 22 Batch 720/1077 - Train Accuracy: 0.9700, Validation Accuracy: 0.9581, Loss: 0.0655\n", + "Epoch 22 Batch 730/1077 - Train Accuracy: 0.9543, Validation Accuracy: 0.9673, Loss: 0.0751\n", + "Epoch 22 Batch 740/1077 - Train Accuracy: 0.9754, Validation Accuracy: 0.9627, Loss: 0.0401\n", + "Epoch 22 Batch 750/1077 - Train Accuracy: 0.9449, Validation Accuracy: 0.9631, Loss: 0.0568\n", + "Epoch 22 Batch 760/1077 - Train Accuracy: 0.9586, Validation Accuracy: 0.9620, Loss: 0.0569\n", + "Epoch 22 Batch 770/1077 - Train Accuracy: 0.9323, Validation Accuracy: 0.9602, Loss: 0.0555\n", + "Epoch 22 Batch 780/1077 - Train Accuracy: 0.9379, Validation Accuracy: 0.9673, Loss: 0.0822\n", + "Epoch 22 Batch 790/1077 - Train Accuracy: 0.9336, Validation Accuracy: 0.9524, Loss: 0.0595\n", + "Epoch 22 Batch 800/1077 - Train Accuracy: 0.9570, Validation Accuracy: 0.9659, Loss: 0.0504\n", + "Epoch 22 Batch 810/1077 - Train Accuracy: 0.9673, Validation Accuracy: 0.9698, Loss: 0.0487\n", + "Epoch 22 Batch 820/1077 - Train Accuracy: 0.9715, Validation Accuracy: 0.9542, Loss: 0.0589\n", + "Epoch 22 Batch 830/1077 - Train Accuracy: 0.9316, Validation Accuracy: 0.9616, Loss: 0.0527\n", + "Epoch 22 Batch 840/1077 - Train Accuracy: 0.9883, Validation Accuracy: 0.9638, Loss: 0.0563\n", + "Epoch 22 Batch 850/1077 - Train Accuracy: 0.9594, Validation Accuracy: 0.9624, Loss: 0.0634\n", + "Epoch 22 Batch 860/1077 - Train Accuracy: 0.9624, Validation Accuracy: 0.9691, Loss: 0.0625\n", + "Epoch 22 Batch 870/1077 - Train Accuracy: 0.9601, Validation Accuracy: 0.9719, Loss: 0.0540\n", + "Epoch 22 Batch 880/1077 - Train Accuracy: 0.9582, Validation Accuracy: 0.9695, Loss: 0.0748\n", + "Epoch 22 Batch 890/1077 - Train Accuracy: 0.9676, Validation Accuracy: 0.9780, Loss: 0.0525\n", + "Epoch 22 Batch 900/1077 - Train Accuracy: 0.9406, Validation Accuracy: 0.9631, Loss: 0.0610\n", + "Epoch 22 Batch 910/1077 - Train Accuracy: 0.9736, Validation Accuracy: 0.9659, Loss: 0.0519\n", + "Epoch 22 Batch 920/1077 - Train Accuracy: 0.9664, Validation Accuracy: 0.9691, Loss: 0.0498\n", + "Epoch 22 Batch 930/1077 - Train Accuracy: 0.9539, Validation Accuracy: 0.9684, Loss: 0.0512\n", + "Epoch 22 Batch 940/1077 - Train Accuracy: 0.9738, Validation Accuracy: 0.9659, Loss: 0.0522\n", + "Epoch 22 Batch 950/1077 - Train Accuracy: 0.9654, Validation Accuracy: 0.9730, Loss: 0.0471\n", + "Epoch 22 Batch 960/1077 - Train Accuracy: 0.9505, Validation Accuracy: 0.9599, Loss: 0.0535\n", + "Epoch 22 Batch 970/1077 - Train Accuracy: 0.9645, Validation Accuracy: 0.9503, Loss: 0.0613\n", + "Epoch 22 Batch 980/1077 - Train Accuracy: 0.9383, Validation Accuracy: 0.9727, Loss: 0.0658\n", + "Epoch 22 Batch 990/1077 - Train Accuracy: 0.9737, Validation Accuracy: 0.9620, Loss: 0.0542\n", + "Epoch 22 Batch 1000/1077 - Train Accuracy: 0.9319, Validation Accuracy: 0.9588, Loss: 0.0578\n", + "Epoch 22 Batch 1010/1077 - Train Accuracy: 0.9699, Validation Accuracy: 0.9737, Loss: 0.0520\n", + "Epoch 22 Batch 1020/1077 - Train Accuracy: 0.9863, Validation Accuracy: 0.9585, Loss: 0.0524\n", + "Epoch 22 Batch 1030/1077 - Train Accuracy: 0.9836, Validation Accuracy: 0.9577, Loss: 0.0538\n", + "Epoch 22 Batch 1040/1077 - Train Accuracy: 0.9663, Validation Accuracy: 0.9545, Loss: 0.0518\n", + "Epoch 22 Batch 1050/1077 - Train Accuracy: 0.9746, Validation Accuracy: 0.9545, Loss: 0.0503\n", + "Epoch 22 Batch 1060/1077 - Train Accuracy: 0.9539, Validation Accuracy: 0.9634, Loss: 0.0365\n", + "Epoch 22 Batch 1070/1077 - Train Accuracy: 0.9516, Validation Accuracy: 0.9535, Loss: 0.0477\n", + "Epoch 23 Batch 10/1077 - Train Accuracy: 0.9511, Validation Accuracy: 0.9510, Loss: 0.0542\n", + "Epoch 23 Batch 20/1077 - Train Accuracy: 0.9719, Validation Accuracy: 0.9513, Loss: 0.0485\n", + "Epoch 23 Batch 30/1077 - Train Accuracy: 0.9754, Validation Accuracy: 0.9581, Loss: 0.0456\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 23 Batch 40/1077 - Train Accuracy: 0.9719, Validation Accuracy: 0.9641, Loss: 0.0512\n", + "Epoch 23 Batch 50/1077 - Train Accuracy: 0.9703, Validation Accuracy: 0.9588, Loss: 0.0426\n", + "Epoch 23 Batch 60/1077 - Train Accuracy: 0.9773, Validation Accuracy: 0.9549, Loss: 0.0347\n", + "Epoch 23 Batch 70/1077 - Train Accuracy: 0.9486, Validation Accuracy: 0.9609, Loss: 0.0550\n", + "Epoch 23 Batch 80/1077 - Train Accuracy: 0.9633, Validation Accuracy: 0.9592, Loss: 0.0394\n", + "Epoch 23 Batch 90/1077 - Train Accuracy: 0.9605, Validation Accuracy: 0.9648, Loss: 0.0508\n", + "Epoch 23 Batch 100/1077 - Train Accuracy: 0.9590, Validation Accuracy: 0.9645, Loss: 0.0526\n", + "Epoch 23 Batch 110/1077 - Train Accuracy: 0.9848, Validation Accuracy: 0.9684, Loss: 0.0402\n", + "Epoch 23 Batch 120/1077 - Train Accuracy: 0.9602, Validation Accuracy: 0.9734, Loss: 0.0592\n", + "Epoch 23 Batch 130/1077 - Train Accuracy: 0.9732, Validation Accuracy: 0.9737, Loss: 0.0429\n", + "Epoch 23 Batch 140/1077 - Train Accuracy: 0.9630, Validation Accuracy: 0.9759, Loss: 0.0429\n", + "Epoch 23 Batch 150/1077 - Train Accuracy: 0.9665, Validation Accuracy: 0.9652, Loss: 0.0513\n", + "Epoch 23 Batch 160/1077 - Train Accuracy: 0.9656, Validation Accuracy: 0.9517, Loss: 0.0489\n", + "Epoch 23 Batch 170/1077 - Train Accuracy: 0.9563, Validation Accuracy: 0.9631, Loss: 0.0530\n", + "Epoch 23 Batch 180/1077 - Train Accuracy: 0.9754, Validation Accuracy: 0.9521, Loss: 0.0465\n", + "Epoch 23 Batch 190/1077 - Train Accuracy: 0.9641, Validation Accuracy: 0.9542, Loss: 0.0508\n", + "Epoch 23 Batch 200/1077 - Train Accuracy: 0.9648, Validation Accuracy: 0.9599, Loss: 0.0641\n", + "Epoch 23 Batch 210/1077 - Train Accuracy: 0.9319, Validation Accuracy: 0.9531, Loss: 0.0624\n", + "Epoch 23 Batch 220/1077 - Train Accuracy: 0.9478, Validation Accuracy: 0.9499, Loss: 0.0459\n", + "Epoch 23 Batch 230/1077 - Train Accuracy: 0.9654, Validation Accuracy: 0.9613, Loss: 0.0506\n", + "Epoch 23 Batch 240/1077 - Train Accuracy: 0.9617, Validation Accuracy: 0.9570, Loss: 0.0509\n", + "Epoch 23 Batch 250/1077 - Train Accuracy: 0.9418, Validation Accuracy: 0.9634, Loss: 0.0489\n", + "Epoch 23 Batch 260/1077 - Train Accuracy: 0.9565, Validation Accuracy: 0.9638, Loss: 0.0486\n", + "Epoch 23 Batch 270/1077 - Train Accuracy: 0.9648, Validation Accuracy: 0.9588, Loss: 0.0564\n", + "Epoch 23 Batch 280/1077 - Train Accuracy: 0.9480, Validation Accuracy: 0.9620, Loss: 0.0553\n", + "Epoch 23 Batch 290/1077 - Train Accuracy: 0.9441, Validation Accuracy: 0.9542, Loss: 0.0814\n", + "Epoch 23 Batch 300/1077 - Train Accuracy: 0.9589, Validation Accuracy: 0.9641, Loss: 0.0498\n", + "Epoch 23 Batch 310/1077 - Train Accuracy: 0.9625, Validation Accuracy: 0.9581, Loss: 0.0476\n", + "Epoch 23 Batch 320/1077 - Train Accuracy: 0.9605, Validation Accuracy: 0.9574, Loss: 0.0627\n", + "Epoch 23 Batch 330/1077 - Train Accuracy: 0.9437, Validation Accuracy: 0.9599, Loss: 0.0564\n", + "Epoch 23 Batch 340/1077 - Train Accuracy: 0.9799, Validation Accuracy: 0.9595, Loss: 0.0518\n", + "Epoch 23 Batch 350/1077 - Train Accuracy: 0.9453, Validation Accuracy: 0.9616, Loss: 0.0465\n", + "Epoch 23 Batch 360/1077 - Train Accuracy: 0.9598, Validation Accuracy: 0.9503, Loss: 0.0495\n", + "Epoch 23 Batch 370/1077 - Train Accuracy: 0.9661, Validation Accuracy: 0.9560, Loss: 0.0444\n", + "Epoch 23 Batch 380/1077 - Train Accuracy: 0.9824, Validation Accuracy: 0.9521, Loss: 0.0535\n", + "Epoch 23 Batch 390/1077 - Train Accuracy: 0.9516, Validation Accuracy: 0.9585, Loss: 0.0647\n", + "Epoch 23 Batch 400/1077 - Train Accuracy: 0.9574, Validation Accuracy: 0.9606, Loss: 0.0647\n", + "Epoch 23 Batch 410/1077 - Train Accuracy: 0.9334, Validation Accuracy: 0.9616, Loss: 0.0639\n", + "Epoch 23 Batch 420/1077 - Train Accuracy: 0.9695, Validation Accuracy: 0.9712, Loss: 0.0458\n", + "Epoch 23 Batch 430/1077 - Train Accuracy: 0.9707, Validation Accuracy: 0.9581, Loss: 0.0450\n", + "Epoch 23 Batch 440/1077 - Train Accuracy: 0.9437, Validation Accuracy: 0.9592, Loss: 0.0765\n", + "Epoch 23 Batch 450/1077 - Train Accuracy: 0.9727, Validation Accuracy: 0.9517, Loss: 0.0655\n", + "Epoch 23 Batch 460/1077 - Train Accuracy: 0.9578, Validation Accuracy: 0.9499, Loss: 0.0595\n", + "Epoch 23 Batch 470/1077 - Train Accuracy: 0.9700, Validation Accuracy: 0.9428, Loss: 0.0536\n", + "Epoch 23 Batch 480/1077 - Train Accuracy: 0.9412, Validation Accuracy: 0.9545, Loss: 0.0524\n", + "Epoch 23 Batch 490/1077 - Train Accuracy: 0.9570, Validation Accuracy: 0.9577, Loss: 0.0670\n", + "Epoch 23 Batch 500/1077 - Train Accuracy: 0.9555, Validation Accuracy: 0.9407, Loss: 0.0405\n", + "Epoch 23 Batch 510/1077 - Train Accuracy: 0.9559, Validation Accuracy: 0.9442, Loss: 0.0520\n", + "Epoch 23 Batch 520/1077 - Train Accuracy: 0.9881, Validation Accuracy: 0.9677, Loss: 0.0453\n", + "Epoch 23 Batch 530/1077 - Train Accuracy: 0.9664, Validation Accuracy: 0.9673, Loss: 0.0651\n", + "Epoch 23 Batch 540/1077 - Train Accuracy: 0.9617, Validation Accuracy: 0.9588, Loss: 0.0417\n", + "Epoch 23 Batch 550/1077 - Train Accuracy: 0.9398, Validation Accuracy: 0.9585, Loss: 0.0499\n", + "Epoch 23 Batch 560/1077 - Train Accuracy: 0.9461, Validation Accuracy: 0.9538, Loss: 0.0538\n", + "Epoch 23 Batch 570/1077 - Train Accuracy: 0.9490, Validation Accuracy: 0.9585, Loss: 0.0676\n", + "Epoch 23 Batch 580/1077 - Train Accuracy: 0.9762, Validation Accuracy: 0.9556, Loss: 0.0411\n", + "Epoch 23 Batch 590/1077 - Train Accuracy: 0.9400, Validation Accuracy: 0.9627, Loss: 0.0678\n", + "Epoch 23 Batch 600/1077 - Train Accuracy: 0.9680, Validation Accuracy: 0.9528, Loss: 0.0466\n", + "Epoch 23 Batch 610/1077 - Train Accuracy: 0.9428, Validation Accuracy: 0.9698, Loss: 0.0611\n", + "Epoch 23 Batch 620/1077 - Train Accuracy: 0.9645, Validation Accuracy: 0.9709, Loss: 0.0526\n", + "Epoch 23 Batch 630/1077 - Train Accuracy: 0.9699, Validation Accuracy: 0.9702, Loss: 0.0539\n", + "Epoch 23 Batch 640/1077 - Train Accuracy: 0.9505, Validation Accuracy: 0.9688, Loss: 0.0523\n", + "Epoch 23 Batch 650/1077 - Train Accuracy: 0.9617, Validation Accuracy: 0.9545, Loss: 0.0471\n", + "Epoch 23 Batch 660/1077 - Train Accuracy: 0.9785, Validation Accuracy: 0.9613, Loss: 0.0463\n", + "Epoch 23 Batch 670/1077 - Train Accuracy: 0.9492, Validation Accuracy: 0.9641, Loss: 0.0562\n", + "Epoch 23 Batch 680/1077 - Train Accuracy: 0.9423, Validation Accuracy: 0.9616, Loss: 0.0515\n", + "Epoch 23 Batch 690/1077 - Train Accuracy: 0.9488, Validation Accuracy: 0.9506, Loss: 0.0528\n", + "Epoch 23 Batch 700/1077 - Train Accuracy: 0.9699, Validation Accuracy: 0.9581, Loss: 0.0412\n", + "Epoch 23 Batch 710/1077 - Train Accuracy: 0.9582, Validation Accuracy: 0.9553, Loss: 0.0499\n", + "Epoch 23 Batch 720/1077 - Train Accuracy: 0.9585, Validation Accuracy: 0.9645, Loss: 0.0632\n", + "Epoch 23 Batch 730/1077 - Train Accuracy: 0.9625, Validation Accuracy: 0.9691, Loss: 0.0753\n", + "Epoch 23 Batch 740/1077 - Train Accuracy: 0.9625, Validation Accuracy: 0.9648, Loss: 0.0398\n", + "Epoch 23 Batch 750/1077 - Train Accuracy: 0.9441, Validation Accuracy: 0.9606, Loss: 0.0517\n", + "Epoch 23 Batch 760/1077 - Train Accuracy: 0.9688, Validation Accuracy: 0.9585, Loss: 0.0617\n", + "Epoch 23 Batch 770/1077 - Train Accuracy: 0.9528, Validation Accuracy: 0.9631, Loss: 0.0498\n", + "Epoch 23 Batch 780/1077 - Train Accuracy: 0.9520, Validation Accuracy: 0.9656, Loss: 0.0703\n", + "Epoch 23 Batch 790/1077 - Train Accuracy: 0.9527, Validation Accuracy: 0.9656, Loss: 0.0605\n", + "Epoch 23 Batch 800/1077 - Train Accuracy: 0.9547, Validation Accuracy: 0.9702, Loss: 0.0580\n", + "Epoch 23 Batch 810/1077 - Train Accuracy: 0.9699, Validation Accuracy: 0.9631, Loss: 0.0404\n", + "Epoch 23 Batch 820/1077 - Train Accuracy: 0.9828, Validation Accuracy: 0.9698, Loss: 0.0511\n", + "Epoch 23 Batch 830/1077 - Train Accuracy: 0.9207, Validation Accuracy: 0.9556, Loss: 0.0720\n", + "Epoch 23 Batch 840/1077 - Train Accuracy: 0.9785, Validation Accuracy: 0.9677, Loss: 0.0428\n", + "Epoch 23 Batch 850/1077 - Train Accuracy: 0.9639, Validation Accuracy: 0.9631, Loss: 0.0616\n", + "Epoch 23 Batch 860/1077 - Train Accuracy: 0.9643, Validation Accuracy: 0.9606, Loss: 0.0603\n", + "Epoch 23 Batch 870/1077 - Train Accuracy: 0.9593, Validation Accuracy: 0.9727, Loss: 0.0482\n", + "Epoch 23 Batch 880/1077 - Train Accuracy: 0.9598, Validation Accuracy: 0.9677, Loss: 0.0587\n", + "Epoch 23 Batch 890/1077 - Train Accuracy: 0.9777, Validation Accuracy: 0.9677, Loss: 0.0485\n", + "Epoch 23 Batch 900/1077 - Train Accuracy: 0.9523, Validation Accuracy: 0.9638, Loss: 0.0563\n", + "Epoch 23 Batch 910/1077 - Train Accuracy: 0.9740, Validation Accuracy: 0.9652, Loss: 0.0524\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 23 Batch 920/1077 - Train Accuracy: 0.9633, Validation Accuracy: 0.9645, Loss: 0.0441\n", + "Epoch 23 Batch 930/1077 - Train Accuracy: 0.9559, Validation Accuracy: 0.9567, Loss: 0.0435\n", + "Epoch 23 Batch 940/1077 - Train Accuracy: 0.9688, Validation Accuracy: 0.9631, Loss: 0.0412\n", + "Epoch 23 Batch 950/1077 - Train Accuracy: 0.9673, Validation Accuracy: 0.9624, Loss: 0.0369\n", + "Epoch 23 Batch 960/1077 - Train Accuracy: 0.9654, Validation Accuracy: 0.9581, Loss: 0.0460\n", + "Epoch 23 Batch 970/1077 - Train Accuracy: 0.9625, Validation Accuracy: 0.9570, Loss: 0.0585\n", + "Epoch 23 Batch 980/1077 - Train Accuracy: 0.9582, Validation Accuracy: 0.9641, Loss: 0.0580\n", + "Epoch 23 Batch 990/1077 - Train Accuracy: 0.9790, Validation Accuracy: 0.9638, Loss: 0.0519\n", + "Epoch 23 Batch 1000/1077 - Train Accuracy: 0.9401, Validation Accuracy: 0.9666, Loss: 0.0652\n", + "Epoch 23 Batch 1010/1077 - Train Accuracy: 0.9711, Validation Accuracy: 0.9606, Loss: 0.0411\n", + "Epoch 23 Batch 1020/1077 - Train Accuracy: 0.9867, Validation Accuracy: 0.9684, Loss: 0.0520\n", + "Epoch 23 Batch 1030/1077 - Train Accuracy: 0.9594, Validation Accuracy: 0.9560, Loss: 0.0717\n", + "Epoch 23 Batch 1040/1077 - Train Accuracy: 0.9581, Validation Accuracy: 0.9709, Loss: 0.0686\n", + "Epoch 23 Batch 1050/1077 - Train Accuracy: 0.9773, Validation Accuracy: 0.9673, Loss: 0.0443\n", + "Epoch 23 Batch 1060/1077 - Train Accuracy: 0.9691, Validation Accuracy: 0.9762, Loss: 0.0422\n", + "Epoch 23 Batch 1070/1077 - Train Accuracy: 0.9492, Validation Accuracy: 0.9645, Loss: 0.0516\n", + "Epoch 24 Batch 10/1077 - Train Accuracy: 0.9498, Validation Accuracy: 0.9627, Loss: 0.0565\n", + "Epoch 24 Batch 20/1077 - Train Accuracy: 0.9707, Validation Accuracy: 0.9734, Loss: 0.0537\n", + "Epoch 24 Batch 30/1077 - Train Accuracy: 0.9652, Validation Accuracy: 0.9609, Loss: 0.0561\n", + "Epoch 24 Batch 40/1077 - Train Accuracy: 0.9715, Validation Accuracy: 0.9695, Loss: 0.0375\n", + "Epoch 24 Batch 50/1077 - Train Accuracy: 0.9664, Validation Accuracy: 0.9595, Loss: 0.0564\n", + "Epoch 24 Batch 60/1077 - Train Accuracy: 0.9710, Validation Accuracy: 0.9673, Loss: 0.0555\n", + "Epoch 24 Batch 70/1077 - Train Accuracy: 0.9482, Validation Accuracy: 0.9688, Loss: 0.0580\n", + "Epoch 24 Batch 80/1077 - Train Accuracy: 0.9625, Validation Accuracy: 0.9698, Loss: 0.0491\n", + "Epoch 24 Batch 90/1077 - Train Accuracy: 0.9535, Validation Accuracy: 0.9570, Loss: 0.0480\n", + "Epoch 24 Batch 100/1077 - Train Accuracy: 0.9641, Validation Accuracy: 0.9581, Loss: 0.0462\n", + "Epoch 24 Batch 110/1077 - Train Accuracy: 0.9734, Validation Accuracy: 0.9624, Loss: 0.0416\n", + "Epoch 24 Batch 120/1077 - Train Accuracy: 0.9602, Validation Accuracy: 0.9652, Loss: 0.0574\n", + "Epoch 24 Batch 130/1077 - Train Accuracy: 0.9643, Validation Accuracy: 0.9716, Loss: 0.0435\n", + "Epoch 24 Batch 140/1077 - Train Accuracy: 0.9515, Validation Accuracy: 0.9609, Loss: 0.0472\n", + "Epoch 24 Batch 150/1077 - Train Accuracy: 0.9632, Validation Accuracy: 0.9567, Loss: 0.0500\n", + "Epoch 24 Batch 160/1077 - Train Accuracy: 0.9742, Validation Accuracy: 0.9467, Loss: 0.0416\n", + "Epoch 24 Batch 170/1077 - Train Accuracy: 0.9480, Validation Accuracy: 0.9513, Loss: 0.0719\n", + "Epoch 24 Batch 180/1077 - Train Accuracy: 0.9852, Validation Accuracy: 0.9613, Loss: 0.0353\n", + "Epoch 24 Batch 190/1077 - Train Accuracy: 0.9758, Validation Accuracy: 0.9688, Loss: 0.0419\n", + "Epoch 24 Batch 200/1077 - Train Accuracy: 0.9727, Validation Accuracy: 0.9688, Loss: 0.0577\n", + "Epoch 24 Batch 210/1077 - Train Accuracy: 0.9379, Validation Accuracy: 0.9645, Loss: 0.0497\n", + "Epoch 24 Batch 220/1077 - Train Accuracy: 0.9634, Validation Accuracy: 0.9698, Loss: 0.0574\n", + "Epoch 24 Batch 230/1077 - Train Accuracy: 0.9803, Validation Accuracy: 0.9691, Loss: 0.0475\n", + "Epoch 24 Batch 240/1077 - Train Accuracy: 0.9836, Validation Accuracy: 0.9659, Loss: 0.0436\n", + "Epoch 24 Batch 250/1077 - Train Accuracy: 0.9531, Validation Accuracy: 0.9627, Loss: 0.0469\n", + "Epoch 24 Batch 260/1077 - Train Accuracy: 0.9568, Validation Accuracy: 0.9677, Loss: 0.0337\n", + "Epoch 24 Batch 270/1077 - Train Accuracy: 0.9484, Validation Accuracy: 0.9663, Loss: 0.0586\n", + "Epoch 24 Batch 280/1077 - Train Accuracy: 0.9422, Validation Accuracy: 0.9616, Loss: 0.0658\n", + "Epoch 24 Batch 290/1077 - Train Accuracy: 0.9383, Validation Accuracy: 0.9581, Loss: 0.0609\n", + "Epoch 24 Batch 300/1077 - Train Accuracy: 0.9708, Validation Accuracy: 0.9592, Loss: 0.0354\n", + "Epoch 24 Batch 310/1077 - Train Accuracy: 0.9660, Validation Accuracy: 0.9574, Loss: 0.0485\n", + "Epoch 24 Batch 320/1077 - Train Accuracy: 0.9637, Validation Accuracy: 0.9506, Loss: 0.0520\n", + "Epoch 24 Batch 330/1077 - Train Accuracy: 0.9520, Validation Accuracy: 0.9556, Loss: 0.0498\n", + "Epoch 24 Batch 340/1077 - Train Accuracy: 0.9745, Validation Accuracy: 0.9691, Loss: 0.0510\n", + "Epoch 24 Batch 350/1077 - Train Accuracy: 0.9520, Validation Accuracy: 0.9695, Loss: 0.0480\n", + "Epoch 24 Batch 360/1077 - Train Accuracy: 0.9617, Validation Accuracy: 0.9631, Loss: 0.0401\n", + "Epoch 24 Batch 370/1077 - Train Accuracy: 0.9647, Validation Accuracy: 0.9602, Loss: 0.0491\n", + "Epoch 24 Batch 380/1077 - Train Accuracy: 0.9723, Validation Accuracy: 0.9659, Loss: 0.0415\n", + "Epoch 24 Batch 390/1077 - Train Accuracy: 0.9504, Validation Accuracy: 0.9613, Loss: 0.0546\n", + "Epoch 24 Batch 400/1077 - Train Accuracy: 0.9609, Validation Accuracy: 0.9620, Loss: 0.0631\n", + "Epoch 24 Batch 410/1077 - Train Accuracy: 0.9215, Validation Accuracy: 0.9631, Loss: 0.0662\n", + "Epoch 24 Batch 420/1077 - Train Accuracy: 0.9699, Validation Accuracy: 0.9606, Loss: 0.0417\n", + "Epoch 24 Batch 430/1077 - Train Accuracy: 0.9758, Validation Accuracy: 0.9620, Loss: 0.0435\n", + "Epoch 24 Batch 440/1077 - Train Accuracy: 0.9430, Validation Accuracy: 0.9688, Loss: 0.0584\n", + "Epoch 24 Batch 450/1077 - Train Accuracy: 0.9730, Validation Accuracy: 0.9624, Loss: 0.0545\n", + "Epoch 24 Batch 460/1077 - Train Accuracy: 0.9605, Validation Accuracy: 0.9652, Loss: 0.0483\n", + "Epoch 24 Batch 470/1077 - Train Accuracy: 0.9712, Validation Accuracy: 0.9450, Loss: 0.0517\n", + "Epoch 24 Batch 480/1077 - Train Accuracy: 0.9461, Validation Accuracy: 0.9545, Loss: 0.0444\n", + "Epoch 24 Batch 490/1077 - Train Accuracy: 0.9684, Validation Accuracy: 0.9499, Loss: 0.0451\n", + "Epoch 24 Batch 500/1077 - Train Accuracy: 0.9598, Validation Accuracy: 0.9609, Loss: 0.0425\n", + "Epoch 24 Batch 510/1077 - Train Accuracy: 0.9594, Validation Accuracy: 0.9634, Loss: 0.0441\n", + "Epoch 24 Batch 520/1077 - Train Accuracy: 0.9836, Validation Accuracy: 0.9638, Loss: 0.0423\n", + "Epoch 24 Batch 530/1077 - Train Accuracy: 0.9727, Validation Accuracy: 0.9616, Loss: 0.0473\n", + "Epoch 24 Batch 540/1077 - Train Accuracy: 0.9707, Validation Accuracy: 0.9663, Loss: 0.0429\n", + "Epoch 24 Batch 550/1077 - Train Accuracy: 0.9492, Validation Accuracy: 0.9677, Loss: 0.0496\n", + "Epoch 24 Batch 560/1077 - Train Accuracy: 0.9570, Validation Accuracy: 0.9506, Loss: 0.0567\n", + "Epoch 24 Batch 570/1077 - Train Accuracy: 0.9408, Validation Accuracy: 0.9645, Loss: 0.0744\n", + "Epoch 24 Batch 580/1077 - Train Accuracy: 0.9762, Validation Accuracy: 0.9656, Loss: 0.0361\n", + "Epoch 24 Batch 590/1077 - Train Accuracy: 0.9490, Validation Accuracy: 0.9695, Loss: 0.0549\n", + "Epoch 24 Batch 600/1077 - Train Accuracy: 0.9557, Validation Accuracy: 0.9631, Loss: 0.0574\n", + "Epoch 24 Batch 610/1077 - Train Accuracy: 0.9527, Validation Accuracy: 0.9666, Loss: 0.0503\n", + "Epoch 24 Batch 620/1077 - Train Accuracy: 0.9695, Validation Accuracy: 0.9691, Loss: 0.0507\n", + "Epoch 24 Batch 630/1077 - Train Accuracy: 0.9805, Validation Accuracy: 0.9659, Loss: 0.0500\n", + "Epoch 24 Batch 640/1077 - Train Accuracy: 0.9580, Validation Accuracy: 0.9613, Loss: 0.0516\n", + "Epoch 24 Batch 650/1077 - Train Accuracy: 0.9715, Validation Accuracy: 0.9712, Loss: 0.0494\n", + "Epoch 24 Batch 660/1077 - Train Accuracy: 0.9797, Validation Accuracy: 0.9659, Loss: 0.0465\n", + "Epoch 24 Batch 670/1077 - Train Accuracy: 0.9624, Validation Accuracy: 0.9588, Loss: 0.0506\n", + "Epoch 24 Batch 680/1077 - Train Accuracy: 0.9632, Validation Accuracy: 0.9592, Loss: 0.0456\n", + "Epoch 24 Batch 690/1077 - Train Accuracy: 0.9539, Validation Accuracy: 0.9652, Loss: 0.0485\n", + "Epoch 24 Batch 700/1077 - Train Accuracy: 0.9750, Validation Accuracy: 0.9602, Loss: 0.0400\n", + "Epoch 24 Batch 710/1077 - Train Accuracy: 0.9707, Validation Accuracy: 0.9631, Loss: 0.0411\n", + "Epoch 24 Batch 720/1077 - Train Accuracy: 0.9618, Validation Accuracy: 0.9634, Loss: 0.0594\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 24 Batch 730/1077 - Train Accuracy: 0.9727, Validation Accuracy: 0.9560, Loss: 0.0782\n", + "Epoch 24 Batch 740/1077 - Train Accuracy: 0.9688, Validation Accuracy: 0.9517, Loss: 0.0417\n", + "Epoch 24 Batch 750/1077 - Train Accuracy: 0.9527, Validation Accuracy: 0.9716, Loss: 0.0466\n", + "Epoch 24 Batch 760/1077 - Train Accuracy: 0.9695, Validation Accuracy: 0.9528, Loss: 0.0484\n", + "Epoch 24 Batch 770/1077 - Train Accuracy: 0.9501, Validation Accuracy: 0.9599, Loss: 0.0498\n", + "Epoch 24 Batch 780/1077 - Train Accuracy: 0.9488, Validation Accuracy: 0.9716, Loss: 0.0722\n", + "Epoch 24 Batch 790/1077 - Train Accuracy: 0.9594, Validation Accuracy: 0.9595, Loss: 0.0620\n", + "Epoch 24 Batch 800/1077 - Train Accuracy: 0.9672, Validation Accuracy: 0.9666, Loss: 0.0527\n", + "Epoch 24 Batch 810/1077 - Train Accuracy: 0.9702, Validation Accuracy: 0.9595, Loss: 0.0418\n", + "Epoch 24 Batch 820/1077 - Train Accuracy: 0.9789, Validation Accuracy: 0.9620, Loss: 0.0521\n", + "Epoch 24 Batch 830/1077 - Train Accuracy: 0.9367, Validation Accuracy: 0.9549, Loss: 0.0615\n", + "Epoch 24 Batch 840/1077 - Train Accuracy: 0.9859, Validation Accuracy: 0.9545, Loss: 0.0406\n", + "Epoch 24 Batch 850/1077 - Train Accuracy: 0.9617, Validation Accuracy: 0.9634, Loss: 0.0709\n", + "Epoch 24 Batch 860/1077 - Train Accuracy: 0.9606, Validation Accuracy: 0.9609, Loss: 0.0503\n", + "Epoch 24 Batch 870/1077 - Train Accuracy: 0.9576, Validation Accuracy: 0.9499, Loss: 0.0456\n", + "Epoch 24 Batch 880/1077 - Train Accuracy: 0.9574, Validation Accuracy: 0.9606, Loss: 0.0714\n", + "Epoch 24 Batch 890/1077 - Train Accuracy: 0.9714, Validation Accuracy: 0.9620, Loss: 0.0513\n", + "Epoch 24 Batch 900/1077 - Train Accuracy: 0.9516, Validation Accuracy: 0.9613, Loss: 0.0700\n", + "Epoch 24 Batch 910/1077 - Train Accuracy: 0.9568, Validation Accuracy: 0.9734, Loss: 0.0464\n", + "Epoch 24 Batch 920/1077 - Train Accuracy: 0.9574, Validation Accuracy: 0.9645, Loss: 0.0446\n", + "Epoch 24 Batch 930/1077 - Train Accuracy: 0.9578, Validation Accuracy: 0.9574, Loss: 0.0442\n", + "Epoch 24 Batch 940/1077 - Train Accuracy: 0.9621, Validation Accuracy: 0.9553, Loss: 0.0389\n", + "Epoch 24 Batch 950/1077 - Train Accuracy: 0.9524, Validation Accuracy: 0.9556, Loss: 0.0474\n", + "Epoch 24 Batch 960/1077 - Train Accuracy: 0.9568, Validation Accuracy: 0.9659, Loss: 0.0522\n", + "Epoch 24 Batch 970/1077 - Train Accuracy: 0.9563, Validation Accuracy: 0.9439, Loss: 0.0530\n", + "Epoch 24 Batch 980/1077 - Train Accuracy: 0.9445, Validation Accuracy: 0.9634, Loss: 0.0615\n", + "Epoch 24 Batch 990/1077 - Train Accuracy: 0.9873, Validation Accuracy: 0.9716, Loss: 0.0603\n", + "Epoch 24 Batch 1000/1077 - Train Accuracy: 0.9315, Validation Accuracy: 0.9666, Loss: 0.0538\n", + "Epoch 24 Batch 1010/1077 - Train Accuracy: 0.9719, Validation Accuracy: 0.9645, Loss: 0.0515\n", + "Epoch 24 Batch 1020/1077 - Train Accuracy: 0.9898, Validation Accuracy: 0.9581, Loss: 0.0406\n", + "Epoch 24 Batch 1030/1077 - Train Accuracy: 0.9848, Validation Accuracy: 0.9652, Loss: 0.0516\n", + "Epoch 24 Batch 1040/1077 - Train Accuracy: 0.9741, Validation Accuracy: 0.9570, Loss: 0.0540\n", + "Epoch 24 Batch 1050/1077 - Train Accuracy: 0.9816, Validation Accuracy: 0.9648, Loss: 0.0369\n", + "Epoch 24 Batch 1060/1077 - Train Accuracy: 0.9707, Validation Accuracy: 0.9549, Loss: 0.0417\n", + "Epoch 24 Batch 1070/1077 - Train Accuracy: 0.9629, Validation Accuracy: 0.9588, Loss: 0.0474\n", + "Epoch 25 Batch 10/1077 - Train Accuracy: 0.9601, Validation Accuracy: 0.9652, Loss: 0.0596\n", + "Epoch 25 Batch 20/1077 - Train Accuracy: 0.9715, Validation Accuracy: 0.9677, Loss: 0.0519\n", + "Epoch 25 Batch 30/1077 - Train Accuracy: 0.9777, Validation Accuracy: 0.9616, Loss: 0.0414\n", + "Epoch 25 Batch 40/1077 - Train Accuracy: 0.9656, Validation Accuracy: 0.9585, Loss: 0.0548\n", + "Epoch 25 Batch 50/1077 - Train Accuracy: 0.9844, Validation Accuracy: 0.9766, Loss: 0.0440\n", + "Epoch 25 Batch 60/1077 - Train Accuracy: 0.9702, Validation Accuracy: 0.9716, Loss: 0.0390\n", + "Epoch 25 Batch 70/1077 - Train Accuracy: 0.9568, Validation Accuracy: 0.9737, Loss: 0.0498\n", + "Epoch 25 Batch 80/1077 - Train Accuracy: 0.9723, Validation Accuracy: 0.9680, Loss: 0.0501\n", + "Epoch 25 Batch 90/1077 - Train Accuracy: 0.9574, Validation Accuracy: 0.9641, Loss: 0.0437\n", + "Epoch 25 Batch 100/1077 - Train Accuracy: 0.9602, Validation Accuracy: 0.9641, Loss: 0.0463\n", + "Epoch 25 Batch 110/1077 - Train Accuracy: 0.9691, Validation Accuracy: 0.9673, Loss: 0.0395\n", + "Epoch 25 Batch 120/1077 - Train Accuracy: 0.9617, Validation Accuracy: 0.9606, Loss: 0.0509\n", + "Epoch 25 Batch 130/1077 - Train Accuracy: 0.9561, Validation Accuracy: 0.9624, Loss: 0.0369\n", + "Epoch 25 Batch 140/1077 - Train Accuracy: 0.9700, Validation Accuracy: 0.9602, Loss: 0.0424\n", + "Epoch 25 Batch 150/1077 - Train Accuracy: 0.9673, Validation Accuracy: 0.9616, Loss: 0.0526\n", + "Epoch 25 Batch 160/1077 - Train Accuracy: 0.9617, Validation Accuracy: 0.9606, Loss: 0.0403\n", + "Epoch 25 Batch 170/1077 - Train Accuracy: 0.9520, Validation Accuracy: 0.9574, Loss: 0.0568\n", + "Epoch 25 Batch 180/1077 - Train Accuracy: 0.9652, Validation Accuracy: 0.9560, Loss: 0.0488\n", + "Epoch 25 Batch 190/1077 - Train Accuracy: 0.9707, Validation Accuracy: 0.9574, Loss: 0.0437\n", + "Epoch 25 Batch 200/1077 - Train Accuracy: 0.9645, Validation Accuracy: 0.9624, Loss: 0.0505\n", + "Epoch 25 Batch 210/1077 - Train Accuracy: 0.9379, Validation Accuracy: 0.9513, Loss: 0.0532\n", + "Epoch 25 Batch 220/1077 - Train Accuracy: 0.9634, Validation Accuracy: 0.9631, Loss: 0.0509\n", + "Epoch 25 Batch 230/1077 - Train Accuracy: 0.9747, Validation Accuracy: 0.9602, Loss: 0.0498\n", + "Epoch 25 Batch 240/1077 - Train Accuracy: 0.9844, Validation Accuracy: 0.9602, Loss: 0.0491\n", + "Epoch 25 Batch 250/1077 - Train Accuracy: 0.9521, Validation Accuracy: 0.9659, Loss: 0.0531\n", + "Epoch 25 Batch 260/1077 - Train Accuracy: 0.9609, Validation Accuracy: 0.9634, Loss: 0.0404\n", + "Epoch 25 Batch 270/1077 - Train Accuracy: 0.9523, Validation Accuracy: 0.9684, Loss: 0.0570\n", + "Epoch 25 Batch 280/1077 - Train Accuracy: 0.9539, Validation Accuracy: 0.9698, Loss: 0.0665\n", + "Epoch 25 Batch 290/1077 - Train Accuracy: 0.9437, Validation Accuracy: 0.9592, Loss: 0.0788\n", + "Epoch 25 Batch 300/1077 - Train Accuracy: 0.9535, Validation Accuracy: 0.9652, Loss: 0.0464\n", + "Epoch 25 Batch 310/1077 - Train Accuracy: 0.9699, Validation Accuracy: 0.9553, Loss: 0.0428\n", + "Epoch 25 Batch 320/1077 - Train Accuracy: 0.9656, Validation Accuracy: 0.9467, Loss: 0.0562\n", + "Epoch 25 Batch 330/1077 - Train Accuracy: 0.9406, Validation Accuracy: 0.9599, Loss: 0.0615\n", + "Epoch 25 Batch 340/1077 - Train Accuracy: 0.9733, Validation Accuracy: 0.9609, Loss: 0.0477\n", + "Epoch 25 Batch 350/1077 - Train Accuracy: 0.9496, Validation Accuracy: 0.9581, Loss: 0.0527\n", + "Epoch 25 Batch 360/1077 - Train Accuracy: 0.9527, Validation Accuracy: 0.9567, Loss: 0.0366\n", + "Epoch 25 Batch 370/1077 - Train Accuracy: 0.9706, Validation Accuracy: 0.9574, Loss: 0.0477\n", + "Epoch 25 Batch 380/1077 - Train Accuracy: 0.9586, Validation Accuracy: 0.9599, Loss: 0.0587\n", + "Epoch 25 Batch 390/1077 - Train Accuracy: 0.9508, Validation Accuracy: 0.9648, Loss: 0.0728\n", + "Epoch 25 Batch 400/1077 - Train Accuracy: 0.9660, Validation Accuracy: 0.9677, Loss: 0.0597\n", + "Epoch 25 Batch 410/1077 - Train Accuracy: 0.9449, Validation Accuracy: 0.9666, Loss: 0.0642\n", + "Epoch 25 Batch 420/1077 - Train Accuracy: 0.9578, Validation Accuracy: 0.9751, Loss: 0.0419\n", + "Epoch 25 Batch 430/1077 - Train Accuracy: 0.9664, Validation Accuracy: 0.9716, Loss: 0.0506\n", + "Epoch 25 Batch 440/1077 - Train Accuracy: 0.9477, Validation Accuracy: 0.9769, Loss: 0.0652\n", + "Epoch 25 Batch 450/1077 - Train Accuracy: 0.9672, Validation Accuracy: 0.9670, Loss: 0.0571\n", + "Epoch 25 Batch 460/1077 - Train Accuracy: 0.9531, Validation Accuracy: 0.9652, Loss: 0.0567\n", + "Epoch 25 Batch 470/1077 - Train Accuracy: 0.9700, Validation Accuracy: 0.9588, Loss: 0.0465\n", + "Epoch 25 Batch 480/1077 - Train Accuracy: 0.9560, Validation Accuracy: 0.9634, Loss: 0.0472\n", + "Epoch 25 Batch 490/1077 - Train Accuracy: 0.9570, Validation Accuracy: 0.9510, Loss: 0.0485\n", + "Epoch 25 Batch 500/1077 - Train Accuracy: 0.9602, Validation Accuracy: 0.9592, Loss: 0.0306\n", + "Epoch 25 Batch 510/1077 - Train Accuracy: 0.9633, Validation Accuracy: 0.9634, Loss: 0.0494\n", + "Epoch 25 Batch 520/1077 - Train Accuracy: 0.9773, Validation Accuracy: 0.9574, Loss: 0.0488\n", + "Epoch 25 Batch 530/1077 - Train Accuracy: 0.9684, Validation Accuracy: 0.9616, Loss: 0.0479\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 25 Batch 540/1077 - Train Accuracy: 0.9672, Validation Accuracy: 0.9688, Loss: 0.0441\n", + "Epoch 25 Batch 550/1077 - Train Accuracy: 0.9461, Validation Accuracy: 0.9613, Loss: 0.0468\n", + "Epoch 25 Batch 560/1077 - Train Accuracy: 0.9484, Validation Accuracy: 0.9549, Loss: 0.0377\n", + "Epoch 25 Batch 570/1077 - Train Accuracy: 0.9548, Validation Accuracy: 0.9545, Loss: 0.0584\n", + "Epoch 25 Batch 580/1077 - Train Accuracy: 0.9654, Validation Accuracy: 0.9606, Loss: 0.0408\n", + "Epoch 25 Batch 590/1077 - Train Accuracy: 0.9568, Validation Accuracy: 0.9634, Loss: 0.0666\n", + "Epoch 25 Batch 600/1077 - Train Accuracy: 0.9754, Validation Accuracy: 0.9730, Loss: 0.0491\n", + "Epoch 25 Batch 610/1077 - Train Accuracy: 0.9465, Validation Accuracy: 0.9641, Loss: 0.0528\n", + "Epoch 25 Batch 620/1077 - Train Accuracy: 0.9809, Validation Accuracy: 0.9585, Loss: 0.0471\n", + "Epoch 25 Batch 630/1077 - Train Accuracy: 0.9750, Validation Accuracy: 0.9624, Loss: 0.0464\n", + "Epoch 25 Batch 640/1077 - Train Accuracy: 0.9613, Validation Accuracy: 0.9634, Loss: 0.0444\n", + "Epoch 25 Batch 650/1077 - Train Accuracy: 0.9773, Validation Accuracy: 0.9602, Loss: 0.0455\n", + "Epoch 25 Batch 660/1077 - Train Accuracy: 0.9785, Validation Accuracy: 0.9528, Loss: 0.0553\n", + "Epoch 25 Batch 670/1077 - Train Accuracy: 0.9638, Validation Accuracy: 0.9730, Loss: 0.0529\n", + "Epoch 25 Batch 680/1077 - Train Accuracy: 0.9468, Validation Accuracy: 0.9478, Loss: 0.0450\n", + "Epoch 25 Batch 690/1077 - Train Accuracy: 0.9551, Validation Accuracy: 0.9542, Loss: 0.0517\n", + "Epoch 25 Batch 700/1077 - Train Accuracy: 0.9781, Validation Accuracy: 0.9677, Loss: 0.0443\n", + "Epoch 25 Batch 710/1077 - Train Accuracy: 0.9809, Validation Accuracy: 0.9652, Loss: 0.0388\n", + "Epoch 25 Batch 720/1077 - Train Accuracy: 0.9667, Validation Accuracy: 0.9524, Loss: 0.0531\n", + "Epoch 25 Batch 730/1077 - Train Accuracy: 0.9574, Validation Accuracy: 0.9517, Loss: 0.0644\n", + "Epoch 25 Batch 740/1077 - Train Accuracy: 0.9703, Validation Accuracy: 0.9602, Loss: 0.0409\n", + "Epoch 25 Batch 750/1077 - Train Accuracy: 0.9574, Validation Accuracy: 0.9688, Loss: 0.0464\n", + "Epoch 25 Batch 760/1077 - Train Accuracy: 0.9699, Validation Accuracy: 0.9680, Loss: 0.0630\n", + "Epoch 25 Batch 770/1077 - Train Accuracy: 0.9628, Validation Accuracy: 0.9709, Loss: 0.0531\n", + "Epoch 25 Batch 780/1077 - Train Accuracy: 0.9441, Validation Accuracy: 0.9613, Loss: 0.0661\n", + "Epoch 25 Batch 790/1077 - Train Accuracy: 0.9508, Validation Accuracy: 0.9656, Loss: 0.0556\n", + "Epoch 25 Batch 800/1077 - Train Accuracy: 0.9660, Validation Accuracy: 0.9648, Loss: 0.0452\n", + "Epoch 25 Batch 810/1077 - Train Accuracy: 0.9706, Validation Accuracy: 0.9670, Loss: 0.0335\n", + "Epoch 25 Batch 820/1077 - Train Accuracy: 0.9824, Validation Accuracy: 0.9659, Loss: 0.0520\n", + "Epoch 25 Batch 830/1077 - Train Accuracy: 0.9430, Validation Accuracy: 0.9531, Loss: 0.0470\n", + "Epoch 25 Batch 840/1077 - Train Accuracy: 0.9773, Validation Accuracy: 0.9641, Loss: 0.0517\n", + "Epoch 25 Batch 850/1077 - Train Accuracy: 0.9658, Validation Accuracy: 0.9680, Loss: 0.0717\n", + "Epoch 25 Batch 860/1077 - Train Accuracy: 0.9613, Validation Accuracy: 0.9684, Loss: 0.0718\n", + "Epoch 25 Batch 870/1077 - Train Accuracy: 0.9507, Validation Accuracy: 0.9709, Loss: 0.0466\n", + "Epoch 25 Batch 880/1077 - Train Accuracy: 0.9605, Validation Accuracy: 0.9638, Loss: 0.0710\n", + "Epoch 25 Batch 890/1077 - Train Accuracy: 0.9691, Validation Accuracy: 0.9684, Loss: 0.0533\n", + "Epoch 25 Batch 900/1077 - Train Accuracy: 0.9613, Validation Accuracy: 0.9627, Loss: 0.0545\n", + "Epoch 25 Batch 910/1077 - Train Accuracy: 0.9650, Validation Accuracy: 0.9638, Loss: 0.0575\n", + "Epoch 25 Batch 920/1077 - Train Accuracy: 0.9625, Validation Accuracy: 0.9691, Loss: 0.0492\n", + "Epoch 25 Batch 930/1077 - Train Accuracy: 0.9531, Validation Accuracy: 0.9648, Loss: 0.0590\n", + "Epoch 25 Batch 940/1077 - Train Accuracy: 0.9723, Validation Accuracy: 0.9709, Loss: 0.0355\n", + "Epoch 25 Batch 950/1077 - Train Accuracy: 0.9784, Validation Accuracy: 0.9744, Loss: 0.0344\n", + "Epoch 25 Batch 960/1077 - Train Accuracy: 0.9699, Validation Accuracy: 0.9709, Loss: 0.0471\n", + "Epoch 25 Batch 970/1077 - Train Accuracy: 0.9676, Validation Accuracy: 0.9641, Loss: 0.0559\n", + "Epoch 25 Batch 980/1077 - Train Accuracy: 0.9688, Validation Accuracy: 0.9716, Loss: 0.0629\n", + "Epoch 25 Batch 990/1077 - Train Accuracy: 0.9815, Validation Accuracy: 0.9755, Loss: 0.0494\n", + "Epoch 25 Batch 1000/1077 - Train Accuracy: 0.9356, Validation Accuracy: 0.9705, Loss: 0.0568\n", + "Epoch 25 Batch 1010/1077 - Train Accuracy: 0.9723, Validation Accuracy: 0.9759, Loss: 0.0496\n", + "Epoch 25 Batch 1020/1077 - Train Accuracy: 0.9855, Validation Accuracy: 0.9759, Loss: 0.0472\n", + "Epoch 25 Batch 1030/1077 - Train Accuracy: 0.9707, Validation Accuracy: 0.9755, Loss: 0.0467\n", + "Epoch 25 Batch 1040/1077 - Train Accuracy: 0.9811, Validation Accuracy: 0.9712, Loss: 0.0512\n", + "Epoch 25 Batch 1050/1077 - Train Accuracy: 0.9852, Validation Accuracy: 0.9670, Loss: 0.0422\n", + "Epoch 25 Batch 1060/1077 - Train Accuracy: 0.9598, Validation Accuracy: 0.9744, Loss: 0.0323\n", + "Epoch 25 Batch 1070/1077 - Train Accuracy: 0.9637, Validation Accuracy: 0.9652, Loss: 0.0451\n", + "Epoch 26 Batch 10/1077 - Train Accuracy: 0.9539, Validation Accuracy: 0.9648, Loss: 0.0583\n", + "Epoch 26 Batch 20/1077 - Train Accuracy: 0.9645, Validation Accuracy: 0.9741, Loss: 0.0428\n", + "Epoch 26 Batch 30/1077 - Train Accuracy: 0.9758, Validation Accuracy: 0.9755, Loss: 0.0400\n", + "Epoch 26 Batch 40/1077 - Train Accuracy: 0.9824, Validation Accuracy: 0.9705, Loss: 0.0323\n", + "Epoch 26 Batch 50/1077 - Train Accuracy: 0.9738, Validation Accuracy: 0.9691, Loss: 0.0386\n", + "Epoch 26 Batch 60/1077 - Train Accuracy: 0.9721, Validation Accuracy: 0.9691, Loss: 0.0372\n", + "Epoch 26 Batch 70/1077 - Train Accuracy: 0.9585, Validation Accuracy: 0.9631, Loss: 0.0493\n", + "Epoch 26 Batch 80/1077 - Train Accuracy: 0.9762, Validation Accuracy: 0.9553, Loss: 0.0305\n", + "Epoch 26 Batch 90/1077 - Train Accuracy: 0.9723, Validation Accuracy: 0.9684, Loss: 0.0507\n", + "Epoch 26 Batch 100/1077 - Train Accuracy: 0.9680, Validation Accuracy: 0.9673, Loss: 0.0460\n", + "Epoch 26 Batch 110/1077 - Train Accuracy: 0.9699, Validation Accuracy: 0.9727, Loss: 0.0383\n", + "Epoch 26 Batch 120/1077 - Train Accuracy: 0.9660, Validation Accuracy: 0.9641, Loss: 0.0526\n", + "Epoch 26 Batch 130/1077 - Train Accuracy: 0.9758, Validation Accuracy: 0.9645, Loss: 0.0345\n", + "Epoch 26 Batch 140/1077 - Train Accuracy: 0.9659, Validation Accuracy: 0.9698, Loss: 0.0404\n", + "Epoch 26 Batch 150/1077 - Train Accuracy: 0.9661, Validation Accuracy: 0.9641, Loss: 0.0428\n", + "Epoch 26 Batch 160/1077 - Train Accuracy: 0.9816, Validation Accuracy: 0.9645, Loss: 0.0405\n", + "Epoch 26 Batch 170/1077 - Train Accuracy: 0.9398, Validation Accuracy: 0.9638, Loss: 0.0612\n", + "Epoch 26 Batch 180/1077 - Train Accuracy: 0.9812, Validation Accuracy: 0.9631, Loss: 0.0380\n", + "Epoch 26 Batch 190/1077 - Train Accuracy: 0.9723, Validation Accuracy: 0.9609, Loss: 0.0420\n", + "Epoch 26 Batch 200/1077 - Train Accuracy: 0.9566, Validation Accuracy: 0.9634, Loss: 0.0595\n", + "Epoch 26 Batch 210/1077 - Train Accuracy: 0.9524, Validation Accuracy: 0.9663, Loss: 0.0562\n", + "Epoch 26 Batch 220/1077 - Train Accuracy: 0.9753, Validation Accuracy: 0.9599, Loss: 0.0365\n", + "Epoch 26 Batch 230/1077 - Train Accuracy: 0.9747, Validation Accuracy: 0.9585, Loss: 0.0417\n", + "Epoch 26 Batch 240/1077 - Train Accuracy: 0.9840, Validation Accuracy: 0.9666, Loss: 0.0433\n", + "Epoch 26 Batch 250/1077 - Train Accuracy: 0.9577, Validation Accuracy: 0.9670, Loss: 0.0509\n", + "Epoch 26 Batch 260/1077 - Train Accuracy: 0.9710, Validation Accuracy: 0.9751, Loss: 0.0316\n", + "Epoch 26 Batch 270/1077 - Train Accuracy: 0.9496, Validation Accuracy: 0.9613, Loss: 0.0448\n", + "Epoch 26 Batch 280/1077 - Train Accuracy: 0.9488, Validation Accuracy: 0.9521, Loss: 0.0600\n", + "Epoch 26 Batch 290/1077 - Train Accuracy: 0.9652, Validation Accuracy: 0.9634, Loss: 0.0749\n", + "Epoch 26 Batch 300/1077 - Train Accuracy: 0.9700, Validation Accuracy: 0.9663, Loss: 0.0417\n", + "Epoch 26 Batch 310/1077 - Train Accuracy: 0.9578, Validation Accuracy: 0.9542, Loss: 0.0473\n", + "Epoch 26 Batch 320/1077 - Train Accuracy: 0.9633, Validation Accuracy: 0.9670, Loss: 0.0482\n", + "Epoch 26 Batch 330/1077 - Train Accuracy: 0.9488, Validation Accuracy: 0.9659, Loss: 0.0482\n", + "Epoch 26 Batch 340/1077 - Train Accuracy: 0.9836, Validation Accuracy: 0.9602, Loss: 0.0418\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 26 Batch 350/1077 - Train Accuracy: 0.9590, Validation Accuracy: 0.9592, Loss: 0.0433\n", + "Epoch 26 Batch 360/1077 - Train Accuracy: 0.9586, Validation Accuracy: 0.9656, Loss: 0.0406\n", + "Epoch 26 Batch 370/1077 - Train Accuracy: 0.9621, Validation Accuracy: 0.9627, Loss: 0.0370\n", + "Epoch 26 Batch 380/1077 - Train Accuracy: 0.9609, Validation Accuracy: 0.9656, Loss: 0.0365\n", + "Epoch 26 Batch 390/1077 - Train Accuracy: 0.9578, Validation Accuracy: 0.9780, Loss: 0.0518\n", + "Epoch 26 Batch 400/1077 - Train Accuracy: 0.9641, Validation Accuracy: 0.9727, Loss: 0.0596\n", + "Epoch 26 Batch 410/1077 - Train Accuracy: 0.9531, Validation Accuracy: 0.9602, Loss: 0.0584\n", + "Epoch 26 Batch 420/1077 - Train Accuracy: 0.9766, Validation Accuracy: 0.9624, Loss: 0.0376\n", + "Epoch 26 Batch 430/1077 - Train Accuracy: 0.9648, Validation Accuracy: 0.9691, Loss: 0.0451\n", + "Epoch 26 Batch 440/1077 - Train Accuracy: 0.9430, Validation Accuracy: 0.9620, Loss: 0.0634\n", + "Epoch 26 Batch 450/1077 - Train Accuracy: 0.9629, Validation Accuracy: 0.9670, Loss: 0.0670\n", + "Epoch 26 Batch 460/1077 - Train Accuracy: 0.9609, Validation Accuracy: 0.9723, Loss: 0.0719\n", + "Epoch 26 Batch 470/1077 - Train Accuracy: 0.9864, Validation Accuracy: 0.9719, Loss: 0.0452\n", + "Epoch 26 Batch 480/1077 - Train Accuracy: 0.9494, Validation Accuracy: 0.9595, Loss: 0.0451\n", + "Epoch 26 Batch 490/1077 - Train Accuracy: 0.9594, Validation Accuracy: 0.9606, Loss: 0.0445\n", + "Epoch 26 Batch 500/1077 - Train Accuracy: 0.9477, Validation Accuracy: 0.9663, Loss: 0.0417\n", + "Epoch 26 Batch 510/1077 - Train Accuracy: 0.9590, Validation Accuracy: 0.9656, Loss: 0.0534\n", + "Epoch 26 Batch 520/1077 - Train Accuracy: 0.9866, Validation Accuracy: 0.9570, Loss: 0.0382\n", + "Epoch 26 Batch 530/1077 - Train Accuracy: 0.9504, Validation Accuracy: 0.9645, Loss: 0.0533\n", + "Epoch 26 Batch 540/1077 - Train Accuracy: 0.9746, Validation Accuracy: 0.9606, Loss: 0.0441\n", + "Epoch 26 Batch 550/1077 - Train Accuracy: 0.9543, Validation Accuracy: 0.9709, Loss: 0.0441\n", + "Epoch 26 Batch 560/1077 - Train Accuracy: 0.9535, Validation Accuracy: 0.9663, Loss: 0.0450\n", + "Epoch 26 Batch 570/1077 - Train Accuracy: 0.9535, Validation Accuracy: 0.9595, Loss: 0.0601\n", + "Epoch 26 Batch 580/1077 - Train Accuracy: 0.9721, Validation Accuracy: 0.9688, Loss: 0.0359\n", + "Epoch 26 Batch 590/1077 - Train Accuracy: 0.9535, Validation Accuracy: 0.9727, Loss: 0.0574\n", + "Epoch 26 Batch 600/1077 - Train Accuracy: 0.9784, Validation Accuracy: 0.9759, Loss: 0.0422\n", + "Epoch 26 Batch 610/1077 - Train Accuracy: 0.9457, Validation Accuracy: 0.9766, Loss: 0.0548\n", + "Epoch 26 Batch 620/1077 - Train Accuracy: 0.9750, Validation Accuracy: 0.9783, Loss: 0.0563\n", + "Epoch 26 Batch 630/1077 - Train Accuracy: 0.9758, Validation Accuracy: 0.9766, Loss: 0.0499\n", + "Epoch 26 Batch 640/1077 - Train Accuracy: 0.9568, Validation Accuracy: 0.9766, Loss: 0.0425\n", + "Epoch 26 Batch 650/1077 - Train Accuracy: 0.9664, Validation Accuracy: 0.9790, Loss: 0.0444\n", + "Epoch 26 Batch 660/1077 - Train Accuracy: 0.9801, Validation Accuracy: 0.9567, Loss: 0.0449\n", + "Epoch 26 Batch 670/1077 - Train Accuracy: 0.9641, Validation Accuracy: 0.9602, Loss: 0.0506\n", + "Epoch 26 Batch 680/1077 - Train Accuracy: 0.9643, Validation Accuracy: 0.9631, Loss: 0.0401\n", + "Epoch 26 Batch 690/1077 - Train Accuracy: 0.9488, Validation Accuracy: 0.9645, Loss: 0.0486\n", + "Epoch 26 Batch 700/1077 - Train Accuracy: 0.9719, Validation Accuracy: 0.9695, Loss: 0.0454\n", + "Epoch 26 Batch 710/1077 - Train Accuracy: 0.9789, Validation Accuracy: 0.9787, Loss: 0.0365\n", + "Epoch 26 Batch 720/1077 - Train Accuracy: 0.9659, Validation Accuracy: 0.9737, Loss: 0.0576\n", + "Epoch 26 Batch 730/1077 - Train Accuracy: 0.9703, Validation Accuracy: 0.9695, Loss: 0.0634\n", + "Epoch 26 Batch 740/1077 - Train Accuracy: 0.9707, Validation Accuracy: 0.9716, Loss: 0.0554\n", + "Epoch 26 Batch 750/1077 - Train Accuracy: 0.9504, Validation Accuracy: 0.9719, Loss: 0.0504\n", + "Epoch 26 Batch 760/1077 - Train Accuracy: 0.9777, Validation Accuracy: 0.9759, Loss: 0.0508\n", + "Epoch 26 Batch 770/1077 - Train Accuracy: 0.9531, Validation Accuracy: 0.9698, Loss: 0.0429\n", + "Epoch 26 Batch 780/1077 - Train Accuracy: 0.9504, Validation Accuracy: 0.9759, Loss: 0.0699\n", + "Epoch 26 Batch 790/1077 - Train Accuracy: 0.9535, Validation Accuracy: 0.9719, Loss: 0.0462\n", + "Epoch 26 Batch 800/1077 - Train Accuracy: 0.9863, Validation Accuracy: 0.9698, Loss: 0.0341\n", + "Epoch 26 Batch 810/1077 - Train Accuracy: 0.9684, Validation Accuracy: 0.9755, Loss: 0.0400\n", + "Epoch 26 Batch 820/1077 - Train Accuracy: 0.9879, Validation Accuracy: 0.9719, Loss: 0.0474\n", + "Epoch 26 Batch 830/1077 - Train Accuracy: 0.9270, Validation Accuracy: 0.9727, Loss: 0.0642\n", + "Epoch 26 Batch 840/1077 - Train Accuracy: 0.9680, Validation Accuracy: 0.9670, Loss: 0.0537\n", + "Epoch 26 Batch 850/1077 - Train Accuracy: 0.9684, Validation Accuracy: 0.9798, Loss: 0.0685\n", + "Epoch 26 Batch 860/1077 - Train Accuracy: 0.9673, Validation Accuracy: 0.9751, Loss: 0.0553\n", + "Epoch 26 Batch 870/1077 - Train Accuracy: 0.9572, Validation Accuracy: 0.9695, Loss: 0.0402\n", + "Epoch 26 Batch 880/1077 - Train Accuracy: 0.9629, Validation Accuracy: 0.9790, Loss: 0.0523\n", + "Epoch 26 Batch 890/1077 - Train Accuracy: 0.9766, Validation Accuracy: 0.9808, Loss: 0.0410\n", + "Epoch 26 Batch 900/1077 - Train Accuracy: 0.9570, Validation Accuracy: 0.9734, Loss: 0.0489\n", + "Epoch 26 Batch 910/1077 - Train Accuracy: 0.9647, Validation Accuracy: 0.9751, Loss: 0.0460\n", + "Epoch 26 Batch 920/1077 - Train Accuracy: 0.9762, Validation Accuracy: 0.9755, Loss: 0.0444\n", + "Epoch 26 Batch 930/1077 - Train Accuracy: 0.9570, Validation Accuracy: 0.9716, Loss: 0.0405\n", + "Epoch 26 Batch 940/1077 - Train Accuracy: 0.9684, Validation Accuracy: 0.9719, Loss: 0.0381\n", + "Epoch 26 Batch 950/1077 - Train Accuracy: 0.9777, Validation Accuracy: 0.9702, Loss: 0.0355\n", + "Epoch 26 Batch 960/1077 - Train Accuracy: 0.9684, Validation Accuracy: 0.9659, Loss: 0.0542\n", + "Epoch 26 Batch 970/1077 - Train Accuracy: 0.9656, Validation Accuracy: 0.9663, Loss: 0.0564\n", + "Epoch 26 Batch 980/1077 - Train Accuracy: 0.9543, Validation Accuracy: 0.9638, Loss: 0.0482\n", + "Epoch 26 Batch 990/1077 - Train Accuracy: 0.9745, Validation Accuracy: 0.9638, Loss: 0.0520\n", + "Epoch 26 Batch 1000/1077 - Train Accuracy: 0.9390, Validation Accuracy: 0.9581, Loss: 0.0616\n", + "Epoch 26 Batch 1010/1077 - Train Accuracy: 0.9754, Validation Accuracy: 0.9606, Loss: 0.0441\n", + "Epoch 26 Batch 1020/1077 - Train Accuracy: 0.9949, Validation Accuracy: 0.9659, Loss: 0.0410\n", + "Epoch 26 Batch 1030/1077 - Train Accuracy: 0.9672, Validation Accuracy: 0.9652, Loss: 0.0516\n", + "Epoch 26 Batch 1040/1077 - Train Accuracy: 0.9823, Validation Accuracy: 0.9688, Loss: 0.0453\n", + "Epoch 26 Batch 1050/1077 - Train Accuracy: 0.9879, Validation Accuracy: 0.9631, Loss: 0.0335\n", + "Epoch 26 Batch 1060/1077 - Train Accuracy: 0.9648, Validation Accuracy: 0.9680, Loss: 0.0371\n", + "Epoch 26 Batch 1070/1077 - Train Accuracy: 0.9523, Validation Accuracy: 0.9666, Loss: 0.0585\n", + "Epoch 27 Batch 10/1077 - Train Accuracy: 0.9474, Validation Accuracy: 0.9659, Loss: 0.0531\n", + "Epoch 27 Batch 20/1077 - Train Accuracy: 0.9656, Validation Accuracy: 0.9570, Loss: 0.0452\n", + "Epoch 27 Batch 30/1077 - Train Accuracy: 0.9766, Validation Accuracy: 0.9638, Loss: 0.0391\n", + "Epoch 27 Batch 40/1077 - Train Accuracy: 0.9777, Validation Accuracy: 0.9670, Loss: 0.0500\n", + "Epoch 27 Batch 50/1077 - Train Accuracy: 0.9859, Validation Accuracy: 0.9670, Loss: 0.0447\n", + "Epoch 27 Batch 60/1077 - Train Accuracy: 0.9699, Validation Accuracy: 0.9616, Loss: 0.0387\n", + "Epoch 27 Batch 70/1077 - Train Accuracy: 0.9519, Validation Accuracy: 0.9677, Loss: 0.0479\n", + "Epoch 27 Batch 80/1077 - Train Accuracy: 0.9840, Validation Accuracy: 0.9684, Loss: 0.0391\n", + "Epoch 27 Batch 90/1077 - Train Accuracy: 0.9582, Validation Accuracy: 0.9695, Loss: 0.0507\n", + "Epoch 27 Batch 100/1077 - Train Accuracy: 0.9688, Validation Accuracy: 0.9691, Loss: 0.0484\n", + "Epoch 27 Batch 110/1077 - Train Accuracy: 0.9711, Validation Accuracy: 0.9759, Loss: 0.0442\n", + "Epoch 27 Batch 120/1077 - Train Accuracy: 0.9582, Validation Accuracy: 0.9759, Loss: 0.0467\n", + "Epoch 27 Batch 130/1077 - Train Accuracy: 0.9650, Validation Accuracy: 0.9688, Loss: 0.0380\n", + "Epoch 27 Batch 140/1077 - Train Accuracy: 0.9613, Validation Accuracy: 0.9691, Loss: 0.0494\n", + "Epoch 27 Batch 150/1077 - Train Accuracy: 0.9829, Validation Accuracy: 0.9609, Loss: 0.0461\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 27 Batch 160/1077 - Train Accuracy: 0.9684, Validation Accuracy: 0.9585, Loss: 0.0380\n", + "Epoch 27 Batch 170/1077 - Train Accuracy: 0.9418, Validation Accuracy: 0.9588, Loss: 0.0524\n", + "Epoch 27 Batch 180/1077 - Train Accuracy: 0.9785, Validation Accuracy: 0.9719, Loss: 0.0335\n", + "Epoch 27 Batch 190/1077 - Train Accuracy: 0.9711, Validation Accuracy: 0.9574, Loss: 0.0386\n", + "Epoch 27 Batch 200/1077 - Train Accuracy: 0.9703, Validation Accuracy: 0.9684, Loss: 0.0432\n", + "Epoch 27 Batch 210/1077 - Train Accuracy: 0.9572, Validation Accuracy: 0.9641, Loss: 0.0479\n", + "Epoch 27 Batch 220/1077 - Train Accuracy: 0.9774, Validation Accuracy: 0.9620, Loss: 0.0477\n", + "Epoch 27 Batch 230/1077 - Train Accuracy: 0.9807, Validation Accuracy: 0.9656, Loss: 0.0417\n", + "Epoch 27 Batch 240/1077 - Train Accuracy: 0.9758, Validation Accuracy: 0.9634, Loss: 0.0413\n", + "Epoch 27 Batch 250/1077 - Train Accuracy: 0.9553, Validation Accuracy: 0.9734, Loss: 0.0486\n", + "Epoch 27 Batch 260/1077 - Train Accuracy: 0.9587, Validation Accuracy: 0.9737, Loss: 0.0313\n", + "Epoch 27 Batch 270/1077 - Train Accuracy: 0.9402, Validation Accuracy: 0.9723, Loss: 0.0562\n", + "Epoch 27 Batch 280/1077 - Train Accuracy: 0.9598, Validation Accuracy: 0.9826, Loss: 0.0572\n", + "Epoch 27 Batch 290/1077 - Train Accuracy: 0.9605, Validation Accuracy: 0.9673, Loss: 0.0594\n", + "Epoch 27 Batch 300/1077 - Train Accuracy: 0.9642, Validation Accuracy: 0.9695, Loss: 0.0463\n", + "Epoch 27 Batch 310/1077 - Train Accuracy: 0.9641, Validation Accuracy: 0.9712, Loss: 0.0448\n", + "Epoch 27 Batch 320/1077 - Train Accuracy: 0.9531, Validation Accuracy: 0.9790, Loss: 0.0479\n", + "Epoch 27 Batch 330/1077 - Train Accuracy: 0.9512, Validation Accuracy: 0.9741, Loss: 0.0475\n", + "Epoch 27 Batch 340/1077 - Train Accuracy: 0.9774, Validation Accuracy: 0.9616, Loss: 0.0492\n", + "Epoch 27 Batch 350/1077 - Train Accuracy: 0.9500, Validation Accuracy: 0.9577, Loss: 0.0538\n", + "Epoch 27 Batch 360/1077 - Train Accuracy: 0.9563, Validation Accuracy: 0.9712, Loss: 0.0309\n", + "Epoch 27 Batch 370/1077 - Train Accuracy: 0.9509, Validation Accuracy: 0.9663, Loss: 0.0465\n", + "Epoch 27 Batch 380/1077 - Train Accuracy: 0.9598, Validation Accuracy: 0.9581, Loss: 0.0511\n", + "Epoch 27 Batch 390/1077 - Train Accuracy: 0.9570, Validation Accuracy: 0.9773, Loss: 0.0585\n", + "Epoch 27 Batch 400/1077 - Train Accuracy: 0.9664, Validation Accuracy: 0.9801, Loss: 0.0551\n", + "Epoch 27 Batch 410/1077 - Train Accuracy: 0.9363, Validation Accuracy: 0.9737, Loss: 0.0626\n", + "Epoch 27 Batch 420/1077 - Train Accuracy: 0.9723, Validation Accuracy: 0.9762, Loss: 0.0403\n", + "Epoch 27 Batch 430/1077 - Train Accuracy: 0.9652, Validation Accuracy: 0.9634, Loss: 0.0311\n", + "Epoch 27 Batch 440/1077 - Train Accuracy: 0.9445, Validation Accuracy: 0.9609, Loss: 0.0579\n", + "Epoch 27 Batch 450/1077 - Train Accuracy: 0.9648, Validation Accuracy: 0.9691, Loss: 0.0572\n", + "Epoch 27 Batch 460/1077 - Train Accuracy: 0.9773, Validation Accuracy: 0.9659, Loss: 0.0514\n", + "Epoch 27 Batch 470/1077 - Train Accuracy: 0.9827, Validation Accuracy: 0.9648, Loss: 0.0447\n", + "Epoch 27 Batch 480/1077 - Train Accuracy: 0.9601, Validation Accuracy: 0.9627, Loss: 0.0470\n", + "Epoch 27 Batch 490/1077 - Train Accuracy: 0.9543, Validation Accuracy: 0.9627, Loss: 0.0551\n", + "Epoch 27 Batch 500/1077 - Train Accuracy: 0.9629, Validation Accuracy: 0.9663, Loss: 0.0396\n", + "Epoch 27 Batch 510/1077 - Train Accuracy: 0.9648, Validation Accuracy: 0.9702, Loss: 0.0442\n", + "Epoch 27 Batch 520/1077 - Train Accuracy: 0.9892, Validation Accuracy: 0.9702, Loss: 0.0404\n", + "Epoch 27 Batch 530/1077 - Train Accuracy: 0.9656, Validation Accuracy: 0.9645, Loss: 0.0437\n", + "Epoch 27 Batch 540/1077 - Train Accuracy: 0.9758, Validation Accuracy: 0.9695, Loss: 0.0479\n", + "Epoch 27 Batch 550/1077 - Train Accuracy: 0.9539, Validation Accuracy: 0.9638, Loss: 0.0465\n", + "Epoch 27 Batch 560/1077 - Train Accuracy: 0.9488, Validation Accuracy: 0.9684, Loss: 0.0491\n", + "Epoch 27 Batch 570/1077 - Train Accuracy: 0.9626, Validation Accuracy: 0.9680, Loss: 0.0606\n", + "Epoch 27 Batch 580/1077 - Train Accuracy: 0.9807, Validation Accuracy: 0.9744, Loss: 0.0384\n", + "Epoch 27 Batch 590/1077 - Train Accuracy: 0.9527, Validation Accuracy: 0.9744, Loss: 0.0517\n", + "Epoch 27 Batch 600/1077 - Train Accuracy: 0.9784, Validation Accuracy: 0.9702, Loss: 0.0502\n", + "Epoch 27 Batch 610/1077 - Train Accuracy: 0.9470, Validation Accuracy: 0.9698, Loss: 0.0568\n", + "Epoch 27 Batch 620/1077 - Train Accuracy: 0.9758, Validation Accuracy: 0.9769, Loss: 0.0452\n", + "Epoch 27 Batch 630/1077 - Train Accuracy: 0.9719, Validation Accuracy: 0.9769, Loss: 0.0491\n", + "Epoch 27 Batch 640/1077 - Train Accuracy: 0.9587, Validation Accuracy: 0.9719, Loss: 0.0561\n", + "Epoch 27 Batch 650/1077 - Train Accuracy: 0.9754, Validation Accuracy: 0.9741, Loss: 0.0406\n", + "Epoch 27 Batch 660/1077 - Train Accuracy: 0.9793, Validation Accuracy: 0.9762, Loss: 0.0492\n", + "Epoch 27 Batch 670/1077 - Train Accuracy: 0.9521, Validation Accuracy: 0.9751, Loss: 0.0507\n", + "Epoch 27 Batch 680/1077 - Train Accuracy: 0.9628, Validation Accuracy: 0.9659, Loss: 0.0430\n", + "Epoch 27 Batch 690/1077 - Train Accuracy: 0.9598, Validation Accuracy: 0.9712, Loss: 0.0480\n", + "Epoch 27 Batch 700/1077 - Train Accuracy: 0.9730, Validation Accuracy: 0.9691, Loss: 0.0413\n", + "Epoch 27 Batch 710/1077 - Train Accuracy: 0.9840, Validation Accuracy: 0.9712, Loss: 0.0299\n", + "Epoch 27 Batch 720/1077 - Train Accuracy: 0.9675, Validation Accuracy: 0.9666, Loss: 0.0587\n", + "Epoch 27 Batch 730/1077 - Train Accuracy: 0.9559, Validation Accuracy: 0.9666, Loss: 0.0655\n", + "Epoch 27 Batch 740/1077 - Train Accuracy: 0.9797, Validation Accuracy: 0.9673, Loss: 0.0369\n", + "Epoch 27 Batch 750/1077 - Train Accuracy: 0.9441, Validation Accuracy: 0.9801, Loss: 0.0455\n", + "Epoch 27 Batch 760/1077 - Train Accuracy: 0.9789, Validation Accuracy: 0.9773, Loss: 0.0582\n", + "Epoch 27 Batch 770/1077 - Train Accuracy: 0.9501, Validation Accuracy: 0.9780, Loss: 0.0459\n", + "Epoch 27 Batch 780/1077 - Train Accuracy: 0.9551, Validation Accuracy: 0.9631, Loss: 0.0635\n", + "Epoch 27 Batch 790/1077 - Train Accuracy: 0.9492, Validation Accuracy: 0.9634, Loss: 0.0434\n", + "Epoch 27 Batch 800/1077 - Train Accuracy: 0.9617, Validation Accuracy: 0.9751, Loss: 0.0500\n", + "Epoch 27 Batch 810/1077 - Train Accuracy: 0.9743, Validation Accuracy: 0.9702, Loss: 0.0354\n", + "Epoch 27 Batch 820/1077 - Train Accuracy: 0.9930, Validation Accuracy: 0.9634, Loss: 0.0443\n", + "Epoch 27 Batch 830/1077 - Train Accuracy: 0.9180, Validation Accuracy: 0.9631, Loss: 0.0561\n", + "Epoch 27 Batch 840/1077 - Train Accuracy: 0.9805, Validation Accuracy: 0.9688, Loss: 0.0484\n", + "Epoch 27 Batch 850/1077 - Train Accuracy: 0.9706, Validation Accuracy: 0.9709, Loss: 0.0687\n", + "Epoch 27 Batch 860/1077 - Train Accuracy: 0.9554, Validation Accuracy: 0.9819, Loss: 0.0488\n", + "Epoch 27 Batch 870/1077 - Train Accuracy: 0.9556, Validation Accuracy: 0.9688, Loss: 0.0461\n", + "Epoch 27 Batch 880/1077 - Train Accuracy: 0.9641, Validation Accuracy: 0.9712, Loss: 0.0604\n", + "Epoch 27 Batch 890/1077 - Train Accuracy: 0.9766, Validation Accuracy: 0.9677, Loss: 0.0462\n", + "Epoch 27 Batch 900/1077 - Train Accuracy: 0.9652, Validation Accuracy: 0.9688, Loss: 0.0442\n", + "Epoch 27 Batch 910/1077 - Train Accuracy: 0.9583, Validation Accuracy: 0.9684, Loss: 0.0495\n", + "Epoch 27 Batch 920/1077 - Train Accuracy: 0.9730, Validation Accuracy: 0.9688, Loss: 0.0455\n", + "Epoch 27 Batch 930/1077 - Train Accuracy: 0.9563, Validation Accuracy: 0.9616, Loss: 0.0396\n", + "Epoch 27 Batch 940/1077 - Train Accuracy: 0.9785, Validation Accuracy: 0.9570, Loss: 0.0397\n", + "Epoch 27 Batch 950/1077 - Train Accuracy: 0.9665, Validation Accuracy: 0.9737, Loss: 0.0330\n", + "Epoch 27 Batch 960/1077 - Train Accuracy: 0.9643, Validation Accuracy: 0.9737, Loss: 0.0447\n", + "Epoch 27 Batch 970/1077 - Train Accuracy: 0.9668, Validation Accuracy: 0.9787, Loss: 0.0555\n", + "Epoch 27 Batch 980/1077 - Train Accuracy: 0.9410, Validation Accuracy: 0.9698, Loss: 0.0480\n", + "Epoch 27 Batch 990/1077 - Train Accuracy: 0.9716, Validation Accuracy: 0.9677, Loss: 0.0542\n", + "Epoch 27 Batch 1000/1077 - Train Accuracy: 0.9386, Validation Accuracy: 0.9709, Loss: 0.0515\n", + "Epoch 27 Batch 1010/1077 - Train Accuracy: 0.9719, Validation Accuracy: 0.9659, Loss: 0.0463\n", + "Epoch 27 Batch 1020/1077 - Train Accuracy: 0.9832, Validation Accuracy: 0.9688, Loss: 0.0407\n", + "Epoch 27 Batch 1030/1077 - Train Accuracy: 0.9824, Validation Accuracy: 0.9734, Loss: 0.0422\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 27 Batch 1040/1077 - Train Accuracy: 0.9622, Validation Accuracy: 0.9759, Loss: 0.0476\n", + "Epoch 27 Batch 1050/1077 - Train Accuracy: 0.9879, Validation Accuracy: 0.9698, Loss: 0.0350\n", + "Epoch 27 Batch 1060/1077 - Train Accuracy: 0.9535, Validation Accuracy: 0.9719, Loss: 0.0557\n", + "Epoch 27 Batch 1070/1077 - Train Accuracy: 0.9539, Validation Accuracy: 0.9734, Loss: 0.0419\n", + "Epoch 28 Batch 10/1077 - Train Accuracy: 0.9502, Validation Accuracy: 0.9716, Loss: 0.0659\n", + "Epoch 28 Batch 20/1077 - Train Accuracy: 0.9746, Validation Accuracy: 0.9737, Loss: 0.0424\n", + "Epoch 28 Batch 30/1077 - Train Accuracy: 0.9715, Validation Accuracy: 0.9737, Loss: 0.0392\n", + "Epoch 28 Batch 40/1077 - Train Accuracy: 0.9812, Validation Accuracy: 0.9734, Loss: 0.0429\n", + "Epoch 28 Batch 50/1077 - Train Accuracy: 0.9820, Validation Accuracy: 0.9680, Loss: 0.0498\n", + "Epoch 28 Batch 60/1077 - Train Accuracy: 0.9747, Validation Accuracy: 0.9759, Loss: 0.0429\n", + "Epoch 28 Batch 70/1077 - Train Accuracy: 0.9539, Validation Accuracy: 0.9648, Loss: 0.0492\n", + "Epoch 28 Batch 80/1077 - Train Accuracy: 0.9805, Validation Accuracy: 0.9702, Loss: 0.0357\n", + "Epoch 28 Batch 90/1077 - Train Accuracy: 0.9590, Validation Accuracy: 0.9673, Loss: 0.0422\n", + "Epoch 28 Batch 100/1077 - Train Accuracy: 0.9648, Validation Accuracy: 0.9627, Loss: 0.0436\n", + "Epoch 28 Batch 110/1077 - Train Accuracy: 0.9848, Validation Accuracy: 0.9616, Loss: 0.0392\n", + "Epoch 28 Batch 120/1077 - Train Accuracy: 0.9680, Validation Accuracy: 0.9723, Loss: 0.0507\n", + "Epoch 28 Batch 130/1077 - Train Accuracy: 0.9751, Validation Accuracy: 0.9695, Loss: 0.0307\n", + "Epoch 28 Batch 140/1077 - Train Accuracy: 0.9683, Validation Accuracy: 0.9719, Loss: 0.0372\n", + "Epoch 28 Batch 150/1077 - Train Accuracy: 0.9725, Validation Accuracy: 0.9684, Loss: 0.0484\n", + "Epoch 28 Batch 160/1077 - Train Accuracy: 0.9832, Validation Accuracy: 0.9680, Loss: 0.0328\n", + "Epoch 28 Batch 170/1077 - Train Accuracy: 0.9520, Validation Accuracy: 0.9670, Loss: 0.0481\n", + "Epoch 28 Batch 180/1077 - Train Accuracy: 0.9703, Validation Accuracy: 0.9698, Loss: 0.0386\n", + "Epoch 28 Batch 190/1077 - Train Accuracy: 0.9723, Validation Accuracy: 0.9641, Loss: 0.0497\n", + "Epoch 28 Batch 200/1077 - Train Accuracy: 0.9809, Validation Accuracy: 0.9688, Loss: 0.0496\n", + "Epoch 28 Batch 210/1077 - Train Accuracy: 0.9580, Validation Accuracy: 0.9673, Loss: 0.0466\n", + "Epoch 28 Batch 220/1077 - Train Accuracy: 0.9618, Validation Accuracy: 0.9666, Loss: 0.0554\n", + "Epoch 28 Batch 230/1077 - Train Accuracy: 0.9874, Validation Accuracy: 0.9627, Loss: 0.0374\n", + "Epoch 28 Batch 240/1077 - Train Accuracy: 0.9770, Validation Accuracy: 0.9673, Loss: 0.0416\n", + "Epoch 28 Batch 250/1077 - Train Accuracy: 0.9563, Validation Accuracy: 0.9673, Loss: 0.0417\n", + "Epoch 28 Batch 260/1077 - Train Accuracy: 0.9699, Validation Accuracy: 0.9695, Loss: 0.0365\n", + "Epoch 28 Batch 270/1077 - Train Accuracy: 0.9434, Validation Accuracy: 0.9709, Loss: 0.0535\n", + "Epoch 28 Batch 280/1077 - Train Accuracy: 0.9629, Validation Accuracy: 0.9730, Loss: 0.0526\n", + "Epoch 28 Batch 290/1077 - Train Accuracy: 0.9496, Validation Accuracy: 0.9702, Loss: 0.0629\n", + "Epoch 28 Batch 300/1077 - Train Accuracy: 0.9634, Validation Accuracy: 0.9691, Loss: 0.0512\n", + "Epoch 28 Batch 310/1077 - Train Accuracy: 0.9648, Validation Accuracy: 0.9652, Loss: 0.0430\n", + "Epoch 28 Batch 320/1077 - Train Accuracy: 0.9691, Validation Accuracy: 0.9744, Loss: 0.0452\n", + "Epoch 28 Batch 330/1077 - Train Accuracy: 0.9500, Validation Accuracy: 0.9691, Loss: 0.0485\n", + "Epoch 28 Batch 340/1077 - Train Accuracy: 0.9811, Validation Accuracy: 0.9638, Loss: 0.0449\n", + "Epoch 28 Batch 350/1077 - Train Accuracy: 0.9551, Validation Accuracy: 0.9641, Loss: 0.0403\n", + "Epoch 28 Batch 360/1077 - Train Accuracy: 0.9668, Validation Accuracy: 0.9723, Loss: 0.0333\n", + "Epoch 28 Batch 370/1077 - Train Accuracy: 0.9714, Validation Accuracy: 0.9627, Loss: 0.0426\n", + "Epoch 28 Batch 380/1077 - Train Accuracy: 0.9766, Validation Accuracy: 0.9705, Loss: 0.0456\n", + "Epoch 28 Batch 390/1077 - Train Accuracy: 0.9566, Validation Accuracy: 0.9670, Loss: 0.0581\n", + "Epoch 28 Batch 400/1077 - Train Accuracy: 0.9711, Validation Accuracy: 0.9648, Loss: 0.0463\n", + "Epoch 28 Batch 410/1077 - Train Accuracy: 0.9363, Validation Accuracy: 0.9577, Loss: 0.0515\n", + "Epoch 28 Batch 420/1077 - Train Accuracy: 0.9723, Validation Accuracy: 0.9620, Loss: 0.0341\n", + "Epoch 28 Batch 430/1077 - Train Accuracy: 0.9703, Validation Accuracy: 0.9606, Loss: 0.0327\n", + "Epoch 28 Batch 440/1077 - Train Accuracy: 0.9551, Validation Accuracy: 0.9634, Loss: 0.0642\n", + "Epoch 28 Batch 450/1077 - Train Accuracy: 0.9691, Validation Accuracy: 0.9680, Loss: 0.0589\n", + "Epoch 28 Batch 460/1077 - Train Accuracy: 0.9742, Validation Accuracy: 0.9652, Loss: 0.0457\n", + "Epoch 28 Batch 470/1077 - Train Accuracy: 0.9877, Validation Accuracy: 0.9602, Loss: 0.0420\n", + "Epoch 28 Batch 480/1077 - Train Accuracy: 0.9404, Validation Accuracy: 0.9606, Loss: 0.0506\n", + "Epoch 28 Batch 490/1077 - Train Accuracy: 0.9570, Validation Accuracy: 0.9627, Loss: 0.0406\n", + "Epoch 28 Batch 500/1077 - Train Accuracy: 0.9621, Validation Accuracy: 0.9698, Loss: 0.0405\n", + "Epoch 28 Batch 510/1077 - Train Accuracy: 0.9652, Validation Accuracy: 0.9652, Loss: 0.0457\n", + "Epoch 28 Batch 520/1077 - Train Accuracy: 0.9777, Validation Accuracy: 0.9670, Loss: 0.0386\n", + "Epoch 28 Batch 530/1077 - Train Accuracy: 0.9703, Validation Accuracy: 0.9691, Loss: 0.0467\n", + "Epoch 28 Batch 540/1077 - Train Accuracy: 0.9750, Validation Accuracy: 0.9695, Loss: 0.0422\n", + "Epoch 28 Batch 550/1077 - Train Accuracy: 0.9605, Validation Accuracy: 0.9702, Loss: 0.0595\n", + "Epoch 28 Batch 560/1077 - Train Accuracy: 0.9523, Validation Accuracy: 0.9673, Loss: 0.0445\n", + "Epoch 28 Batch 570/1077 - Train Accuracy: 0.9494, Validation Accuracy: 0.9688, Loss: 0.0545\n", + "Epoch 28 Batch 580/1077 - Train Accuracy: 0.9803, Validation Accuracy: 0.9666, Loss: 0.0349\n", + "Epoch 28 Batch 590/1077 - Train Accuracy: 0.9593, Validation Accuracy: 0.9709, Loss: 0.0491\n", + "Epoch 28 Batch 600/1077 - Train Accuracy: 0.9762, Validation Accuracy: 0.9730, Loss: 0.0462\n", + "Epoch 28 Batch 610/1077 - Train Accuracy: 0.9523, Validation Accuracy: 0.9780, Loss: 0.0585\n", + "Epoch 28 Batch 620/1077 - Train Accuracy: 0.9781, Validation Accuracy: 0.9751, Loss: 0.0467\n", + "Epoch 28 Batch 630/1077 - Train Accuracy: 0.9719, Validation Accuracy: 0.9592, Loss: 0.0448\n", + "Epoch 28 Batch 640/1077 - Train Accuracy: 0.9695, Validation Accuracy: 0.9734, Loss: 0.0445\n", + "Epoch 28 Batch 650/1077 - Train Accuracy: 0.9711, Validation Accuracy: 0.9744, Loss: 0.0457\n", + "Epoch 28 Batch 660/1077 - Train Accuracy: 0.9746, Validation Accuracy: 0.9688, Loss: 0.0330\n", + "Epoch 28 Batch 670/1077 - Train Accuracy: 0.9577, Validation Accuracy: 0.9656, Loss: 0.0491\n", + "Epoch 28 Batch 680/1077 - Train Accuracy: 0.9635, Validation Accuracy: 0.9705, Loss: 0.0455\n", + "Epoch 28 Batch 690/1077 - Train Accuracy: 0.9578, Validation Accuracy: 0.9759, Loss: 0.0462\n", + "Epoch 28 Batch 700/1077 - Train Accuracy: 0.9750, Validation Accuracy: 0.9709, Loss: 0.0455\n", + "Epoch 28 Batch 710/1077 - Train Accuracy: 0.9711, Validation Accuracy: 0.9783, Loss: 0.0346\n", + "Epoch 28 Batch 720/1077 - Train Accuracy: 0.9692, Validation Accuracy: 0.9702, Loss: 0.0614\n", + "Epoch 28 Batch 730/1077 - Train Accuracy: 0.9742, Validation Accuracy: 0.9609, Loss: 0.0683\n", + "Epoch 28 Batch 740/1077 - Train Accuracy: 0.9645, Validation Accuracy: 0.9595, Loss: 0.0402\n", + "Epoch 28 Batch 750/1077 - Train Accuracy: 0.9656, Validation Accuracy: 0.9659, Loss: 0.0385\n", + "Epoch 28 Batch 760/1077 - Train Accuracy: 0.9742, Validation Accuracy: 0.9627, Loss: 0.0479\n", + "Epoch 28 Batch 770/1077 - Train Accuracy: 0.9513, Validation Accuracy: 0.9723, Loss: 0.0441\n", + "Epoch 28 Batch 780/1077 - Train Accuracy: 0.9418, Validation Accuracy: 0.9634, Loss: 0.0768\n", + "Epoch 28 Batch 790/1077 - Train Accuracy: 0.9539, Validation Accuracy: 0.9705, Loss: 0.0467\n", + "Epoch 28 Batch 800/1077 - Train Accuracy: 0.9777, Validation Accuracy: 0.9620, Loss: 0.0477\n", + "Epoch 28 Batch 810/1077 - Train Accuracy: 0.9688, Validation Accuracy: 0.9670, Loss: 0.0319\n", + "Epoch 28 Batch 820/1077 - Train Accuracy: 0.9777, Validation Accuracy: 0.9677, Loss: 0.0451\n", + "Epoch 28 Batch 830/1077 - Train Accuracy: 0.9512, Validation Accuracy: 0.9691, Loss: 0.0536\n", + "Epoch 28 Batch 840/1077 - Train Accuracy: 0.9785, Validation Accuracy: 0.9695, Loss: 0.0311\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 28 Batch 850/1077 - Train Accuracy: 0.9695, Validation Accuracy: 0.9719, Loss: 0.0606\n", + "Epoch 28 Batch 860/1077 - Train Accuracy: 0.9628, Validation Accuracy: 0.9535, Loss: 0.0446\n", + "Epoch 28 Batch 870/1077 - Train Accuracy: 0.9601, Validation Accuracy: 0.9673, Loss: 0.0360\n", + "Epoch 28 Batch 880/1077 - Train Accuracy: 0.9598, Validation Accuracy: 0.9716, Loss: 0.0457\n", + "Epoch 28 Batch 890/1077 - Train Accuracy: 0.9688, Validation Accuracy: 0.9645, Loss: 0.0428\n", + "Epoch 28 Batch 900/1077 - Train Accuracy: 0.9543, Validation Accuracy: 0.9641, Loss: 0.0515\n", + "Epoch 28 Batch 910/1077 - Train Accuracy: 0.9702, Validation Accuracy: 0.9787, Loss: 0.0427\n", + "Epoch 28 Batch 920/1077 - Train Accuracy: 0.9734, Validation Accuracy: 0.9716, Loss: 0.0329\n", + "Epoch 28 Batch 930/1077 - Train Accuracy: 0.9688, Validation Accuracy: 0.9613, Loss: 0.0404\n", + "Epoch 28 Batch 940/1077 - Train Accuracy: 0.9750, Validation Accuracy: 0.9609, Loss: 0.0352\n", + "Epoch 28 Batch 950/1077 - Train Accuracy: 0.9788, Validation Accuracy: 0.9602, Loss: 0.0400\n", + "Epoch 28 Batch 960/1077 - Train Accuracy: 0.9665, Validation Accuracy: 0.9702, Loss: 0.0401\n", + "Epoch 28 Batch 970/1077 - Train Accuracy: 0.9602, Validation Accuracy: 0.9730, Loss: 0.0541\n", + "Epoch 28 Batch 980/1077 - Train Accuracy: 0.9551, Validation Accuracy: 0.9730, Loss: 0.0547\n", + "Epoch 28 Batch 990/1077 - Train Accuracy: 0.9819, Validation Accuracy: 0.9734, Loss: 0.0493\n", + "Epoch 28 Batch 1000/1077 - Train Accuracy: 0.9382, Validation Accuracy: 0.9727, Loss: 0.0642\n", + "Epoch 28 Batch 1010/1077 - Train Accuracy: 0.9793, Validation Accuracy: 0.9656, Loss: 0.0391\n", + "Epoch 28 Batch 1020/1077 - Train Accuracy: 0.9949, Validation Accuracy: 0.9709, Loss: 0.0301\n", + "Epoch 28 Batch 1030/1077 - Train Accuracy: 0.9859, Validation Accuracy: 0.9780, Loss: 0.0509\n", + "Epoch 28 Batch 1040/1077 - Train Accuracy: 0.9790, Validation Accuracy: 0.9762, Loss: 0.0489\n", + "Epoch 28 Batch 1050/1077 - Train Accuracy: 0.9805, Validation Accuracy: 0.9688, Loss: 0.0348\n", + "Epoch 28 Batch 1060/1077 - Train Accuracy: 0.9805, Validation Accuracy: 0.9730, Loss: 0.0355\n", + "Epoch 28 Batch 1070/1077 - Train Accuracy: 0.9703, Validation Accuracy: 0.9719, Loss: 0.0389\n", + "Epoch 29 Batch 10/1077 - Train Accuracy: 0.9527, Validation Accuracy: 0.9705, Loss: 0.0527\n", + "Epoch 29 Batch 20/1077 - Train Accuracy: 0.9621, Validation Accuracy: 0.9666, Loss: 0.0460\n", + "Epoch 29 Batch 30/1077 - Train Accuracy: 0.9629, Validation Accuracy: 0.9790, Loss: 0.0479\n", + "Epoch 29 Batch 40/1077 - Train Accuracy: 0.9805, Validation Accuracy: 0.9648, Loss: 0.0324\n", + "Epoch 29 Batch 50/1077 - Train Accuracy: 0.9809, Validation Accuracy: 0.9695, Loss: 0.0437\n", + "Epoch 29 Batch 60/1077 - Train Accuracy: 0.9754, Validation Accuracy: 0.9656, Loss: 0.0277\n", + "Epoch 29 Batch 70/1077 - Train Accuracy: 0.9539, Validation Accuracy: 0.9652, Loss: 0.0571\n", + "Epoch 29 Batch 80/1077 - Train Accuracy: 0.9766, Validation Accuracy: 0.9759, Loss: 0.0379\n", + "Epoch 29 Batch 90/1077 - Train Accuracy: 0.9719, Validation Accuracy: 0.9684, Loss: 0.0430\n", + "Epoch 29 Batch 100/1077 - Train Accuracy: 0.9730, Validation Accuracy: 0.9648, Loss: 0.0417\n", + "Epoch 29 Batch 110/1077 - Train Accuracy: 0.9742, Validation Accuracy: 0.9712, Loss: 0.0293\n", + "Epoch 29 Batch 120/1077 - Train Accuracy: 0.9629, Validation Accuracy: 0.9766, Loss: 0.0513\n", + "Epoch 29 Batch 130/1077 - Train Accuracy: 0.9810, Validation Accuracy: 0.9659, Loss: 0.0342\n", + "Epoch 29 Batch 140/1077 - Train Accuracy: 0.9655, Validation Accuracy: 0.9620, Loss: 0.0331\n", + "Epoch 29 Batch 150/1077 - Train Accuracy: 0.9539, Validation Accuracy: 0.9616, Loss: 0.0430\n", + "Epoch 29 Batch 160/1077 - Train Accuracy: 0.9750, Validation Accuracy: 0.9570, Loss: 0.0363\n", + "Epoch 29 Batch 170/1077 - Train Accuracy: 0.9563, Validation Accuracy: 0.9762, Loss: 0.0537\n", + "Epoch 29 Batch 180/1077 - Train Accuracy: 0.9828, Validation Accuracy: 0.9666, Loss: 0.0320\n", + "Epoch 29 Batch 190/1077 - Train Accuracy: 0.9684, Validation Accuracy: 0.9709, Loss: 0.0407\n", + "Epoch 29 Batch 200/1077 - Train Accuracy: 0.9656, Validation Accuracy: 0.9624, Loss: 0.0512\n", + "Epoch 29 Batch 210/1077 - Train Accuracy: 0.9483, Validation Accuracy: 0.9620, Loss: 0.0497\n", + "Epoch 29 Batch 220/1077 - Train Accuracy: 0.9646, Validation Accuracy: 0.9719, Loss: 0.0506\n", + "Epoch 29 Batch 230/1077 - Train Accuracy: 0.9866, Validation Accuracy: 0.9741, Loss: 0.0375\n", + "Epoch 29 Batch 240/1077 - Train Accuracy: 0.9762, Validation Accuracy: 0.9716, Loss: 0.0358\n", + "Epoch 29 Batch 250/1077 - Train Accuracy: 0.9602, Validation Accuracy: 0.9705, Loss: 0.0476\n", + "Epoch 29 Batch 260/1077 - Train Accuracy: 0.9617, Validation Accuracy: 0.9751, Loss: 0.0386\n", + "Epoch 29 Batch 270/1077 - Train Accuracy: 0.9559, Validation Accuracy: 0.9723, Loss: 0.0550\n", + "Epoch 29 Batch 280/1077 - Train Accuracy: 0.9563, Validation Accuracy: 0.9634, Loss: 0.0523\n", + "Epoch 29 Batch 290/1077 - Train Accuracy: 0.9563, Validation Accuracy: 0.9656, Loss: 0.0611\n", + "Epoch 29 Batch 300/1077 - Train Accuracy: 0.9659, Validation Accuracy: 0.9634, Loss: 0.0414\n", + "Epoch 29 Batch 310/1077 - Train Accuracy: 0.9797, Validation Accuracy: 0.9727, Loss: 0.0367\n", + "Epoch 29 Batch 320/1077 - Train Accuracy: 0.9652, Validation Accuracy: 0.9684, Loss: 0.0398\n", + "Epoch 29 Batch 330/1077 - Train Accuracy: 0.9598, Validation Accuracy: 0.9656, Loss: 0.0423\n", + "Epoch 29 Batch 340/1077 - Train Accuracy: 0.9782, Validation Accuracy: 0.9677, Loss: 0.0455\n", + "Epoch 29 Batch 350/1077 - Train Accuracy: 0.9656, Validation Accuracy: 0.9627, Loss: 0.0393\n", + "Epoch 29 Batch 360/1077 - Train Accuracy: 0.9777, Validation Accuracy: 0.9719, Loss: 0.0354\n", + "Epoch 29 Batch 370/1077 - Train Accuracy: 0.9818, Validation Accuracy: 0.9666, Loss: 0.0440\n", + "Epoch 29 Batch 380/1077 - Train Accuracy: 0.9762, Validation Accuracy: 0.9688, Loss: 0.0433\n", + "Epoch 29 Batch 390/1077 - Train Accuracy: 0.9512, Validation Accuracy: 0.9759, Loss: 0.0574\n", + "Epoch 29 Batch 400/1077 - Train Accuracy: 0.9656, Validation Accuracy: 0.9666, Loss: 0.0475\n", + "Epoch 29 Batch 410/1077 - Train Accuracy: 0.9396, Validation Accuracy: 0.9759, Loss: 0.0643\n", + "Epoch 29 Batch 420/1077 - Train Accuracy: 0.9746, Validation Accuracy: 0.9680, Loss: 0.0358\n", + "Epoch 29 Batch 430/1077 - Train Accuracy: 0.9727, Validation Accuracy: 0.9705, Loss: 0.0392\n", + "Epoch 29 Batch 440/1077 - Train Accuracy: 0.9551, Validation Accuracy: 0.9705, Loss: 0.0451\n", + "Epoch 29 Batch 450/1077 - Train Accuracy: 0.9570, Validation Accuracy: 0.9634, Loss: 0.0584\n", + "Epoch 29 Batch 460/1077 - Train Accuracy: 0.9730, Validation Accuracy: 0.9634, Loss: 0.0468\n", + "Epoch 29 Batch 470/1077 - Train Accuracy: 0.9856, Validation Accuracy: 0.9684, Loss: 0.0440\n", + "Epoch 29 Batch 480/1077 - Train Accuracy: 0.9609, Validation Accuracy: 0.9645, Loss: 0.0376\n", + "Epoch 29 Batch 490/1077 - Train Accuracy: 0.9625, Validation Accuracy: 0.9574, Loss: 0.0513\n", + "Epoch 29 Batch 500/1077 - Train Accuracy: 0.9617, Validation Accuracy: 0.9698, Loss: 0.0399\n", + "Epoch 29 Batch 510/1077 - Train Accuracy: 0.9746, Validation Accuracy: 0.9695, Loss: 0.0431\n", + "Epoch 29 Batch 520/1077 - Train Accuracy: 0.9859, Validation Accuracy: 0.9549, Loss: 0.0331\n", + "Epoch 29 Batch 530/1077 - Train Accuracy: 0.9680, Validation Accuracy: 0.9400, Loss: 0.0503\n", + "Epoch 29 Batch 540/1077 - Train Accuracy: 0.9785, Validation Accuracy: 0.9599, Loss: 0.0505\n", + "Epoch 29 Batch 550/1077 - Train Accuracy: 0.9555, Validation Accuracy: 0.9716, Loss: 0.0499\n", + "Epoch 29 Batch 560/1077 - Train Accuracy: 0.9523, Validation Accuracy: 0.9673, Loss: 0.0374\n", + "Epoch 29 Batch 570/1077 - Train Accuracy: 0.9412, Validation Accuracy: 0.9641, Loss: 0.0592\n", + "Epoch 29 Batch 580/1077 - Train Accuracy: 0.9821, Validation Accuracy: 0.9741, Loss: 0.0287\n", + "Epoch 29 Batch 590/1077 - Train Accuracy: 0.9572, Validation Accuracy: 0.9695, Loss: 0.0584\n", + "Epoch 29 Batch 600/1077 - Train Accuracy: 0.9754, Validation Accuracy: 0.9723, Loss: 0.0546\n", + "Epoch 29 Batch 610/1077 - Train Accuracy: 0.9593, Validation Accuracy: 0.9677, Loss: 0.0525\n", + "Epoch 29 Batch 620/1077 - Train Accuracy: 0.9707, Validation Accuracy: 0.9737, Loss: 0.0459\n", + "Epoch 29 Batch 630/1077 - Train Accuracy: 0.9762, Validation Accuracy: 0.9695, Loss: 0.0460\n", + "Epoch 29 Batch 640/1077 - Train Accuracy: 0.9587, Validation Accuracy: 0.9712, Loss: 0.0472\n", + "Epoch 29 Batch 650/1077 - Train Accuracy: 0.9605, Validation Accuracy: 0.9798, Loss: 0.0443\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch 29 Batch 660/1077 - Train Accuracy: 0.9789, Validation Accuracy: 0.9730, Loss: 0.0419\n", + "Epoch 29 Batch 670/1077 - Train Accuracy: 0.9744, Validation Accuracy: 0.9741, Loss: 0.0461\n", + "Epoch 29 Batch 680/1077 - Train Accuracy: 0.9650, Validation Accuracy: 0.9798, Loss: 0.0467\n", + "Epoch 29 Batch 690/1077 - Train Accuracy: 0.9566, Validation Accuracy: 0.9744, Loss: 0.0496\n", + "Epoch 29 Batch 700/1077 - Train Accuracy: 0.9730, Validation Accuracy: 0.9712, Loss: 0.0402\n", + "Epoch 29 Batch 710/1077 - Train Accuracy: 0.9723, Validation Accuracy: 0.9766, Loss: 0.0263\n", + "Epoch 29 Batch 720/1077 - Train Accuracy: 0.9642, Validation Accuracy: 0.9702, Loss: 0.0480\n", + "Epoch 29 Batch 730/1077 - Train Accuracy: 0.9773, Validation Accuracy: 0.9755, Loss: 0.0639\n", + "Epoch 29 Batch 740/1077 - Train Accuracy: 0.9852, Validation Accuracy: 0.9741, Loss: 0.0370\n", + "Epoch 29 Batch 750/1077 - Train Accuracy: 0.9570, Validation Accuracy: 0.9759, Loss: 0.0531\n", + "Epoch 29 Batch 760/1077 - Train Accuracy: 0.9805, Validation Accuracy: 0.9656, Loss: 0.0412\n", + "Epoch 29 Batch 770/1077 - Train Accuracy: 0.9598, Validation Accuracy: 0.9666, Loss: 0.0435\n", + "Epoch 29 Batch 780/1077 - Train Accuracy: 0.9461, Validation Accuracy: 0.9691, Loss: 0.0623\n", + "Epoch 29 Batch 790/1077 - Train Accuracy: 0.9660, Validation Accuracy: 0.9648, Loss: 0.0484\n", + "Epoch 29 Batch 800/1077 - Train Accuracy: 0.9836, Validation Accuracy: 0.9723, Loss: 0.0400\n", + "Epoch 29 Batch 810/1077 - Train Accuracy: 0.9699, Validation Accuracy: 0.9680, Loss: 0.0427\n", + "Epoch 29 Batch 820/1077 - Train Accuracy: 0.9871, Validation Accuracy: 0.9680, Loss: 0.0366\n", + "Epoch 29 Batch 830/1077 - Train Accuracy: 0.9461, Validation Accuracy: 0.9712, Loss: 0.0431\n", + "Epoch 29 Batch 840/1077 - Train Accuracy: 0.9715, Validation Accuracy: 0.9727, Loss: 0.0436\n", + "Epoch 29 Batch 850/1077 - Train Accuracy: 0.9702, Validation Accuracy: 0.9705, Loss: 0.0636\n", + "Epoch 29 Batch 860/1077 - Train Accuracy: 0.9635, Validation Accuracy: 0.9766, Loss: 0.0465\n", + "Epoch 29 Batch 870/1077 - Train Accuracy: 0.9613, Validation Accuracy: 0.9744, Loss: 0.0480\n", + "Epoch 29 Batch 880/1077 - Train Accuracy: 0.9684, Validation Accuracy: 0.9744, Loss: 0.0554\n", + "Epoch 29 Batch 890/1077 - Train Accuracy: 0.9788, Validation Accuracy: 0.9716, Loss: 0.0405\n", + "Epoch 29 Batch 900/1077 - Train Accuracy: 0.9539, Validation Accuracy: 0.9659, Loss: 0.0434\n", + "Epoch 29 Batch 910/1077 - Train Accuracy: 0.9758, Validation Accuracy: 0.9638, Loss: 0.0462\n", + "Epoch 29 Batch 920/1077 - Train Accuracy: 0.9668, Validation Accuracy: 0.9670, Loss: 0.0490\n", + "Epoch 29 Batch 930/1077 - Train Accuracy: 0.9633, Validation Accuracy: 0.9673, Loss: 0.0398\n", + "Epoch 29 Batch 940/1077 - Train Accuracy: 0.9656, Validation Accuracy: 0.9734, Loss: 0.0350\n", + "Epoch 29 Batch 950/1077 - Train Accuracy: 0.9632, Validation Accuracy: 0.9620, Loss: 0.0331\n", + "Epoch 29 Batch 960/1077 - Train Accuracy: 0.9639, Validation Accuracy: 0.9581, Loss: 0.0405\n", + "Epoch 29 Batch 970/1077 - Train Accuracy: 0.9656, Validation Accuracy: 0.9659, Loss: 0.0501\n", + "Epoch 29 Batch 980/1077 - Train Accuracy: 0.9551, Validation Accuracy: 0.9705, Loss: 0.0511\n", + "Epoch 29 Batch 990/1077 - Train Accuracy: 0.9704, Validation Accuracy: 0.9723, Loss: 0.0472\n", + "Epoch 29 Batch 1000/1077 - Train Accuracy: 0.9513, Validation Accuracy: 0.9666, Loss: 0.0435\n", + "Epoch 29 Batch 1010/1077 - Train Accuracy: 0.9684, Validation Accuracy: 0.9595, Loss: 0.0455\n", + "Epoch 29 Batch 1020/1077 - Train Accuracy: 0.9898, Validation Accuracy: 0.9741, Loss: 0.0371\n", + "Epoch 29 Batch 1030/1077 - Train Accuracy: 0.9914, Validation Accuracy: 0.9691, Loss: 0.0346\n", + "Epoch 29 Batch 1040/1077 - Train Accuracy: 0.9770, Validation Accuracy: 0.9641, Loss: 0.0498\n", + "Epoch 29 Batch 1050/1077 - Train Accuracy: 0.9934, Validation Accuracy: 0.9577, Loss: 0.0343\n", + "Epoch 29 Batch 1060/1077 - Train Accuracy: 0.9688, Validation Accuracy: 0.9723, Loss: 0.0298\n", + "Epoch 29 Batch 1070/1077 - Train Accuracy: 0.9711, Validation Accuracy: 0.9648, Loss: 0.0368\n", + "Model Trained and Saved\n" + ] + } + ], + "source": [ + "\"\"\"\n", + "DON'T MODIFY ANYTHING IN THIS CELL\n", + "\"\"\"\n", + "def get_accuracy(target, logits, _type):\n", + " \"\"\"\n", + " Calculate accuracy\n", + " \"\"\"\n", + " max_seq = max(target.shape[1], logits.shape[1])\n", + " if max_seq - target.shape[1]:\n", + " target = np.pad(\n", + " target,\n", + " [(0,0),(0,max_seq - target.shape[1])],\n", + " 'constant')\n", + " if max_seq - logits.shape[1]:\n", + " logits = np.pad(\n", + " logits,\n", + " [(0,0),(0,max_seq - logits.shape[1])],\n", + " 'constant')\n", + " \n", + " acc = np.mean(np.equal(target, logits))\n", + " if _type is 'train':\n", + " with tf.name_scope('optimization'):\n", + " summary = tf.Summary(value=[tf.Summary.Value(tag=\"accuracy\", simple_value=acc)])\n", + " else:\n", + " with tf.name_scope('validation'):\n", + " summary = tf.Summary(value=[tf.Summary.Value(tag=\"accuracy\", simple_value=acc)])\n", + " return summary, acc\n", + "\n", + "# Split data to training and validation sets\n", + "train_source = source_int_text[batch_size:]\n", + "train_target = target_int_text[batch_size:]\n", + "valid_source = source_int_text[:batch_size]\n", + "valid_target = target_int_text[:batch_size]\n", + "(valid_sources_batch, valid_targets_batch, valid_sources_lengths, valid_targets_lengths ) = next(get_batches(valid_source,\n", + " valid_target,\n", + " batch_size,\n", + " source_vocab_to_int[''],\n", + " target_vocab_to_int[''])) \n", + "with tf.Session(graph=train_graph) as sess:\n", + " sess.run(tf.global_variables_initializer())\n", + " \n", + " saver = tf.train.Saver(keep_checkpoint_every_n_hours=0.5)\n", + " \n", + " \n", + " for epoch_i in range(epochs):\n", + " \n", + " \n", + " n_batches = len(train_source)//batch_size \n", + " for batch_i, (source_batch, target_batch, sources_lengths, targets_lengths) in enumerate(\n", + " get_batches(train_source, train_target, batch_size,\n", + " source_vocab_to_int[''],\n", + " target_vocab_to_int[''])):\n", + " \n", + " iteration = epoch_i*n_batches + batch_i\n", + " \n", + " summary, _, loss = sess.run(\n", + " [merged, train_op, cost],\n", + " {input_data: source_batch,\n", + " targets: target_batch,\n", + " lr: learning_rate,\n", + " target_sequence_length: targets_lengths,\n", + " source_sequence_length: sources_lengths,\n", + " keep_prob: keep_probability})\n", + " \n", + " train_writer.add_summary(summary, iteration)\n", + " \n", + " if epoch_i % 5 == 0:\n", + " saver.save(sess, save_path + 'ckpt', global_step=epoch_i)\n", + "\n", + " if batch_i % display_step == 0 and batch_i > 0:\n", + "\n", + "\n", + " batch_train_logits = sess.run(\n", + " inference_logits,\n", + " {input_data: source_batch,\n", + " source_sequence_length: sources_lengths,\n", + " target_sequence_length: targets_lengths,\n", + " keep_prob: 1.0})\n", + "\n", + "\n", + " batch_valid_logits = sess.run(\n", + " inference_logits,\n", + " {input_data: valid_sources_batch,\n", + " source_sequence_length: valid_sources_lengths,\n", + " target_sequence_length: valid_targets_lengths,\n", + " keep_prob: 1.0})\n", + "\n", + " train_acc_sum, train_acc = get_accuracy(target_batch, batch_train_logits, _type='train')\n", + " \n", + " valid_acc_sum, valid_acc = get_accuracy(valid_targets_batch, batch_valid_logits, _type='test')\n", + " \n", + " \n", + " train_writer.add_summary(train_acc_sum, iteration)\n", + " test_writer.add_summary(valid_acc_sum, iteration)\n", + " \n", + " \n", + "\n", + " print('Epoch {:>3} Batch {:>4}/{} - Train Accuracy: {:>6.4f}, Validation Accuracy: {:>6.4f}, Loss: {:>6.4f}'\n", + " .format(epoch_i, batch_i, len(source_int_text) // batch_size, train_acc, valid_acc, loss))\n", + " \n", + "\n", + " # Save Model\n", + " saver = tf.train.Saver()\n", + " saver.save(sess, save_path)\n", + " print('Model Trained and Saved')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Save Parameters\n", + "Save the `batch_size` and `save_path` parameters for inference." + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "\"\"\"\n", + "DON'T MODIFY ANYTHING IN THIS CELL\n", + "\"\"\"\n", + "# Save parameters for checkpoint\n", + "helper.save_params(save_path)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Checkpoint" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "\"\"\"\n", + "DON'T MODIFY ANYTHING IN THIS CELL\n", + "\"\"\"\n", + "import tensorflow as tf\n", + "import numpy as np\n", + "import helper\n", + "import problem_unittests as tests\n", + "\n", + "_, (source_vocab_to_int, target_vocab_to_int), (source_int_to_vocab, target_int_to_vocab) = helper.load_preprocess()\n", + "load_path = helper.load_params()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Sentence to Sequence\n", + "To feed a sentence into the model for translation, you first need to preprocess it. Implement the function `sentence_to_seq()` to preprocess new sentences.\n", + "\n", + "- Convert the sentence to lowercase\n", + "- Convert words into ids using `vocab_to_int`\n", + " - Convert words not in the vocabulary, to the `` word id." + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Tests Passed\n" + ] + } + ], + "source": [ + "def sentence_to_seq(sentence, vocab_to_int):\n", + " \"\"\"\n", + " Convert a sentence to a sequence of ids\n", + " :param sentence: String\n", + " :param vocab_to_int: Dictionary to go from the words to an id\n", + " :return: List of word ids\n", + " \"\"\"\n", + " \n", + " \n", + " return [vocab_to_int.get(word, vocab_to_int['']) for word in sentence.split(' ')]\n", + "\n", + "\n", + "\n", + "\"\"\"\n", + "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n", + "\"\"\"\n", + "tests.test_sentence_to_seq(sentence_to_seq)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Translate\n", + "This will translate `translate_sentence` from English to French." + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "INFO:tensorflow:Restoring parameters from /output/run_1/checkpoints/\n", + "Input\n", + " Word Ids: [212, 60, 210, 140, 87, 123, 216]\n", + " English Words: ['he', 'saw', 'a', 'old', 'yellow', 'truck', '.']\n", + "\n", + "Prediction\n", + " Word Ids: [16, 354, 351, 246, 192, 256, 157, 185, 1]\n", + " French Words: il a pas la nouveau camion bleu . \n" + ] + } + ], + "source": [ + "translate_sentence = 'he saw a old yellow truck .'\n", + "\n", + "\n", + "\"\"\"\n", + "DON'T MODIFY ANYTHING IN THIS CELL\n", + "\"\"\"\n", + "translate_sentence = sentence_to_seq(translate_sentence, source_vocab_to_int)\n", + "\n", + "loaded_graph = tf.Graph()\n", + "with tf.Session(graph=loaded_graph) as sess:\n", + " # Load saved model\n", + " loader = tf.train.import_meta_graph(load_path + '.meta')\n", + " loader.restore(sess, load_path)\n", + "\n", + " input_data = loaded_graph.get_tensor_by_name('input:0')\n", + " logits = loaded_graph.get_tensor_by_name('predictions:0')\n", + " target_sequence_length = loaded_graph.get_tensor_by_name('target_sequence_length:0')\n", + " source_sequence_length = loaded_graph.get_tensor_by_name('source_sequence_length:0')\n", + " keep_prob = loaded_graph.get_tensor_by_name('keep_prob:0')\n", + "\n", + " translate_logits = sess.run(logits, {input_data: [translate_sentence]*batch_size,\n", + " target_sequence_length: [len(translate_sentence)*2]*batch_size,\n", + " source_sequence_length: [len(translate_sentence)]*batch_size,\n", + " keep_prob: 1.0})[0]\n", + "\n", + "print('Input')\n", + "print(' Word Ids: {}'.format([i for i in translate_sentence]))\n", + "print(' English Words: {}'.format([source_int_to_vocab[i] for i in translate_sentence]))\n", + "\n", + "print('\\nPrediction')\n", + "print(' Word Ids: {}'.format([i for i in translate_logits]))\n", + "print(' French Words: {}'.format(\" \".join([target_int_to_vocab[i] for i in translate_logits])))\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Imperfect Translation\n", + "You might notice that some sentences translate better than others. Since the dataset you're using only has a vocabulary of 227 English words of the thousands that you use, you're only going to see good results using these words. For this project, you don't need a perfect translation. However, if you want to create a better translation model, you'll need better data.\n", + "\n", + "You can train on the [WMT10 French-English corpus](http://www.statmt.org/wmt10/training-giga-fren.tar). This dataset has more vocabulary and richer in topics discussed. However, this will take you days to train, so make sure you've a GPU and the neural network is performing well on dataset we provided. Just make sure you play with the WMT10 corpus after you've submitted this project.\n", + "## Submitting This Project\n", + "When submitting this project, make sure to run all the cells before saving the notebook. Save the notebook file as \"dlnd_language_translation.ipynb\" and save it as a HTML file under \"File\" -> \"Download as\". Include the \"helper.py\" and \"problem_unittests.py\" files in your submission." + ] + } + ], + "metadata": { + "anaconda-cloud": {}, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.5.1" + } + }, + "nbformat": 4, + "nbformat_minor": 1 +} diff --git a/helper.py b/helper.py new file mode 100644 index 0000000..c2b92fa --- /dev/null +++ b/helper.py @@ -0,0 +1,99 @@ +import os +import pickle +import copy +import numpy as np + + +CODES = {'': 0, '': 1, '': 2, '': 3 } + + +def load_data(path): + """ + Load Dataset from File + """ + input_file = os.path.join(path) + with open(input_file, 'r', encoding='utf-8') as f: + return f.read() + + +def preprocess_and_save_data(source_path, target_path, text_to_ids): + """ + Preprocess Text Data. Save to to file. + """ + # Preprocess + source_text = load_data(source_path) + target_text = load_data(target_path) + + source_text = source_text.lower() + target_text = target_text.lower() + + source_vocab_to_int, source_int_to_vocab = create_lookup_tables(source_text) + target_vocab_to_int, target_int_to_vocab = create_lookup_tables(target_text) + + source_text, target_text = text_to_ids(source_text, target_text, source_vocab_to_int, target_vocab_to_int) + + # Save Data + with open('/output/preprocess.p', 'wb') as out_file: + pickle.dump(( + (source_text, target_text), + (source_vocab_to_int, target_vocab_to_int), + (source_int_to_vocab, target_int_to_vocab)), out_file) + + +def load_preprocess(): + """ + Load the Preprocessed Training data and return them in batches of or less + """ + with open('/output/preprocess.p', mode='rb') as in_file: + return pickle.load(in_file) + + +def create_lookup_tables(text): + """ + Create lookup tables for vocabulary + """ + vocab = set(text.split()) + vocab_to_int = copy.copy(CODES) + + for v_i, v in enumerate(vocab, len(CODES)): + vocab_to_int[v] = v_i + + int_to_vocab = {v_i: v for v, v_i in vocab_to_int.items()} + + return vocab_to_int, int_to_vocab + + +def save_params(params): + """ + Save parameters to file + """ + with open('/output/params.p', 'wb') as out_file: + pickle.dump(params, out_file) + + +def load_params(): + """ + Load parameters from file + """ + with open('/output/params.p', mode='rb') as in_file: + return pickle.load(in_file) + + +def batch_data(source, target, batch_size): + """ + Batch source and target together + """ + for batch_i in range(0, len(source)//batch_size): + start_i = batch_i * batch_size + source_batch = source[start_i:start_i + batch_size] + target_batch = target[start_i:start_i + batch_size] + yield np.array(pad_sentence_batch(source_batch)), np.array(pad_sentence_batch(target_batch)) + + +def pad_sentence_batch(sentence_batch): + """ + Pad sentence with id + """ + max_sentence = max([len(sentence) for sentence in sentence_batch]) + return [sentence + [CODES['']] * (max_sentence - len(sentence)) + for sentence in sentence_batch] diff --git a/params.p b/params.p new file mode 100644 index 0000000..b4ad426 Binary files /dev/null and b/params.p differ diff --git a/problem_unittests.py b/problem_unittests.py new file mode 100644 index 0000000..6dcde85 --- /dev/null +++ b/problem_unittests.py @@ -0,0 +1,413 @@ +import numpy as np +import tensorflow as tf +from tensorflow.python.layers.core import Dense +import itertools +import collections +import helper + + +def _print_success_message(): + print('Tests Passed') + + +def test_text_to_ids(text_to_ids): + test_source_text = 'new jersey is sometimes quiet during autumn , and it is snowy in april .\nthe united states is usually chilly during july , and it is usually freezing in november .\ncalifornia is usually quiet during march , and it is usually hot in june .\nthe united states is sometimes mild during june , and it is cold in september .' + test_target_text = 'new jersey est parfois calme pendant l\' automne , et il est neigeux en avril .\nles états-unis est généralement froid en juillet , et il gèle habituellement en novembre .\ncalifornia est généralement calme en mars , et il est généralement chaud en juin .\nles états-unis est parfois légère en juin , et il fait froid en septembre .' + + test_source_text = test_source_text.lower() + test_target_text = test_target_text.lower() + + source_vocab_to_int, source_int_to_vocab = helper.create_lookup_tables(test_source_text) + target_vocab_to_int, target_int_to_vocab = helper.create_lookup_tables(test_target_text) + + test_source_id_seq, test_target_id_seq = text_to_ids(test_source_text, test_target_text, source_vocab_to_int, target_vocab_to_int) + + assert len(test_source_id_seq) == len(test_source_text.split('\n')),\ + 'source_id_text has wrong length, it should be {}.'.format(len(test_source_text.split('\n'))) + assert len(test_target_id_seq) == len(test_target_text.split('\n')), \ + 'target_id_text has wrong length, it should be {}.'.format(len(test_target_text.split('\n'))) + + target_not_iter = [type(x) for x in test_source_id_seq if not isinstance(x, collections.Iterable)] + assert not target_not_iter,\ + 'Element in source_id_text is not iteratable. Found type {}'.format(target_not_iter[0]) + target_not_iter = [type(x) for x in test_target_id_seq if not isinstance(x, collections.Iterable)] + assert not target_not_iter, \ + 'Element in target_id_text is not iteratable. Found type {}'.format(target_not_iter[0]) + + source_changed_length = [(words, word_ids) + for words, word_ids in zip(test_source_text.split('\n'), test_source_id_seq) + if len(words.split()) != len(word_ids)] + assert not source_changed_length,\ + 'Source text changed in size from {} word(s) to {} id(s): {}'.format( + len(source_changed_length[0][0].split()), len(source_changed_length[0][1]), source_changed_length[0][1]) + + target_missing_end = [word_ids for word_ids in test_target_id_seq if word_ids[-1] != target_vocab_to_int['']] + assert not target_missing_end,\ + 'Missing id at the end of {}'.format(target_missing_end[0]) + + target_bad_size = [(words.split(), word_ids) + for words, word_ids in zip(test_target_text.split('\n'), test_target_id_seq) + if len(word_ids) != len(words.split()) + 1] + assert not target_bad_size,\ + 'Target text incorrect size. {} should be length {}'.format( + target_bad_size[0][1], len(target_bad_size[0][0]) + 1) + + source_bad_id = [(word, word_id) + for word, word_id in zip( + [word for sentence in test_source_text.split('\n') for word in sentence.split()], + itertools.chain.from_iterable(test_source_id_seq)) + if source_vocab_to_int[word] != word_id] + assert not source_bad_id,\ + 'Source word incorrectly converted from {} to id {}.'.format(source_bad_id[0][0], source_bad_id[0][1]) + + target_bad_id = [(word, word_id) + for word, word_id in zip( + [word for sentence in test_target_text.split('\n') for word in sentence.split()], + [word_id for word_ids in test_target_id_seq for word_id in word_ids[:-1]]) + if target_vocab_to_int[word] != word_id] + assert not target_bad_id,\ + 'Target word incorrectly converted from {} to id {}.'.format(target_bad_id[0][0], target_bad_id[0][1]) + + _print_success_message() + + +def test_model_inputs(model_inputs): + with tf.Graph().as_default(): + input_data, targets, lr, keep_prob, target_sequence_length, max_target_sequence_length, source_sequence_length = model_inputs() + + # Check type + assert input_data.op.type == 'Placeholder',\ + 'Input is not a Placeholder.' + assert targets.op.type == 'Placeholder',\ + 'Targets is not a Placeholder.' + assert lr.op.type == 'Placeholder',\ + 'Learning Rate is not a Placeholder.' + assert keep_prob.op.type == 'Placeholder', \ + 'Keep Probability is not a Placeholder.' + assert target_sequence_length.op.type == 'Placeholder', \ + 'Target Sequence Length is not a Placeholder.' + assert max_target_sequence_length.op.type == 'Max', \ + 'Max Target Sequence Length is not a Placeholder.' + assert source_sequence_length.op.type == 'Placeholder', \ + 'Source Sequence Length is not a Placeholder.' + + # Check name + assert input_data.name == 'input:0',\ + 'Input has bad name. Found name {}'.format(input_data.name) + assert target_sequence_length.name == 'target_sequence_length:0',\ + 'Target Sequence Length has bad name. Found name {}'.format(target_sequence_length.name) + assert source_sequence_length.name == 'source_sequence_length:0',\ + 'Source Sequence Length has bad name. Found name {}'.format(source_sequence_length.name) + assert keep_prob.name == 'keep_prob:0', \ + 'Keep Probability has bad name. Found name {}'.format(keep_prob.name) + + assert tf.assert_rank(input_data, 2, message='Input data has wrong rank') + assert tf.assert_rank(targets, 2, message='Targets has wrong rank') + assert tf.assert_rank(lr, 0, message='Learning Rate has wrong rank') + assert tf.assert_rank(keep_prob, 0, message='Keep Probability has wrong rank') + assert tf.assert_rank(target_sequence_length, 1, message='Target Sequence Length has wrong rank') + assert tf.assert_rank(max_target_sequence_length, 0, message='Max Target Sequence Length has wrong rank') + assert tf.assert_rank(source_sequence_length, 1, message='Source Sequence Lengthhas wrong rank') + + _print_success_message() + + +def test_encoding_layer(encoding_layer): + rnn_size = 512 + batch_size = 64 + num_layers = 3 + source_sequence_len = 22 + source_vocab_size = 20 + encoding_embedding_size = 30 + + with tf.Graph().as_default(): + rnn_inputs = tf.placeholder(tf.int32, [batch_size, + source_sequence_len]) + source_sequence_length = tf.placeholder(tf.int32, + (None,), + name='source_sequence_length') + keep_prob = tf.placeholder(tf.float32) + + enc_output, states = encoding_layer(rnn_inputs, rnn_size, num_layers, keep_prob, + source_sequence_length, source_vocab_size, + encoding_embedding_size) + + + assert len(states) == num_layers,\ + 'Found {} state(s). It should be {} states.'.format(len(states), num_layers) + + bad_types = [type(state) for state in states if not isinstance(state, tf.contrib.rnn.LSTMStateTuple)] + assert not bad_types,\ + 'Found wrong type: {}'.format(bad_types[0]) + + bad_shapes = [state_tensor.get_shape() + for state in states + for state_tensor in state + if state_tensor.get_shape().as_list() not in [[None, rnn_size], [batch_size, rnn_size]]] + assert not bad_shapes,\ + 'Found wrong shape: {}'.format(bad_shapes[0]) + + _print_success_message() + + +def test_decoding_layer(decoding_layer): + batch_size = 64 + vocab_size = 1000 + embedding_size = 200 + sequence_length = 22 + rnn_size = 512 + num_layers = 3 + target_vocab_to_int = {'': 1, '': 3} + + with tf.Graph().as_default(): + + + target_sequence_length_p = tf.placeholder(tf.int32, (None,), name='target_sequence_length') + max_target_sequence_length = tf.reduce_max(target_sequence_length_p, name='max_target_len') + + dec_input = tf.placeholder(tf.int32, [batch_size, sequence_length]) + dec_embed_input = tf.placeholder(tf.float32, [batch_size, sequence_length, embedding_size]) + dec_embeddings = tf.placeholder(tf.float32, [vocab_size, embedding_size]) + keep_prob = tf.placeholder(tf.float32) + state = tf.contrib.rnn.LSTMStateTuple( + tf.placeholder(tf.float32, [None, rnn_size]), + tf.placeholder(tf.float32, [None, rnn_size])) + encoder_state = (state, state, state) + + train_decoder_output, infer_logits_output = decoding_layer( dec_input, + encoder_state, + target_sequence_length_p, + max_target_sequence_length, + rnn_size, + num_layers, + target_vocab_to_int, + vocab_size, + batch_size, + keep_prob, + embedding_size) + + + + assert isinstance(train_decoder_output, tf.contrib.seq2seq.BasicDecoderOutput),\ + 'Found wrong type: {}'.format(type(train_decoder_output)) + assert isinstance(infer_logits_output, tf.contrib.seq2seq.BasicDecoderOutput),\ + 'Found wrong type: {}'.format(type(infer_logits_output)) + + assert train_decoder_output.rnn_output.get_shape().as_list() == [batch_size, None, vocab_size], \ + 'Wrong shape returned. Found {}'.format(train_decoder_output.rnn_output.get_shape()) + assert infer_logits_output.sample_id.get_shape().as_list() == [batch_size, None], \ + 'Wrong shape returned. Found {}'.format(infer_logits_output.sample_id.get_shape()) + + + + _print_success_message() + + +def test_seq2seq_model(seq2seq_model): + batch_size = 64 + vocab_size = 300 + embedding_size = 100 + sequence_length = 22 + rnn_size = 512 + num_layers = 3 + target_vocab_to_int = {'': 1, '': 3} + + with tf.Graph().as_default(): + + dec_input = tf.placeholder(tf.int32, [batch_size, sequence_length]) + dec_embed_input = tf.placeholder(tf.float32, [batch_size, sequence_length, embedding_size]) + dec_embeddings = tf.placeholder(tf.float32, [vocab_size, embedding_size]) + keep_prob = tf.placeholder(tf.float32) + enc_state = tf.contrib.rnn.LSTMStateTuple( + tf.placeholder(tf.float32, [None, rnn_size]), + tf.placeholder(tf.float32, [None, rnn_size])) + + input_data = tf.placeholder(tf.int32, [batch_size, sequence_length]) + target_data = tf.placeholder(tf.int32, [batch_size, sequence_length]) + keep_prob = tf.placeholder(tf.float32) + source_sequence_length = tf.placeholder(tf.int32, (None,), name='source_sequence_length') + target_sequence_length_p = tf.placeholder(tf.int32, (None,), name='target_sequence_length') + max_target_sequence_length = tf.reduce_max(target_sequence_length_p, name='max_target_len') + + train_decoder_output, infer_logits_output = seq2seq_model( input_data, + target_data, + keep_prob, + batch_size, + source_sequence_length, + target_sequence_length_p, + max_target_sequence_length, + vocab_size, + vocab_size, + embedding_size, + embedding_size, + rnn_size, + num_layers, + target_vocab_to_int) + + # input_data, target_data, keep_prob, batch_size, sequence_length, + # 200, target_vocab_size, 64, 80, rnn_size, num_layers, target_vocab_to_int) + + assert isinstance(train_decoder_output, tf.contrib.seq2seq.BasicDecoderOutput),\ + 'Found wrong type: {}'.format(type(train_decoder_output)) + assert isinstance(infer_logits_output, tf.contrib.seq2seq.BasicDecoderOutput),\ + 'Found wrong type: {}'.format(type(infer_logits_output)) + + assert train_decoder_output.rnn_output.get_shape().as_list() == [batch_size, None, vocab_size], \ + 'Wrong shape returned. Found {}'.format(train_decoder_output.rnn_output.get_shape()) + assert infer_logits_output.sample_id.get_shape().as_list() == [batch_size, None], \ + 'Wrong shape returned. Found {}'.format(infer_logits_output.sample_id.get_shape()) + + _print_success_message() + + +def test_sentence_to_seq(sentence_to_seq): + sentence = 'this is a test sentence' + vocab_to_int = {'': 0, '': 1, '': 2, 'this': 3, 'is': 6, 'a': 5, 'sentence': 4} + + output = sentence_to_seq(sentence, vocab_to_int) + + assert len(output) == 5,\ + 'Wrong length. Found a length of {}'.format(len(output)) + + assert output[3] == 2,\ + 'Missing id.' + + assert np.array_equal(output, [3, 6, 5, 2, 4]),\ + 'Incorrect ouput. Found {}'.format(output) + + _print_success_message() + + +def test_process_encoding_input(process_encoding_input): + batch_size = 2 + seq_length = 3 + target_vocab_to_int = {'': 3} + with tf.Graph().as_default(): + target_data = tf.placeholder(tf.int32, [batch_size, seq_length]) + dec_input = process_encoding_input(target_data, target_vocab_to_int, batch_size) + + assert dec_input.get_shape() == (batch_size, seq_length),\ + 'Wrong shape returned. Found {}'.format(dec_input.get_shape()) + + test_target_data = [[10, 20, 30], [40, 18, 23]] + with tf.Session() as sess: + test_dec_input = sess.run(dec_input, {target_data: test_target_data}) + + assert test_dec_input[0][0] == target_vocab_to_int[''] and\ + test_dec_input[1][0] == target_vocab_to_int[''],\ + 'Missing GO Id.' + + _print_success_message() + + +def test_decoding_layer_train(decoding_layer_train): + batch_size = 64 + vocab_size = 1000 + embedding_size = 200 + sequence_length = 22 + rnn_size = 512 + num_layers = 3 + + with tf.Graph().as_default(): + with tf.variable_scope("decoding") as decoding_scope: + # dec_cell = tf.contrib.rnn.MultiRNNCell([tf.contrib.rnn.BasicLSTMCell(rnn_size)] * num_layers) + + dec_embed_input = tf.placeholder(tf.float32, [batch_size, sequence_length, embedding_size]) + keep_prob = tf.placeholder(tf.float32) + target_sequence_length_p = tf.placeholder(tf.int32, (None,), name='target_sequence_length') + max_target_sequence_length = tf.reduce_max(target_sequence_length_p, name='max_target_len') + + for layer in range(num_layers): + with tf.variable_scope('decoder_{}'.format(layer)): + lstm = tf.contrib.rnn.LSTMCell(rnn_size, + initializer=tf.random_uniform_initializer(-0.1, 0.1, seed=2)) + dec_cell = tf.contrib.rnn.DropoutWrapper(lstm, + input_keep_prob=keep_prob) + + output_layer = Dense(vocab_size, + kernel_initializer=tf.truncated_normal_initializer(mean=0.0, stddev=0.1), + name='output_layer') + # output_fn = lambda x: tf.contrib.layers.fully_connected(x, vocab_size, None, scope=decoding_scope) + + + encoder_state = tf.contrib.rnn.LSTMStateTuple( + tf.placeholder(tf.float32, [None, rnn_size]), + tf.placeholder(tf.float32, [None, rnn_size])) + + train_decoder_output = decoding_layer_train(encoder_state, dec_cell, + dec_embed_input, + target_sequence_length_p, + max_target_sequence_length, + output_layer, + keep_prob) + + # encoder_state, dec_cell, dec_embed_input, sequence_length, + # decoding_scope, output_fn, keep_prob) + + + assert isinstance(train_decoder_output, tf.contrib.seq2seq.BasicDecoderOutput),\ + 'Found wrong type: {}'.format(type(train_decoder_output)) + + assert train_decoder_output.rnn_output.get_shape().as_list() == [batch_size, None, vocab_size], \ + 'Wrong shape returned. Found {}'.format(train_decoder_output.rnn_output.get_shape()) + + _print_success_message() + + +def test_decoding_layer_infer(decoding_layer_infer): + batch_size = 64 + vocab_size = 1000 + sequence_length = 22 + embedding_size = 200 + rnn_size = 512 + num_layers = 3 + + with tf.Graph().as_default(): + with tf.variable_scope("decoding") as decoding_scope: + + dec_embeddings = tf.Variable(tf.random_uniform([vocab_size, embedding_size])) + + dec_embed_input = tf.placeholder(tf.float32, [batch_size, sequence_length, embedding_size]) + keep_prob = tf.placeholder(tf.float32) + target_sequence_length_p = tf.placeholder(tf.int32, (None,), name='target_sequence_length') + max_target_sequence_length = tf.reduce_max(target_sequence_length_p, name='max_target_len') + + for layer in range(num_layers): + with tf.variable_scope('decoder_{}'.format(layer)): + lstm = tf.contrib.rnn.LSTMCell(rnn_size, + initializer=tf.random_uniform_initializer(-0.1, 0.1, seed=2)) + dec_cell = tf.contrib.rnn.DropoutWrapper(lstm, + input_keep_prob=keep_prob) + + output_layer = Dense(vocab_size, + kernel_initializer=tf.truncated_normal_initializer(mean=0.0, stddev=0.1), + name='output_layer') + # output_fn = lambda x: tf.contrib.layers.fully_connected(x, vocab_size, None, scope=decoding_scope) + + + encoder_state = tf.contrib.rnn.LSTMStateTuple( + tf.placeholder(tf.float32, [None, rnn_size]), + tf.placeholder(tf.float32, [None, rnn_size])) + + infer_logits_output = decoding_layer_infer( encoder_state, + dec_cell, + dec_embeddings, + 1, + 2, + max_target_sequence_length, + vocab_size, + output_layer, + batch_size, + keep_prob) + + # encoder_state, dec_cell, dec_embeddings, 10, 20, + # sequence_length, vocab_size, decoding_scope, output_fn, keep_prob) + + + assert isinstance(infer_logits_output, tf.contrib.seq2seq.BasicDecoderOutput),\ + 'Found wrong type: {}'.format(type(infer_logits_output)) + + assert infer_logits_output.sample_id.get_shape().as_list() == [batch_size, None], \ + 'Wrong shape returned. Found {}'.format(infer_logits_output.sample_id.get_shape()) + + _print_success_message() diff --git a/run_1/checkpoints/checkpoint b/run_1/checkpoints/checkpoint new file mode 100644 index 0000000..7f429a3 --- /dev/null +++ b/run_1/checkpoints/checkpoint @@ -0,0 +1,2 @@ +model_checkpoint_path: "/output/run_1/checkpoints/" +all_model_checkpoint_paths: "/output/run_1/checkpoints/" diff --git a/run_1/checkpoints/ckpt-10.data-00000-of-00001 b/run_1/checkpoints/ckpt-10.data-00000-of-00001 new file mode 100644 index 0000000..59c4e48 Binary files /dev/null and b/run_1/checkpoints/ckpt-10.data-00000-of-00001 differ diff --git a/run_1/checkpoints/ckpt-10.index b/run_1/checkpoints/ckpt-10.index new file mode 100644 index 0000000..3dcecc9 Binary files /dev/null and b/run_1/checkpoints/ckpt-10.index differ diff --git a/run_1/checkpoints/ckpt-10.meta b/run_1/checkpoints/ckpt-10.meta new file mode 100644 index 0000000..cca4598 Binary files /dev/null and b/run_1/checkpoints/ckpt-10.meta differ diff --git a/run_1/checkpoints/ckpt-15.data-00000-of-00001 b/run_1/checkpoints/ckpt-15.data-00000-of-00001 new file mode 100644 index 0000000..4565144 Binary files /dev/null and b/run_1/checkpoints/ckpt-15.data-00000-of-00001 differ diff --git a/run_1/checkpoints/ckpt-15.index b/run_1/checkpoints/ckpt-15.index new file mode 100644 index 0000000..a99aff0 Binary files /dev/null and b/run_1/checkpoints/ckpt-15.index differ diff --git a/run_1/checkpoints/ckpt-15.meta b/run_1/checkpoints/ckpt-15.meta new file mode 100644 index 0000000..83b8e2f Binary files /dev/null and b/run_1/checkpoints/ckpt-15.meta differ diff --git a/run_1/checkpoints/ckpt-20.data-00000-of-00001 b/run_1/checkpoints/ckpt-20.data-00000-of-00001 new file mode 100644 index 0000000..534bfd8 Binary files /dev/null and b/run_1/checkpoints/ckpt-20.data-00000-of-00001 differ diff --git a/run_1/checkpoints/ckpt-20.index b/run_1/checkpoints/ckpt-20.index new file mode 100644 index 0000000..a368ebc Binary files /dev/null and b/run_1/checkpoints/ckpt-20.index differ diff --git a/run_1/checkpoints/ckpt-20.meta b/run_1/checkpoints/ckpt-20.meta new file mode 100644 index 0000000..80ea4bb Binary files /dev/null and b/run_1/checkpoints/ckpt-20.meta differ diff --git a/run_1/checkpoints/ckpt-25.data-00000-of-00001 b/run_1/checkpoints/ckpt-25.data-00000-of-00001 new file mode 100644 index 0000000..414f855 Binary files /dev/null and b/run_1/checkpoints/ckpt-25.data-00000-of-00001 differ diff --git a/run_1/checkpoints/ckpt-25.index b/run_1/checkpoints/ckpt-25.index new file mode 100644 index 0000000..95f575f Binary files /dev/null and b/run_1/checkpoints/ckpt-25.index differ diff --git a/run_1/checkpoints/ckpt-25.meta b/run_1/checkpoints/ckpt-25.meta new file mode 100644 index 0000000..e22f906 Binary files /dev/null and b/run_1/checkpoints/ckpt-25.meta differ diff --git a/run_1/checkpoints/ckpt-5.data-00000-of-00001 b/run_1/checkpoints/ckpt-5.data-00000-of-00001 new file mode 100644 index 0000000..a4e3efb Binary files /dev/null and b/run_1/checkpoints/ckpt-5.data-00000-of-00001 differ diff --git a/run_1/checkpoints/ckpt-5.index b/run_1/checkpoints/ckpt-5.index new file mode 100644 index 0000000..eefc6da Binary files /dev/null and b/run_1/checkpoints/ckpt-5.index differ diff --git a/run_1/checkpoints/ckpt-5.meta b/run_1/checkpoints/ckpt-5.meta new file mode 100644 index 0000000..5ddd39f Binary files /dev/null and b/run_1/checkpoints/ckpt-5.meta differ diff --git a/run_1/checkpoints/index.html b/run_1/checkpoints/index.html new file mode 100644 index 0000000..e854e42 --- /dev/null +++ b/run_1/checkpoints/index.html @@ -0,0 +1,22 @@ + +Index of /viewer/data/Hz6f7dxuUUi5phtaMUsCvH/YQ8jPSubysLQWyikKxwT3n/run_1/checkpoints/ + +

Index of /viewer/data/Hz6f7dxuUUi5phtaMUsCvH/YQ8jPSubysLQWyikKxwT3n/run_1/checkpoints/

../
+checkpoint                                         01-Sep-2017 10:56                 109
+ckpt-10.data-00000-of-00001                        01-Sep-2017 09:41             6929504
+ckpt-10.index                                      01-Sep-2017 09:41                1966
+ckpt-10.meta                                       01-Sep-2017 09:41             1272733
+ckpt-15.data-00000-of-00001                        01-Sep-2017 10:02             6929504
+ckpt-15.index                                      01-Sep-2017 10:02                1966
+ckpt-15.meta                                       01-Sep-2017 10:03             1272733
+ckpt-20.data-00000-of-00001                        01-Sep-2017 10:24             6929504
+ckpt-20.index                                      01-Sep-2017 10:24                1966
+ckpt-20.meta                                       01-Sep-2017 10:24             1272733
+ckpt-25.data-00000-of-00001                        01-Sep-2017 10:46             6929504
+ckpt-25.index                                      01-Sep-2017 10:46                1966
+ckpt-25.meta                                       01-Sep-2017 10:46             1272733
+ckpt-5.data-00000-of-00001                         01-Sep-2017 09:19             6929504
+ckpt-5.index                                       01-Sep-2017 09:19                1966
+ckpt-5.meta                                        01-Sep-2017 09:19             1272733
+
+ diff --git a/run_1/index.html b/run_1/index.html new file mode 100644 index 0000000..9723e35 --- /dev/null +++ b/run_1/index.html @@ -0,0 +1,8 @@ + +Index of /viewer/data/Hz6f7dxuUUi5phtaMUsCvH/YQ8jPSubysLQWyikKxwT3n/run_1/ + +

Index of /viewer/data/Hz6f7dxuUUi5phtaMUsCvH/YQ8jPSubysLQWyikKxwT3n/run_1/

../
+checkpoints/                                       01-Sep-2017 10:56                   -
+logs/                                              01-Sep-2017 08:45                   -
+
+ diff --git a/run_1/logs/index.html b/run_1/logs/index.html new file mode 100644 index 0000000..6770a86 --- /dev/null +++ b/run_1/logs/index.html @@ -0,0 +1,8 @@ + +Index of /viewer/data/Hz6f7dxuUUi5phtaMUsCvH/YQ8jPSubysLQWyikKxwT3n/run_1/logs/ + +

Index of /viewer/data/Hz6f7dxuUUi5phtaMUsCvH/YQ8jPSubysLQWyikKxwT3n/run_1/logs/

../
+test/                                              01-Sep-2017 08:45                   -
+train/                                             01-Sep-2017 08:45                   -
+
+ diff --git a/run_1/logs/test/events.out.tfevents.1504255553.task-instance-container b/run_1/logs/test/events.out.tfevents.1504255553.task-instance-container new file mode 100644 index 0000000..385db4f Binary files /dev/null and b/run_1/logs/test/events.out.tfevents.1504255553.task-instance-container differ diff --git a/run_1/logs/test/index.html b/run_1/logs/test/index.html new file mode 100644 index 0000000..9d4b9e1 --- /dev/null +++ b/run_1/logs/test/index.html @@ -0,0 +1,7 @@ + +Index of /viewer/data/Hz6f7dxuUUi5phtaMUsCvH/YQ8jPSubysLQWyikKxwT3n/run_1/logs/test/ + +

Index of /viewer/data/Hz6f7dxuUUi5phtaMUsCvH/YQ8jPSubysLQWyikKxwT3n/run_1/logs/test/

../
+events.out.tfevents.1504255553.task-instance-co..> 01-Sep-2017 10:56              152431
+
+ diff --git a/run_1/logs/train/events.out.tfevents.1504255543.task-instance-container b/run_1/logs/train/events.out.tfevents.1504255543.task-instance-container new file mode 100644 index 0000000..1a49276 Binary files /dev/null and b/run_1/logs/train/events.out.tfevents.1504255543.task-instance-container differ diff --git a/run_1/logs/train/index.html b/run_1/logs/train/index.html new file mode 100644 index 0000000..3369fc5 --- /dev/null +++ b/run_1/logs/train/index.html @@ -0,0 +1,7 @@ + +Index of /viewer/data/Hz6f7dxuUUi5phtaMUsCvH/YQ8jPSubysLQWyikKxwT3n/run_1/logs/train/ + +

Index of /viewer/data/Hz6f7dxuUUi5phtaMUsCvH/YQ8jPSubysLQWyikKxwT3n/run_1/logs/train/

../
+events.out.tfevents.1504255543.task-instance-co..> 01-Sep-2017 10:56             4266556
+
+