Experimental evaluation of memory capacity of recurrent neural networks

Abstract

In this article, the importance of correct representation of input data for recurrent neural network is experimentally analysed on the basis of the task for recognizing handwritten digits and task for incrementing an integer. In order to solve this task, the same information in a different form is provided for the neural network and quality of classification is evaluated. It was received, that a simple permutation of inputs has caused the decrease of quality from several percentage points (for short sequences, e.g. incrementing 32-bit integer in binary) up to 15% for long ones (784 steps). In addition, the phenomena that models examining the depiction of handwritten digits, presented in a horizontal way converge on average faster than analogue models with vertical digit representation.