FPGA Acceleration on Multilayer Perceptron (MLP) Neural Network for Handwritten Digit Recognition

This dissertation presents a hardware implementation of a Multi-Layer Perceptron (MLP) network used for the purpose of low-latency, high-accuracy digit recognition. The accuracy of various network designs was compared in Python, and the final network design was comprised of 784 input neurons, a single hidden-layer of 12 neurons, and an output layer of 10 neurons. The weights and biases of this network were then trained using the Modified National Institute of Standards and Technology (MNIST) handwritten digit data in Python using the stochastic gradient descent method. This network design was then tested in software for the digit recognition accuracy of half (16-bit), single (32-bit), and double (64-bit) precision inputs. These all gave nearly the same results of (93.26, 93.25, and 93.25%) digit recognition accuracy respectively. This design was then implemented in hardware using the Verilog Hardware Description Language (HDL). This novel design uses a custom timing structure along with single-precision, floating-point IPs from Vivado for multiplication, addition, subtraction, accumulation, exponential, and reciprocal. Results show a speedup of 40.3967 over the fastest software execution, and 127.219 over the slowest software execution. The results of the synthesis were found for the Kintex-Ultrascale FPGA, part xcku035-sfva784-1LV-I. These results showed a utilization of 44,668 Look Up Tables (LUT), 14,274 Flip Flops (FF), and 604 Digital Signal Processors (DSP), for utilization of 21.99%, 3.51%, and 35.53% respectively. Compared with related works, our proposed work provides the lowest latency for digit recognition, with a speedup of 61 and 42 over these works. Further compared to these related works, our design is between the two in accuracy and resource utilization, showing a tradeoff between design complexity and digit recognition accuracy. Conclusions of our research are that our proposed design presents a high-accuracy, low-latency digit recognition network. Our proposed design allows for further customization to fit with a future user’s needs.