Restaurant and Café Menu Image Classification Based on Deep Neural Networks.

This paper proposes a framework based on Neural Networks for classifying restaurant and Café menus and extracting information on them using Optical Character Recognition and performing text-to-speech for the recognized menu text. The proposed framework can be used in applications related to helping blind people and people with vision difficulties figure out the public place (restaurant or Café for example) from the menu and know the shown items in the displayed menu. The framework uses transfer learning on pre-trained Deep Neural Networks which are Googlenet, Densenet201, Resnet18, Resnet50, Resnet101, Xception, Inceptionresnetv2, Shufflenet, Nasnetmobile, Darknet19, and Vgg19. The dataset comprises 346 restaurant and Café menu images from a website that contained numerous restaurant and Café menu images. The comparisons and tests showed that the average classification time is 0.039255262 seconds.The best values of Accuracy, Error, Recall, F1_score, Matthew’s Correlation Coefficient, and Kappa are 0.9422, 0.0578,1,0.9454,0.8904, and 0.8844 respectively, they are obtained with the Resnet101 network. The best values of Specificity, Precision, and False Positive Rate are 0.8844, 0.8964, and 0.1156 respectively, they are obtained with the Shufflenet network. The best case in the proposed framework is Resnet 101.