A key challenge in renal diagnosis using digital pathology has been the scarcity of reliable annotated datasets that can act as a benchmark for histological investigations. This paper introduces a novel medical image dataset, titled Glomeruli Classification Database (GCDB), consisting of renal glomeruli images bifurcated into binary classes of normal and abnormal morphology. Based on this dataset, we direct our pioneering efforts to explore suitable deep neural network (DNN) techniques related to kidney tissue slide imaging so as to establish a state of the art in this relatively unexplored domain. The paper focuses on classifying normal and abnormal categories of glomeruli which are the vital blood filtration units of the kidney. The results obtained using publicly available transfer learning models are held in comparison with supervised classifiers configured with image features extracted from the last layers of pre-trained image classifiers. Contrary to popular belief, transfer learning models such as ResNet50 and InceptionV3 are empirically proved to under-perform for this particular task whereas the Logistic Regression model augmented with features from the InceptionResNetV2 show the most promising results on the GCDB dataset.