Efficacy of Two Hidden Layers Artificial Neural Network Synapticity for Deep Learning: A Case of Pattern Recognition
Abstract
Most research works in Artificial Neural Network (ANN) are accustomed with the use of single hidden layer (SHL) topology without giving considerations to the problem type, its complexity and desired depth of supervised or unsupervised learning. This could be partly due to the inherent complexities associated with the use of more than one hidden layer which in turn affects solution efficiency. However, the trade-off occasionally is between efficiency and effectiveness of result. When effectiveness is prioritized perhaps for sensitive or mission critical systems, then multiple hidden layers can become advantageous. This research has investigated the ability of an Artificial Neural Network (ANN) with two hidden layer topology to exhibit deep learning behaviour in comparison with a single hidden layer architecture ANN system. A two hidden layer (THL) Neural Network was developed and implemented using Microsoft Visual Studio programming suite and applied to a pattern recognition problem. The gradient descent optimization of the back propagation algorithm in a feed forward scheme was used in the development of the supervised ANN which consisted of thirty inputs at the input layer, two hidden layers with five nodes and a single output layer with one node for a Boolean response. Normalized images mapped into a pattern extraction template using principal component analysis (PCA) of the original images served as pre-processed inputs to the two hidden layer architecture with an initial learning rate of η = 0.1 and maximum tolerable rate of η = 0.4 for fast convergence. Iterations for validation of the feed forward back propagation algorithm using three image patterns showed that over 96% recognition of presented data was recorded. Graphical comparison of the results obtained from separate iterative sessions of the One Hidden Layer (OHL) and (THL) architectures under same input-output dataset revealed more visible traits of attained deep learning by the two hidden layer architecture due to enhanced synapticity of additional nodes.Downloads
References
Agarwal, M., Jain, N., Kumar, M. & Agrawal, H. (2010). Face Recognition Using Eigen Faces and Artificial Neural Network. International Journal of Computer Theory and Engineering, 2(4), 624 – 629.
Aires, F., Prigent, C., Rossow, W. (2004). Neural Network Uncertainty Assessment Using Bayesian Statistics: A Remote Sensing Application. Neural Computation 16(11), 2415-2458.
Awolusi, T.F., Oke, O.L., Akinkurolere, O.O., Sojobi, A.O., & Aluko, O.G. (2019). Performance Comparison of Neural Network Training Algorithms in the Modeling Properties of Steel Fiber Reinforced Concrete. Heliyon. 5(1): e01115. Available from: doi: 10.1016/j.heliyon.2018.e01115
Barron AR. 1993. Universal Approximation Bounds for Superpositions of a Sigmoidal Function. IEEE Transaction on Information Theory. 39(1): 930-945. Available from: doi: 10.1109/18.256500.
Basu, J.K., Bhattacharyya, D. & Kim, T. (2010). Use of Artificial Neural Network in Pattern Recognition. International Journal of Software Engineering and Its Applications, 4(2), 23 – 33.
Bishop, C.M., (1995). Neural Networks for Pattern Recognition. Clarendon Press, Oxford
Bishop, C.M., (1996). Neural Networks: A Pattern Recognition Perspective. In Friesler, E. & Beale, R. (eds) Handbook of Neural Computation, Oxford University Press.
Bovik, A.C., Clark, M.& Geisler, W. S. (1990) Multichannel Texture Analysis Using Localized Spatial Filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(1), 55 – 73.
Carlson, T.N. (1967). Synoptic histories of three African disturbances that developed into Atlantic hurricanes. Monthly weather review, 7(3), 256 – 276.
Chicca, E., Badoni, D., Dante, V., D’Andreagiovanni, M., Salina, G., Carota, L., Fusi, S., Giudice, P. (2003) A VLSI Recurrent Network of Integrate-and-Fire Neurons Connected by Plastic Synapses with Long-Term Memory. IEEE Transactions on Neural Networks, 14(5), 1297 – 1307.
Gallagher, M.R. (2000). Multi-layer Perceptron Error Surfaces: Visualization, Structure and Modelling, Ph.D. Thesis from the Department of Computer Science and Electrical Engineering, University of Queensland, Australia.
Greenspan, H., Goodman, R. & Chellappa, R. (1991). Texture Analysis via Unsupervised and Supervised Learning. Proceedings of the 1991 International Joint Conference on Neural Networks, 1, 639 – 644.
Friedl, E. (2024). Hurricanes: Types, Formation, Causes & Effects. Retrieved from https://study.com/academy/lesson/hurricanes-types-formation-causes-effects.html on 24th August, 2024.
Hagan, M.T, Demuth, H.B, Beale, M.H. & DeJesus, O. (2013). Neural Network Design, 2nd Edition.
Hashimoto, S., Yoshiki, S., Saeki, R., Mimura, Y., Ando, R. & Nanba, S. (2016). Development and Application of Traffic Accident Density Estimation Models using Kernel Density Estimation. Journal of Traffic and Transportation Engineering (English Ed.), 3(3), 262 – 270.
Hebb, D.O. (1949). The Organization of Behaviour. Wiley, New York.
Hopfield, J.J. (1984). Neurons with Graded Response have Collective Computational Properties like Those of Two-state Neurons. Proceedings of National Academy of Science. 81, 3088 – 3092.
Janglová, D. (2004). Neural Networks in Mobile Robot Motion. International Journal of Advanced Robotic Systems, 1(1), 15 – 22.
Kisi, O., & Uncuoglu, E. (2005). Comparison of Three Back-propagation Training Algorithms for Two Case Studies. Indian Journal of Engineering and Materials Science. 12, 434 – 442.Larose, D.T. (2005). Discovering Knowledge in Data: An Introduction to Data Mining (pp 60 – 214). New Jersey, U.S.A: John Wiley & Sons, Inc.
Larose, D.T. (2006). Data Mining Methods and Models (pp 204 – 239). New Jersey, U.S.A: John Wiley & Sons, Inc.
Latha, P., Ganesan, L. &Annadurai, S. (2009). Face Recognition using Neural Networks, Signal Processing: An International Journal (SPIJ), 3(5), 153 – 160.
Lawrence, S., Giles, C.L., Tsoi, A.C. & Back, A.D. (1993). Face Recognition: A Convolutional Neural Network Approach. IEEE Transactions on Neural Networks, Special Issue on Neural Networks and Pattern Recognition, 8(1), 98 – 113.
Lu, H., Setiono, R. & Liu, H. (1996). Effective Data Mining using Neural Networks. IEEE Transactions on Knowledge and Data Engineering, 6(6), 957 – 961.
Mitchell, M.T. (1997). Machine Learning. McGraw – Hill Science/Engineering/Math Publishing, USA. pp 81 – 127, 154 – 200.
Radiology Assistant (2024). Available from: https://radiologyassistant.nl/neuroradiology/brain-tumor/systematic-approach
Nazeer, S.A., Omar, N., Jumari, K.F. & Khalid, M. (2007). Face Detecting using Artificial Neural Networks Approach. First Asia International Conference on Modeling & Simulation.
Nisbet, R., Elder, J., & Miner, G. (2009). Handbook of Statistical Analysis and Data Mining Applications. Burlington, Massachusetts: Academic Press-Elsevier Inc. Burlington.
Osigbemeh, M.S., Ohaneme, C.O., Inyiama, H.C. (2017) “An Algorithm for Characterizing Pre-fuzzified Linguistic Nuance using Neural Network”. International Journal of Speech Technology. 20(2), 355 – 362. DOI.10.1007/s10772-017-9413-5.
Osigbemeh, M.S., Okezie, C.C., Inyiama, H.C. (2016) “Performance Metrics of various Topologies of a Feed Forward Error-Back Propagation Neural Network”. 2016 Unizik Faculty of Engineering International Conference held at the University auditorium in Awka.
Osigbemeh, M.S., Osuji, C.C., Onyesolu, M.O., Onochie, U.P (2024). “Comparison of the Artificial Neural Network’s Approximations Based on the Levenberg-Marquardt Algorithm and the Gradient Descent Optimization Method on Datasets.” Artificial Intelligence Evolution, 5(1), 24–38. DOI: https://doi.org/10.37256/aie.5120243781.
Rudin, C. & Wagstaff, K. L. (2013). Machine Learning, Special Issue on Machine Learning for Science and Society. Springer. DOI 10.1007/s10994-013-5425-9
Suzuki, K., Shiraishi, J., Abe, H., MacMahon, H. & Doi, K. (2005). False-positive Reduction in Computer-aided Diagnostic Scheme for Detecting Nodules in Chest Radiographs by Means of Massive Training Artificial Neural Network Academic Radiology, 12(2), 191 – 201. doi:10.1016/j.acra.2004.11.017.
Swets, J. (1988). Measuring The Accuracy of Diagnostic Systems. Science.; 240(4857): 1285-1293.
Tank, D.W. & Hopfield, J.J. (1986). Simple “Neural” Optimization Networks: An A/D Converter, Signal Decision Circuit and a Linear Programming Circuit. IEEE Transactions on Circuits and Systems, 33(5), 533 – 541.
Turk, M. & Pentland,A. (1991). Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3, 71– 86.
Witten, I.H., Frank, E., & Hall, M.A. (2011). Data Mining: Practical Machine Learning Tools and Techniques. 3rd ed. Burlington, Massachusetts: Morgan Kaufmann Publishers-Elsevier Inc.
Yegnanarayana, B. (1994). Artificial Neural Networks for Pattern Recognition. Sadhana,19(2), 189 – 238.
Zhou, Z., Chawla, N.V., Jin, Y., & Williams, G.J. (2015). Big Data Opportunities and Challenges: Discussions from Data Analytics Perspectives. IEEE Computational Intelligence Magazine.; 9(4), 62 – 73. Available from: doi: 10.1109/ MCI.2014.2350953
Published
How to Cite
Issue
Section
Copyright (c) 2025 Michael Osigbemeh, Augustine Azubogu, Michael Ayomoh, Alpheus Okahu
This work is licensed under a Creative Commons Attribution 4.0 International License.
