Tutorial AM5

Deep Learning - From Theory to Practice

Lawrence (Larry) Hall, University of South Florida

Abstract: Whenever there is a lot of image, text or audio data there is the possibility of effectively applying deep learning. Deep learning encompasses different types of multiple hidden layer (deep) neural networks. This tutorial will introduce neural networks and backpropagation for learning, which enables the development of many types of deep neural networks. A discussion of several types of deep neural networks including convolutional neural networks, deep autoencoders and recurrent neural networks will be provided. A focus on the many deep neural network parameters and good ways to set them will be included, as well as data augmentation approaches. Generative adversarial networks will be introduced from a data augmentation perspective. Transfer learning, in which existing deep neural networks are used in new domains, will be discussed. Some examples of both transfer learning and training with small medical image data sets will be given. The participant in this workshop will come away with the knowledge to begin to build and utilize deep neural networks as well as read and understand some of the literature.


Lawrence O. Hall

He is a Distinguished University Professor in the Department of Computer Science and Engineering at University of South Florida. He served as Dept. Chair from 2008-2015. In the Fall of 2015 he was the Melchor Visiting Professor in the Department of Computer Science and Engineering and Distinguished Fellow of the Notre Dame Institute for Advanced Study at the University of Notre Dame. He received his Ph.D. in Computer Science from the Florida State University in 1986 and a B.S. in Applied Mathematics from the Florida Institute of Technology in 1980. He is a fellow of the IEEE. He is a fellow of the AAAS and IAPR. He is a member of the National Academy of Inventors. He received the Norbert Wiener award in 2012 from the IEEE SMC Society. He was named the 2017 Theodore and Venette Askounes-Ashford Distinguished Scholar Award winner at USF. He received the Joseph G. Wohl award in 2017 for the IEEE SMC Society. His research interests lie in distributed machine learning, extreme data mining, bioinformatics, pattern recognition and integrating AI into image processing. The exploitation of imprecision with the use of fuzzy logic in pattern recognition, AI and learning is a research theme. He has authored or co-authored over 85 publications in journals, as well as many conference papers and book chapters. He has received over $5.2M in research funding from agencies such as the National Science Foundation, National Institutes of Health, Department of Energy (DOE), DARPA, NASA, etc.

His neural network research stems from the late 1980’s when his group worked on “A Hybrid Connectionist, Symbolic Learning System” (AAAI – 90), and self-configuring neural networks (Divide and Conquer Neural Networks, Neural Networks). His group did early work on big data with the DOE. As part of that work, anomaly detectors were developed. The groups work on dealing with imbalanced data sets has been cited over 5,000 times. Some recent work has been on leveraging deep learning for medical image understanding which include several conference publications (Combining Deep Neural Network and Traditional Image Features to Improve Survival Prediction Accuracy for Lung Cancer Patients from Diagnostic CT, IEEE International Conference on SMC, Oct. 2016., Fine-Tuning Convolutional Deep Features For MRI Based Brain Tumor Classification, SPIE Medical Imaging 2017, Stability of deep features across CT scanners and field of view using a physical phantom, Computer Aided Diagnosis SPIE 2018) and journal papers such as “Deep Feature Transfer Learning in Combination with Traditional Features Predicts Survival Among Patients with Lung Adenocarcinoma”, Tomography 2016,”Predicting Malignant Nodules by Fusing Deep Features with Classical Radiomics Features” – to appear.

He teaches graduate level courses on Data Mining and Machine Learning both of which have sections on deep neural networks. A currently funded DARPA grant involves deep learning in networks to predict information flow. In Sept. 2017 he gave a tutorial on deep learning at the Computational Intelligence Society Summer School held at the Indian Statistical Institute in Kolkata, India.

He is a past president of NAFIPS. The former vice president for membership of the SMC society. He was the President of the IEEE Systems, Man and Cybernetics society for 2006-7. He was the Editor-In-Chief of the IEEE Transactions on Systems, Man and Cybernetics, Part B, 2002-05. He served as the first Vice President for Publications of the IEEE Biometrics Council. He is currently on the IEEE Publications Services and Products Board and the finance chair. He is a member of the IEEE publishing conduct committee. His a past member of the IEEE Periodicals Committee and the IEEE periodicals review and advisory committee. His is a member of the IEEE SMC Society operations and planning committee. He was on the Editorial board of IEEE Access (2013-17), an Associate Editor for the International Journal of Intelligent Data Analysis, the International Journal of Pattern Recognition and Artificial Intelligence and International Journal of Approximate Reasoning.