Abstract
Modern Artificial Intelligence often relies on learning from vast amounts of “labeled” data – examples where the correct answer is already known. But what was machine learning like before this era of big data? In this “last lecture” as a UW Professor, Les Atlas will explore that early period when researchers faced a critical shortage of training data.
This scarcity demanded innovation. Concepts like “shift-invariance” – the ability to recognize patterns regardless of their exact position – became crucial. In 1987, collaborating with T. Homma and R. Marks, Prof. Atlas proposed replacing simple connections in neural networks with a “convolution” operation. This replacement allowed learning components (“weights”) to be shared across different input locations, drastically reducing the amount of information the network needed to learn from limited data. Inspired by unsupervised approaches (e.g., Hubel & Weisel and Fujusaki’s Neocognitron), this convolution-based approach, applied initially to speech sounds, became foundational for what are now known as “convolutional neural networks” (CNNs). We validated this approach using synthesized speech to create enough training examples, publishing at the first Neural Information Processing Systems (NIPS) conference in 1987. Notably, Prof. Alex Waibel and colleagues independently developed a mathematically identical concept, the “time-delay neural network,” around the same time.
Even with these advances, real-world data remained scarce. Inspired by research from UW’s Profs. Kuhl and Meltzoff on how infants efficiently learn speech, we developed strategies for systems to intelligently select the most informative data points to learn from. These concepts, known as “selective sampling” (published 1989) and “active learning” (published 1994) with UW CSE colleagues Dr. David Cohn and Prof. Richard Ladner, offered other ways to cope with limited data.
Prof. Atlas will reflect on these developments and share key lessons learned about research and achieving technical impact.
Bio
Les Atlas was born and raised in Milwaukee, Wisconsin. He received his Bachelor’s degree from the University of Wisconsin, and then went to Stanford for his masters and Ph.D., in Electrical Engineering. Since then he has been on the faculty at the University of Washington Department of Electrical and Computer Engineering, where he is also an adjunct professor of Computer Science and Engineering. He is a Life Fellow of the IEEE. He becomes an Emeritus Professor summer of 2025 and will continue to enjoy his research collaborations. The research discussed here was initially funded by a National Science Foundation Presidential Young Investigator Award.
