2021 Bower Science Award Theme: Neural Networks for Machine Learning
Our brains recognize patterns. When we see an animal, we can identify the image in a flash, instantly matching size, shape, and past experiences with similar creatures through a complex series of neuronal connections to arrive at a classification. Thus, we know a gorilla from a greyhound. In 1979, Kunihiko Fukushima designed an artificial multi-layered neural network with learning capabilities that could mimic the brain’s visual network, an insight that has become the basis of modern artificial intelligence technology. Fukushima’s work bridging electrical engineering and neuroscience has led to an array of practical applications from self-driving cars to facial recognition, from cancer detection to flood prediction, with many more to come.
Kunihiko Fukushima, the man who first developed the “Neocognitron,” a machine-learning and pattern-recognition technology that sparked a revolution in what we now call artificial intelligence, came of age in war-blasted Japan where things were in short supply. Curiosity, however, was plentiful. Fukushima became fascinated by the analogy of electrical circuitry and the human brain. He wondered if the way that electrical components—switches, transistors, and capacitors—were wired together could mimic the way that our neuroanatomy is set up to process patterns. His formal training as an electrical engineer gave Fukushima the scope to pursue his observation and invent the “Neocognitron,” an artificial neural network, similar to those in the visual cortex of the brain. Fukushima’s pioneering approach, successfully combining aspects of neuroscience and electrical engineering, laid the foundation for today’s artificial intelligence technology.
Fukushima was born in 1936 in Taiwan—a Japanese territory at that time—and lived there with his family until the end of World War II. With the end of the war came the end of Japanese rule over Taiwan, and Japanese people living there were forced to return to Japan, leaving all of their belongings behind. Upon their return, the family found a devastated country and had little money to spare for toys. An uncle gave Fukushima a surplus transformer and a disassembled electric motor to tinker with. Wires and circuits became Fukushima’s passion. From scrap, he built electric trains and a radio. Unsurprisingly, he would go on to pursue electronics for his 1958 bachelor’s degree and earn a Ph.D. in electrical engineering in 1966, both at Kyoto University.
After graduating from Kyoto University, Fukushima joined the research division of NHK, the Japan Broadcasting Corporation, where his research on efficient coding of television signals led to his Ph.D. thesis. In 1965, NHK established a basic research facility, named the Broadcasting Science Research Laboratories, where Fukushima joined a research group on visual and auditory information processing. The group included not only engineers, but also neurophysiologists and psychologists. Its aim was to investigate the biological brain—the final destination of signals broadcast through TV and radio.
If you see something moving, your brain uses a series of hierarchical categorizations to ultimately decide what it is. First, your brain might process the object’s size, then your brain narrows the identification through layers of defining features like color and shape while bringing together other associated memories and past experiences. For example, we can distinguish a cat from a dog instantly because of this layered pattern processing and our personal life experiences. As toddlers, we may incorrectly call a fox a dog or a hamster a mouse, but our brains are quick to reclassify new patterns of size or color, noting key differences as well as incorporating input from others. Fukushima’s plan was to build and “train” an electronic version of that system.
At NHK, Fukushima worked closely with neurophysiologists and psychologists to assemble artificial neural networks. His first artificial network, “Cognitron,” demonstrated an ability to recognize patterns, but it could do little with patterns that were shifted, rotated, or partially obscured. Based on what was known about neurophysiology, Fukushima decided that he would need a larger, self-organizing network with more layers to achieve advanced pattern-recognition capabilities.
In 1979, Fukushima published a description of this artificial neural network, which he named “Neocognitron.” It was inspired by two kinds of neural cells known to exist in the primary visual cortex, a “simple” or “S” cell and a “complex” or “C” cell, arranged in a cascading order for pattern recognition tasks. Fukushima’s Neocognitron would achieve flexible pattern recognition and machine learning through multiple layers of artificial S and C cells and by incorporating learning rules to modify interactions amongst the cells and these internally hidden layers. To demonstrate capabilities of this first deep learning neural network machine, Fukushima successfully trained the Neocognitron to read handwritten numbers from zero to nine, accommodating variation in the writing.
The Neocognitron’s structure is at the heart of artificial intelligence’s deep neural networks, especially those that employ visual pattern recognition. This pattern recognition function is fundamental to how self-driving cars detect hazards, how security cameras recognize faces, and how medical software pinpoints abnormalities in biopsy samples. Pattern recognition also enables flood predictions based on satellite images. It unlocks your cell phone. It is behind new technologies that help those with severe visual impairments to navigate the world.
After moving to Osaka University in 1989, Fukushima expanded his research on neural networks and machine learning to make neural network models not only for visual pattern recognition, but also for many other functions of the biological brain. In 1999, he moved to the University of Electro-Communications in Tokyo and then, in 2001, to the Tokyo University of Technology. In 2006, he took his present position as a senior research scientist at Fuzzy Logic Systems Institute in Fukuoka. Today Fukushima works mostly from his home in Tokyo, developing new training methods and architectures for neural networks, including the Neocognitron, to recognize deformed and shifted patterns more robustly with smaller computational requirements. While most people find themselves well into retirement at the age of 83, Fukushima published his latest paper in November 2019.
Fukushima’s work has been highly honored. His accolades include Japan’s Science and Technology Agency Award (1985), the Institute of Electrical and Electronics Engineers Neural Networks Pioneer Award (2003), the Asia-Pacific Neural Network Assembly Outstanding Achievement Award (2005), the International Neural Network Society Helmholtz Award (2012), the Institute of Electronics, Information and Communication Engineers Distinguished Achievement and Contributions Award (2017), the Japanese Neural Network Society Academic Award (2017), and the Kenjiro Takayanagi Award (2020), among others. He was the founding president of the Japanese Neural Network Society and a founding member of the board of governors of the International Neural Network Society. Fukushima is a former president of the Asia-Pacific Neural Network Assembly.
Because of Kunihiko Fukushima’s ability to recognize patterns in the brain and integrate them into artificial neural networks, we now have incredible pattern recognition capabilities in the technology that surrounds us. The next time your phone unlocks by recognizing your face, you can thank Kunihiko Fukushima.
Watch "Neural Networks for Machine Learning"
Information as of March 2020