NTT research scientist creates model for discontinuous learning

NTT Research, Inc., a division of NTT (TYO:9432), announced today that a paper by Physics & Informatics Laboratory Scientist (PHI), Dr. Gautam Reddy, has closed A new contribution to the understanding of learning, published in Proceedings of the National Academy of Sciences (PNAS), one of the most comprehensive and cited interdisciplinary scientific journals in the world. gender. The paper, titled «A reinforcement-based mechanism for discontinuous learning,» uses data on the spatial navigation of labyrinthine labyrinths to provide a simple explanation. for a sudden “a-ha” moment of understanding. Dr. Reddy shows that the results of reinforcement learning (RL) algorithms, which operate gradually, can appear discontinuous for some types of tasks. In particular, this is true when the artificial RL agent learns the direct path from point A to point B in certain structured environments.

A maze or maze has a complex structure similar to a knowledge graph, showing how humans explore and navigate knowledge databases with their minds. A recent paper by neuroscientists at Caltech describes how rats that move through mazes in search of water show «rapid learning, sudden insights, and efficient exploration». Dr. Reddy analyzed the data in that paper and came up with a learning rule to explain the sudden insight phenomenon. He quantifies the relationship between learning rules, derived from biologically-based learning interruptions; learning curve, which represents the speed of the reinforcement wave; and the propensity to explore or the probability that the rat will take a certain action at a given intersection. The novel contribution of the paper is that the RL algorithms can reproduce the discontinuity of the “a-ha” moment in the simulations.

PHI Laboratory Director Yoshihisa Yamamoto said: “Congratulations to Dr. Reddy for his publication in this prestigious journal and for his innovative discovery. “His key findings will advance our understanding of how we reason, solve problems, and gain insights.”

Aged RL algorithms reinforce rewarding actions step by step through accumulated experience. However, past and recent experiments show a marked disruption in the learning process. Dr. Reddy elucidated the underlying mechanism.

“We show that the non-linear dynamics of RL-based learning, together with continuous discovery, lead to discontinuous learning curves in structured environments,” said Dr. Reddy. like a tree». “We develop a quantitative theory that explains the origin and highlights the generality of the phenomenon. Theory explains existing data and makes specific predictions that can be tested.”

Going forward, Dr. Reddy plans to extend the testing framework beyond tree graphs to looped topologies. He also hopes to test how an animal learns a neural representation of its position in a maze, a prerequisite for it to appreciate different actions at a particular location.

Along with these research initiatives, Dr. Reddy is also leading the PHI Laboratory of NTT Research in a five-year joint research project with scientists at Harvard University to study neural responses of animals, with the hope of informing future artificial intelligence (AI) systems. . The mission of PHI Laboratories is to discover fundamental principles and new technologies that drive information processing beyond modern technology. Its members and partners explore the space between quantum information science, neuroscience, and optical technology. In October 2020, Yamamoto, Director of the PHI Laboratory, co-authored a paper in the journal Applied Physics (APL) titled, “Coherent Ising Machines: Quantum Optics and Perspectives on neural networks”, underscoring the interdisciplinary nature of the groundbreaking research program of PHI Laboratories.

In addition to Harvard, eight other universities have agreed to conduct joint research with the PHI Laboratory. These include California Institute of Technology (Caltech), Cornell University, Massachusetts Institute of Technology (MIT), University of Notre Dame, Stanford University, Swinburne University of Technology, Tokyo Institute of Technology, and the University of Michigan. NASA’s Ames Research Center in Silicon Valley also entered into a joint research agreement with PHI Laboratories.

Introduction to NTT . Research

NTT Research opened an office in July 2019 as a new Silicon Valley startup to conduct fundamental research and advance technology to drive positive change for humanity. Currently, three laboratories are located at NTT Research’s facilities in Sunnyvale: the Physics and Informatics Laboratory (PHI), the Cryptography and Information Security (CIS) Laboratory, and the Informatics Laboratory. Medical and Health (MEI). The organization aims to upgrade reality in three areas: 1) quantum information, neuroscience and quantum light; 2) cryptography and information security; and 3) medical and health informatics. NTT Research is part of NTT, a global technology and business solutions provider with an annual R&D budget of $3.6 billion.

NTT and the NTT logo are trademarks or registered trademarks of NIPPON PHONE AND TELEPHONE CORPORATION and/or its affiliates. All other referenced product names are trademarks of their respective owners. © 2023 NIPPON PHONE AND PHONE JOINT STOCK COMPANY

See the source version on businesswire.com: https://www.businesswire.com/news/home/20230712487718/en/