Current and Future of Meta-Learning

The State of the Art of the young domain of Meta-Learning [3] is held by the connectionist approach. Several techniques of such an inspiration have recently shown promising results in automatically designing neural network architectures [1]. However, apart from back-propagation, only a few applications of other learning techniques are used for these purposes. The back-propagation process takes advantage of specific optimization techniques that are best suited to some fields of applications (e.g., Computer Vision and Natural Language Processing). Hence the need for a more general learning approach, namely, a basic algorithm able to make inference in different contexts with different properties. In our research work, we deal with the problem from a scientific and epistemological point of view. We believe that this is needed to fully understand the mechanisms and dynamics underlying human learning [2]. To this aim, we define some elementary inference operations and show how modern architectures can be built by a combination of those elementary methods. We analyze each method in different settings and find the best-suited application context for each learning algorithm. Furthermore, we discuss experimental findings and compare them with human learning. The discrepancy is particularly evident between unsupervised and reinforcement learning. Then, we determine which elementary learning rules are best suited for those systems and, finally, we propose some improvements in reinforcement learning architectures.