Adaptive Learning

    Adaptive Learning

    Overview

    To build a deep learning model that can generalize well to new environments (novel data distributions), we need to ensure that two things happen. First, the model should be complex enough to capture the characteristics of every new environment. And second, the training data should be a diverse dataset representing the properties of different data distributions.

    However, in some applications, we cannot meet these two conditions because of some constraints. For instance, some memory limitations apply when working with mobile and edge devices. Moreover, labeling data for every new environment imposes data annotation costs. In such cases, we need an annotation-free approach to train a lightweight model for every novel data distribution. Here is where Adaptive Learning comes to play.

    How it works

    Adaptive learning is a teacher-student method that trains a student network for each new environment based on the labels generated by a teacher model. This model builds a student model that adapts to new environments.

    In this method, the teacher model is a deep neural network trained on a rich dataset and can generalize well to new environments. We feed the data gathered from the new domain to the teacher model to get the predicted labels. Then, we use these labels to train a student model. Finally, we can use the lightweight student model for inference on edge devices.

    Adaptive learning can increase model performance significantly. By applying adaptive learning to the Smart Social Distancing application as a use case, we have achieved a 72.20 mAP score compared to a 25.02 baseline. Since this approach uses automatically generated labels, it cuts data annotation costs. Also, adaptive learning builds a lightweight model that is deployable on edge devices.

    Supported devices

    You can now try adaptive learning on several platforms, such as:

    • X86 CPU
    • X86 GPU

    Support for a variety of edge devices is coming soon.

    Explore more

    We faced several challenges, such as catastrophic forgetting and some network instabilities when we wanted to apply adaptive learning to the Smart Social Distancing application. You can read theĀ adaptive learning article by Neuralet to learn more about how adaptive learning works and see how we solved the challenges along the way. The source code is available on our GitHub repository.

    Get In Touch

    Have a question? Send us a message and we will respond as soon as possible.