Thesis Proposal: Alternative learning methods for tiny AI devices - in Collaboration with ST Microelectronics

Nowadays, AI is being deployed not only on large workstations but also on memory-constrained low-power devices, from mobile phones to tiny computers and microcontrollers, and down to the sensors themselves - like IMUs and battery fuel gauges. However, pre-trained (offline) models often suffer a large accuracy drop when transferred to these devices due to the shift in data distribution (known as concept drift) caused by hardware and/or environmental differences. At the same time, existing adaptation methods, such as continual learning and on-device learning, require large amounts of resources for the backpropagation step. While some recent approaches managed to lower the backpropagation cost, e.g. using sparsity to only update a fraction of weights, this computational load is still too high for tiny devices. Because of this, the need for alternative, lighter learning methods is rising.
Novel neural processing units are capable of efficiently running multiple inferences per second in a forward pass. These accelerators, however, are currently limited to speeding up pre-trained models and do not support backpropagation. The potential of forward-only learning approaches to fully utilize these devices for model training and adaptation is a key area of interest. This raises important questions: What are the most effective learning methods for such forward-optimized devices with limited resources and energy budgets? Are we forever reliant on backpropagation? What are the trade-offs between forward-only learning methods and classical or sparse backpropagation?

Project Overview:

The primary focus of this internship is to investigate and develop forward learning methods tailored for deployment on tiny embedded neural processing units with limited memory and energy budgets. Specifically, the candidate will explore lightweight learning techniques that can mitigate the accuracy loss experienced by pre-trained models when deployed on resource-constrained devices. Some examples of these models are the one standardized by MLCommons/Tiny [6]. By leveraging these alternative methods, the aim is to train efficient AI models directly on the target embedded systems. ST Microelectronics has recently introduced new smart sensors and devices integrating small, on-board energy-efficient AI accelerators. However, on such small devices, it becomes power hungry to accommodate the huge computational and memory load of classical backpropagation. Can these inference-only accelerators also be used for training with some novel approaches?

Thesis Work:

Literature Review and Research: Conduct an in-depth review of state-of-the-art lightweight learning methods suitable for embedded deployment. Investigate techniques such as Forward-Forward propagation [1], PEPITA [2], and Sparse backpropagation [3]

Model Development, Training, and Optimization: Implement and train lightweight deep learning models using PyTorch, to solve computer vision problems. Optimize the trained models for deployment on various embedded platforms, considering factors such as memory footprint, computational complexity, and energy efficiency - accounting for the computational overhead of the different training algorithms.

On-Device Learning Implementation: Implement the different updating strategies analyzed during the literature review, comparing the overhead in terms of memory and operations needed. Compare the models' final accuracy, latency, and resource utilization to identify the most suitable solutions for real-world deployment. Analyze the tradeoff between resource utilization and final accuracy for different levels of hardware resources.

Qualifications:

Knowledge of machine learning, deep learning, and computer vision.
Programming languages: Python and C/C++.
Experience with deep learning frameworks like TensorFlow or PyTorch.
Familiarity with embedded systems and hardware platforms is a plus

We Offer:

Curricular internship/ thesis work;
Duration: around 6 months, depending on the candidate’s needs and preparation;
Canteen (except for UniTN students);
Support for the search for accommodation at the affiliated structures (no allowance).
Internship in STMicroelectronics to further prove own abilities, with monthly salary, after a 1st preparatory phase in UniTN

For further details on the activities, please contact Elisabetta Farella efarella@fbk.eu
The internship will be co-supervised by Danilo Pau, STMicroelectronics.

For any support, further clarification and/or information, the Human Resources Department will remain available via the address: jobs@fbk.eu

References:

[1] Geoffrey Hinton, “The Forward-Forward Algorithm: Some Preliminary Investigations”, ArXiv. /abs/2212.13345, 2022

[2] Giorgia Dellaferrera, Gabriel Kreiman, “Error-driven Input Modulation: Solving the Credit Assignment Problem without a Backward Pass”, ICML 2022

[3] Torsten Hoefler, Dan Alistarh, Tal Ben-Nun, Nikoli Dryden, Alexandra Peste, “Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks”, ArXiv. /abs/2102.00554, 2021

[4] Ravi Srinivasan, Francesca Mignacco, Martino Sorbaro, Maria Refinetti, Avi Cooper, Gabriel Kreiman, Giorgia Dellaferrera, “Forward learning with top-down feedback: empirical and analytical characterization”, ICLR 2024

[5] Pau, Danilo Pietro and Aymone, Fabrizio Maria, “Suitability of Forward-Forward and PEPITA Learning to MLCommons-Tiny benchmarks”, 2023 IEEE COINS
https://github.com/fabrizioaymone/suitability-of-Forward-Forward-and-PEPITA-learning

[6] https://github.com/mlcommons/tiny

Recruitment Type	Standard
Business Unit	Centro Digital Society/E3DA
Locations	Science and Technology Hub - Trento