Lightweight Conditional Model Extrapolation for Streaming Data under Class Prior Shift (LIMES)

Опубликовано: 03 Январь 2025
на канале: ML in PL

Lightweight Conditional Model Extrapolation for Streaming Data under Class Prior Shift (LIMES) by Paulina Tomaszewska (Warsaw University of Technology)

Many Machine Learning models don’t work well under large class imbalance. The problem gets even more difficult when the class distribution changes over time. It may happen in non-stationary data streams. We address this issue in the proposed method called LIMES which stands for Lightweight Model Extrapolation for Streaming data under Class-Prior Shift. The model works in a continuous manner. It is lightweight as it adds no trainable parameters and almost no memory or computational overhead compared to training a single model. The solution is inspired by the MAML network where one model that can be easily adjusted to different scenarios. In the case of MAML, the adaptation is done using few gradient updates, in the case of LIMES, we apply analytical formula derived from the Bayesian rule. The core of the method is bias correction term that allows to shift from the reference class distribution to the target one. At this point, the extrapolation step is also needed in the case when the exact target class distribution is not a priori known. We evaluated our method on Twitter data where the goal was to find out the country where the tweet was issued. The results that we got show that LIMES outperforms baselines especially in the most difficult scenarios.

The talk was delivered during ML in PL Conference 2022 as a part of Contributed Talks. The conference was organized by a non-profit NGO called ML in PL Association.

ML in PL Association website: https://mlinpl.org/
ML in PL Conference 2022 website: https://conference2022.mlinpl.org/
ML In PL Conference 2023 website: https://conference2023.mlinpl.org/

---

ML in PL Association was founded based on the experiences in organizing of the ML in PL Conference (formerly PL in ML), the ML in PL Association is a non-profit organization devoted to fostering the machine learning community in Poland and Europe and promoting a deep understanding of ML methods. Even though ML in PL is based in Poland, it seeks to provide opportunities for international cooperation.