The 2nd e-Prevention challenge: Psychotic and Non-Psychotic Relapse Detection using Wearable-Based Digital Phenotyping

Update 01/03/2024: We release the final leaderboard for both Tracks of the e-Prevention SPGC ICASSP 2024 Challenge

Track 1 – Winning Teams in the Non-Psychotic Relapse Detection Track

Track 2 – Winning Teams in the Psychotic Relapse Detection Track

Congratulations to all participants ! Detailed results can be found further below in this page.

Details about the Session in ICASSP 2024 will be announced soon. Stay Tuned!

Update 02/10/2023: Data and baselines are released! Check https://github.com/filby89/spgc-eprevention-icassp2024 for baseline code and register for the challenge by e-mail in order to receive a link to the data!

The 2nd e-Prevention challenge has been accepted at ICASSP 2024 !

Registration is now open!

To register for the challenge, participants are required to send an e-mail to the below contacts with the team name, the names of their team members, as well as their emails and affiliations.

Contacts

P. P. Filntisis, School of ECE NTUA & Institute of Robotics, AthenaRC, filby@central.ntua.gr

N. Efthymiou, School of ECE NTUA & Institute of Robotics, AthenaRC, nefthymiou@central.ntua.gr

Organizing Team

P. P. Filntisis¹, N. Efthymiou¹, G. Retsinas¹, A. Zlatintsi¹, C. Garoufis¹, T. Sounapoglou², P. Tsanakas¹, N. Smyrnis³, and P. Maragos¹

1 School of ECE (CVSP / IRAL Group), National Technical University of Athens, Athens, Greece
2 BLOCKACHAIN PC, Thessaloniki, Greece
3 National & Kapodistrian University of Athens, Medical School, Athens, Greece

Important Dates:

• Grand Challenge Dataset Release and Starting Date: 27 September 2023
• Grand Challenge Solution Submission Deadline: 3 January 2024 (Anywhere on Earth)
• Grand Challenge Leaderboard Released and Invitation for 2-page Papers: 5 January 2023
• Grand Challenge 2-page Papers (invitation only): Tuesday, 9 January 2024
• Grand Challenge 2-page Paper Acceptance Notification: Tuesday, 23 January 2024
• Camera-ready Grand Challenge 2-page Papers: Tuesday, 30 January 2024

Challenge Overview

The objective of the 2nd e-Prevention challenge is to stimulate innovative research on the prediction and identification of mental health relapses via the analysis and processing of the digital phenotype of patients in the psychotic spectrum. The challenge will offer participants access to long-term continuous recordings of raw biosignals captured from wearable sensors – namely accelerometers, gyroscopes and heart rate monitors embedded in a smartwatch. Supplemental data such as sleep schedules, daily step count, and demographics will also be made available.

Participants will be evaluated on their ability to use this data to extract digital phenotypes that can effectively quantify behavioral patterns and traits. This will be assessed across two distinct tasks: 1) Detection of non-psychotic relapses, and 2) Detection of psychotic relapses, both in patients within the psychotic spectrum.

The extensive data that will be used in this challenge have been sourced from the e-Prevention project [1] (https://eprevention.gr/), an innovative integrated system for medical support that facilitates effective monitoring and relapse prevention in patients with mental disorders.

Data subsets from the e-Prevention project were previously used in the 1st e-Prevention challenge during ICASSP 2023 (https://robotics.ntua.gr/eprevention-sp-challenge/). The challenge saw participation from 15 teams, some of which achieved outstanding state-of-the-art results on the person identification task and encouraging outcomes on the relapse detection task. Drawing from these results, the current challenge narrows its focus on both psychotic and non-psychotic relapses. Our goal is to inspire participants to design and refine solutions for relapse detection, thereby pushing boundaries in this important area of mental health care.

The rules for participation

Participating teams are allowed to compete in any or both tracks; however, all participants should not be included in more than one team. For the official submission, the participating teams will evaluate their proposed algorithms in the provided testing subset of each track. In more detail:

• First Track: Participants will upload to the challenge website a .csv file with two columns: The daily timestamp of the recordings and the corresponding non-psychotic relapse anomaly scores, over which the PR-AUC and ROC-AUC metrics will be calculated.
• Second Track: Participants will upload to the challenge website a .csv file with two columns: The daily timestamp of the recordings and the corresponding psychotic relapse anomaly scores, over which the PR-AUC and ROC-AUC metrics will be calculated.

Participating teams are allowed to update their submissions multiple times. Αfter the completion of the challenge, the top-scoring teams for each track will be declared the winners of their respective track. Furthermore, the top-5 performing teams will be required to provide a synopsis of their proposed methodology and results in a two-page paper and present it in person to the Special Session dedicated to this challenge at the ICASSP-2024 conference.

Permission is granted to use the data, given that you agree: 1. To include a reference to the e-Prevention Dataset in any work that makes use of the dataset. For research papers, cite our preferred publication as it will be listed on our website and our challenge overview paper (to be released later); for other media, cite our preferred publication as it will be listed on our website. 2. That you do not distribute this dataset or modified versions. 3. That you may not use the dataset or any derivative work for commercial purposes, such as, for example, licensing or selling the data or using the data with the purpose of procuring a commercial gain. 4. That all rights not expressly granted to you are reserved by the e-Prevention SP Grand Challenge 2024 organizers.

The criteria that will be used to evaluate the challenge submissions

For both tracks, the evaluation of the state of the patient as stable or relapsing will be carried out on a daily basis, either by suitable preprocessing of the input features or via post-hoc aggregation of predictions over smaller segments. Since this is an anomaly detection task, the average of the PR-AUC and ROC-AUC scores over the daily predictions will be utilized as the final evaluation metrics.

Participants can evaluate the effectiveness of their approach using the same metrics on the validation set.

Detailed Results

Track1 – Non-Psychotic Relapse Detection

Track2 – Psychotic Relapse Detection

Contacts

P. P. Filntisis, School of ECE NTUA & Institute of Robotics, AthenaRC, filby@central.ntua.gr

N. Efthymiou, School of ECE NTUA & Institute of Robotics, AthenaRC, nefthymiou@central.ntua.gr

Reference

Zlatintsi, A.; Filntisis, P.P.; Garoufis, C.; Efthymiou, N.; Maragos, P.; Menychtas, A.; Maglogiannis, I.; Tsanakas, P.; Sounapoglou, T.; Kalisperakis, E.; Karantinos, T.; Lazaridi, M.; Garyfalli, V.; Mantas, A.; Mantonakis, L.; Smyrnis, N. E-Prevention: Advanced Support System for Monitoring and Relapse Prevention in Patients with Psychotic Disorders Analyzing Long-Term Multimodal Data from Wearables and Video Captures. Sensors 2022, 22, 7544. https://doi.org/10.3390/s22197544

Funding: This research has been financed by the European Regional Development Fund of the European Union and Greek national funds through the Operational Program Competitiveness, Entrepreneurship and Innovation, under the call RESEARCH–CREATE–INNOVATE (project acronym: e-Prevention, code: T1EDK-02890/MIS: 5032797).