The 2nd e-Prevention challenge: Psychotic and Non-Psychotic Relapse Detection using Wearable-Based Digital Phenotyping

---

Final Update: We release the dataset for both Tracks of the e-Prevention SPGC ICASSP 2024 Challenge as well as the test labels at:

https://drive.google.com/drive/u/0/folders/1TzFV0Q0jm1nmT4f3UTmgYeWcjkA9sPrN

Update 01/03/2024: We release the final leaderboard for both Tracks of the e-Prevention SPGC ICASSP 2024 Challenge

Track 1 – Winning Teams in the Non-Psychotic Relapse Detection Track

Position	Team	Score	Paper Title	Authors	Institution
1	Samsung R&D Institute Poland	0.6656	BUMBLEBEE YOUR WAY TO RECOVERY: TRANSFORMING THE APPROACH TO DETECTION OF MENTAL HEALTH RELAPSES	Kamil Górzyński, Anna Plęs, Ivan Ryzhankow, Bartłomiej Zych	Samsung
2	MagCIL	0.6464	A SELF-SUPERVISED LEARNING APPROACH FOR DETECTING NON-PSYCHOTIC RELAPSES USING WEARABLE-BASED DIGITAL PHENOTYPING	Panagiotis Kaliosis, Sofia Eleftheriou, Christos Nikou, Theodoros Giannakopoulos	NCSR ”DEMOKRITOS”
3	Jackalope	0.5844	PATIENT-SPECIFIC MODELING OF DAILY ACTIVITY PATTERNS FOR UNSUPERVISED DETECTION OF PSYCHOTIC AND NON-PSYCHOTIC RELAPSES	Alice Hein, Sven Gronauer, Klaus Diepold	Technical University of Munich

 

Track 2 – Winning Teams in the Psychotic Relapse Detection Track

Position	Team	Score	Paper Title	Authors	Institution
1	Jackalope	0.5037	PATIENT-SPECIFIC MODELING OF DAILY ACTIVITY PATTERNS FOR UNSUPERVISED DETECTION OF PSYCHOTIC AND NON-PSYCHOTIC RELAPSES	Alice Hein, Sven Gronauer, Klaus Diepold	Technical University of Munich
2	CHI-EIHW	0.4991	PERSONALISED ANOMALY DETECTORS AND PROTOTYPICAL REPRESENTATIONS FOR RELAPSE DETECTION FROM WEARABLE-BASED DIGITAL PHENOTYPING	Adria Mallol-Ragolta, Anika Spiesberger, Andreas Triantafyllopoulos, Björn Schuller	Τechnical University of Munich, University of Augsburg, Ιmperial College London
3	SCRB-LUL	0.4964	UNSUPERVISED RELAPSE DETECTION USING WEARABLE-BASED DIGITAL PHENOTYPING FOR THE 2ND E-PREVENTION CHALLENGE	Jinting Wu, Mei Tu	Samsung

 

Congratulations to all participants ! Detailed results can be found further below in this page.

Details about the Session in ICASSP 2024 will be announced soon. Stay Tuned!

 

---

Update 02/10/2023: Data and baselines are released! Check https://github.com/filby89/spgc-eprevention-icassp2024 for baseline code and register for the challenge by e-mail in order to receive a link to the data!

The 2nd e-Prevention challenge has been accepted at ICASSP 2024 !

Registration is now open!

To register for the challenge, participants are required to send an e-mail to the below contacts with the team name, the names of their team members, as well as their emails and affiliations.

Contacts

P. P. Filntisis, School of ECE NTUA & Institute of Robotics, AthenaRC, filby@central.ntua.gr

N. Efthymiou, School of ECE NTUA & Institute of Robotics, AthenaRC, nefthymiou@central.ntua.gr

Organizing Team

P. P. Filntisis¹, N. Efthymiou¹, G. Retsinas¹, A. Zlatintsi¹, C. Garoufis¹, T. Sounapoglou², P. Tsanakas¹, N. Smyrnis³, and P. Maragos¹

1 School of ECE (CVSP / IRAL Group), National Technical University of Athens, Athens, Greece
2 BLOCKACHAIN PC, Thessaloniki, Greece
3 National & Kapodistrian University of Athens, Medical School, Athens, Greece

Important Dates:

• Grand Challenge Dataset Release and Starting Date: 27 September 2023
• Grand Challenge Solution Submission Deadline: 3 January 2024 (Anywhere on Earth)
• Grand Challenge Leaderboard Released and Invitation for 2-page Papers: 5 January 2023
• Grand Challenge 2-page Papers (invitation only): Tuesday, 9 January 2024
• Grand Challenge 2-page Paper Acceptance Notification: Tuesday, 23 January 2024
• Camera-ready Grand Challenge 2-page Papers: Tuesday, 30 January 2024

Challenge Overview

The objective of the 2nd e-Prevention challenge is to stimulate innovative research on the prediction and identification of mental health relapses via the analysis and processing of the digital phenotype of patients in the psychotic spectrum. The challenge will offer participants access to long-term continuous recordings of raw biosignals captured from wearable sensors – namely accelerometers, gyroscopes and heart rate monitors embedded in a smartwatch. Supplemental data such as sleep schedules, daily step count, and demographics will also be made available.

Participants will be evaluated on their ability to use this data to extract digital phenotypes that can effectively quantify behavioral patterns and traits. This will be assessed across two distinct tasks: 1) Detection of non-psychotic relapses, and 2) Detection of psychotic relapses, both in patients within the psychotic spectrum.

The extensive data that will be used in this challenge have been sourced from the e-Prevention project [1] (https://eprevention.gr/), an innovative integrated system for medical support that facilitates effective monitoring and relapse prevention in patients with mental disorders.

Data subsets from the e-Prevention project were previously used in the 1st e-Prevention challenge during ICASSP 2023 (https://robotics.ntua.gr/eprevention-sp-challenge/). The challenge saw participation from 15 teams, some of which achieved outstanding state-of-the-art results on the person identification task and encouraging outcomes on the relapse detection task. Drawing from these results, the current challenge narrows its focus on both psychotic and non-psychotic relapses. Our goal is to inspire participants to design and refine solutions for relapse detection, thereby pushing boundaries in this important area of mental health care.

The rules for participation

Participating teams are allowed to compete in any or both tracks; however, all participants should not be included in more than one team. For the official submission, the participating teams will evaluate their proposed algorithms in the provided testing subset of each track. In more detail:

• First Track: Participants will upload to the challenge website a .csv file with two columns: The daily timestamp of the recordings and the corresponding non-psychotic relapse anomaly scores, over which the PR-AUC and ROC-AUC metrics will be calculated.
• Second Track: Participants will upload to the challenge website a .csv file with two columns: The daily timestamp of the recordings and the corresponding psychotic relapse anomaly scores, over which the PR-AUC and ROC-AUC metrics will be calculated.

Participating teams are allowed to update their submissions multiple times. Αfter the completion of the challenge, the top-scoring teams for each track will be declared the winners of their respective track. Furthermore, the top-5 performing teams will be required to provide a synopsis of their proposed methodology and results in a two-page paper and present it in person to the Special Session dedicated to this challenge at the ICASSP-2024 conference.

Permission is granted to use the data, given that you agree: 1. To include a reference to the e-Prevention Dataset in any work that makes use of the dataset. For research papers, cite our preferred publication as it will be listed on our website and our challenge overview paper (to be released later); for other media, cite our preferred publication as it will be listed on our website. 2. That you do not distribute this dataset or modified versions. 3. That you may not use the dataset or any derivative work for commercial purposes, such as, for example, licensing or selling the data or using the data with the purpose of procuring a commercial gain. 4. That all rights not expressly granted to you are reserved by the e-Prevention SP Grand Challenge 2024 organizers.

The criteria that will be used to evaluate the challenge submissions

For both tracks, the evaluation of the state of the patient as stable or relapsing will be carried out on a daily basis, either by suitable preprocessing of the input features or via post-hoc aggregation of predictions over smaller segments. Since this is an anomaly detection task, the average of the PR-AUC and ROC-AUC scores over the daily predictions will be utilized as the final evaluation metrics.

Participants can evaluate the effectiveness of their approach using the same metrics on the validation set.

Detailed Results

Track1 – Non-Psychotic Relapse Detection

Position	Team	AUROC	AUPRC	Total AVG	Institution	Country
1	Samsung R&D Institute Poland	0.7110	0.6202	0.6656	Samsung	Poland
2	MagCIL	0.6512	0.6416	0.6464	NCSR ”DEMOKRITOS”	Greece
3	Jackalope	0.5949	0.5740	0.5844	Technical University of Munich	Germany
4	SCRB-LUL	0.6102	0.5255	0.5678	Samsung	China
5	CHI-EIHW	0.5796	0.5549	0.5673	Τechnical University of Munich, University of Augsburg, Ιmperial College London	Germany & UK
6	SLTUoS	0.5595	0.5067	0.5331	University of Sheffield	UK
7	PerCeiVE	0.5626	0.5029	0.5328	University of Catania	Italy
8	ABCDZ	0.4848	0.5124	0.4986	University Politechnica of Bucharest	Romania
	Baseline	0.5606	0.4851	0.5228
	Random	0.5	0.4298	0.4649

Track2 – Psychotic Relapse Detection

Position	Team	AUROC	AUPRC	Total AVG	Institution	Country
1	Jackalope	0.5632	0.4443	0.5037	Technical University of Munich	Germany
2	CHI-EIHW	0.4930	0.5053	0.4991	Τechnical University of Munich, University of Augsburg, Ιmperial College London	Germany & UK
3	SCRB-LUL	0.5690	0.4237	0.4964	Samsung	China
4	Samsung R&D Institute Poland	0.5110	0.4775	0.4943	Samsung	Poland
5	ABCDZ	0.5480	0.4111	0.4795	University Politechnica of Bucharest	Romania
6	SLTUoS	0.5261	0.3976	0.4618	University of Sheffield	UK
7	MagCIL	0.4497	0.3876	0.4187	NCSR ”DEMOKRITOS”	Greece
8	PerCeiVE	0.4527	0.3678	0.4103	University of Catania	Italy
	Baseline	0.5477	0.4116	0.4797
	Random	0.5	0.3471	0.424

Contacts

P. P. Filntisis, School of ECE NTUA & Institute of Robotics, AthenaRC, filby@central.ntua.gr

N. Efthymiou, School of ECE NTUA & Institute of Robotics, AthenaRC, nefthymiou@central.ntua.gr

Reference

Zlatintsi, A.; Filntisis, P.P.; Garoufis, C.; Efthymiou, N.; Maragos, P.; Menychtas, A.; Maglogiannis, I.; Tsanakas, P.; Sounapoglou, T.; Kalisperakis, E.; Karantinos, T.; Lazaridi, M.; Garyfalli, V.; Mantas, A.; Mantonakis, L.; Smyrnis, N. E-Prevention: Advanced Support System for Monitoring and Relapse Prevention in Patients with Psychotic Disorders Analyzing Long-Term Multimodal Data from Wearables and Video Captures. Sensors 2022, 22, 7544. https://doi.org/10.3390/s22197544

Funding: This research has been financed by the European Regional Development Fund of the European Union and Greek national funds through the Operational Program Competitiveness, Entrepreneurship and Innovation, under the call RESEARCH–CREATE–INNOVATE (project acronym: e-Prevention, code: T1EDK-02890/MIS: 5032797).