Workshop
Workshop program and overall results
Workshop
The SPEAR Challenge Workshop was held online on Thursday 30th of March. Below you can find videos of the presentations and a summary of the results.
Programme
UK time (BST)
Session 1 (3:00 pm  4:15 pm)
 3:00 pm Welcome & Introduction to the SPEAR Challenge
 3:30 pm Zhongweiyang Xu (Univeristy of Illinois) (paper)
 3:45 pm Benjamin Stahl (IEM, Gratz) (paper)

4:00 pm Iko Pieper (audifon) (paper)
 4:15 Break (15 min)
Session 2 (4:30 pm  5:45 pm)
 4:30pm Invited talk: Augmented audio at Meta and the future of human communication: Technology, challenges, and opportunities
 5:00 pm Evaluation & results
 5:30 pm Q&A
 5:45 pm Wrap up
Results
Teams
Results are presented using the annoymised algorithm labels. The teams contributing each algorithm are given in the table below.
ID  Team name  Algorithm name  Affiliation  Description 

alg_A    passthrough    
alg_B    baseline    
alg_C  audifon  aud1  audifon GmbH & Co. KG  paper 
alg_D  audifon  aud2  audifon GmbH & Co. KG  paper 
alg_E  IEM  iem  University of Music and Performing Arts Graz  paper 
alg_F  IIPHIS  iip1  Sogang University  paper 
alg_G  IIPHIS  iip2  Sogang University  paper 
alg_H  IIPHIS  iip3  Sogang University  paper 
alg_I  IIPHIS  iip4  Sogang University  paper 
alg_J  UIUC  uiuc1  Univeristy of Illinois  paper 
alg_K  UIUC  uiuc2  Univeristy of Illinois  paper 
alg_L  UIUC  uiuc3  Univeristy of Illinois  paper 
alg_M  UIUC  uiuc4  Univeristy of Illinois  paper 
alg_N  UIUC  uiuc5  Univeristy of Illinois  paper 
alg_O  ICL  icl1  Imperial College London  
alg_P  ICL  icl2  Imperial College London 
Metrics
Each segment of enhanced audio was compared to the direct path inear signals using a large number of objective metrics (see Evaluation metrics). Here we show the difference in metric between each algorithm and the unprocessed noisy mixture (alg_A  passthrough).
Listening tests
Each pair of algorithms was compared by at least 10 participants. In each experiment 20 stimuli were presented.
Using a BradleyTerry model as a hierarchical generalized model under a Bayesian framework to analyze the pair comparison data, the probability of the second algorithm in a specified ordered pair is dependent on the stimulus and the participant.
In the following figure, points represent median probability estimate of the 2nd algorithm being preferred; bars represent the 89% highest density credible interval surrounding the median estimate; asterisks indicate statistically significant probabilities (different from 0.5).
Considering all the pairwise data, the model predicts the number of times a particular algorithm would win in a fully balanced set of pairwise comparison trials. From this we obtain the rank order as shown below.