Multi-Armed Bandits in Brain-Computer Interfaces

Frida Heskebeck; Carolina Bergeling; Bo Bernhardsson

doi:10.3389/fnhum.2022.931085

Multi-Armed Bandits in Brain-Computer Interfaces

Frida Heskebeck, Carolina Bergeling, Bo Bernhardsson

Blekinge Institute of Technology

Research output: Contribution to journal › Review article › peer-review

Abstract

The multi-armed bandit (MAB) problem models a decision-maker that optimizes its actions based on current and acquired new knowledge to maximize its reward. This type of online decision is prominent in many procedures of Brain-Computer Interfaces (BCIs) and MAB has previously been used to investigate, e.g., what mental commands to use to optimize BCI performance. However, MAB optimization in the context of BCI is still relatively unexplored, even though it has the potential to improve BCI performance during both calibration and real-time implementation. Therefore, this review aims to further describe the fruitful area of MABs to the BCI community. The review includes a background on MAB problems and standard solution methods, and interpretations related to BCI systems. Moreover, it includes state-of-the-art concepts of MAB in BCI and suggestions for future research.

Original language	English
Article number	931085
Journal	Frontiers in Human Neuroscience
Volume	16
DOIs	https://doi.org/10.3389/fnhum.2022.931085
Publication status	Published - 2022 Jul 5

Bibliographical note

Funding Information:
This work was partially supported by the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation. All authors are also members of the ELLIIT Strategic Research Area.

Publisher Copyright:
Copyright © 2022 Heskebeck, Bergeling and Bernhardsson.

Subject classification (UKÄ)

Control Engineering

Free keywords

Brain-Computer Interface (BCI)
calibration
multi-armed bandit (MAB)
real-time optimization
reinforcement learning

Access to Document

10.3389/fnhum.2022.931085Licence: CC BY

1 Licentiate Thesis

On Calibration Algorithms for Real-Time Brain-Computer Interfaces
Heskebeck, F., 2023 Oct 13, Lund: Department of Automatic Control, Lund Institute of Technology, Lund University.
Research output: Thesis › Licentiate Thesis

Open Access
File

1 Finished

Optimizing the Next Generation Brain Computer Interfaces using Cloud Computing
Heskebeck, F. (Researcher), Bernhardsson, B. (Supervisor), Bergeling, C. (Assistant supervisor) & Hägglund, T. (Assistant supervisor)
2019/08/05 → 2024/08/01
Project: Dissertation

Cite this

@article{eec4fad54b674c43b0c2e77628410098,

title = "Multi-Armed Bandits in Brain-Computer Interfaces",

abstract = "The multi-armed bandit (MAB) problem models a decision-maker that optimizes its actions based on current and acquired new knowledge to maximize its reward. This type of online decision is prominent in many procedures of Brain-Computer Interfaces (BCIs) and MAB has previously been used to investigate, e.g., what mental commands to use to optimize BCI performance. However, MAB optimization in the context of BCI is still relatively unexplored, even though it has the potential to improve BCI performance during both calibration and real-time implementation. Therefore, this review aims to further describe the fruitful area of MABs to the BCI community. The review includes a background on MAB problems and standard solution methods, and interpretations related to BCI systems. Moreover, it includes state-of-the-art concepts of MAB in BCI and suggestions for future research.",

keywords = "Brain-Computer Interface (BCI), calibration, multi-armed bandit (MAB), real-time optimization, reinforcement learning",

author = "Frida Heskebeck and Carolina Bergeling and Bo Bernhardsson",

note = "Funding Information: This work was partially supported by the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation. All authors are also members of the ELLIIT Strategic Research Area. Publisher Copyright: Copyright {\textcopyright} 2022 Heskebeck, Bergeling and Bernhardsson.",

year = "2022",

month = jul,

day = "5",

doi = "10.3389/fnhum.2022.931085",

language = "English",

volume = "16",

journal = "Frontiers in Human Neuroscience",

issn = "1662-5161",

publisher = "Frontiers Media S. A.",

}

TY - JOUR

T1 - Multi-Armed Bandits in Brain-Computer Interfaces

AU - Heskebeck, Frida

AU - Bergeling, Carolina

AU - Bernhardsson, Bo

N1 - Funding Information: This work was partially supported by the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation. All authors are also members of the ELLIIT Strategic Research Area. Publisher Copyright: Copyright © 2022 Heskebeck, Bergeling and Bernhardsson.

PY - 2022/7/5

Y1 - 2022/7/5

N2 - The multi-armed bandit (MAB) problem models a decision-maker that optimizes its actions based on current and acquired new knowledge to maximize its reward. This type of online decision is prominent in many procedures of Brain-Computer Interfaces (BCIs) and MAB has previously been used to investigate, e.g., what mental commands to use to optimize BCI performance. However, MAB optimization in the context of BCI is still relatively unexplored, even though it has the potential to improve BCI performance during both calibration and real-time implementation. Therefore, this review aims to further describe the fruitful area of MABs to the BCI community. The review includes a background on MAB problems and standard solution methods, and interpretations related to BCI systems. Moreover, it includes state-of-the-art concepts of MAB in BCI and suggestions for future research.

AB - The multi-armed bandit (MAB) problem models a decision-maker that optimizes its actions based on current and acquired new knowledge to maximize its reward. This type of online decision is prominent in many procedures of Brain-Computer Interfaces (BCIs) and MAB has previously been used to investigate, e.g., what mental commands to use to optimize BCI performance. However, MAB optimization in the context of BCI is still relatively unexplored, even though it has the potential to improve BCI performance during both calibration and real-time implementation. Therefore, this review aims to further describe the fruitful area of MABs to the BCI community. The review includes a background on MAB problems and standard solution methods, and interpretations related to BCI systems. Moreover, it includes state-of-the-art concepts of MAB in BCI and suggestions for future research.

KW - Brain-Computer Interface (BCI)

KW - calibration

KW - multi-armed bandit (MAB)

KW - real-time optimization

KW - reinforcement learning

U2 - 10.3389/fnhum.2022.931085

DO - 10.3389/fnhum.2022.931085

M3 - Review article

C2 - 35874164

AN - SCOPUS:85134513425

SN - 1662-5161

VL - 16

JO - Frontiers in Human Neuroscience

JF - Frontiers in Human Neuroscience

M1 - 931085

ER -

Multi-Armed Bandits in Brain-Computer Interfaces

Abstract

Bibliographical note

Subject classification (UKÄ)

Free keywords

Access to Document

Fingerprint

Research output

On Calibration Algorithms for Real-Time Brain-Computer Interfaces

Projects

Optimizing the Next Generation Brain Computer Interfaces using Cloud Computing

Cite this