Compacting Singleshot Multi-Plane Image via Scale Adjustment

Max Bergfelt; Viktor Larsson; Hideo Saito; Shohei Mori

doi:10.1109/ISMAR-Adjunct60411.2023.00117

Compacting Singleshot Multi-Plane Image via Scale Adjustment

Max Bergfelt, Viktor Larsson, Hideo Saito, Shohei Mori

Research output: Chapter in Book/Report/Conference proceeding › Paper in conference proceeding › peer-review

Abstract

A recent singleshot multiplane image (MPI) generation enables to copy an observed reality within a camera frame into other reality domains via view synthesis. While the scene scale is unknown due to the nature of singleshot MPI processing, camera tracking algorithms can estimate depth within the application world coordinate system. Given such depth information, we propose to adjust the scale of singleshot MPI to that of the currently observed scene. We find the individual scales of the MPI layers by minimizing the differences between the depth of MPI rendering and that of camera tracking. We eventually found that many layers fall within a close depth. Therefore, we merge such layers into one to compact the MPI representation. We compared our method with baselines using real and synthetic datasets with dense and sparse depth inputs. Our results demonstrate that our algorithm achieves higher scores in image metrics and reduces MPI data amount by up to 78%.

Original language	English
Title of host publication	Proceedings - 2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2023
Editors	Gerd Bruder, Anne-Helene Olivier, Andrew Cunningham, Evan Yifan Peng, Jens Grubert, Ian Williams
Publisher	IEEE - Institute of Electrical and Electronics Engineers Inc.
Pages	549-554
Number of pages	6
ISBN (Electronic)	9798350328912
DOIs	https://doi.org/10.1109/ISMAR-Adjunct60411.2023.00117
Publication status	Published - 2023
Event	2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2023 - Sydney, Australia Duration: 2023 Oct 16 → 2023 Oct 20

Conference

Conference	2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2023
Country/Territory	Australia
City	Sydney
Period	2023/10/16 → 2023/10/20

Subject classification (UKÄ)

Computer Vision and Robotics (Autonomous Systems)

Free keywords

Computer graphics
Computing methodologies
Human computer interaction (HCI)
Human-centered computing
Image manipulation
Image-based rendering
Interaction paradigms
Mixed / augmented reality

Access to Document

10.1109/ISMAR-Adjunct60411.2023.00117

Cite this

Bergfelt, M., Larsson, V., Saito, H., & Mori, S. (2023). Compacting Singleshot Multi-Plane Image via Scale Adjustment. In G. Bruder, A.-H. Olivier, A. Cunningham, E. Y. Peng, J. Grubert, & I. Williams (Eds.), Proceedings - 2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2023 (pp. 549-554). IEEE - Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISMAR-Adjunct60411.2023.00117

Bergfelt, Max ; Larsson, Viktor ; Saito, Hideo et al. / Compacting Singleshot Multi-Plane Image via Scale Adjustment. Proceedings - 2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2023. editor / Gerd Bruder ; Anne-Helene Olivier ; Andrew Cunningham ; Evan Yifan Peng ; Jens Grubert ; Ian Williams. IEEE - Institute of Electrical and Electronics Engineers Inc., 2023. pp. 549-554

@inproceedings{0add231016d740e58ca339918594b5fe,

title = "Compacting Singleshot Multi-Plane Image via Scale Adjustment",

abstract = "A recent singleshot multiplane image (MPI) generation enables to copy an observed reality within a camera frame into other reality domains via view synthesis. While the scene scale is unknown due to the nature of singleshot MPI processing, camera tracking algorithms can estimate depth within the application world coordinate system. Given such depth information, we propose to adjust the scale of singleshot MPI to that of the currently observed scene. We find the individual scales of the MPI layers by minimizing the differences between the depth of MPI rendering and that of camera tracking. We eventually found that many layers fall within a close depth. Therefore, we merge such layers into one to compact the MPI representation. We compared our method with baselines using real and synthetic datasets with dense and sparse depth inputs. Our results demonstrate that our algorithm achieves higher scores in image metrics and reduces MPI data amount by up to 78%.",

keywords = "Computer graphics, Computing methodologies, Human computer interaction (HCI), Human-centered computing, Image manipulation, Image-based rendering, Interaction paradigms, Mixed / augmented reality",

author = "Max Bergfelt and Viktor Larsson and Hideo Saito and Shohei Mori",

year = "2023",

doi = "10.1109/ISMAR-Adjunct60411.2023.00117",

language = "English",

pages = "549--554",

editor = "Gerd Bruder and Anne-Helene Olivier and Andrew Cunningham and Peng, {Evan Yifan} and Jens Grubert and Ian Williams",

booktitle = "Proceedings - 2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2023",

publisher = "IEEE - Institute of Electrical and Electronics Engineers Inc.",

address = "United States",

note = "2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2023 ; Conference date: 16-10-2023 Through 20-10-2023",

}

Bergfelt, M, Larsson, V, Saito, H & Mori, S 2023, Compacting Singleshot Multi-Plane Image via Scale Adjustment. in G Bruder, A-H Olivier, A Cunningham, EY Peng, J Grubert & I Williams (eds), Proceedings - 2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2023. IEEE - Institute of Electrical and Electronics Engineers Inc., pp. 549-554, 2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2023, Sydney, Australia, 2023/10/16. https://doi.org/10.1109/ISMAR-Adjunct60411.2023.00117

Compacting Singleshot Multi-Plane Image via Scale Adjustment. / Bergfelt, Max; Larsson, Viktor; Saito, Hideo et al.
Proceedings - 2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2023. ed. / Gerd Bruder; Anne-Helene Olivier; Andrew Cunningham; Evan Yifan Peng; Jens Grubert; Ian Williams. IEEE - Institute of Electrical and Electronics Engineers Inc., 2023. p. 549-554.

Research output: Chapter in Book/Report/Conference proceeding › Paper in conference proceeding › peer-review

TY - GEN

T1 - Compacting Singleshot Multi-Plane Image via Scale Adjustment

AU - Bergfelt, Max

AU - Larsson, Viktor

AU - Saito, Hideo

AU - Mori, Shohei

PY - 2023

Y1 - 2023

N2 - A recent singleshot multiplane image (MPI) generation enables to copy an observed reality within a camera frame into other reality domains via view synthesis. While the scene scale is unknown due to the nature of singleshot MPI processing, camera tracking algorithms can estimate depth within the application world coordinate system. Given such depth information, we propose to adjust the scale of singleshot MPI to that of the currently observed scene. We find the individual scales of the MPI layers by minimizing the differences between the depth of MPI rendering and that of camera tracking. We eventually found that many layers fall within a close depth. Therefore, we merge such layers into one to compact the MPI representation. We compared our method with baselines using real and synthetic datasets with dense and sparse depth inputs. Our results demonstrate that our algorithm achieves higher scores in image metrics and reduces MPI data amount by up to 78%.

AB - A recent singleshot multiplane image (MPI) generation enables to copy an observed reality within a camera frame into other reality domains via view synthesis. While the scene scale is unknown due to the nature of singleshot MPI processing, camera tracking algorithms can estimate depth within the application world coordinate system. Given such depth information, we propose to adjust the scale of singleshot MPI to that of the currently observed scene. We find the individual scales of the MPI layers by minimizing the differences between the depth of MPI rendering and that of camera tracking. We eventually found that many layers fall within a close depth. Therefore, we merge such layers into one to compact the MPI representation. We compared our method with baselines using real and synthetic datasets with dense and sparse depth inputs. Our results demonstrate that our algorithm achieves higher scores in image metrics and reduces MPI data amount by up to 78%.

KW - Computer graphics

KW - Computing methodologies

KW - Human computer interaction (HCI)

KW - Human-centered computing

KW - Image manipulation

KW - Image-based rendering

KW - Interaction paradigms

KW - Mixed / augmented reality

U2 - 10.1109/ISMAR-Adjunct60411.2023.00117

DO - 10.1109/ISMAR-Adjunct60411.2023.00117

M3 - Paper in conference proceeding

AN - SCOPUS:85180366446

SP - 549

EP - 554

BT - Proceedings - 2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2023

A2 - Bruder, Gerd

A2 - Olivier, Anne-Helene

A2 - Cunningham, Andrew

A2 - Peng, Evan Yifan

A2 - Grubert, Jens

A2 - Williams, Ian

PB - IEEE - Institute of Electrical and Electronics Engineers Inc.

T2 - 2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2023

Y2 - 16 October 2023 through 20 October 2023

ER -

Bergfelt M, Larsson V, Saito H, Mori S. Compacting Singleshot Multi-Plane Image via Scale Adjustment. In Bruder G, Olivier AH, Cunningham A, Peng EY, Grubert J, Williams I, editors, Proceedings - 2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct 2023. IEEE - Institute of Electrical and Electronics Engineers Inc. 2023. p. 549-554 doi: 10.1109/ISMAR-Adjunct60411.2023.00117

Compacting Singleshot Multi-Plane Image via Scale Adjustment

Abstract

Conference

Subject classification (UKÄ)

Free keywords

Access to Document

Fingerprint

Cite this