Ironies of Programming Automation: Exploring the Experience of Code Synthesis via Large Language Models

Alan McCabe; Moa Björkman; Joel Engström; Peng Kuang; Emma Söderberg; Luke Church

doi:10.1145/3660829.3660835

Ironies of Programming Automation: Exploring the Experience of Code Synthesis via Large Language Models

Alan McCabe, Moa Björkman, Joel Engström, Peng Kuang, Emma Söderberg, Luke Church

Lund University

Research output: Chapter in Book/Report/Conference proceeding › Paper in conference proceeding › peer-review

Abstract

The widespread availability of large language models (LLMs) has presented the opportunity for novice programmers to make use of them for the purpose of understanding and synthesising code. In this paper, we discuss a small pilot study intended to explore the user experience of doing so in a limited way, and the attitudes of a group of novice programmers towards this style of programming. We also draw parallels to the seminal work of Lisanne Bainbridge, and discuss the way in which her "ironies of automation" are also present when attempting to automate the activity of programming.

Original language	English
Title of host publication	Programming '24: Companion
Subtitle of host publication	Proceedings of the 8th International Conference on the Art, Science, and Engineering of Programming
Publisher	Association for Computing Machinery (ACM)
Pages	12-21
ISBN (Print)	9798400706349
DOIs	https://doi.org/10.1145/3660829.3660835
Publication status	Published - 2024
Event	8th International Conference on the Art, Science, and Engineering of Programming - Lund, Sweden Duration: 2024 Mar 11 → 2024 Mar 14

Conference

Conference	8th International Conference on the Art, Science, and Engineering of Programming
Country/Territory	Sweden
City	Lund
Period	2024/03/11 → 2024/03/14

Subject classification (UKÄ)

Software Engineering

Access to Document

10.1145/3660829.3660835Licence: CC BY

ADAPT2: Adaptive Developer Tools
Söderberg, E. (PI), Höst, M. (CoI), Church, L. (CoI), Niehorster, D. C. (CoI), Nyström, M. (CoI), Hedin, G. (Supervisor), McCabe, A. (Research student) & Kuang, P. (Research student)
Swedish Foundation for Strategic Research, SSF
2020/04/01 → 2026/03/31
Project: Research

Cite this

McCabe, A., Björkman, M., Engström, J., Kuang, P., Söderberg, E., & Church, L. (2024). Ironies of Programming Automation: Exploring the Experience of Code Synthesis via Large Language Models. In Programming '24: Companion : Proceedings of the 8th International Conference on the Art, Science, and Engineering of Programming (pp. 12-21). Association for Computing Machinery (ACM). https://doi.org/10.1145/3660829.3660835

@inproceedings{084bc1f1557946e98c616efd26887871,

title = "Ironies of Programming Automation: Exploring the Experience of Code Synthesis via Large Language Models",

abstract = "The widespread availability of large language models (LLMs) has presented the opportunity for novice programmers to make use of them for the purpose of understanding and synthesising code. In this paper, we discuss a small pilot study intended to explore the user experience of doing so in a limited way, and the attitudes of a group of novice programmers towards this style of programming. We also draw parallels to the seminal work of Lisanne Bainbridge, and discuss the way in which her {"}ironies of automation{"} are also present when attempting to automate the activity of programming.",

author = "Alan McCabe and Moa Bj{\"o}rkman and Joel Engstr{\"o}m and Peng Kuang and Emma S{\"o}derberg and Luke Church",

year = "2024",

doi = "10.1145/3660829.3660835",

language = "English",

isbn = "9798400706349",

pages = "12--21",

booktitle = "Programming '24: Companion",

publisher = "Association for Computing Machinery (ACM)",

address = "United States",

note = "8th International Conference on the Art, Science, and Engineering of Programming ; Conference date: 11-03-2024 Through 14-03-2024",

}

McCabe, A, Björkman, M, Engström, J, Kuang, P , Söderberg, E & Church, L 2024, Ironies of Programming Automation: Exploring the Experience of Code Synthesis via Large Language Models. in Programming '24: Companion : Proceedings of the 8th International Conference on the Art, Science, and Engineering of Programming. Association for Computing Machinery (ACM), pp. 12-21, 8th International Conference on the Art, Science, and Engineering of Programming, Lund, Sweden, 2024/03/11. https://doi.org/10.1145/3660829.3660835

Ironies of Programming Automation: Exploring the Experience of Code Synthesis via Large Language Models. / McCabe, Alan; Björkman, Moa; Engström, Joel et al.
Programming '24: Companion : Proceedings of the 8th International Conference on the Art, Science, and Engineering of Programming. Association for Computing Machinery (ACM), 2024. p. 12-21.

Research output: Chapter in Book/Report/Conference proceeding › Paper in conference proceeding › peer-review

TY - GEN

T1 - Ironies of Programming Automation: Exploring the Experience of Code Synthesis via Large Language Models

AU - McCabe, Alan

AU - Björkman, Moa

AU - Engström, Joel

AU - Kuang, Peng

AU - Söderberg, Emma

AU - Church, Luke

PY - 2024

Y1 - 2024

N2 - The widespread availability of large language models (LLMs) has presented the opportunity for novice programmers to make use of them for the purpose of understanding and synthesising code. In this paper, we discuss a small pilot study intended to explore the user experience of doing so in a limited way, and the attitudes of a group of novice programmers towards this style of programming. We also draw parallels to the seminal work of Lisanne Bainbridge, and discuss the way in which her "ironies of automation" are also present when attempting to automate the activity of programming.

AB - The widespread availability of large language models (LLMs) has presented the opportunity for novice programmers to make use of them for the purpose of understanding and synthesising code. In this paper, we discuss a small pilot study intended to explore the user experience of doing so in a limited way, and the attitudes of a group of novice programmers towards this style of programming. We also draw parallels to the seminal work of Lisanne Bainbridge, and discuss the way in which her "ironies of automation" are also present when attempting to automate the activity of programming.

U2 - 10.1145/3660829.3660835

DO - 10.1145/3660829.3660835

M3 - Paper in conference proceeding

SN - 9798400706349

SP - 12

EP - 21

BT - Programming '24: Companion

PB - Association for Computing Machinery (ACM)

T2 - 8th International Conference on the Art, Science, and Engineering of Programming

Y2 - 11 March 2024 through 14 March 2024

ER -

McCabe A, Björkman M, Engström J, Kuang P , Söderberg E, Church L. Ironies of Programming Automation: Exploring the Experience of Code Synthesis via Large Language Models. In Programming '24: Companion : Proceedings of the 8th International Conference on the Art, Science, and Engineering of Programming. Association for Computing Machinery (ACM). 2024. p. 12-21 doi: 10.1145/3660829.3660835

Ironies of Programming Automation: Exploring the Experience of Code Synthesis via Large Language Models

Abstract

Conference

Subject classification (UKÄ)

Access to Document

Fingerprint

Projects

ADAPT2: Adaptive Developer Tools

Cite this