Can machines (do) language? A cognitive semiotic exploration of Large Language Model-based systems, user practices and conceptions
Author
Summary, in English
Large Language Model-based systems, such as ChatGPT, have gained tremendous popularity in recent years. These computational systems generate natural language text in a conversation-like manner, relying on “deep learning” to produce statistically probable language patterns. Many studies have focused on the potentials of such systems, including tasks that are intended to test for natural language understanding. I argue that such efforts are misdirected: generated text may at first glance seem indistinguishable from that produced by people, but there is no understanding of language embodied in such models. The first research question provides support to this argument with the help of two cognitive semiotic frameworks: the Semiotic Hierarchy, the Motivation & Sedimentation Model and the phenomenological approach to language that is consistent with it.
The existing literature highlights the risks posed by inaccuracies and biases encoded in these systems, but there is surprisingly limited research on how users actually engage with them. This gap is the scope of the second and third research questions, which explore user practices and conceptions, respectively. To examine this, interviews were conducted and analysed inspired by Giorgi’s approach to descriptive phenomenology, ultimately seeking to identify invariant structures of experience. The analysis was organised and presented in five categories, informed by the research questions. The categories of (1) tasks and reasons, (2) interaction and (3) evaluation address the first research question on user practices, while (4) conceptions and (5) perceived language capacity relate to the second research question on user conceptions.
The empirical analysis found that (1) participants tended to utilise ChatGPT for textual, analytical, inquisitional (information inquiry) and computational tasks for the reasons of time efficiency, compensating or extending abilities or knowledge, inspiration, validation, reducing mental load or effort, and entertainment. Further, (2) they interacted with ChatGPT iteratively in a conversational manner, which also entailed politeness for several reasons, including beliefs that it may lead to improved functionality. (3) Participants considered it inappropriate to use ChatGPT for anti-social and deceitful purposes, whereas using it to assist in performing daily tasks was deemed appropriate. Maintaining authenticity was important to participants in matters they personally engaged in. Moreover, they emphasised the importance of critically evaluating the quality of the generated text, especially for significant tasks. Participants made such evaluations based on their previous knowledge, their intuition and by cross-referencing. Moreover, (4) participants conceived of ChatGPT as a multifaceted entity that can assist in solving various tasks. Although they did not attribute subjective experience to it explicitly, they tended to unintentionally personify it through use of language that implies intrinsic meaning, as well as sometimes intentionally for the sake of entertainment. Lastly, (5) their explicit understanding of ChatGPT’s language capabilities aligned with the theoretically grounded answer in the first research question, though expressed in ordinary language. They acknowledged that the software lacks the capacity for intrinsic meaning and merely “simulates” language. They also recognised its lacking capacity for creative expression (i.e., originality) and situatedness.
The existing literature highlights the risks posed by inaccuracies and biases encoded in these systems, but there is surprisingly limited research on how users actually engage with them. This gap is the scope of the second and third research questions, which explore user practices and conceptions, respectively. To examine this, interviews were conducted and analysed inspired by Giorgi’s approach to descriptive phenomenology, ultimately seeking to identify invariant structures of experience. The analysis was organised and presented in five categories, informed by the research questions. The categories of (1) tasks and reasons, (2) interaction and (3) evaluation address the first research question on user practices, while (4) conceptions and (5) perceived language capacity relate to the second research question on user conceptions.
The empirical analysis found that (1) participants tended to utilise ChatGPT for textual, analytical, inquisitional (information inquiry) and computational tasks for the reasons of time efficiency, compensating or extending abilities or knowledge, inspiration, validation, reducing mental load or effort, and entertainment. Further, (2) they interacted with ChatGPT iteratively in a conversational manner, which also entailed politeness for several reasons, including beliefs that it may lead to improved functionality. (3) Participants considered it inappropriate to use ChatGPT for anti-social and deceitful purposes, whereas using it to assist in performing daily tasks was deemed appropriate. Maintaining authenticity was important to participants in matters they personally engaged in. Moreover, they emphasised the importance of critically evaluating the quality of the generated text, especially for significant tasks. Participants made such evaluations based on their previous knowledge, their intuition and by cross-referencing. Moreover, (4) participants conceived of ChatGPT as a multifaceted entity that can assist in solving various tasks. Although they did not attribute subjective experience to it explicitly, they tended to unintentionally personify it through use of language that implies intrinsic meaning, as well as sometimes intentionally for the sake of entertainment. Lastly, (5) their explicit understanding of ChatGPT’s language capabilities aligned with the theoretically grounded answer in the first research question, though expressed in ordinary language. They acknowledged that the software lacks the capacity for intrinsic meaning and merely “simulates” language. They also recognised its lacking capacity for creative expression (i.e., originality) and situatedness.
Department/s
- Cognitive Semiotics
- Master's Programme: Language and Linguistics
Publishing year
2024
Language
English
Full text
Document type
Student publication for Master's degree (two years)
Topic
- Languages and Literatures
Keywords
- Cognitive semiotics
- ChatGPT
- Artificial Intelligence
- Semiotic Hierarchy
- Motivation & Sedimentation Model
- phenomenology
- user practices
- user conceptions
- descriptive phenomenology
Supervisor
- Jordan Zlatev (Docent)