Keywords: ChatGPT, LLM, Higher Education, AI in academic research, AI ethics, learning with AI
With the increasing proliferation of Artificial Intelligence (AI), universities face new challenges and opportunities. This study by the Research and Innovation Lab for Digital Teaching (FIDL) investigates the use, benefits, and limitations of ChatGPT and other Large Language Models (LLMs) at Bavarian Universities of Applied Sciences (HAWs). Based on a written online survey and guided interviews with lecturers, students, and administrative staff, the study provides valuable insights into the current state and perspectives of using LLMs in higher education.
Summary
The study analyses the use, benefits, and limitations of ChatGPT and LLMs in teaching at Bavarian HAWs. A total of 1,570 individuals participated in the online survey, and 27 interviews were conducted. The results show that most participants have a limited understanding of the usability of LLMs and their technological functionality. Additionally, the frequency of using ChatGPT and LLMs in teaching, studying, and daily professional life is still low, primarily due to the lack of meaningful application possibilities, poor quality of results, and legal concerns. A ban on ChatGPT is predominantly rejected. Instead, lecturers and students desire more information about the opportunities and limitations of ChatGPT and similar tools, as well as financial, technical, and didactic support from universities.
Application scenarios and usefulness
The perceived usefulness of ChatGPT application scenarios varies by version and target group. Text summarisation and improvement are considered useful. Study participants view ChatGPT as an opportunity to improve teaching but also as a challenge for examination practices, legal security, and competency development.
- Text summarisation and improvement: These functions are particularly valued and can significantly ease the writing process in higher education.
- Individual learning support: ChatGPT can act as a personal, interactive learning coach, aiding students in preparing for exams and writing academic papers.
- Didactic tools: Lecturers can use ChatGPT to create teaching materials, plan lessons, and support students.
Challenges and limitations
The study highlights several challenges and limitations in using ChatGPT and LLMs:
- Loss of competencies: There is a risk of students losing competencies as certain skills may no longer be regularly practised.
- Legal uncertainties: Data protection and copyright issues, as well as examination practices, pose significant challenges.
- Quality of results: The results from ChatGPT and other LLMs are not always reliable and can be erroneous.
Recommendations and future developments
The study provides several recommendations for the future use of ChatGPT and LLMs at Bavarian HAWs:
- Targeted training programs: There is a significant need for targeted training programs for lecturers and students to enhance understanding and effective use of LLMs.
- Technical and didactic support: Universities should provide financial, technical, and didactic support to facilitate the meaningful use of LLMs.
- Adaptation of examination culture: The methods of examination need to be reconsidered and adapted to meet the challenges posed by the use of LLMs.
Conclusion
The study shows that ChatGPT and other LLMs offer great potential for higher education but also bring significant challenges. It is in the interest of all higher education stakeholders to continuously improve teaching through the reflective use of innovative technologies. This study makes an important contribution to clarifying questions about the use, benefits, and limitations of ChatGPT and other LLMs and provides valuable insights for the future design of higher education teaching.
Personal observation: The study balances theoretical insights with practical application, offering a nuanced perspective on both the potential and the challenges of integrating AI into higher education.
Citation: Gärtner, C., Moraß, A., Koss, S., Garbusa, S., Matern, S. & Innermann, I. (2024). Einsatz, Nutzen und Grenzen von ChatGPT und anderen Large Language Modellen an den bayerischen HAWs
(Die Studien- und Schriftenreihe des Forschungs- und Innovationslabors Digitale Lehre – FIDL, Bd. 5).
München/Nürnberg: FIDL – Forschungs- und Innovationslabor Digitale Lehre.
https://doi.org/10.34646/thn/ohmdok-1466
Read the full article here (available in German).