_A Workshop at the DeLFI 2024
Monday 9 September 2024 14:00-17:30 HS Fulda Room 307 Notes

Programme

14:00 Introduction

recording

 

14:10 Thematic discovery for occupational therapist e-portfolios

Finlay Thomson and Yvan Richard (late breaking result)

We applied a range of machine learning techniques to reveal the thematic structure of e-portfolios provided to the OT Board in NZ. Clustering was applied to vector embeddings learnt from the portfolio text. This identified broad themes, and how they have changed over the four two-year cycles of e-portfolio guided reflection. Initial findings confirm that the e-portfolios have reflected changes in guidance to the sector over these years. The embeddings also allowed the development of a general purpose search and review tool that has proven to be useful for the administration of the e-portfolios more broadly.

14:25 Szenarien für eine automatische Analyse von E-Portfolios (slides)

Andreas Isking and Paul Libbrecht

(Presentation in English) Wir verwenden Scenario Based Design, um automatische Analysen von E-Portfolios in übersichtlichen Dashboards zu visualisieren. Die Ergebnisse dieser Analysen können auf unterschiedlichste Arten dargestellt werden. In Szenarien wird versucht, die Situationen zu identifizieren, in denen es Probleme oder Verbesserungspotenzial gibt, damit Bewertungs- und Schreibprozesse verbessert werden. Wir präsentieren die angewandte Methode und ausgewählte Szenarien, die bei der Entwicklung einer Web-Applikation geholfen haben.

14:40 Transmitting ePortfolio Content for (AI) Analysis (slides)

Kristina Hoeppner, Alexander Gantikow, Robert Lyon and Paul Libbrecht

The world of artificial intelligence (AI) tools is exploding and every day a new tool shows up, but equally, many drop off and are never heard of again. This poses a risk for organisations that want to integrate AI meaningfully into their portfolio work because there could ultimately be a lot of change very rapidly. Therefore, in this workshop, we want to initiate a discussion what a technical interface may look like between a portfolio platform and systems that can analyse portfolio content such as those powered by AI. The ultimate goal would be to define a standard of a data-space for ePortfolios that could be used by different providers to easily connect their AI platform to their portfolio platform. Please follow the invitation.

14:55 Discussion

15:30 - Break -

 

16:00 Explainable feedback for learning based on rubric based multimodal assessment analytics with AI (slides)

Karsten Detlef Wolf, Fatima Maya and Lisanne Heilmann

Providing timely formative feedback to students is very important to support self-regulated learning and deep learning strategies. Feedback has been shown to increase student engagement, satisfaction and learning outcomes, especially in generative learning tasks such as ePortfolios and other forms of multimodal compositions. However, the provision of detailed formative feedback places high demands on teachers’ resources. It would be highly beneficial if Large Language Models (LLM) could be used to support the feedback process. Therefore, this paper first describes a general architecture for multimodal formative assessment analysis and feedback generation. It is based on assessment rubrics, which are then used to build task-specific AI analysis pipelines to generate explainable assessment metrics, which are then used to produce helpful feedback. An example of a feedback pipeline for student video submissions in an ePortfolio is given, along with a prompting chain for feedback generation. The paper describes further steps necessary to evaluate and optimise this process in real classroom scenarios..

16:15 Implementierung von Textanalyse in der E-Portfolio-Bewertung (slides)

Pierre-Olivier Günthner and Sandra Rebholz

(Presentation in English) Die Implementierung von Textanalyse in die Bewertung von E-Portfolios bietet eine Möglichkeit, die Bewertung formaler Kriterien von E-Portfolios zu unterstützen. Durch den Einsatz von natürlicher Sprachverarbeitung (NLP) und statistischen Methoden werden verschiedene linguistische Merkmale erfasst und analysiert, darunter die Textkomplexität, lexikalische Vielfalt und Kohärenz. Herausforderungen werden in der Interpretation der ermittelten Kennzahlen sowie in der Eignung von statistischen Methoden und Standardmodellen des NLP für spezifische Themen wie z.B. Informatik gesehen.

16:30 Supporting essay construction and writing: the GPT essay navigator

Kerstin Vogel and Kamal Bhattacharya, Supporting (late breaking result)

16:45 Discussion

17:30 - Closing -

---

Dates

• Paper submission: 10 June 17th of June 2024 (extended)
• Acceptance notifications: 10 July 2024
• Camera-ready version: 26 August 2024 (changed)
• Workshop and conference: 9 September 2024

About

For decades, digital portfolios have offered a strongly personalised way of assessing learning outcomes, be that in the artistic, technical, or even literary domains. They give the opportunity for learners to frame their knowledge close to their real world, thus reaching deep learning effects through reflection.

The growing availability of tools that can analyse texts has opened new possibilities to assess, navigate, and revisit e-portfolios which, nowadays, are mostly web-based: Students can receive feedback on where to deepen, on language to adjust, on degrees of appropriation or reflective generalisation to render in their portfolios.

Such automated feedback, which is based on readily available general chat systems, on external services, on self built text models, or on simple statistical evaluations can support educators assessing the competencies evidenced in portfolios. Such feedback can also invite students to continuously review their work and improve on it. The perspectives open brilliant avenues for students to become more confident in their writing and reflecting as they gain more practice through consistent feedback.

These new possibilities lead the way to more dynamic and more creative writing and reading processes, for both learners and educators.

Committee

The committee is made of:

• Laura Farinetti (Politecnico Milano, Italy)
• Armin Grasnick (IU Internationale Hochschule, Germany)
• Thanasis Hadzilacos (The Cyprus Institute, Cyprus)
• Kristina Höppner (Catalyst IT, New Zealand) (organizer)
• Paul Libbrecht (MeVIS, PH Weingarten & IU Internationale Hochschule, Germany, Cabrilog SAS, France) (organizer)
• Anna Mavroudi (Institutt for pedagogikk, University of Oslo, Germany)
• Wolfgang Müller (MeVIS, PH Weingarten, Germany) (organizer)
• Simon Ostermann (Sprachtechnologie und Multilingualität, DFKI Saarbrücken, Germany)
• Christina Stollner (OTH Amberg-Weiden, Germany)
• Elvira Popescu (Computers and Electronics, University of Craiova, Romania)
• Sandra Rebholz (MeVIS, PH Weingarten & OTH Amberg-Weiden, Germany) (organizer)
• Leonhard Riedl (OTH Amberg-Weiden, Germany)
• Karsten D. Wolf (ZeMKI, Uni Bremen, Germany)

(Past) Call for Submissions

We invite submissions that depict experiences, tools, expectations, or methodologies with analysis tools of e-portfolios and, broader, that sketch the evolution of this modern assessment tool.

The submissions should sketch the didactical settings employed, the type of portfolios obtained and analysed as well as the technology proposed to support the pedagogical process effectively.

(Past) Format, submission and publication

(Past) Please submit a PDF document describing your approach in up to 6 pages in length (excluding references) until June 17, 2024. Papers can be submitted in English or German.

(Updated) Please use the updated GI-Edition LNI LaTex template (2023) when preparing your submission. Author guidelines and templates are available from the DELFI conference site Link.

Please submit your blinded paper via Easychair.

The workshop committee will evaluate the submissions using the following criteria:

• thematic relevance,
• innovative character,
• real world applicability of the didactical setting,
• soundness of the proposal 

All accepted contributions will be published in the DELFI workshop proceedings. For publication, we require the complete set of manuscript source files bundled in a single .ZIP (including images, references, .bbl files, etc.). Please submit the final version of your contribution by 16th August 2024.

Questions and suggestions? please write us.