Large Language Models (LLMs) are powerful — but are they consistent? Can they reliably produce the same output when presented with inputs that have the same meaning? This is a key question for trustworthiness in AI systems that is being addressed within the KomMKonLLM project (Netidee: Förderjahr 2024 / Projekt Call #19 / ProjectID: 7409).
On Tuesday, February 25, 2025, Ludwig Kampel and Bernhard Garn joined the event “SBA Security Meetup hosted by Dynatrace!” to present a combinatorial approach to consistency testing of LLMs that is being implemented within KomMKonLLM. The presentation about KomMKonLLM – given jointly by Ludwig and Bernhard – generated much interest from the audience and the ensuing discussion covered several different aspects on the topic of consistency (testing) of LLMs.
You can find more information on the official project homepage of KomMKonLLM here: https://www.netidee.at/kommkonllm and you can get in contact about KomMKonLLM at (n o sp ac es): “K omMKon LLM@s ba-resear ch.o r g ”.
