Artificial intelligence and its enormous developments and possibilities are on everyone’s lips. Students and staff at Hof University of Applied Sciences have recently been provided with their own AI tools for text and image generation (“Lisa and Luis”) , which are also being used more and more. They were specially developed and made available by the AI User Center at the Institute for Information Systems (iisys). We spoke to Head Prof. Dr. René Peinl about the tools and the initial user feedback.
Prof. Peinl, why did you decide to offer your own tools within the university?
“Generative AI has been known to many at least from hearsay since ChatGPT and is often directly associated with the market leaders. Although these actually work well, they are by no means without an alternative, as is often assumed. For each individual, using the cloud offerings of US companies is very convenient and not a bad thing. For society as a whole, however, we are voluntarily fueling the next monopoly and ensuring that our domestic companies are left behind. After all, AI is based on data and by using the services, we generate new data that benefits the provider.”
What areas of application are “Lisa and Luis” designed for?
“In principle, the tools are relatively versatile and not limited to individual use cases. Incidentally, this is also a feature of advanced AI. However, they are particularly suitable for applications where factual accuracy is not so important. In an advertising text, a little exaggeration or flowery paraphrasing is fine. An illustrative image for a website does not have to correspond 100% to the facts in every detail. A shadow can contradict the lighting conditions or a cat can have an unusual pattern of spots without it being a bad thing.”
For those who haven’t discovered it yet: How can you actually use “Lisa and Luis”?
“After registering, you can access the university’s AI services, which are constantly being expanded, via the homepage ai.hof-university.de. The same identifier is used for this as for all other university services such as Moodle or Primuss.”
Source: Hof University of Applied Sciences;
In your opinion, what is the current status of the tools?
“The development of AI is incredibly fast. A model that is still state of the art today can be almost obsolete in two months’ time. That’s why it’s important that we update regularly. For example, we are currently testing the Llama 3 model, which achieves significantly better results in some cases than the model currently in use, which is based on the Mistral model from France, due to the slightly larger model size and significantly more extensive training data. The image generator is currently based on Stable Diffusion XL Lightning, which is version 2.5 with additional acceleration, so to speak. It is currently at the cutting edge. However, the release of the successor v3 has been announced for 12.6.24, so we will certainly be offering a new model in the fall. We have also recently introduced a text model for generating software code, which should help computer science students.”
Who was involved in the creation of our own AI tools?
“On the professorial side, our colleague Sven Rill from the Faculty of Computer Science and Vice President Dietmar Wolff were involved. However, we first had to do a lot of convincing not to take the supposedly easy route to OpenAI like many others. On the staff side, Thomas Herpich, Thomas Weber and Johannes Wirth, as well as Noah Lehmann, deserve special mention. Finally, we also had student support, for example from Hannes Steinel.”
Prof. Dr. Sven Rill and Vice President Prof. Dr.-Ing. Dietmar Wolff; Image: Hof University of Applied Sciences;
What steps are planned for the future in this field?
“The task is to keep the existing offers up to date, i.e. to update them every 3-6 months, and to incorporate further offers. We are currently discussing how the university’s existing data can be integrated into the chatbot so that students can then have questions about the university answered via the chatbot or employees can ask questions about processes or contact persons. We are also planning an extension to image understanding so that questions about images can be asked. Voice input and output would also be possible. However, it is a question of how much time we can spend on this, as our main task is teaching and research and the development of infrastructure is not originally part of either field.”
First questions from users to the developers:
Dear Prof. Peinl, below we have now collected some questions and user observations from students and would like to ask you to respond to them, as we think this will benefit all users:
–On text generation with “Lisa”:
Can AI tools already be used to generate texts on niche topics or does it make more sense to only query more general topics at first?
“It depends a bit on what you see as a niche topic, but I think that the vast majority of topics are included in the training data often enough to be useful for queries.”
Sometimes answers to current topics are outdated or not up to date – will this improve?
“No, this is a fundamental problem. The training of the AI models ends on a certain date (currently in October 2023 for many models, for example) and nothing more is “learned” afterwards. Up-to-date data can only be fed into the chatbot by linking it to a search engine. Microsoft and Google, for example, do this in some cases.”
What is the general trend in the area of AI creativity?
“For a long time, it was thought that creativity would remain the domain of humans. Today we know that, just as playing chess is only one form of intelligence, creativity also has many aspects. AI can already do much of what was previously considered creative. This is why creativity is currently being redefined, but this also means that only very few people are truly creative. Most people don’t even like completely new things that much. The innovations that are best received build on the familiar and just vary it. If you look at the charts at the moment, there’s a 90s revival wave going on and with “Stumblin in”, a song from 1978 has even made it back into the charts in a very carefully updated form. If that’s creative, then so is the AI. If not, then very few people are either.”
Which approach produces better, less “machine” sounding lyrics?
a) An English-language prompt (command line) requests an English-language text from the text generation AI, which is then translated into German manually or by another AI tool (e.g. DeepL). Or…
b) A German prompt directly requests a German-language text from the text generation AI, thereby allowing the AI, which is probably more likely to “think” in English, to perform a translation service in addition to the content service.
“This is a very good question that would require more detailed investigation. Our “Lisa” currently jumps from English to German from time to time in dialog because she has been trained to do so. With many other models, it’s exactly the opposite. What’s more, the answer is always just a snapshot. I would say b) for the best models and a) for the average ones, although even the translators are not error-free.”
Will the result of the text generation AI be more accurate if you first create specific personas and then refer to these personas (as the alleged text creator or target group) when creating the text?
“Yes, I would say so. In general, it’s always good to create an explicit context on which the chatbot can work. It’s like writing down a few key points on a piece of paper before giving a speech. You don’t even have to type much. The chatbot itself can help build the context by asking it appropriate initial questions.”
ChatGPT often gives very comprehensive answers, albeit often in a rather monotonous sentence structure. “Lisa”, on the other hand, usually answers very briefly. Can you influence the depth of the answers via the prompts?
“Yes, this is possible to a certain extent, but beyond that it is again a question of the training data. Models that have been trained on many long passages tend to be chatty. Others, which mainly had short answers in the training data, find it difficult to give detailed answers. However, it is always a good idea to include instructions such as “be brief”, “the answer should have a maximum of 200 characters” or “answer in detail and at length” in the prompt.”
Currently, the text generation AI “Lisa” still has some spelling weaknesses. How can the developers influence this?
“In my opinion, these are more grammatical than spelling weaknesses. Sometimes words are formed incorrectly. For example, I once read something about “Mutterin” because the AI doesn’t know all the exceptions to the formation of feminine forms in German. This is a consequence of the fact that over 90% of the training data is English and German only accounts for around 1%, depending on the model. The problem is that there are not enough publicly available German texts to compensate for the large amount of English information. A BayernGPT, as Markus Söder would like to see, or even the results of the OpenGPT-X research project could improve this situation with a more balanced mix of training data.”
–On image generation with “Luis”:
How can we get our AI tools to generate text in the images (currently, spelling errors or distortions can sometimes still be observed in the image)?
“This is a known problem with image generators. In the past, they were hardly ever trained with images that contained text. The v3 model from Stable Diffusion mentioned above should be much better here. However, it will probably also be the case that English words are more likely to be “painted” without errors than German words. This is a consequence of the fact that we in Germany failed to invest in research into this type of AI early enough, and the publicly available models are almost all trained with predominantly English-language data, which is also a problem for images without text. For example, images of Joe Biden and Taylor Swift are generated quite well, but German celebrities like Lena Meyer-Landrut are rather unknown.”
Luis throws out good quality images, but he often doesn’t understand what is meant in the prompt. How can we work on recognition skills and what prospects are there here?
“So far, you still have to describe exactly what the picture should look like (e.g. garbage can with a symbol for the circular economy). If an image is to be generated for an abstract topic (e.g. recycling), the AI is overwhelmed because there are many possibilities and no clear instructions. In the future, the researchers hope that close interaction between chatbots and image generators will improve this situation. To a certain extent, this already works today. You can simply ask Lisa to generate a prompt for Luis.”
Faces and hands often still look very fake. As with drawing, they also seem to be the biggest challenge for the AI. Why is that?
“With faces, this is only true if they are not the main subject of the photo. The grossest errors arise due to the models’ lack of physical and biological knowledge. They don’t “know” that hands always have five fingers. The training images sometimes show more or fewer fingers, so this seems to be variable for the AI. AI is not good at counting anyway. We’ve managed to get to grips with this to a certain extent with chatbots, but it’s still the case with image generators. The fact that arms cannot be in any position, but are restricted by bones and joints, is also unknown to the image AI. Therefore, the visual patterns are simply generated as they seem to “fit” or as the random pattern from which the image is refined seems to dictate.”
Will it soon be possible to output entire layouts for websites as well as photos?
“You can already generate HTML and CSS source code for websites today and the good models can even generate suitable JavaScript to implement simple games such as tic-tac-toe. The researchers are working towards general AI. This means that it should be able to take over all human activities (especially office jobs). The only question is when this will be soon, and experts are arguing about this, with opinions ranging from 1-2 years to decades. Personally, I tend to favor shorter periods of time rather than longer ones. Yann LeCun, the chief AI researcher at Meta, claimed 11 months before the release of ChatGPT that everyday physics cannot be learned from text alone. The example he gave was then disproved in less than a year, although LeCun is certainly one of the best AI experts in the world.”
Thank you very much for your answers and the wealth of information!
