University websites are a wealth of information, but they present many challenges. How can they present content to meet the needs of a diverse range of site visitors? How can you optimise content to align with a school-leaver prospective student as well as an academic expert? How can you train hundreds of staff to publish web content consistently? At the University of Edinburgh, we partnered with open-source Drupal AI experts to investigate what AI-powered tools could do to improve the way we publish and manage our web content. Taking a UX-research-led approach, we begun a series of experiments to understand the possibilities for applying AI and LLMs and to appreciate the associated risks and implications. What we found surprised us. Who knew AI Agents could help content editors learn a system in real-time, or that you could use prompt engineering to apply the rules of a style guide? In this session, I'll talk through the approaches I used to carry out controlled experiments with open-source AI, pulling together concepts from conversational design, learning design, interaction design, service design and AI configuration. I'll share a toolkit to design your own AI experiments and offer a fresh perspective on improving institutional websites and their content.
