Generating Automatic Feedback on UI Mockups with Large Language Models
Proceedings of the CHI Conference on Human Factors in Computing Systems(2024)
Abstract
Feedback on user interface (UI) mockups is crucial in design. However, human
feedback is not always readily available. We explore the potential of using
large language models for automatic feedback. Specifically, we focus on
applying GPT-4 to automate heuristic evaluation, which currently entails a
human expert assessing a UI's compliance with a set of design guidelines. We
implemented a Figma plugin that takes in a UI design and a set of written
heuristics, and renders automatically-generated feedback as constructive
suggestions. We assessed performance on 51 UIs using three sets of guidelines,
compared GPT-4-generated design suggestions with those from human experts, and
conducted a study with 12 expert designers to understand fit with existing
practice. We found that GPT-4-based feedback is useful for catching subtle
errors, improving text, and considering UI semantics, but feedback also
decreased in utility over iterations. Participants described several uses for
this plugin despite its imperfect suggestions.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined