Investigating the Influence of Different Factors on the UX Evaluation of a Mobile Application

User eXperience (UX) evaluations play an essential role in the software development process. As the results from such evaluations can drive future releases, it is necessary to identify which factors can substantially change users' judgments about their experience to have more precise results and understand UX better. This article investigates how interaction sequencing, previous experience, and the number of problems could affect overall satisfaction and the two main UX dimensions: pragmatic and hedonic. We employed three different evaluation methods to evaluate a chatbot-based mobile shopping application. The results revealed that participants with previous experience with similar apps tended to give lower ratings. We also found that as inspectors identify more problems, they tend to rate the pragmatic dimension lower. Finally, we did not identify a significant influence of interaction sequencing on UX evaluation. We discuss the reasons for these results, the implications for practitioners and researchers, and research opportunities.