Usability Evaluation of an E-Commerce Integrated with a Fan Community Platform Using Cognitive Walkthrough

Aida Khalisatifa; Ali Ibrahim

doi:10.30871/jaic.v9i1.8865

Aida Khalisatifa Information Systems, Faculty of Computer Science, Universitas Sriwijaya
Ali Ibrahim Master of Computer Science, Faculty of Computer Science, Universitas Sriwijaya

DOI: https://doi.org/10.30871/jaic.v9i1.8865

Keywords: Cognitive Walkthrough, E-Commerce, PSSUQ, Usability evaluation, User experience

Abstract

This study aims to evaluate the usability and user experience of Weverse Shop e-commerce app after integration into Weverse app using the Cognitive Walkthrough and Post Study System Usability Questionnaire (PSSUQ) methods. Cognitive Walkthrough is used to identify usability issues from an expert perspective, while PSSUQ is used to quantitatively measure user experience through three subscales: System Usefulness, Information Quality, and Interface Quality. Participants in this study ran 7 task scenarios relevant to the application features. Based on the analysis results, the average scores for the PSSUQ subscales were 2.99 for System Usefulness, 2.98 for Information Quality, and 2.87 for Interface Quality, with an overall score of 2.95. These results indicate that the application interface still needs improvement, especially in the aspects of navigation and information delivery. This research provides recommendations for improvements to usability elements to increase user satisfaction.

Downloads

Download data is not yet available.

References

D. Sinaga and P. Peniarsih, “Menghadapi Perubahan Dunia Melalui Transformasi Digital Menuju Kesuksesan pada Era Digitalisasi,” JSI (Jurnal sistem Informasi) Universitas Suryadarma, vol. 11, no. 2, pp. 51–58, 2024.

Statista, “Number of users of e-commerce in Indonesia 2020-2029,” Statista Research Department. Accessed: Aug. 05, 2024. [Online]. Available: https://www.statista.com/forecasts/251635/e-commerce-users-in-indonesia#statisticContainer

I. M. Insani, M. Iqbal, and M. A. Rakhman, “Dampak Budaya Korean Pop Dalam Perilaku Konsumerisme di Kalangan Penggemar K-Pop UKM Hallyu-Up Edutainment,” The Indonesian Journal of Social Studies, vol. 7, no. 1, pp. 143–157, 2024.

E. Alexandrina, L. Ramonita, and A. Anindita, “Understanding the motivation of purchase decision making of KPOP Seventeen’s merchandise in Weverse Shop,” Jurnal Mantik, vol. 8, no. 1, pp. 704–713, 2024.

W. F. Putra and P. Febriana, “Penggunaan Aplikasi Digital Weverse sebagai New Media Interaksi antara Artis/Idol K-Pop dengan Para Penggemarnya Using the Weverse Digital Application as a New Media for Interaction between K-Pop Artists/Idols and their Fans,” Social Sciences (JEHSS), vol. 5, no. 3, pp. 1649–1659, 2023.

A. Kumar, “UX Competitive Analysis — Weverse Shop,” Medium. Accessed: Jul. 07, 2024. [Online]. Available: https://medium.com/@abhigyakumar/ux-competitive-analysis-weverse-shop-10a89d3740f6

Weverse Company, “Weverse Shop: Notices,” Weverse Shop. Accessed: Aug. 05, 2024. [Online]. Available: https://shop.weverse.io/en/notices/5588

C. J. Hisyam, A. Nikita, A. N. Chotimah, J. F. Pasha, and N. S. Mulia, “Peran Platform Weverse Sebagai Sarana Interaksi Dan Ekonomi Dalam Industri K-Pop,” Jurnal Ekonomi dan Pembangunan Indonesia, vol. 2, no. 3, pp. 76–86, 2024.

T. Mastery, “Weverse – Top Ten Most Important Things You Need To Know,” DotCom Magazine. Accessed: Jul. 07, 2024. [Online]. Available: https://dotcommagazine.com/about-dotcom-magazine/

T. T. Van Ho, M. J. Noh, Y. N. Lee, and Y. S. Kim, “Analyzing User Feedback on a Fan Community Platform’Weverse’: A Text Mining Approach,” Smart Media Journal, vol. 13, no. 6, pp. 62–71, 2024.

Z. B. Agustyn, S. Wibowo, and A. J. Furqon, “Usability Testing Aplikasi Getcontact Menggunakan Metode Post Study System Usability Questionnaire (PSSUQ),” Jurnal Informatika dan Teknik Elektro Terapan, vol. 12, no. 2, 2024.

Jakob Nielsen, “Usability 101: Introduction to Usability,” Nielsen Norman Group. Accessed: May 28, 2024. [Online]. Available: https://www.nngroup.com/articles/usability-101-introduction-to-usability/

ISO 9241-210, “Ergonomics of human-system interaction—Part 210: Human-centred design for interactive systems,” 2019.

B. V. Angela, T. T. Wulansari, R. Riyayatsyah, Y. Fitrianto, and A. Rahim, “User Interface and User Experience Analysis of Kejar Mimpi Mobile Application using the User-Centered Design Method,” ILKOM Jurnal Ilmiah, vol. 15, no. 1, pp. 1–10, 2023.

K. Hornbæk and E. L.-C. Law, “Meta-analysis of correlations among usability measures,” in Proceedings of the SIGCHI conference on Human factors in computing systems, 2007, pp. 617–626.

F. Bagheri, F. Abbasi, M. Sadeghi, and R. Khajouei, “Evaluating the usability of a cancer registry system using Cognitive Walkthrough, and assessing user agreement with its problems,” BMC Med Inform Decis Mak, vol. 23, no. 1, p. 23, 2023.

M. Ridwan, M. Maulana, and D. Nurdiana, “Usability Testing Website MY UT Menggunakan Metode Post-Study System Usability Questionnaire Berdasarkan Pandangan Mahasiswa Universitas Terbuka,” Jurnal Sistem Informasi dan Informatika (Simika), vol. 7, no. 2, Jul. 2024, doi: 10.47080/simika.v7i2.3389.

N.-C. Chi, A. Shanahan, K. Nguyen, I. Demir, Y.-K. Fu, and K. Herr, “Usability Testing of the PACE App to Support Family Caregivers in Managing Pain for People with Dementia,” Innov Aging, vol. 7, no. Suppl 1, p. 857, 2023.

P. S. Levy and S. Lemeshow, Sampling of Populations: Methods and Applications. in Wiley Series in Survey Methodology. Wiley, 2013. [Online]. Available: https://books.google.co.id/books?id=XU9ZmLe5k1IC

D. Sugiyono, “Metode penelitian pendidikan pendekatan kuantitatif, kualitatif dan R&D,” 2013.

W. (Bill) Albert and T. S. (Tom) Tullis, “Chapter 4 - Performance Metrics,” in Measuring the User Experience (Third Edition), W. (Bill) Albert and T. S. (Tom) Tullis, Eds., Morgan Kaufmann, 2023, pp. 71–107. doi: https://doi.org/10.1016/B978-0-12-818080-8.00004-2.

J. R. Lewis, “Usability testing,” Handbook of human factors and ergonomics, pp. 1267–1312, 2012.

J. R. Lewis, “Psychometric Evaluation of the PSSUQ Using Data from Five Years of Usability Studies,” Int J Hum Comput Interact, vol. 14, no. 3–4, pp. 463–488, Sep. 2002, doi: 10.1080/10447318.2002.9669130.

J. Sauro and J. R. Lewis, “Chapter 8 - Standardized Usability Questionnaires,” in Quantifying the User Experience, J. Sauro and J. R. Lewis, Eds., Boston: Morgan Kaufmann, 2012, pp. 185–240. doi: https://doi.org/10.1016/B978-0-12-384968-7.00008-4.

R. Suwandy, S. H. Marpaung, and C. Caroline, “Evaluasi Pengalaman Pengguna dengan Menggunakan Post Study System Usability Questionnaire (PSSUQ) Perpustakaan Digital Universitas Mikroskil,” Jurnal Pendidikan Tambusai, vol. 6, no. 1, pp. 4190–4206, 2022.

L.-O. Bligård and A.-L. Osvalder, “An Analytical Approach for Predicting and Identifying Use Error and Usability Problem,” in HCI and Usability for Medicine and Health Care, A. Holzinger, Ed., Berlin, Heidelberg: Springer Berlin Heidelberg, 2007, pp. 427–440.