An evaluation framework for information systems from the users’ perspective: a scoping review
DOI:
https://doi.org/10.59681/2175-4411.v18.2026.1539Keywords:
Evaluation models, Information System, Information TechnologyAbstract
This scoping review aims to synthesize scientific literature on information systems evaluation methods from the users’ perspective, focusing on their characteristics, construction stages, and measurement properties. The search was conducted in 2024 across the following databases: ACM Digital Library, Embase, Engineering Village, ERIC, IEEE Xplore, LILACS, MEDLINE (via PubMed), Scopus, and Web of Science. The study selection process was independently carried out by two reviewers. A total of 18,837 articles were identified in the searches, and 11 were ultimately included in the review. The studies presented various evaluation frameworks specifically for health information systems, with one study also evaluating information systems from other fields. Only five studies reported the frameworks’ validation procedures, and very few provided comprehensive results of the validations or sufficiently thorough analyses. This scoping review underscores the methodological diversity in information systems evaluation models and highlights the limitations of current use. It emphasizes the need for standardized approaches to enhance comparability, transparency, and long-term system development.
Downloads
References
Pleskach V, Pleskach M, Zelikovska O. Information Security Management System in Distributed Information Systems. In: 2019 IEEE International Conference on Advanced Trends in Information Theory, ATIT 2019 - Proceedings. 2019.
Gregersen E. 5 Components of Information Systems | Britannica. Britannica2021;
Stoykova S, Shakev N. Artificial Intelligence for Management Information Systems: Opportunities, Challenges, and Future Directions. Algorithms2023;16(8).
Maia MAQ, Barbosa RR, Williams P. Usabilidade e experiência do usuário de sistemas de informação: em busca de limites e relações. Ciência da Informação em Revista 2020;6(3).
BSI. ISO 9241-210:2019, Ergonomics of human-system interaction - Part 210: Human-centred design for interactive systems. 2019.
Mie L, Cintho M, Rodrigues Machado R, Cabral Moro CM, Correspondente A. Métodos para Avaliação de Sistema de Informação em Saúde. 2016.
Osborne SE. Evaluating Health Care Information Systems: Methods and Applications. J Adv Nurs 1994;20(1).
de Morais RM, Costa AL. Um modelo para avaliação de sistemas de informação do SUS de abrangência nacional: O processo de seleção e estruturação de indicadores. Revista de Administracao Publica 2014;48(3).
Garcia-Smith D, Effken JA. Development and initial evaluation of the Clinical Information Systems Success Model (CISSM). Int J Med Inform 2013;82(6).
Gürsel G, Zayim N, Gülkesen KH, Arifoǧlu A, Saka O. A new approach in the evaluation of hospital information systems. Turkish Journal of Electrical Engineering and Computer Sciences 2014;22(1).
Kanungo S, Duda S, Srinivas Y. A structured model for evaluating information systems effectiveness. Syst Res Behav Sci 1999;16(6).
Kukhareva P V., Weir C, Del Fiol G, Aarons GA, Taft TY, Schlechter CR, et al. Evaluation in Life Cycle of Information Technology (ELICIT) framework: Supporting the innovation life cycle from business case assessment to summative evaluation. J Biomed Inform 2022;127.
Lau F, Hagens S, Muttitt S. A proposed benefits evaluation framework for health information systems in Canada. Healthc Q 2007;10(1).
Norouzi H, Mehrolhassani MH, Hajesmaeel-Gohari S, Ahmadian L, Ghaemi MM, Mohammadi M, et al. Identification and classification of indicators for evaluating health information systems. Health Policy Technol 2024;13(2).
Sockolow PS, Weiner JP, Bowles KH, Lehmann HP. A new instrument for measuring clinician satisfaction with electronic health records. CIN - Computers Informatics Nursing 2011;29(10).
Stockdale R, Standing C. An interpretive approach to evaluating information systems: A content, context, process framework. Eur J Oper Res 2006;173(3).
Stylianides A, Mantas J, Roupa Z, Yamasaki EN. Development of an evaluation framework for health information systems (DIPSA). Acta Informatica Medica 2018;26(4).
Yusof MM, Kuljis J, Papazafeiropoulou A, Stergioulas LK. An evaluation framework for Health Information Systems: human, organization and technology-fit factors (HOT-fit). Int J Med Inform 2008;77(6).
Reichenheim M, Bastos JL. What, what for and how? Developing measurement instruments in epidemiology. Rev Saude Publica 2021;55.
Barzekar H, Ebrahimzadeh F, Luo J, Karami M, Robati Z, Goodarzi P. Adoption of hospital information system among nurses: A technology acceptance model approach. Acta Informatica Medica 2019;27(5).
Ammenwerth E. Technology Acceptance Models in ealth nformatics: TAM and UTAUT. Stud Health Technol Inform 2019;263.
Rana RH, Alam K, Gow J. Health outcome and expenditure in low-income countries: does increasing diffusion of information and communication technology matter? Inf Technol Dev 2020;26(3).
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Vanessa Pereira Corrêa Rampinelli, Catiele Raquel Schmidt, Júlia Meller Dias de Oliveira, Ianka Cristina Celuppi, Arthur Thives Mello, Jades Fernando Hammes, Célio Luiz Cunha, Raul Sidnei Wazlawick, Danielle Soares Rocha Vieira, Eduardo Monguilhott Dalmarco
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Submission of a paper to Journal of Health Informatics is understood to imply that it is not being considered for publication elsewhere and that the author(s) permission to publish his/her (their) article(s) in this Journal implies the exclusive authorization of the publishers to deal with all issues concerning the copyright therein. Upon the submission of an article, authors will be asked to sign a Copyright Notice. Acceptance of the agreement will ensure the widest possible dissemination of information. An e-mail will be sent to the corresponding author confirming receipt of the manuscript and acceptance of the agreement.
