An evaluation framework for information systems from the users’ perspective: a scoping review

Estrutura de avaliação para sistemas de informação na perspectiva dos usuários: uma revisão de escopo

Un marco de evaluación para sistemas de información desde la perspectiva de los usuarios: una revisión de alcance

Voltar ao Sumário

DOI: 

https://doi.org/10.59681/2175-4411.v18.2026.1539

Autor correspondente: Vanessa Pereira Corrêa Rampinelli

E-mail: fisio.vpc@gmail.com

  

Abstract

This scoping review aimed to synthesize scientific literature on information systems evaluation methods from the users’ perspective, focusing on their characteristics, construction stages, and measurement properties. The search was conducted in 2024 across the following databases: ACM Digital Library, Embase, Engineering Village, ERIC, IEEE Xplore, LILACS, MEDLINE (via PubMed), Scopus, and Web of Science. The study selection process was carried out independently by two reviewers. A total of 18,837 articles were identified, and 11 were ultimately included in the review. The studies presented various evaluation frameworks specific to health information systems, with one study also evaluating information systems from other fields. Only five studies reported the frameworks' validation procedures, and very few provided comprehensive validation results or sufficiently thorough analyses. This scoping review underscores the methodological diversity in information systems evaluation models and highlights the limitations of current use. It emphasizes the need for standardized approaches to enhance comparability, transparency, and long-term system development. 

Keywords: Evaluation models; Information systems; Information technology.

 

Resumo

Esta revisão de escopo teve como objetivo sintetizar a literatura científica sobre estruturas de avaliação de sistemas de informação sob a perspectiva dos usuários, com foco em suas características, etapas de construção e propriedades de mensuração. A busca foi realizada em 2024 nas seguintes bases de dados: ACM Digital Library, Embase, Engineering Village, ERIC, IEEE Xplore, LILACS, MEDLINE (via PubMed), Scopus e Web of Science. O processo de seleção dos estudos foi realizado de forma independente por dois revisores. Um total de 18.837 artigos foram identificados nas buscas, e 11 foram incluídos na revisão. Os estudos apresentaram diversas estruturas de avaliação específicas para sistemas de informação em saúde, com um estudo também avaliando sistemas de informação de outras áreas. Apenas cinco estudos relataram os procedimentos de validação das estruturas, e poucos forneceram resultados abrangentes das validações ou análises suficientemente completas. Esta revisão de escopo destaca a diversidade metodológica nos modelos de avaliação de sistemas de informação e destaca as limitações do uso atual. Enfatiza a necessidade de abordagens padronizadas para aprimorar a comparabilidade, a transparência e o desenvolvimento de sistemas a longo prazo.

Palavras-chave: Modelos de avaliação; Sistemas de informação; Tecnologia da informação.

 

Resumen

Esta revisión de alcance busca sintetizar la literatura científica sobre métodos de evaluación de sistemas de información desde la perspectiva de los usuarios, centrándose en sus características, etapas de construcción y propiedades de medición. La búsqueda se realizó en 2024 en las siguientes bases de datos: ACM Digital Library, Embase, Engineering Village, ERIC, IEEE Xplore, LILACS, MEDLINE (vía PubMed), Scopus y Web of Science. El proceso de selección de estudios fue realizado de forma independiente por dos revisores. Se identificaron 18.837 artículos en las búsquedas, de los cuales 11 se incluyeron en la revisión. Los estudios presentaron diversos marcos de evaluación específicos para sistemas de información en salud, y un estudio también evaluó sistemas de información de otros campos. Solo cinco estudios informaron sobre los procedimientos de validación de los marcos, y muy pocos proporcionaron resultados completos de las validaciones o análisis suficientemente exhaustivos. Esta revisión de alcance subraya la diversidad metodológica de los modelos de evaluación de sistemas de información y destaca las limitaciones de su uso actual. Enfatiza la necesidad de enfoques estandarizados para mejorar la comparabilidad, la transparencia y el desarrollo de sistemas a largo plazo. 

Palabras clave: Modelos de evaluación; Sistemas de información; Tecnología de la información.

 

1. Introduction

Information systems (IS) are organized sets of components designed to collect, process, store, and distribute information. They support decision-making, coordination, control, analysis, and visualization within organizations (1). These components include hardware, software, networks, data, and personnel, which work together to streamline operations and deliver relevant information (2). Health information systems (HIS) are well established within this domain, and IS also play key roles in public sectors such as education, mobility, security, and geolocation.

Evaluating the quality of IS is complex due to the multitude of factors involved. Deploying these systems across various sectors introduces further challenges in aligning multidimensional quality aspects with specific user needs. Understanding both their advantages and challenges, especially from the user’s perspective, is essential (3). IS use refers to how individuals apply computer systems in their professional activities, reflecting both adoption and effectiveness. 

Users, whether individuals, groups, or organizations, interact with these systems to perform tasks, access information, conduct transactions, or make decisions. They can be categorized as end, administrative, technical, managerial, or external users (4). End users rely on the system for daily tasks; examples include healthcare professionals using electronic health records (EHR) or employees accessing enterprise management systems. Defining user categories is essential for designing, implementing, and evaluating systems, as each group’s needs and expectations influence usability, efficiency, and overall success (5). 

Despite the importance of user-centered evaluation, few studies present comprehensive frameworks from this perspective. Most available studies come from the health sector or are over thirty years old (6-7). Many focus on technical or organizational aspects rather than the user experience, prioritizing performance, implementation, or economic impacts over acceptance, satisfaction, and ease of use. Evaluations also tend to be system-specific, limiting generalizability. Thus, a significant gap remains regarding frameworks that incorporate the user perspective.

This manuscript examines key sectors of national governance: the economy, agriculture, health, education, the environment, and infrastructure, which are essential for resource management, policy formulation, and addressing societal needs equitably and sustainably. IS in these areas support data collection, analysis, and dissemination, thereby improving decision-making, transparency, and efficiency. This study addresses the current gap in the literature by synthesizing IS evaluation methods from the user perspective, with a specific focus on the public sector. Accordingly, this scoping review aimed to synthesize the scientific literature on evaluating IS from a user perspective, examining the characteristics, development stages, and measurement properties of these evaluation frameworks.

 

2. Methods

This scoping review was conducted in accordance with Joanna Briggs Institute guidelines and the PRISMA Extension for Scoping Reviews (PRISMA-ScR). The study protocol was registered on the Open Science Framework (https://doi.org/10.17605/OSF.IO/H89T2). In August 2024, a comprehensive search was conducted across several databases, specifically: the ACM Digital Library, Embase, Engineering Village, ERIC, IEEE Xplore, Scientific Health Information from Latin America and the Caribbean countries (LILACS), Medical Literature Analysis and Retrieval System Online (MEDLINE) via PubMed, Scopus, and Web of Science (Appendix 1). The search strategy focused on three main concepts: “information systems,” “information technology,” and “evaluation framework.” Initially, the strategy was implemented in MEDLINE (via PubMed); the search terms were subsequently adapted to meet the specific requirements of each database. Additionally, two separate searches were conducted on Google Scholar, one in English and the other in Portuguese, screening only the first 100 articles from each search. No restrictions or filters were applied during the database searches. The retrieved articles were then exported to Rayyan® (Qatar Computing Research Institute, Qatar).

Afterward, Reviewer 1 excluded duplicate records by utilizing the platform’s automatic deduplication feature, followed by manual verification. Subsequently, Reviewer 2 was added to the Rayyan project with the blinding feature enabled. In the first eligibility phase, titles and abstracts were screened using the inclusion criteria (Appendix 2). Initially, the two reviewers independently excluded 10 articles, followed by a consensus meeting. Thereafter, all titles and abstracts were screened independently, and disagreements were resolved by consensus. In the second eligibility phase, full texts were evaluated using a similar approach of independent assessment and consensus. Full texts that were not readily available were requested from the corresponding authors via three attempts over a three-week period. The reference lists of the included full texts were also reviewed.

There were no restrictions regarding the language, geographic location, or publication date of the studies. However, gray literature (e.g., letters, book chapters, non-peer-reviewed conference papers, opinion pieces, articles describing techniques, panels, preprints, and clinical or technical guides) was excluded. Full texts that remained unavailable after three attempts to contact the corresponding authors over a three-week period were also excluded.

Following the selection process, the included articles were divided between two reviewers for data extraction. Four reviewers collectively created the extraction spreadsheet. A pilot extraction was conducted to calibrate the reviewers before the final extraction. Following the extraction process, the two reviewers cross-checked each other’s extracted data. To conclude, a consensus meeting was held. 

The extracted characteristics included the first author, year of publication, country of study, study design, name and/or type of IS, name and/or type of evaluation framework, development stage of the evaluation framework, and available psychometric properties. Lastly, the first and second reviewers analyzed the extracted data, followed by verification by all authors. The results are summarized descriptively in the tables and figures provided herein.

 

3. Results

Appendix 4 presents a summary of the search and selection process. Initially, the searches yielded 18,837 articles, of which 6,011 were excluded as duplicates. Upon reviewing the titles and abstracts, we eliminated an additional 12,780 articles. In the subsequent stage, we assessed 46 articles for potential relevance. However, 34 did not satisfy all inclusion criteria (Appendix 3). Consequently, 11 studies were included in the review (8–18).

The studies included in this analysis were conducted across various countries, with the United States contributing the most (n = 3); the remaining studies were from Brazil (n = 1) (8), Canada (n = 1) (13), Cyprus (n = 1) (17), India (n = 1) (11), Iran (n = 1) (14), New Zealand (n = 1) (16), Turkey (n = 1) (10), and the United Kingdom (n = 1) (18). All studies were published in English and from 1999 onward, although most were published after 2006. The primary characteristics of these studies are provided in Appendix 5.

The studies introduced a range of evaluation frameworks (8-18). Among the dimensions described in the papers, those assessing quality were the most common, followed by those evaluating user expectations and satisfaction (Appendix 6). As for the main characteristics of the frameworks (e.g., methods for application or scoring, etc.) in the sample, five reported validating their evaluation frameworks (9-10-15-17-18), four only provided a description of the framework (8-13-14-16), and two developed and applied a framework without validation (11-12). All studies provided an evaluation framework for IS in healthcare (8-10,12-18), except for one (11), which applied its framework in the healthcare sector but also extended its use to IS in various other fields, such as manufacturing, banking and finance, consulting, commerce, travel agencies, and education. 

The Clinical Information Systems Success Model (CISSM) assesses the effectiveness of clinical information systems across four principal dimensions: system performance, information quality, social influence, and facilitating conditions (9). The Human, Organization, and Technology-fit (HOT-fit) evaluation framework for HIS integrates human, organizational, and technological factors to assess IS success, underscoring the critical need for alignment among these components (18). The Development of an Evaluation Framework for Health Information Systems (DIPSA) Framework offers a structured approach to HIS assessment, focusing on performance, organizational impact, and perceived benefits from various stakeholders’ perspectives. Meanwhile, the Electronic Health Record Nurse Satisfaction (EHRNS) framework (15), a questionnaire-based tool, measures nurses’ satisfaction with EHR systems. The Expectation Questionnaire (10) evaluates end users’ expectations of IS to identify discrepancies between expected and actual outcomes.

The IS Effectiveness Model (11) is a more generic framework for evaluating IS effectiveness by analyzing technical quality, user satisfaction, and the impact on organizational goals. This approach incorporates both internal evaluations (by CEOs, IS managers, and users) and external assessments (by IT experts), culminating in a composite score derived from a formula that integrates these perspectives. The Content, Context, and Process evaluation framework focuses on health interventions (16), examining intervention content, implementation context, and the processes executed to achieve outcomes. Despite its detailed theoretical basis, this framework has not been applied or validated in practice. Likewise, the Infoway Benefits Evaluation Framework targets HIS (13), emphasizing accessibility, quality of care, and operational efficiency, although it also remains unvalidated. The Evaluation in Life Cycle of Information Technology (12), designed to assess IS throughout their lifecycle with a particular focus on the EPIC® EHR system in the United States, aims to evaluate systems from planning through implementation, emphasizing the fulfillment of organizational needs. Moreover, two studies introduced frameworks without designated titles. Morais and Costa (8) presented a framework for evaluating the IS of the Brazilian Unified Health System, identifying critical indicators of product quality. In Iran, Norouzi et al. developed a framework based on literature from 2000 to 2018, proposing indicators related to system performance, organizational aspects, system design, and technology.

What is more, among the surveyed studies (Appendix 5), only five reported validating their evaluation frameworks (9-10-15-17-18). However, the study utilizing the HOT-fit framework did not disclose its validation results (18). While all five studies provided details on participant sample size and profession (9-10-15-17-18), only three included information on participants’ sex, age, and professional experience (9-15-17). 

The CISSM framework underwent validation with 234 nurses across four hospitals (9). Construct validity was assessed through factor analysis, which revealed that items loaded onto single factors for each dimension. System Performance explained 63% of the variance, thus supporting Hypothesis 1, while Information Quality, which considers usefulness, completeness, format, and accuracy, accounted for 75% of the variance, validating Hypothesis 2. Social Influence, characterized by social factors, services, and support, constituted 55% of the variance, confirming Hypothesis 3. Facilitating Conditions, encompassing behavioral control and work processes, represented 56% of the variance, supporting Hypothesis 4. During the integration stage, Information Quality and Social Influence emerged as the primary predictors of System Use Dependency, explaining 25% of its variance. Nurse Satisfaction was influenced by Information Quality, System Performance, and Facilitating Conditions, which collectively accounted for 70% of the variance, with Information Quality acting as the largest contributing factor. At the outcome stage, Nurse Satisfaction predicted 54% of the variance in net benefits, while System Use Dependency accounted for 14%. Greater system integration was associated with increased dependency, satisfaction, and perceived benefits across the hospitals. Reliability was assessed using Cronbach’s alpha, yielding values of 0.80 (system performance), 0.89 (information quality), 0.85 (social influence), and 0.89 (facilitating conditions).

The DIPSA Framework (17) was validated in a study involving 309 healthcare professionals across two public hospitals in Cyprus. Construct validity was assessed through factor analysis, simplifying the original 42 questions across seven categories into 27 questions across five categories. Although the initial factor analysis revealed 10 potential categories, only five met the inclusion criteria, which required each category to retain at least five questions with a factor loading of ≥0.41. These final categories included Satisfaction, system quality, collaboration, procedures, and safety. Internal consistency was evaluated using Cronbach’s alpha, yielding the following results: system quality (0.913) and safety (0.855) under the technology factor; collaboration (0.939) and satisfaction (0.916) under the human factor, and procedures (0.898) under the organization factor.

The EHRNS framework (15) was validated in the context of EHR implementation by surveying 37 participants at 11 months post-implementation and 32 participants at 17 months post-implementation. The instrument was deemed clear with relevant items; however, the ‘availability’ item demonstrated a ceiling effect in both samples, indicating a need for revision. Construct validity was partially supported, with significant correlations observed between three item pairs across dimensions (e.g., ‘team communication’ with ‘patient safety’ and ‘patient outcome’), suggesting that the framework would benefit from further validation. Qualitative observations and interviews corroborated the validity of items such as ‘support’ and ‘overall satisfaction.’ Internal consistency, assessed using Kendall’s tau, revealed significant correlations in three of the five related item pairs (e.g., ‘support-resources’ and ‘overall satisfaction-worthwhile’), indicating moderate reliability. The variability in responses reflected diverse user perceptions of satisfaction.

The Expectation Questionnaire (10) was validated by analyzing the importance weights and expectation ratings provided by hospital IS users across two hospitals in Turkey. The validation involved a sample of 600 participants (504 from Hospital A and 96 from Hospital B), with response rates of 76.4% and 96.0%, respectively. The importance weights reflect the relevance users assign to each evaluated variable, measured on a 5-point Likert scale ranging from ‘not important’ to ‘very important.’ Conversely, expectation ratings gauge the extent to which user expectations are met, also evaluated on a 5-point Likert scale from ‘strongly disagree’ to ‘strongly agree.’ To facilitate more nuanced analyses and precise computations of the expectation-meeting index, these data were transformed into fuzzy triangular numbers. The internal consistency of these metrics was assessed using Cronbach’s alpha. For importance weights, the coefficients were 0.871 for Hospital A and 0.942 for Hospital B. For expectation ratings, the scores were 0.966 for Hospital A and 0.959 for Hospital B. The overall Cronbach’s alpha values for these two dimensions were 0.881 and 0.970, respectively.

 

4. Discussion

This review identified 11 evaluation frameworks for IS from the users’ perspective, mainly within the health domain. The frameworks showed diverse characteristics, underscoring the complexity and multidimensionality of IS. Differences in focus and scope reflect varied methodological approaches, demonstrating adaptability to different contexts. However, their practical utility is frequently limited by a lack of rigorous validation. Among the few studies that reported measurement properties, empirical assessments were often incomplete, thereby reducing both the generalizability of the frameworks and confidence in their findings.

Most studies focused on quality, satisfaction, and user expectations, though they differed in their specific approaches. A common feature was the emphasis on multiple dimensions (e.g., technical quality, user satisfaction, and organizational impact), highlights the multifaceted nature of IS. Frameworks such as CISSM (9) and HOT-fit (18) stressed the alignment of human, organizational, and technological factors, recognizing that IS effectiveness depends on the continuous interaction among these elements. Other frameworks, including the IS Effectiveness Model (11) and DIPSA (17), assessed system impact on organizational goals and processes to ensure strategic and operational alignment. Notably, DIPSA (17) explicitly builds upon the core variables established in HOT-fit (18) .

The evaluated frameworks also differ significantly in scope. DIPSA prioritizes performance and stakeholder benefits, whereas the Infoway Benefits Evaluation Framework (13) focuses on accessibility and quality of care. The Content, Context, and Process framework (16) specifically analyzes health interventions, whereas the IS Effectiveness Model (11) adopts a broader perspective applicable outside the healthcare sector. These variations show that although all the frameworks aim to assess IS, their methods and scopes are tailored to address distinct organizational needs.

Only five studies validated their frameworks (9-10-15-17-18), and one of these failed to report its empirical results. Despite showing promise in terms of reliability and construct validity, the validation efforts in these studies often lacked methodological depth and breadth. Although all used established methods (e.g., factor analysis), the underlying rationale for initial dimension definition was often unclear. Furthermore, factor analysis alone does not ensure robust validation, especially given the requirement for ongoing system adaptation (19). 

Content validation was similarly insufficient, typically relying only on expert input without involving end users or accounting for cultural adaptation. This oversight weakens generalizability and undermines confidence in the overarching conclusions, underscoring the need for rigorous empirical validation to ensure accuracy across diverse contexts. The findings also stress the inclusion of core dimensions such as usability, satisfaction, information quality, and security, which collectively reflect both technical efficiency and successful integration into organizational and user environments (20). While IS adoption is often studied through the lens of specific barriers and facilitators, advancing the theoretical understanding of system adoption, dissemination, and long-term maintenance remains essential. Evaluation models have thus evolved to prioritize key determinants such as perceived usefulness and ease of use (21).

The included studies covered a wide geographic range, and all were published in English. A significant portion of the research was conducted in the United States (9-12-15), with the remaining studies being in Brazil, Canada, Cyprus, Iran, New Zealand, Turkey, and India. This distribution highlights a persistent disparity in the geographic representation of user-centered evaluations for HIS (22).

Despite employing broad search strategies, all included studies ultimately focused on HIS (8-18). This dominance suggests a gap in the development of user-centered evaluation frameworks within other IS domains. Although one analyzed framework was structurally generic, most addressed specific healthcare contexts, indicating a clear need to extend these evaluative applications to other sectors, such as education and public administration.

Furthermore, despite core dimensions (e.g., interoperability, standards adoption, semantic consistency, and legacy data integration) being widely recognized as critical for the performance and scalability of HIS, these technical aspects are rarely captured by evaluations grounded in the end-user perspective. User-centered assessments tend to focus on attributes directly experienced during daily system interaction, including quality, functionality, usability, and perceived usefulness. 

Technical and architectural characteristics (e.g., interoperability or semantic alignment) typically require specialized analyses conducted by system developers, IT managers, or interoperability experts; thus, they fall outside the direct evaluative capacity of most end users. Viewed from this perspective, the predominance of generalized evaluation dimensions identified in this review does not necessarily reflect a limitation of the primary studies, but rather an intrinsic characteristic of user-based evaluation methods. 

This scoping review is subject to certain limitations. Extensive methodological variability across studies hindered direct comparability and analytical consistency. Differences in validity assessments and a lack of detailed data (e.g., missing item descriptions, incomplete framework operationalization, and unpublished validation results) further reduced transparency and replicability. Additionally, publication bias may have influenced the synthesized findings. Nonetheless, the review’s strengths include a comprehensive scoping search strategy and the timely relevance of the topic given the accelerating global adoption of health information technologies.

 

5. Conclusion

This scoping review provides a comprehensive analysis of IS evaluation models, particularly within the healthcare sector. It highlights the prevailing methodological diversity, the limited generalizability of existing tools, and the absence of standardized evaluation dimensions. Although the included studies cover varied approaches and contexts, thereby enhancing our understanding of current evaluation practices, significant challenges remain in result comparability, analytical transparency, and generalizability. These limitations emphasize the urgent need for further research utilizing standardized methodologies, broader demographic populations, and longitudinal study designs. Nevertheless, this review notes important strengths in the literature, including the high clinical relevance of the evaluated topics and the broad range of dimensions addressed, advancing knowledge on the reliability and validity of health evaluation models. 

Future research should focus on refining specific evaluation models and would benefit from integrating user-centered evaluation approaches with complementary technical or organizational assessments, such as those related to system integration or interoperability, to provide a more comprehensive understanding of HIS and their downstream implications for healthcare management and service quality.

Lastly, bridging the gap between evaluation frameworks developed in academic research and their practical adoption by the organizations that develop or deploy HIS may contribute to more systematic assessments, improve end-user satisfaction, and enhance overall productivity in real-world clinical settings.

 

Acknowledgment

The authors would like to thank Karyn Munik Lehmkuhl for her support with the search strategies and Atlas Assessoria Linguística for language editing. 

 

References

  1. Pleskach V, Pleskach M, Zelikovska O. Information Security Management System in Distributed Information Systems. In: 2019 IEEE International Conference on Advanced Trends in Information Theory, ATIT 2019 - Proceedings. 2019. 
  2. Gregersen E. 5 Components of Information Systems | Britannica. Britannica2021;
  3. Stoykova S, Shakev N. Artificial Intelligence for Management Information Systems: Opportunities, Challenges, and Future Directions. Algorithms. 2023;16(8). 
  4. Maia MAQ, Barbosa RR, Williams P. Usabilidade e experiência do usuário de sistemas de informação: em busca de limites e relações. Ciência da Informação em Revista. 2020;6(3). 
  5. BSI. ISO 9241-210:2019, Ergonomics of human-system interaction - Part 210: Human-centred design for interactive systems. 2019. 
  6. Mie L, Cintho M, Rodrigues Machado R, Cabral Moro CM, Correspondente A. Métodos para Avaliação de Sistema de Informação em Saúde. 2016. 
  7. Osborne SE. Evaluating Health Care Information Systems: Methods and Applications. J Adv Nurs. 1994;20(1). 
  8. de Morais RM, Costa AL. Um modelo para avaliação de sistemas de informação do SUS de abrangência nacional: O processo de seleção e estruturação de indicadores. Revista de Administracao Publica. 2014;48(3). 
  9. Garcia-Smith D, Effken JA. Development and initial evaluation of the Clinical Information Systems Success Model (CISSM). Int J Med Inform. 2013;82(6). 
  10. Gürsel G, Zayim N, Gülkesen KH, Arifoǧlu A, Saka O. A new approach in the evaluation of hospital information systems. Turkish Journal of Electrical Engineering and Computer Sciences. 2014;22(1). 
  11. Kanungo S, Duda S, Srinivas Y. A structured model for evaluating information systems effectiveness. Syst Res Behav Sci. 1999;16(6). 
  12. Kukhareva P V., Weir C, Del Fiol G, Aarons GA, Taft TY, Schlechter CR, et al. Evaluation in Life Cycle of Information Technology (ELICIT) framework: Supporting the innovation life cycle from business case assessment to summative evaluation. J Biomed Inform. 2022;127. 
  13. Lau F, Hagens S, Muttitt S. A proposed benefits evaluation framework for health information systems in Canada. Healthc Q 2007;10(1). 
  14. Norouzi H, Mehrolhassani MH, Hajesmaeel-Gohari S, Ahmadian L, Ghaemi MM, Mohammadi M, et al. Identification and classification of indicators for evaluating health information systems. Health Policy Technol 2024;13(2). 
  15. Sockolow PS, Weiner JP, Bowles KH, Lehmann HP. A new instrument for measuring clinician satisfaction with electronic health records. CIN - Computers Informatics Nursing 2011;29(10). 
  16. Stockdale R, Standing C. An interpretive approach to evaluating information systems: A content, context, process framework. Eur J Oper Res 2006;173(3). 
  17. Stylianides A, Mantas J, Roupa Z, Yamasaki EN. Development of an evaluation framework for health information systems (DIPSA). Acta Informatica Medica 2018;26(4). 
  18. Yusof MM, Kuljis J, Papazafeiropoulou A, Stergioulas LK. An evaluation framework for Health Information Systems: human, organization and technology-fit factors (HOT-fit). Int J Med Inform 2008;77(6). 
  19. Reichenheim M, Bastos JL. What, what for and how? Developing measurement instruments in epidemiology. Rev Saude Publica 2021;55. 
  20. Barzekar H, Ebrahimzadeh F, Luo J, Karami M, Robati Z, Goodarzi P. Adoption of hospital information system among nurses: A technology acceptance model approach. Acta Informatica Medica 2019;27(5). 
  21. Ammenwerth E. Technology Acceptance Models in ealth nformatics: TAM and UTAUT. Stud Health Technol Inform 2019;263. 
  22. Rana RH, Alam K, Gow J. Health outcome and expenditure in low-income countries: does increasing diffusion of information and communication technology matter? Inf Technol Dev 2020;26(3).