Table 4 Example for the Program Implementation domain, Infrastructure and technology subdomain: evaluation questions, measures, and data specifics

Infrastructure & Technology (functionality, performance, security)^a

To what extent has cybersecurity been reviewed and considered? (e.g., has a threat risk assessment (TRA) been conducted?)

# of access issues, adherence to cybersecurity standards, # of threats or security breaches

Cyber-review by 3rd party, TRA, cybersecurity monitoring tools

Quantitative

Applicable regional or national security guidance or directives (e.g., Canadian Centre for Cybersecurity guidance & directives)

Do providers and/or patients have appropriate access to information required to support care delivery? (e.g., access to whole patient chart, ability to communicate with other providers, or provide follow up information to patients)

Surveys to providers, staff, patients/families, safety reporting system, # of release of information requests

Surveys

Qualitative & quantitative

Other similar programs

To what extent does the technology support virtual care service delivery?

% of planned/unplanned downtime, # of requests for change/optimization, volume of technical issues, needs assessment/requirements evaluation

HelpDesk tickets/IT reports, project intake/enhancement requests, post-encounter questionnaires

Quantitative

Other similar programs