Patient engagement in healthcare: a preliminary set of measures to evaluate patient engagement in the European Reference Networks
Abstract
Aim: The European Reference Networks (ERNs) provide clinicians and patients the opportunity to collaborate at EU level to improve diagnosis, care and treatment for people living with rare and complex conditions. However, building a partnership culture to systematically involve patients in ERN activities and decision-making structures is challenging, partly because the role of patient representatives and the value of this collaboration are not always understood. The objective of this project was to develop an evaluation framework to assess the impact of patient engagement in the ERNs and to provide evidence on the value of patient-clinician partnership.
Methods: The evaluation was developed by EURORDIS and patient representatives involved in the ERNs (ePAG advocates) through a participatory and iterative process. The work was organised in three different phases: (1) clarify roles and identify common goals for ePAG advocates’ engagement in the ERNs; (2) define a set of measures; and (3) test the measures in three different ePAGs (European Patient Advocacy Groups).
Results: The project allowed developing a common understanding among ePAG advocates of their role and goals in the ERNs and defining a patient-driven evaluation framework to assess their level of engagement in the ERNs’ activities and how effectively they were working to fulfil their role.
Conclusion: Engaging with ERN clinicians to refine the framework would probably render it more relevant to the reality and priorities of the specific ERNs and more valuable as a tool to build a strong partnership culture. Such an evaluation framework could be integrated into the ERNs’ quality improvement system to ensure that the networks’ activities are driven by and remain responsive to patients’ needs.
Keywords
INTRODUCTION
Patient-clinician partnerships to transform healthcare services in Europe are in their infancy, compared to other areas, such as medicine development, where there is a long tradition of patient involvement and where the role of patient representatives is recognised and well established[1]. Today, patient involvement in healthcare focuses primarily on their role as partners in their own care, with few countries systematically involving patients in decision-making structures for healthcare design and delivery[1].
The European Reference Networks (ERNs) are virtual networks involving healthcare providers across Europe. They aim to facilitate discussion on complex or rare diseases and conditions that require highly specialised treatment and concentrated knowledge and resources[2]. The first 24 ERNs were launched in March 2017, involving more than 900 highly specialised healthcare units from over 300 hospitals in 26 EU countries. For the first time, ERNs gave clinicians and patient organisations the opportunity to co-design better healthcare services for people living with rare diseases (RDs) at an unprecedented scale. Clinicians and patient representatives from different countries involved in the networks are expected to collaborate in a wide range of activities that range from developing standards of care and quality of life measures to defining data access policies and conducting clinical research. While there are some good examples of good collaboration between ERN clinical leads and ePAG advocates (see Table 1 for a definition)[4-6], there is work ahead to build a similar patient-clinician partnership culture across the 24 ERNs.
Terminology
| Term | Definition |
| European Patient Advocacy Group (ePAG) | A patient group, specific to each European Reference Network (ERN), that coordinates patient involvement in the ERN and is composed of patient advocates who are active in different bodies and structures of the ERN governance framework (ERN Board, ERN Executive Committee, work streams and working groups) |
| ePAG advocates | Patient representatives who are endorsed by a patient organisation to be active in the ERN governance structure. They represent the interests of patients and families living with a rare disease that falls under the scope of a given ERN |
| ePAG Steering Committee | A Committee composed of ePAG advocates from the 24 ePAGs, coordinated by EURORDIS, to share experience and knowledge on patient engagement from across the 24 ERNs |
| ePAG leads | ePAG advocates from ITHACA, ReCONNET and RITA ERNs who committed to lead and coordinate the application of the measures in their own ePAG |
| Patient engagement in ERNs | The active, meaningful and collaborative interaction between patients and clinical leads in the ERNs’ activities and decisions, where the networks’ activities and decision making is guided by patients’ contributions as partners, recognising their specific experiences, values and expertise (definition adapted from[3]) |
| Measures | A set of metrics identified to assess the level and maturity of patient engagement in the ERNs in the context of an evaluation framework that interrelates each metric with a goal, intermediate outcomes and activities |
The lack of formal recognition in the EU legislation of patient representatives as ERN members left a vacuum regarding the mechanisms for patient engagement and the role of patient organisations and its representatives in these networks and the mechanisms to facilitate this collaboration. To address the first shortcoming and structure patient involvement in the ERNs, EURORDIS[7] and the rare disease patient community created in 2017 24 European Patient Advocacy Groups (ePAGs). Today, there are more than 300 ePAG advocates involved on a voluntary basis in these groups and all ERNs have at least one ePAG advocate in their board.
To address the uncertainty around the role of patient organisations and their representatives in the ERNs, EURORDIS and the ePAG Steering Committee, currently composed of 41 ePAG advocates, started to discuss in 2019 potential measures to evaluate the impact of patient engagement in the networks as a way to understand what worked, what was the value of patient-clinician partnerships and what were the areas for improvement. The loose delineation of patient representatives’ role in the ERNs, overall and in each of the specific collaborative activities, emerged as a challenge to measure the impact of the collaboration between patient representatives and clinicians. The lack of clear defined goals for patient engagement it hard to evaluate the patient representatives efforts in a meaningful way[8]. This is why the first step of the process initiated in 2019 was to clarify the role and goals of patient representatives in the ERNs, as a basis to develop a set of measures to evaluate this impact in terms of outcomes and facilitate continuous improvement in this area.
As such, the process had an important positive externality as it served to clarify and build a common understanding of the ePAG advocates’ goals (why they were engaging in the ERNs) and what were the key activities and processes that allowed them to fulfil their role (how they were engaging in the ERNs). Once this was established, EURORDIS supported the ePAG Steering Committee members to define a set of measures to assess progress towards the three main goals that they had identified as being the central reasons for engaging in the ERNs: (1) improve quality of care, diagnosis and treatment for rare diseases; (2) ensure patient-centric ERNs; and (3) ensure a good level of awareness among the patient community about ERNs and their work.
This paper presents the conclusions of this work: a patient-driven evaluation framework that can be used to assess the level of engagement of ePAG advocates in the ERN activities and how effectively they are working to fulfil their role. The framework tested in three different ePAGs to test the data collection tools and the relevance of the measures. This phase has not concluded, and therefore the results of this pilot phase are beyond the scope of this article.
As the commitment and resources of the ERNs to support meaningful patient engagement evolve, we hope to engage with the ERN clinical leads to co-design an evaluation framework to assess the quality and impact of patient engagement in the ERNs. The work described in this paper is an important first step towards co-building a more robust evaluation framework for patient involvement in the ERNs that could eventually be integrated in the ERNs quality improvement system.
Terminology
Throughout the article, the authors use different terms that are relevant to the European Reference Networks and the involvement of patient representatives in these Networks. Definitions for these terms are provided in Table 1.
METHODS
The ePAG Impact Assessment Framework was developed by ePAG advocates and EURORDIS through a participatory and iterative process. EURORDIS provided support with the methodological approach throughout the different phases of the process and facilitated the discussions. The process included three different phases (see Figure 1): (1) identify ePAG advocates’ common goals, intermediate outcomes and activities; (2) identify measures; and (3) test the measures.
Figure 1. Phases and participatory design.
Phase 1: Identify ePAG advocates’ common goals, intermediate outcomes and activities
The aim of this phase was to build a common understanding across ePAG advocates involved in different ERNs on a common set of goals and link them back to intermediate outcomes and to the activities that these patient representatives should perform to achieve the goals identified. In a face-to-face workshop with members of the ePAG Steering Committee, we articulated why they had decided to engage as patient representatives in the ERNs, what were short-term direct results that would show progress towards those goals and what activities they needed to focus on to achieve them (20 ePAG Steering Committee members involved in 14 of the 24 ERNs accepted the invitation to attend this workshop). To facilitate the discussion, participants used the logic framework developed by the UK National Health Service in England to measure the impact of health programmes[9]. The discussions focused on articulating what will change, how and why (Step 1 of the methodology in Figure 1). Some preliminary measures were also identified (Step 2 of the methodology in Figure 1).
Phase 2: Identify measures
A smaller Task and Finish working group composed, composed by five ePAG advocates from four ERNs, was created to review the results of the workshop and identify metrics for the different activities, outcomes and goals. Taking as a basis the results of the face-to-face workshop, four calls were held by the working group members and three with the ePAG Steering Committee to reach consensus on the final set of measures (see Table 2) between October 2019 and March 2020. EURORDIS facilitated these discussions and supported the ePAG advocates on the methodological aspects.
Set of measures grouped by goal
| ePAG advocates’ Goals | Intermediate outcomes | ePAG activities | n | Measure | Evidence of … |
| Improve quality of care, diagnosis and treatment | New knowledge or evidence is synthesised and shared through Clinical Decision Support Tools (CDSTs) and Clinical Practice Guidelines (CPGs) by the ERNs | Involved in reviewing existing or developing new CDSTs and CPGs | 1 | Listed as co-authors of ERN CDSTs and CPGs | ePAG advocates involvement in reviewing/developing ERN CDSTs and CPGs - Activity |
| Ensure patient-centric ERNs that respond to the rare disease patient community needs. Patient needs steer ERN activities and their structure | Represent and share our community needs and experience in the ERNs | Involved in ERN working groups with clinicians | 2 | Number of ERN working groups (WGs) with ePAG advocates participating as members of ERN WGs | ePAG advocates’ involvement in ERN WGs - Activity |
| Patient-centric registries | Involved in the ERN Registry governance | 3 | Patient representatives involved in ERN Registry governance structure or ERN Registry Task Force | ePAGs advocates’ involvement in the ERN Registry governance - Activity | |
| Clear understanding of patient community needs | Conduct surveys/develop patient journeys to understand unmet needs by thematic areas and share results with ERN clinicians | 4.1 | Number of patient journeys or surveys to capture patients’ needs that have been discussed with ERN clinicians | ePAG advocates activities’ to understand unmet needs and share results with ERN clinicians - Activity | |
| 4.2 | Percentage of sub-thematic disease areas where patient needs have been captured through a patient journey (or a survey on patients’ needs) and results have been discussed with clinicians | Coverage of ePAG advocates’ activities to understand unmet needs by thematic areas and share results with ERN clinicians - Activity | |||
| ePAG advocates and ERN clinicians understand advocates’ role and are able to effectively engage | Build teamwork and patient-clinician partnerships in the ERN | 5 | Perception or level of satisfaction on how ePAG advocates are working with clinicians as a team to advance the ERN goals | Teamwork and patient-clinician partnership is well-established in the ERN - Intermediate outcome | |
| ERNs measuring patient experience (PREMs), patient reported outcomes (PROMs) and health outcomes and using this information to inform CDSTs/CPGs | Involved in the identification of PREMs, PROMs and other health outcomes measures | 6 | Percentage of outcome measures identified with the input of ePAG advocates | ePAG advocates’ involvement in the identification of Patient Reported Experience Measures, Patient Reported Outcome Measures and other health outcome measures - Activity | |
| Use effectively own network to disseminate ERN information and developments | Engage with clinicians, HCPs and scientific societies to share ERN updates and activities | 7 | Number of posters/presentations on ERNs delivered in national or international meetings (conferences and workshops) | Engagement with clinical community to share ERN updates and activities - Activity | |
| Ensure a good level of awareness among the patient community about the ERNs and their work (so that all can benefit) | Engage regularly with ePAG Community [patient organisation (PO) not actively involved] | Organise outreach to patient community | 8.1 | Perception on how the ePAG has organised the outreach to its wider patient community (dissemination) | Mechanisms to communicate with patient community - Activity |
| Engage regularly with ePAG Community (PO not actively involved) | Formalise feedback loop | 8.2 | Perception on how the ePAG has engaged with patient community to capture their feedback on different ERN-related topics/work | Mechanisms to elicit patient community views - Activity |
First, the members of the working group discussed a preliminary set of 14 measures, identified by EURORDIS based on the outcomes of the workshop, prioritised and selected nine measures based on three criteria: (1) relevance of the measure in the short-medium term in the context of the work of the ERNs; (2) adequacy of the measure to measure the contribution of the ePAG; and (3) clarity of the measure. In a second step, the members of the working group discussed and rated together the selected measures in terms of their validity, reliability, feasibility and relevance. All measures were rated offline individually on a scale of 1-5, and the group later agreed on a final rating. Finally, each member ranked them according to their own opinion on the importance of each measure. Measurement methods were also discussed within the working group. The working group shared at three different points in time the results of their work with the ePAG Steering Committee to validate their ratings and collective ranking and refine the set of measures with the feedback received. Once the ePAG Steering Committee had reached a consensus on the set of nine measures, all individual ePAG groups were invited to rank the measures and discuss them with their coordinators and ERN project management teams. The feedback collected was discussed and analysed in a call with the ePAG Steering Committee to agree on a final list of eight measures, two of which were broken down into two sub-measures.
Phase 3: Test the measures
The objective of this phase was to apply the framework in several ePAGs to test its relevance and identify areas for improvement. Three groups were selected to test the framework: ERN RITA ePAG (RIPAG), ERN ITHACA ePAG and ERN ReCONNET ePAG. Each ePAG group designated two ePAG advocates to coordinate and lead the application of the framework in their respective ePAG. EURORDIS set up a working group to bring all of them together and provide support throughout where needed. The activities in this phase were organised around four tasks: (1) development of data collection tools; (2) data collection; (3) analysis of the results; and (4) recommendations. The results of the pilot phase will be presented in another article.
RESULTS
The framework (see Table 2) includes goals, activities and intermediate outcomes, metrics and measurement methods to assess the impact of patient engagement in the ERNs:
(1) Improve quality of care, diagnosis and treatment of rare diseases.
(2) Ensure patient-centric ERNs that respond to the rare disease patient community needs. Patient needs steer ERN activities and their structure.
(3) Ensure a good level of awareness among the patient community about ERNs and their work (so that all can benefit).
Initially, the ePAG advocates identified a fourth goal for their involvement in the ERNs as patient representatives: “Equity of access to quality care across Europe for rare disease patients”. However, it was difficult to establish a direct link between the activities developed by the patient advocates in the ERNs and changes in equity of access to healthcare for rare disease patients. Due to this attribution challenge, the consensus was to remove this goal from the framework.
Goal 1: Improve quality of care, diagnosis and treatment of rare diseases
This goal includes metrics to assess the level of involvement of ePAG advocates in the development of ERN guidance on care, diagnosis and treatment. The main tools used by the ERNs to offer this type of guidance are clinical decision support tools (Clinical decision support tools include clinical consensus statements, care pathways, evidence-based protocols, “do’s and don’ts” factsheets and quality measures) and clinical practice guidelines.
The metric “listed as co-authors of Clinical Practice Guidelines and Clinical Decision Support Tools” is complemented with additional information gathered to determine the type of involvement and input provided by ePAG advocates throughout the different stages of the process. For the ePAG advocates who led the application of the framework in their respective groups, this measure was perceived as the most relevant one due to the importance that they place on Goal 1 as well as on having their contributions formally recognised in peer-review publications. Following their suggestion, the initial definition of the measure, which read “listed as co-authors of Consensus statements and Clinical Practice Guidelines”, was modified to include any type of clinician decision support tool, and not just consensus statements. Going forward, after discussing the results of all three ERNs, they suggested further exploring the possibility to formulate the measure in even broader terms; to include any type of ERN publication addressing improvements in quality of care, diagnosis and treatment in general; and to include publications where ePAG advocates’ names appear in the acknowledgement section.
Goal 2: Ensure patient-centric ERNs that meet our community needs
The metrics included under this goal assess the mechanisms in place to facilitate the effective involvement of ePAG advocates in the work of the ERNs, such as registries, as well as their work to capture the patient community care needs and sharing and discussing them with clinicians. Most importantly, this set of metrics includes a subjective measure to assess the perception of patient representatives about their collaboration with clinicians in the respective ERNs.
Goal 3: Ensure a good level of awareness among the patient community about the ERNs and their work (so that all can benefit)
The measures clustered under this goal assess how the ePAG is performing its “bridge” function connecting the patient community to the ERNs and vice versa. Specifically, the measure included under this goal is broken down into two different sub-measures to assess the perception of the ePAG group around two different aspects: (1) how they engage with the patient community to help them understand how to access the ERNs’ information and advice; and (2) how they engage with the patient community to collect their feedback on topics that are under discussion in the ERN.
DISCUSSION
Value derived from developing the framework
While learning and accountability were identified as the main purposes for the framework, the work done to design the framework was served a third purpose, namely to further clarify the objectives of the ePAG advocates in the ERNs and identify the core activities they needed to focus on to fulfil their role as patient representatives. The workshop held in Phase 1 helped to crystalise a common understanding across the ePAG leads from 14 different ERNs of the role and goals of patient representatives in the ERNs. It is worth noting that some participants had difficulties understanding how the outcome of this workshop was later used to identify measures and following the logic of how these linked back to the activities, intermediate outcomes and goals.
Throughout Phase 2, when the measures were being developed, some ePAG Steering Committee members also expressed reservations to use the framework for different reasons. They either did not have the time to apply it or thought it was too early to use it given the limited level of engagement of ePAG advocates in their ERN.
Another important aspect discussed by the ePAG Steering Committee in the early stages was with whom to share the results. The group agreed that, in this first stage, each ePAG would share the results only with their own ERN clinical leads and ERN coordinating teams to engage with them and discuss any shortcomings and improvements on the ERN patient engagement strategy and related activities. In addition, the ERNs have different priorities reflected in the focus of their annual plans, therefore it was not recommended to use the results to compare ERNs’ performance on patient engagement, but rather to use them to share best practices where relevant.
Methodological strengths and limitations
The main strength of this framework is that patient representatives involved in different ERNs developed a common vision on the change that they want to effect as patient representatives in the networks, developed the measures and defined the measurement tools through a participatory process. Overall, this approach ensures that the aspects assessed in this framework are of relevance for the ePAG advocates directly involved in the work of the ERNs.
The main methodological weakness is the lack of meaningful involvement of ERN clinical leads in designing and implementation of the framework. Resources to facilitate their participation in this effort were limited. We invited all ePAG Steering Committee members to share and explain the framework to their ERN coordinators and ERN project managers. However, not all were able to engage with them and only two ERN coordinators provided feedback (the EpiCare and BOND ERN coordinators reviewed and ranked the measures together with their ePAG advocates). As a result, the selected measures are prone to bias, as they only capture aspects relevant to patient representatives.
Using the logic model depicted in Figure 2 helped ePAG advocates understand in a simple way what they were hoping to achieve and what they needed to do to get there. We also used a participatory process to populate the model. However, this measurement and evaluation approach also revealed to be limited in the sense that some of the activities identified could be contributing to different goals. For example, ePAG advocates’ involvement in the development of outcome measures is linked to Goal 2 (patient-centric ERNs), but this activity will certainly contribute to achieve Goal 1 (improve quality of care, diagnosis and treatment). The ePAG advocates piloting this framework have been encouraged to share the results and organise feedback sessions with ERN clinicians, but the model falls short of showing the bigger picture as it cannot capture the value of collective and shared contributions to achieving a certain goal. Going forward, using an alternative evaluation methodology based on a reflexive approach to refine the framework would possibly enhance its value as a tool for shared learning and reflection and be better adapted to capture the complexity of the environment within which the ERNs operate[10,11].
Figure 2. The phases’ logic model used to identify goals, outcomes and activities. f2f: Face to face.
The theoretical framework behind the tool is also weak, since we did not perform a literature review to inform the development of the tool and relied on our experience as ePAG advocates and the work done by EURORDIS to assess the impact of patient engagement in medicines development[11].
We built on the extensive knowledge and experience of EURORDIS and the ePAG Steering Committee members around the ERNs and the specific context of each ERN, but “assumptions” and “contextual factors” were not discussed in depth by the group, therefore the framework lacks specific measures rigorously evaluate the contextual factors and assumptions.
All the measures that were selected focus on the implementation and procedural aspects of patient engagement in the ERNs (e.g., whether ePAG advocates are participating in ERN working groups; whether they felt that they are collaborating effectively with the clinicians; or whether they have put in place outreach mechanisms to connect with their patient community). The literature has identified the lack of outcome measures as a limitation to present concrete evidence on the added value of patient engagement[6]. In the current stage of development of the ERNs, patient engagement is still in its infancy, and many processes and structures to support patient engagement are lacking. This partially explains why the framework does not include any outcome measures but rather focuses on tools and processes to enable patient engagement. As such, the application of the current measures does not allow evaluating how patient engagement contributes to achieving the intermediate outcomes and goals identified in the framework.
In the longer term, we expect the framework to evolve to a more outcome-oriented approach, that will eventually allow to assess how the collaboration between patients and clinicians has specifically contributed to achieve the ERN goals.
Finally, not all 24 ePAG groups were equally involved in the process, and not all ePAG Steering Committee members had the same level of understanding of what was being done and why. For example, when we asked all of them to rate the selected measures with their fellow ERN ePAG advocates in terms of their validity, reliability, relevance and feasibility, most of them were unable to do it without additional support. Therefore, only the ePAG advocates of the Task and Finish group were able to engage in this exercise. Setting up a small working group allowed us to progress at a good pace and ensure a good understanding of the concepts and the methods, but conveying the results of the work to the larger ePAG Steering Committee group and engaging with them required more time than expected.
Evaluation tools and frameworks are increasingly being developed to assess the impact of patient engagement in research and health system decision making. A systematic review conducted in 2018 concluded that most of them featured a limited participation of patients in their design and reporting and were not scientifically robust[12]. Overall, our framework was designed with a high level of involvement of patient representatives who are active in the ERNs and have a very good understanding of the context, but further work is needed to refine it and to improve its scientific rigor.
In conclusion, conducting high-quality patient engagement in the ERNs and evaluating its impact requires a major commitment from the ERNs. Additional and sustained organisational capacities are needed to plan, conduct and evaluate patient engagement in the ERNs. To enable a fruitful patient-clinician partnership, patient organisations, ePAG advocates and clinical leads must co-build the patient partnership strategy and develop together the tools and processes to support this engagement. They must also agree on their approach to evaluating the value of this collaboration. In doing so, they should establish partnerships to collaborate with social researchers, universities and other organisations that could enhance and complement the ERNs evaluation capacities and expertise. The framework and measures described in this paper could be used as a basis to co-design an evaluation framework for patient engagement in the ERNs informed by multiple perspectives. Ideally, such a refined framework could then be integrated into the future ERN quality improvement system as a way to support a meaningful patient-clinician partnership and ensure that the networks’ activities are driven by and remain responsive to patients’ needs.
DECLARATIONS
Acknowledgements
We want to thank the ePAG Steering Committee members for their contribution to this project, their continuous feedback and validation. Furthermore, we would like to thank Eugenia Durante and Julie Power (ERN RITA ePAG advocates); Alain Cornet and Ana Vieira (ERN ReCONNET ePAG advocates); Renée Jopp, Gabor Pogany and Sue Routhledge (ERN ITHACA ePAG advocates); Ruth Biller, Edward Callus and Ester Costafreda (ERN Guard-Heart ePAG advocates) for their contribution to this project; Matt-Bolz Johnson for workshop design and facilitation and Elisa Ferrer and Lidewij Eva Vat for their professional guidance in the project’s inception stage. Finally, we are grateful to our ERN (European Reference Network) Coordinators and coordinating teams, in particular to Anne Hugon and Prof. Alain Verloes (ERN ITHACA), Prof. Marta Mosca (ERN ReCONNET), Prof. Arthur A.M Wilde (ERN Guard-Heart), and Prof. Nico Wulfraat and Bart Uitterhaegen (ERN RITA) for all their support.
Authors’ contributions
Conceived and designed the analysis: Wiehe L, Aslanian AL, Martin IH
Collected the data: Marinello D, Galetti I, Dan D, Sundqvist Andersson A, Aguilera S, Louisse S
Performed the analysis: Marinello D, Galetti I, Dan D, Sundqvist Andersson A, Aguilera S, Louisse S, Martin IH
Wrote the paper: Marinello D, Aguilera S, Wiehe L, Aslanian AL, Martin IH
Availability of data and materials
The authors confirm that the data supporting the findings of this study are available within the article.
Financial support and sponsorship
None.
Conflicts of interest
All authors declared that there are no conflicts of interest.
Ethical approval and consent to participate
Not applicable.
Consent for publication
The manuscript does not contain any individual person’s data in any form.
Copyright
© The Author(s) 2021.
REFERENCES
1. Le Cam Y, Bolz-johnson M. Expert by experience: valuing patient engagement in healthcare. In: Pomey M, Denis J, Dumez V, editors. Patient engagement. Cham: Springer International Publishing; 2019. p. 233-67.
2. Public Health. Available from: https://ec.europa.eu/health/ern/networks_en. [Last accessed on 23 Aug 2021].
3. Harrington RL, Hanna ML, Oehrlein EM, et al. Defining patient engagement in research: results of a systematic review and analysis: report of the ISPOR patient-centered special interest group. Value Heal 2020;23:677-88.
4. Bolz-Johnson M, Meek J, Hoogerbrugge N. “Patient Journeys”: improving care by patient involvement. Eur J Hum Genet 2020;28:141-3.
5. Rosaria T, Sara C, Valentina L, et al. RarERN Path: a methodology towards the optimisation of patients’ care pathways in rare and complex diseases developed within the European Reference Networks. Orphanet J Rare Dis 2020;15:347.
6. Smith M, Alexander E, Marcinkute R, et al. Telemedicine strategy of the European Reference Network ITHACA for the diagnosis and management of patients with rare developmental disorders. Orphanet J Rare Dis 2020;15:103.
7. EURORDIS - The Voice of Rare Disease Patients in Europe. Available from: https://www.eurordis.org/. [Last accessed on 23 Aug 2021].
8. Abelson J, Humphrey A, Syrowatka A, Bidonde J, Judd M. Evaluating patient, family and public engagement in health services improvement and system redesign. Healthc Q 2018;21:61-7.
9. Impact framework. Available from: https://www.england.nhs.uk/sustainableimprovement/impact-framework/. [Last accessed on 23 Aug 2021].
10. van Mierlo B. Reflexive monitoring in action. A guide for monitoring system innovation projects. 2010. Available from: https://library.wur.nl/WebQuery/wurpubs/reports/395732. [Last accessed on 23 Aug 2021].
11. Vat LE, Finlay T, Robinson P, et al. Evaluation of patient engagement in medicine development: A multi-stakeholder framework with metrics. Heal Expect 2021;24:419-506.
Cite This Article
Export citation file: BibTeX | RIS
OAE Style
Marinello D, Galetti I, Dan D, Andersson AS, Aguilera S, Louisse S, Wiehe L, Aslanian AL, Martin IH. Patient engagement in healthcare: a preliminary set of measures to evaluate patient engagement in the European Reference Networks. Rare Dis Orphan Drugs J 2022;1:2. http://dx.doi.org/10.20517/rdodj.2021.001
AMA Style
Marinello D, Galetti I, Dan D, Andersson AS, Aguilera S, Louisse S, Wiehe L, Aslanian AL, Martin IH. Patient engagement in healthcare: a preliminary set of measures to evaluate patient engagement in the European Reference Networks. Rare Disease and Orphan Drugs Journal. 2022; 1(1): 2. http://dx.doi.org/10.20517/rdodj.2021.001
Chicago/Turabian Style
Marinello, Diana, Ilaria Galetti, Dorica Dan, Ammi Sundqvist Andersson, Silvia Aguilera, Simone Louisse, Lenja Wiehe, Anne-Laure Aslanian, Ines Hernando Martin. 2022. "Patient engagement in healthcare: a preliminary set of measures to evaluate patient engagement in the European Reference Networks" Rare Disease and Orphan Drugs Journal. 1, no.1: 2. http://dx.doi.org/10.20517/rdodj.2021.001
ACS Style
Marinello, D.; Galetti I.; Dan D.; Andersson AS.; Aguilera S.; Louisse S.; Wiehe L.; Aslanian A.L.; Martin IH. Patient engagement in healthcare: a preliminary set of measures to evaluate patient engagement in the European Reference Networks. Rare. Dis. Orphan. Drugs. J. 2022, 1, 2. http://dx.doi.org/10.20517/rdodj.2021.001
About This Article
Copyright
Data & Comments
Data
Cite This Article 37 clicks
Like This Article 30
likes
Comments
Comments must be written in English. Spam, offensive content, impersonation, and private information will not be permitted. If any comment is reported and identified as inappropriate content by OAE staff, the comment will be removed without notice. If you have any queries or need any help, please contact us at support@oaepublish.com.