Validation of the German version of the Needs Assessment Tool: Progressive Disease – Heart Failure

Valentina Gonzalez-Jaramillo (  valentina.gonzalez@ispm.unibe.ch ) University of Bern: Universitat Bern https://orcid.org/0000-0002-2470-7022 Jelena Guyer Spital Emmental Nora Luethi University of Bern: Universitat Bern Piotr Sobanski Spital Schwyz Rut Zbinden University of Bern: Universitat Bern Eveline Rodriguez tilia Köniz Lukas Hunziker Inselspital University Hospital Bern: Inselspital Universitatsspital Bern Steffen Eychmüller University of Bern: Universitat Bern Maud Maessen University of Bern: Universitat Bern


Introduction
Heart failure (HF) is a global pandemic currently affecting at least 26 million people worldwide. Its prevalence is growing as the population ages and other risk factors increase (1). Despite optimal guidelines and recommended therapies, and apart from uncontrolled disease-related symptoms such as shortness of breath, pain, sleep disorders, and fatigue (2), people living with HF often suffer from other conditions such as depression and anxiety (3). Moreover, taking care of or living with a person suffering from chronic HF can be stressful and burdensome (4). Both patients and their families are at increased risk of experiencing physical, emotional, and nancial burdens that may impair their quality of life (5).
Palliative care (PC) has been shown to improve the quality of life of patients and their families by preventing and alleviating suffering through early identi cation, evaluation, and treatment of physical, psychosocial, and spiritual problems associated with chronic illnesses such as HF (6). However, physicians and other health personnel face di culties in identifying patient needs for PC in the rst place. This presents a barrier to effective PC (7). Failure to assess such needs routinely and systematically in daily medical practice contributes to undertreatment and unnecessary suffering.
Several tools have been used to assess the need for PC in patients with HF. However, most of them focus on clinical characteristics and prognostic factors and do not take into account the needs of family members or caregivers (8)(9)(10)(11)(12). However, in 2013 Australian researchers developed the "Needs Assessment Tool: Progressive Disease-Heart Failure" (NAT: PD-HF) to help health care personnel identify the needs of patients with HF and their informal caregivers, and match them with the most appropriate services regardless of whether they may be psychology, social work, cardiology, specialized PC, or general medicine (13). The NAT: PD-HF, more than a questionnaire, is a direct, face-to-face interaction guide aimed to increase the attention dedicated to patients and their narratives in a highly e cient and effective way. Clinical environments that optimize this attention, will not only improve the quality of care and patient and staff satisfaction, but also reduce care costs. The implementation of NAT: PD-HF in the clinical practice could be a fundamental basis for strategies focusing on optimizing attention (14,15).
To date the NAT: PD-HF has been available in only English (13) and Dutch (16). We therefore 1) translated the tool into German and culturally adapted it, 2) assessed psychometric characteristics of the translated NAT: PD-HF that include internal consistency, inter-rater reliability, and test-retest reliability, 3) evaluated whether and how patient and health care personnel understood the tool and its utility, and 4) assessed the face validity, applicability, relevance, and acceptability of the German NAT: PD-HF among health care personnel.

Study design
We conducted this work at Inselspital, the University Hospital of Bern, Switzerland, a tertiary academic hospital with a dedicated HF clinic and a specialized palliative care service. Translation and validation of the original tool was performed in accordance with the guidelines of the European Organisation for Research and Treatment of Cancer (17). We performed separate forward and backward translations (Supplementary Material 1), and evaluated internal consistency, inter-rater reliability, and test-retest reliability.
We surveyed patients (Supplementary Material 2) and health care personnel (Supplementary Material 3) to gauge their level of understanding of the NAT: PD-HF and perception of its utility. Face validity, applicability, relevance, and acceptability of the tool by health care personnel were assessed with interviews of personnel specializing in cardiology and palliative care at our institution (Supplementary Material 4, Fig. 1). Sociodemographic and basic clinical information of the participants was extracted from electronic medical records.
The original NAT: PD-HF Tool The NAT: PD-HF is a comprehensive tool designed for health care personnel to assess both a patient's and a main informal caregiver's physical, psychological, social, and spiritual needs (13). It consists of three parts: the rst page is a user guide explaining the purpose of the tool and how to complete it, the second is a questionnaire comprised of 20 items, and the third page has examples of what to address within each item of the questionnaire (Supplementary Material 5). The questionnaire is subdivided into sections addressing four topics: 1) priority referral for further assessment, 2) patient well-being, 3) ability of the informal caregiver or family to care for the patient, and 4) caregiver well-being. For each question, health care personnel selects the level of concern: (1) none, (2) some/potential, (3) signi cant. In case there is some/potential or signi cant concern, the health care worker is prompted to choose an appropriate action: (1) directly manage it, (2) management by another care team member, or (3) refer to other providers such as the patient's general practitioner, social worker, psychologist, specialist PC service, cardiologist, or other.

Translation and cultural adaptation
The NAT: PD-HF was translated from English to German using a forward-backward translation procedure and cultural adaptation by four translators, one of which was a professional medical text translator (Supplementary Material 1). After pilot testing the translated tool with 19 patients, we made small adjustments to the translation, and in the recruitment process. The pilot test also revealed the need to train health care personnel who will use the tool. Because of the adjustments after the pilot test, we excluded pilot participants from the nal analysis.

Participants and recruitment
Eligible participants were adult patients (≥ 18 years of age) with an appointment at the Heart Failure Clinic at Inselspital Bern who had had at least one consultation in the clinic and who could uently communicate in German. No speci c stage or severity of HF was selected to ensure a representative, full spectrum of the disease.
Patients meeting the inclusion criteria received a study invitation package containing an invitation letter and study description, a participant information sheet and consent form, and a removable response card that patients could send back to the research team stating whether they were interested in taking part of the study or not. For those who were interested, we arranged an appointment with a study member immediately before or after their scheduled consultation at the HF clinic.
We evaluated patients' cognitive capacity by asking them three questions about the study after having explained its purpose and the content from the consent form. The questions were 1) what is the aim of the study? 2) In which patients will the study be performed? 3) As part of this study, will you have blood tests or ultrasounds? In case of satisfactory answers to the three questions, the interviewer proceeded with the signing of the consent form and the interview.
Study procedure Using the tool with HF patients Using the German version of the NAT: PD-HF a general practitioner trained in administering the tool directly queried the patient and, if present, an informal caregiver. From these data we estimated internal consistency. The encounter was recorded to allow a second evaluator to provide data for gauging interrater reliability using the audio recording. After 10 to 20 days, a second appointment was scheduled with the patient and the NAT: PD-HF was repeated face-to-face by the same evaluator who used the tool during the rst assessment to provide data for gauging test-retest reliability (Fig. 1).
Assessing understanding of the German NAT: PD-HF and its utility At the end of the study, we asked patients to tell us whether they thought the NAT: PD-HF questions were easy to understand and answer, and whether the questions might lead to better care. We obtained their answers via a ve-point Likert scale ranging from ''strongly agree'' to ''strongly disagree'' (Supplementary Material 2). We also asked about whether such questions are addressed in the course of their regular clinical consultations, and asked them what other questions they thought should be included in the tool.
We asked health care personnel representing all potential patient referral services to complete the same survey that patients completed (Supplementary Material 3).

Face validity, applicability, relevance and acceptability
Interviews were conducted with the group of health care personnel to assess face validity, applicability, relevance, and acceptability of the German NAT: PD-HF (Supplementary Material 4).

Data analysis
Internal consistency and sample size We used Cronbach's alpha to assess internal consistency. Sample size calculations were based on Feldt´s formula (18). To have 80% power at an alpha error of 5%, a lowest acceptable Cronbach´s alpha value of 0.75, and an expected value of 0.85, the sample size calculated was 66. Estimating a dropout rate of 10% we aimed to recruit 75 patients.

Inter-rater reliability and test-retest reliability
To assess the agreement between the results obtained from the physician using the NAT: PD-HF the rst time and the results when the second evaluator lled out the tool listening the recordings, it is the interrater reliability, we used Cohen`s kappa. Since our data were ordinal, we used weight in such a way that the difference between "No concern" and "Some/potential concern" is less than the difference between "No concern" and "Signi cant concern." We similarly regarded the difference between "No concern" and "Some/potential concern" as greater than the difference between "Some/potential concern" and "Signi cant concern" since, in the second case, at least, the professionals agreed that some additional action should be taken. (The NAT: PD-HF instructions state that the professional using the tool should act on each identi ed need). We weighted the agreement of the two evaluators according to the following matrix: No concern Some/potential concern Signi cant concern No concern 1 0.2 0 Some/potential concern 0.2 1 0.8 Signi cant concern 0 0.8 1 The frequency of some/potential and signi cant concerns were low for some of the NAT: PD-HF items (Supplementary Table 1). We therefore decided to additionally report the results of the prevalenceadjusted and bias-adjusted kappa (PABAK) to avoid obtaining in ated agreements due to bias introduced by those low frequencies (19).
To assess the agreement between the results of the physician using the tool the rst time and the results of the same physician using the tool the second time, it is the test-retest reliability, we estimated Cohen`s.
For this calculation, we used the same weights presented in the matrix above, as well as PABAK.
We interpreted inter-rater and test-retest reliabilities as near-perfect agreement if the kappa was greater than 0.81, as substantial if the kappa was between 0.61 and 0.80, moderate if it was between 0.41 and 0.60, and poor if it was less than 0.40 (20).
Unlike the rst evaluator, the second evaluator was not always the same. Therefore, we did a sensitivity analysis to assess the inter-rater reliability for each one of the two second evaluators and de ne whether the data could be analyzed as a whole or if we needed to account for second evaluator differences.

Survey of patients and health care personnel
To summarize surveys results, "Strongly agree" and "Agree," were pooled with each other and "Disagree" and "Strongly disagree" were pooled with each other. The original ve-point Likert scale was thus converted into a three-point scale for the analysis. All analyses were performed using STATA release 15 (Stata Corp, College Station, Texas).

Participants
Between December 2019 and March 2020, 200 patients meeting the inclusion criteria were invited to participate. Among those invited, 70 patients consented to participate, giving a recruitment rate of 35% (Fig. 2). Though men predominated among the invitees, women (12/30, 40%) were slightly more willing to participate than men (58/170, 34%) (Fig. 2). There was no loss to follow-up.
A large majority of the patients included in the study were men (58/70, 83%); the mean age of all participants was 62.0 years (SD 13.7) (   Table 1).

Tool translation
The culturally adapted German translation of the NAT: PD-HF is presented in Supplementary Material 1.
Psychometric characteristics of the German NAT: PD-HF

Internal consistency
With a Cronbach's alpha of 0.83, the internal consistency was high. Furthermore, when removing each question to evaluate how the alpha changed without each item Cronbach's alpha ranged from 0.80 and 0.84. Details of the psychometric characteristics are presented in Table 2. We observed no differences in the performance of the different rst and second evaluators. Therefore, there was no need to account for second evaluators' differences in the analyses (Supplementary Table 2).

Test-retest reliability
The median time between the two appointments was 15 days (IQR [14][15][16][17][18][19][20] and none of the patients reported any signi cant change in his or her condition between the two appointments. Test-retest reliability of the German NAT: PD-HF was very high and all questions had almost perfect agreement, with Cohen`s kappa ranging from 0.82 to 1.00 ( Table 2).

Survey of patients
All of the 70 participants agreed that the tool questions were easy to understand and answer. Sixty-one patients (87%) thought that discussing the issues raised in the questions may improve quality of care. Only three patients reported these questions being routinely asked or discussed during clinical consultations (Table 3). Regarding other questions that should be included in the tool, the most frequently mentioned topics were lifestyle habits and sexuality. More speci cally, patients suggested inquiry about exercise and nutrition habits, and whether there have been changes in a couple's sex life and there are side effects of medication that interfere with it.

Survey of health care personnel
We interviewed 27 health care professionals. These included eight cardiologists, ve cardiology nurses, four PC specialists, three PC nurses, three psychologists from the HF clinic, two general practitioners, and two social workers. Among the 27 interviewees, 21 were women, and 14 of the interviewees have worked in the profession for more than 10 years.
In general, the health care personnel rated the questions easy for patients to understand, but not to answer. Most (23 of the 27 professionals) considered the tool helpful for improving the quality of patient care. Eleven of the health care personnel reported that the questions in the tool are usually asked during a clinical consultation, while the remaining 16 responded in a neutral manner, neither agreeing nor disagreeing. The four PC specialists agreed that topics from the tool are discussed during a routine consultation with a PC specialist. In contrast to that, only two of the cardiologists said topics from the tool are discussed in consultations, while the remaining six said that most "are not addressed," and that "psychosocial aspects are not discussed" (Table 3).
Face validity, applicability, relevance and acceptability

Face validity
Face validity was rated high, with most of the health care personnel (23 of 27) agreeing that the tool appears to measure unmet needs of both patients with HF and their caregivers. The remaining four interviewees partially agreed, but thought that to properly assess caregiver needs the caregiver needs to be present in the interview (Table 4).

Applicability
Among the 27 health care personnel, 17 (63%) consider the tool easy to use, and all but two consider the written instructions on use the tool helpful. However, some thought the tool and its instructions were too long and too detailed.
PC specialists, psychologists, and social workers think that doctors (general practitioners, cardiologists, PC specialists) as well as nurses can use the tool, while cardiology staff think that mainly nurses should use it.

Relevance
Two PC specialists, one general practitioner and a psychologist, each questioned the relevance of some elements of the tool. One of the PC specialist was not sure what question 2.5 should cover. The other PC specialist said that question 2.7 is not very useful since it collapses different factors into a single question, and suggested that section three, which is about the ability of caregiver to care for the patient, and section four, about caregiver well-being, should be merged. The GP thought question 3.6 is not very clear and should be rephrased, and the psychologist thought that questions 2.3 and 2.5 have similar content (Table 4).

Acceptability
There was no consensus on either when or how often the NAT: PD-HF should be employed in clinical practice. Some believe that it should be used early, at the rst or second consultation, while others believe that it is better not to use it early, but only after a basis of trust has been built with the interviewer and the patient has already dealt with the disease. Some respondents think it should be used at regular intervals such as every six months or at annual check-ups, while others think it should be used again after major changes in health status or after hospitalization ( Table 4).
The main concerns about accepting the NAT: PD-HF are lack of time to use it, and that being able to use it requires some training or at least a very thorough introduction to using it and its value (Table 4).
Another reservation about using the NAT: PD-HF reported by cardiologists and cardiology nurses was that some staff members do not feel comfortable discussing emotional, spiritual, or nancial issues with their patients (Table 4).

Discussion
Key results We translated the NAT: PD-HF into German, culturally adapted, and validated it. The validation showed good internal consistency and substantial inter-rater agreement for the majority of the items. Additionally, we were able to assess the test-retest reliability and we found almost perfect agreement between the rst and second assessments. Moreover, patients thought well of the tool, and they agreed that it could help to improve their quality of care and that it covered relevant topics that are not normally addressed in clinical consultations. Similarly, face validity and user-friendliness were rated highly by health care professionals. However, like the participating health care personnel we also believe training is necessary to ensure correct use of the tool.

NAT: PD-HF strengths
Other such tools based on patient prognosis ignore the needs that patients with longer prognoses may have, and how di cult it is to predict disease trajectory in HF (8-10). The NAT: PD-HF evaluates patient needs in a more integrated and profound way. It assesses not only patient well-being and information needs, but also the needs and well-being of family members and informal caregivers. A recently published systematic review of tools to assess PC needs in patients with HF concluded that the NAT: PD-HF was the most appropriate tool to assess the needs of patients with HF (21). The English version of the tool had already demonstrated a good correlation between evaluators (13) and good acceptability by health professionals (13,16). Our study supports the previous ndings and additionally shows that the NAT: PD-HF is stable over time and that patients regard it as useful.

NAT: PD-HF limitations
We observed no consensus across medical staff on when to use the tool for the rst time, nor on how often to use it. However, a recently published position statement from the European Association of Palliative Care proposed an algorithm for when to employ the NAT: PD-HF. The association also recommended assessing PC needs in patients in less advanced stages of HF at annual reviews or after any signi cant health-related event for those in more advanced stages (22).
Although the tool was designed to be used without training, experience with the staff in this study showed that prior training is necessary. Training was mainly on how to discuss the needs of the patients and the informal caregiver in an empathic, respectful and e cient way by applying situation-speci c adaptations of the example sentences of the instruction manual and on how to decide on cut-off points of the scoring system. In a similar study assessing the NAT: PD-HF in the Netherlands, nurses trained in using the tool still requested they referred that more training to assess PC needs (16). A qualitative study that evaluated barriers in the implementation of the Needs Assessment Tool: Progressive Disease-Interstitial Lung Disease (NAT: PD-ILD), also found the need for training for the correct application of that tool (23).
Although the majority of the questions had a substantial inter-rater agreement, the last question of section two, question seven, had a Cohen´s kappa of 0.17, which is a poor correlation. This nding is consistent with the validation of the English version of the NAT: PD-HF and with the validation of the NAT: PD-Cancer (13,24). That question aiming to identify barriers for the health care delivery, tries to assess multiple factors, namely social factors, health beliefs, and cultural beliefs from both the patient and the family.
Implications for future research Its high face validity suggests that the translated tool appears to measure unmet needs of patients with HF and their informal caregivers, and both medical staff and patients think the tool could help to improve quality of care. However, its effectiveness in reducing unmet needs and consequently even improving quality of life has not yet been studied. It would be interesting to prospectively study the effectiveness of this tool in reducing unmet needs and increasing quality of life in clinical practice.

Implications for clinical practice
More than one-third of the patients cited a need for further information. During interviews some staff also mentioned that they did not feel comfortable discussing certain topics. These reports from patients and staff together suggest that improving doctor-patient communication may assist using the NAT: PD-HF more effectively in clinical practice to recognize unmet needs and improve daily life for people living with HF. As attention is linked to higher-quality communication, one way to improve communication would be to adjust the conditions of clinical practice so that patients and their narratives receive su cient attention for the physician to fully understand their situation or need (14).

Conclusion
The German NAT: PD-HF is a reliable, valid, and internally consistent tool that is well accepted by both patients and health care personnel. However, it is important to keep in mind that effective use of the tool requires training of health care personnel. Study procedure. Adobe Illustrator Draw was used create the artwork

Supplementary Files
This is a list of supplementary les associated with this preprint. Click to download. SupplementarymaterialHQLO.docx