Sally Bennett, John Bennett, Mal Boyle, Jeff Coombes, Craig Lockwood, Lisa Nissen, Marie Pirotta, Sharon Sanders and Jemma Skeat
This chapter is an accompaniment to the previous chapter (Chapter 10) where the steps involved in answering a clinical question about patients' experiences and concerns were explained. In order to further help you learn how to appraise the evidence for this type of question, this chapter contains a number of worked examples of questions about patients' experiences and concerns.
The Critical Appraisal Skills Program (CASP) qualitative checklist1 for critically appraising qualitative research has been used as a guide in each of the examples that follow, with additional comments added from the QARI instrument at the end of each example.
The CASP checklist is lengthy, and so for the purpose of brevity we have omitted the two screening items at the beginning of the checklist that ask: ‘Was there a clear statement of the aims?’ and ‘Is a qualitative methodology appropriate?’. The authors who contributed examples to this chapter had already screened their chosen articles, and therefore we have not provided comments on these two items and you will see that each appraisal begins with item 3 of the checklist.
It is important to understand that the CASP checklist for qualitative research provides a technical/pragmatic appraisal rather than a theoretical appraisal (as the QARI instrument does). The CASP tools were developed from guides produced by the Evidence-Based Medicine Working Group and published in the Journal of the American Medical Association.2 One of the guides addresses qualitative research.3 This guide does not consider theoretical perspectives of qualitative research, but asks the reader to judge the methodological rigour of the report by critically appraising the research/study design and analysis. You will note that the third item in the CASP checklist also asks about research design, and this needs clarification as it is subject to interpretation. Specifically it asks: ‘Was the research design appropriate to address the aims of the research?’ It also prompts the reader to consider whether ‘the researcher has justified the research design (for example, have they discussed how they decided which methods to use?)’. There are no specific guidelines about what the CASP tool means by ‘research design’ or ‘methods to use’. One could interpret this as meaning ‘identify which research methodology was used’, e.g. phenomenology, grounded theory, discourse analysis and so on. However, the perspective taken from the User's guide3 is that ‘research design’ is about the practicalities of how the research was carried out, not which type of qualitative research methodology was used. In the examples that follow, we have therefore taken both approaches where relevant information was reported. As the CASP checklist does not ask about the philosophical perspectives of the researcher, we have added relevant comments from the QARI tool at the end of each worked example.
Although there are other checklists available for appraising qualitative evidence that have a slightly different focus, there is no consensus about the ideal approach that should be used. An explanation and comparison of the features of the QARI tool and the CASP checklist was provided in Chapter 10.
Clinical scenario
You have been working as a therapy manager in the multidisciplinary rehabilitation unit of a major metropolitan hospital for people with acquired brain injury, and want to understand the experience of patients of the inpatient rehabilitation setting as you think this might help inform ongoing refinements and developments within the rehabilitation unit. You are planning to meet with all staff on a rolling basis about how things might be improved in the unit, and decide to find studies that have specifically sought patients' views.
How do people who have had an acquired brain injury and their family members perceive the inpatient rehabilitation experience?
Search terms and databases used to find the evidence
Database: PubMed—Health Services Research Queries (with ‘qualitative research’ category and ‘narrow scope’ selected), which is accessed through the ‘Topic-specific queries’ link on PubMed.
Search terms: (brain injury) AND rehabilitation AND inpatient
This search strategy retrieved 10 articles. Two very recent articles focused on inpatient rehabilitation for people who have an acquired brain injury. One article considered a residential rehabilitation unit and the other considered the experience of an inpatient rehabilitation unit from the perspective of patients and their family members. This is the article that you choose to appraise.
Fleming J, Sampson J, Cornwell P, et al. Brain injury rehabilitation: the lived experience of inpatients and their family caregivers. Scand J Occup Ther 2012;19:184–93.
Structured abstract (adapted from the above)
Study design: This study was part of a larger qualitative research study about transitions from hospital to home for people with acquired brain injuries. This study reports on a subset of questions from this larger study, pertaining to their experiences of the rehabilitation unit, using a phenomenological approach.
Study question: To describe and interpret the inpatient brain injury rehabilitation experience from the perspective of patients and their carers.
Context: The inpatient rehabilitation ward was a secured ward in a large metropolitan hospital in Australia. It was a 26-bed ward with shared and single rooms, and had a dining room, recreational space, physiotherapy gym and outdoor courtyards. The multidisciplinary rehabilitation team included medical and nursing staff, occupational therapists, physiotherapists, speech pathologists, a social worker and a neuropsychologist.
Participants: Criterion-based purposeful sampling was used until data saturation was reached, resulting in a sample size of 20 patients and 18 nominated family carers.
Data collection method: Semi-structured interviews using an interview guide were conducted separately with participants and their nominated carers in a quiet room in the final week prior to discharge from the rehabilitation unit. The interview questions informing this study were the initial questions from the larger study intended for building rapport and obtaining background information regarding rehabilitation received up to the point of hospital discharge.
Analysis: Analysis of the interviews was conducted using a ‘manifest content analytic’ approach. The transcribed interview data were approached with the question: ‘How do patients and family perceive the inpatient brain injury rehabilitation unit?’ Analysis occurred as follows. Two researchers independently extracted content related to the question from one interview transcription. Once accuracy of extraction was confirmed, one author extracted relevant data from the remaining transcripts. Data was organised by two researchers independently into core meanings (meaning units), which were then grouped and condensed into representative codes. Codes were then used to identify themes and categories, and all researchers met to establish a framework of themes and categories. This framework was then used by one author to code the remaining interviews. Emerging themes and codes were discussed if they arose.
Key findings: Three themes were identified: ‘rehabilitation context/environment’, ‘activity/occupation’ and ‘support and adjustment’. The rehabilitation context/environment theme encompassed concerns about the limitations/restrictions of the physical environment and positive and negative perceptions related to organisational factors and staff attitudes. The theme of activity/occupation reflected the perceived importance of activities related to therapy sessions and other structured activities contrasting with the boredom experienced by some participants during unstructured time regardless of unstructured activities being available to them. The support/adjustment theme emphasised the pivotal role played by family members and friends in providing support to patients that was unable to be provided by staff, carers' need for information, and barriers experienced by carers in terms of communication and difficulty accessing staff.
Conclusion: This study improves understanding of rehabilitation from the patient's and family members' perspective and can assist rehabilitation teams to better meet the needs of people with acquired brain injury. Specifically highlighted was the need for a more therapeutic environment, meaningful occupations, and family-centredness in inpatient settings.
Is the evidence rigorous and sufficiently reported?
Detailed questions from CASP
3 Was the research design appropriate to address the aims of the research?
Consider:
• Whether the researcher has justified the research design (for example, have they discussed how they decided which method to use?).
The authors clearly indicate that a phenomenological methodology was used and the justification for this was provided in the article. This study utilised semi-structured interviews with patients and nominated family carers in the week prior to patients being discharged from rehabilitation. The design and methods used for this study were clearly justified by the authors and addressed the study aims (‘to describe and interpret the inpatient brain injury rehabilitation experience from the perspective of patients and their caregivers’).
4 Was the recruitment strategy appropriate to the aims of the research?
Consider:
• Whether the researcher has explained how the participants were selected.
Criterion-based purposeful sampling was used to select ‘information-rich’ participants who met a predetermined set of criteria, from whom comprehensive and detailed data regarding the rehabilitation experience were collected. Participants were recruited from a brain injury rehabilitation unit at a large metropolitan hospital in Australia.
• Whether they explained why the participants they selected were the most appropriate to provide access to the type of knowledge sought by the study.
Some information is available about this through the eligibility criteria (for example, they needed to be current inpatients in a brain injury rehabilitation unit and able to communicate satisfactorily); it is also clear that those selected were appropriate for providing information with respect to the aims of this study.
5 Were the data collected in a way that addressed the research issue?
Consider:
• Whether the setting for data collection was justified.
Participants were interviewed in the week prior to discharge home from the brain injury rehabilitation unit. Interviews were undertaken in the occupational therapy department and thus data was collected in a way that could address the research issue.
• Whether it is clear how data were collected (for example, focus group, semi-structured interview etc).
Data were collected using semi-structured interviews separately with patients and a nominated family member.
• Whether the researcher has justified the methods chosen.
Authors indicate that they used interviews in an attempt to gain a deep understanding of the phenomena.
• Whether the researcher has made the methods explicit (for example, for interview method, is there an indication of how interviews were conducted, or did they use a topic guide?)
Although the authors explained the data collection process clearly, this study was part of a larger study and so it was not clear what specific questions were asked of the participants relevant to this component. The authors just state that this article was based on: ‘a set of initial questions based around participants’ experiences in the rehabilitation unit'. Referring to earlier articles did not improve the clarity of this issue.
• Whether methods were modified during the study. If so, has the researcher explained how and why?
There is no mention that the research methods were changed during the study period.
• Whether the form of data is clear (for example, tape recordings, video material, notes, etc).
It is clearly stated that interviews were tape-recorded and field notes taken.
• Whether the researcher has discussed saturation of data
The saturation principle was used to determine the sample size of the study. Saturation was identified as the point at which no new information (that is, major themes) was emerging from the data-collection process, and this was determined through collaborative discussion between members of the research team.
6 Has the relationship between researcher and participants been adequately considered?
Consider:
• Whether the researcher critically examined their own role, potential bias and influence during:
– Formulation of the research questions.
There is no comment about this in the article.
– Data collection, including sample recruitment and choice of location.
The authors only state that ‘recruitment and data collection were completed by an occupational therapist who had experience at the rehabilitation unit, but was not employed by the hospital’. However, there is no actual examination of her role beyond this statement.
• How the researcher responded to events during the study and whether they considered the implications of any changes in the research design.
No changes occurred. The authors note that the injuries of participants may have affected their ability to clearly articulate and reflect upon their experiences as required during the interview process. However, they also note that triangulation with family carer interviews reduced the impact of bias that this might have introduced.
7 Have ethical issues been taken into consideration?
Consider:
• Whether there are sufficient details of how the research was explained to participants for the reader to assess whether ethical standards were maintained.
All participants in the study signed a written consent statement.
• Whether the researcher has discussed issues raised by the study (for example, issues around informed consent or confidentiality or how they have handled the effects of the study on the participants during and after the study).
This is an important issue for a population who have sustained brain injuries and have related cognitive impairments. However, the ability of the participants to provide informed consent was determined by the patient's treating occupational therapist. Patients were excluded from the study if there was evidence of a premorbid psychiatric illness or cognitive impairment that impeded their ability to provide informed consent.
• Whether approval has been sought from the ethics committee.
8 Was the data analysis sufficiently rigorous?
Consider:
• Whether there is an in-depth description of the analysis process. Whether thematic analysis is used. If so, is it clear how the categories/themes were derived from the data?
This is very clearly explained in the paper. Analysis of the interviews was conducted using a ‘manifest content analytic’ approach.
• Whether the researcher explains how the data presented were selected from the original sample, to demonstrate the analysis process.
This was not clearly explained; however, representative quotes were given in table format for each of the themes and categories.
• Whether sufficient data are presented to support the findings. To what extent contradictory data are taken into account.
Sufficient data were presented and both positive and negative perspectives were reported.
• Whether the researcher critically examined their own role, potential bias and influence during analysis and selection of data for presentation.
The researcher did not critically examine their own role or potential biases and influence during analysis and selection of data.
9 Is there a clear statement of findings?
Consider:
• Whether the findings are explicit. Whether there is adequate discussion of the evidence both for and against the researcher's arguments.
The authors presented a balanced discussion of the issues derived from participants' statements, reflecting both their positive and their negative perceptions.
• Whether the researcher has discussed the credibility of their findings (for example, triangulation, respondent validation, more than one analyst).
The authors report features that improve the credibility of their findings, including triangulation of data from family members and patients, using respondent validation and involving more than one analyst in at least a few aspects of data analysis.
• Whether the findings are discussed in relation to the original research question.
The findings were very clearly discussed in relation to the original objective of the research.
10 How valuable is the research?
Consider:
• Whether the researcher discusses the contribution the study makes to existing knowledge or understanding (for example, do they consider the findings in relation to current practice or policy, or relevant research-based literature?).
The authors discuss the implications of this research and how it might inform rehabilitation team members, managers, and those designing rehabilitation units of this type. They also consider their findings in the light of contemporary research findings.
• Whether they identify new areas where research is necessary.
The authors indicate that further research may address the impact of secured wards in brain injury rehabilitation, time use during inpatient rehabilitation settings and the associated issues of boredom and the psychosocial adjustment of individuals during post-acute rehabilitation.
• Whether they identify new areas where research is necessary. Whether the researchers have discussed whether or how the findings can be transferred to other populations or considered other ways the research may be used.
The authors suggest that further research might be necessary to confirm whether these findings are common across brain injury rehabilitation units or specific to the unit in this study. However, the principles of considering environmental design, occupation and family-centred practice are relevant across many different settings.
Additional comments from the QARI instrument
In addition to the CASP criteria, it would also be helpful to consider congruity between the philosophical position adopted in the study, study methodology, representation of the data and interpretation of the results as outlined in QARI. While the study does not explicitly state that an interpretive perspective is taken when trying to understand the experiences and views of people who have received inpatient rehabilitation for acquired brain injuries and their family carers, the authors do state clearly that a phenomenological perspective was taken and this is an appropriate methodology for an interpretive philosophy. The objectives of the study are congruent with this methodology, as are the methods used to collect and analyse the data.
People who receive inpatient rehabilitation following brain injury and their family members perceive the rehabilitation environment as restrictive and that patients' motivation was influenced by both positive and negative organisational factors and staff attitudes. Therapy sessions and related activities were valued as contributing towards recovery, but boredom was experienced by participants during unstructured time, contributing to a feeling of frustration. It was clear that family members and friends provided pivotal support to patients that was unable to be provided by staff. Carers identified a need for information about brain injury, the process of rehabilitation and discharge, specific treatment goals and outcomes, and available community services. However, they experienced a number of barriers in terms of communication during the inpatient rehabilitation time, and difficulty accessing staff.
How might we use this evidence to inform practice?
As the authors indicate, this study improves our understanding of rehabilitation from the patient and family member's perspective and can assist rehabilitation teams to better meet the needs of people with acquired brain injury. Specific issues to consider in future developments of the rehabilitation unit in our clinical scenario (and perhaps other rehabilitation units that might be of a similar nature to that described in this study) are how the physical environment might be enhanced to reduce the sense of restriction while maintaining safety standards, considering how to provide more meaningful and structured activities and occupations during inpatient stays and the importance of enacting a family-centred approach in inpatient settings with systematic processes to support clear and consistent communication between all.
Clinical scenario
You are a member of a recently established multidisciplinary team of health professionals who are involved in the management of patients with diabetic foot complications. During a case conference, you begin a discussion about a patient who has developed a new ulcer on the plantar surface of her foot after a recent digital amputation. While the patient has a pair of therapeutic shoes that can accommodate pressure-deflective devices, she does not wear them. Recognising that many of the patients who are seen by the team do not appear to wear their prescribed footwear or well-fitting, protective non-prescription shoes as advised, the endocrinologist asks whether you have any suggestions about how to tackle this problem. Before coming up with any strategies, you say that first you would like to gain an understanding of people's attitudes to, beliefs about and experiences with prescribed footwear. You think this understanding may help you come up with strategies that might improve patients' concordance with footwear advice. You agree to present your findings at the next case conference.
What are people's experience with prescribed footwear, and what factors determine whether they will follow footwear advice?
Search terms and databases used to find the evidence
Database: PubMed and CINAHL.
Search terms: (footwear OR shoe*) AND (perspective* OR attitude* OR view* OR experience*) in the study title.
This search retrieves 20 articles in PubMed and 17 in CINAHL. Looking through the titles, there are several studies that have examined people's experiences with prescribed footwear or have investigated factors associated with a person's decision to follow footwear advice. You think it will be important that you review them all for your presentation. To begin with, you select the following study which gives the views of patients with diabetes and of health professionals caring for people with diabetes-related foot complications on the use of therapeutic footwear.
Johnson M, Newton P, Goyder E. Patient and professional perspectives on prescribed therapeutic footwear for people with diabetes: a vignette study. Pat Educ Couns 2006;64:167–72.
Structured abstract (adapted from the above)
Study design: Qualitative analysis of semi-structured interviews using the framework approach.
Study question: To explore the experiences and views of people involved in prescribing and wearing footwear designed to reduce diabetes-related complications.
Context: Diabetes foot clinics, North Sheffield, UK.
Participants: 15 health professionals with experience in delivering care to people with diabetes-related foot complications and 15 patients from two diabetes foot clinics who had been prescribed footwear designed to reduce diabetes-related complications.
Data collection method: Participants were interviewed using a vignette depicting the experiences of a fictitious patient with diabetes-related foot complications. Prompts were used after each vignette section to explore participant views.
Analysis: Interviews were taped and analysed using the framework approach.
Key findings: There was incongruence between patients and health professionals in terms of the expectations and reality of preventive behaviour. Health professionals, while advocating preventive measures in order to limit morbidity, are aware of how difficult this can be for patients. For patients, the reality of wearing prescribed footwear conflicted with personal and social norms. Both patients and health professionals expressed concerns about shoe fit when foot shape, size and structure are constantly changing.
Is the evidence rigorous and sufficiently reported?
Detailed questions from CASP
3 Was the research design appropriate to address the aims of the research?
Consider:
• Whether the researcher has justified the research design.
There was no clear statement of the chosen research methodology. This study aimed to explore issues identified by health professionals and patients of diabetic foot care clinics that have impacts on the use of therapeutic footwear. The arrangement of conditions for collecting and analysing data in this study is congruent with the aims of the research.
4 Was the recruitment strategy appropriate to the aims of the research?
Consider:
• Whether the researcher has explained how the participants were selected.
A purposive sampling strategy was used to identify health professionals from a range of disciplines and patients of varying ages from a range of socio-economic circumstances who may or may not have used prescribed footwear. In addition, health professionals identified a purposive sample of patients from the two diabetes foot clinics.
• Whether they explained why the participants they selected were the most appropriate to provide access to the type of knowledge sought by the study.
The researchers explain that these participants provided a broad spectrum of beliefs and attitudes to therapeutic footwear. The recruitment strategy is appropriate to the aims of the research, although whether (and why) any invited participants declined to participate in the study is not discussed.
5 Were the data collected in a way that addressed the research issue?
Consider:
• Whether the setting for data collection was justified.
The setting for data collection was described: health professionals were interviewed in one of two hospital sites or in primary care, and patients were interviewed at their home or at one of the hospital-based diabetes foot clinics; however, the justification for this was not provided. One could assume that these locations were selected for convenience.
• Whether it is clear how data were collected.
In this study, individual interviews were conducted with patients and health professionals using a vignette depicting the experience of a fictitious patient with diabetes-related complications.
• Whether the researcher has justified the methods chosen.
The researchers believe that this approach facilitated the expression of opinions, beliefs and attitudes in a less personal and non-threatening way. The researchers also justified their use of individual interviews as opposed to focus groups.
• Whether the researcher has made the methods explicit.
Individual interviews with health professionals and patients were conducted using a vignette. Prompts were used after each vignette section to explore participant views. Responses were then followed up as necessary to increase the clarity and depth of information.
• Whether methods were modified during the study. If so, has the researcher explained how and why?
There is no mention that the research methods were changed during the study period.
• Whether the form of data is clear.
Yes. It is stated that interviews were tape-recorded.
6 Has the relationship between researcher and participants been adequately considered?
Consider:
• Whether the researcher critically examined their own role, potential bias and influence during:
– Formulation of the research questions.
There is no comment about this issue.
– Data collection, including sample recruitment and choice of location.
The article does not specify who conducted the interviews with study participants, nor the possible professional or personal relationships between the researcher and participants; and how this may have influenced recruitment and data collection is not considered.
• How the researcher responded to events during the study and whether they considered the implications of any changes in the research design.
7 Have ethical issues been taken into consideration?
Consider:
• Whether there are sufficient details of how the research was explained to participants for the reader to assess whether ethical standards were maintained.
Details were not provided in this article but are provided in an earlier article by these authors.
• Whether the researcher has discussed issues raised by the study.
• Whether approval has been sought from the ethics committee.
8 Was the data analysis sufficiently rigorous?
Consider:
• Whether there is an in-depth description of the analysis process. Whether thematic analysis is used. If so, is it clear how the categories/themes were derived from the data?
A detailed description of the process of analysis was provided and the researchers adequately described how the study findings emerged from the data. The interviews were analysed using the framework approach. In this method, after becoming familiar with the data, investigators classified the data according to key themes and concepts. In this study, two researchers carried out this process with a sample of transcripts before applying the framework to the remaining data.
• Whether the researcher explains how the data presented were selected from the original sample, to demonstrate the analysis process.
The researchers drew on quotes from different participants to validate the identified themes; however, the validity of the interpretations does not appear to have been examined.
• Whether sufficient data are presented to support the findings.
The extent to which contradictory data are taken into account is not clear. Sufficient data were provided to support the findings, but there were no clear examples of contradictory data presented.
• Whether the researcher critically examined their own role, potential bias and influence during analysis and selection of data for presentation.
The beliefs and values of researchers are not described and the potential influence of these on data analysis is not known.
9 Is there a clear statement of findings?
Consider:
• Whether the findings are explicit. Whether there is adequate discussion of the evidence both for and against the researcher's arguments.
Yes. The study concluded that patient perspectives need to be taken into account in shoe provision. The researchers' conclusions are clearly drawn from the key themes they identified and inform the people prescribing footwear designed to reduce diabetes-related complications.
• Whether the researcher has discussed the credibility of their findings.
Two researchers developed the thematic framework from which to classify or ‘code’ the data and applied it to the data. No report of respondent validation or triangulation was noted.
• Whether the findings are discussed in relation to the original research question.
Yes. The findings were very clearly discussed in relation to the original objective of the research.
10 How valuable is the research?
Consider:
• Whether the researcher discusses the contribution the study makes to existing knowledge or understanding.
This study makes a useful contribution to the limited body of research which has explored the experiences and views of health professionals and patients regarding therapeutic footwear. The researchers discussed how these findings relate to existing research and current practice.
• Whether they identify new areas where research is necessary.
The authors recommend that further research, as part of research with larger samples, explore the issues involved for patients with diabetes who receive self-care advice.
• Whether the researchers have discussed whether or how the findings can be transferred to other populations or considered other ways the research may be used.
The researchers do mention transferability of the findings, suggesting that the small sample in this study prevents generalisation to a larger population of people with foot complications. However, they stated that the issues raised are transferable to similar settings.
You reflect on your appraisal of this study and, while the design and methods of data collection were generally appropriate, there was inadequate detail within this report to make clear judgments on the relationship of the researchers to participants and how this may have influenced recruitment, data collection and analysis.
Additional comments from the QARI instrument
While the study does not clearly state the philosophical or theoretical premises on which it is based, it appears to take an interpretive perspective in trying to understand the experiences and views of people involved in prescribing and wearing footwear that is designed to reduce diabetes-related complications. It would be suited to phenomenological methodology, although this was not stated as such. The objectives of the study are congruent with this methodology, as are the methods used to collect and analyse the data.
The researchers present two themes which summarise issues relating to prescribed footwear that are important to patients and health professionals. The first theme relates to prescribed versus routine behaviour. There is incongruence between health professional recommendations regarding footwear and patients' behaviour, with patients finding it difficult to carry out advice that does not fit in with usual behaviour or with that of others. The second theme explores issues related to the fitting of footwear when foot shape, size and structure may be constantly changing. This was problematic for prescribers and patients. The researchers concluded that the insensitiveness of prescribed shoes to the varying contexts in which they are to be used, or the varying shapes of the patient's foot, may be the reason for low levels of use of prescribed footwear.
How might we use this evidence to inform practice?
The researchers discuss how the results of this study might be used in practice and suggest that negotiating the extent of use of prescribed footwear, providing varied styles of therapeutic footwear for different contexts, and more-detailed evaluation of shoe fit may improve the acceptance of prescribed footwear. Furthermore, involving patients in footwear design and obtaining views about the acceptability of the recommendations may help in developing more-realistic aims that are relevant to the patient.
There is limited information provided by which you can compare your setting with the study setting. However, the demographic details of the study sample seem similar to those of your patients in terms of their age, duration of diabetes and the type of intervention being received. There is no further detail on the types of footwear that participants in this study had been prescribed.
You reflect on discussions that you have had with patients about their footwear. Some of the experiences described in the study seem consistent with what your patients have talked about. Many patients with prescribed therapeutic shoes (shoes that have features such as extra depth, broad sole, deep toe box) comment negatively on the appearance and many complain that the shoes are not suitable for the hot climate in which they live. They also report difficulty being able to put the shoes on. Before moving on to the next study, you draw up a table which you will use to summarise the methods and results of the studies that you have found. You plan to present this at the next case conference, using it to stimulate discussion on possible strategies to improve concordance with footwear advice.
Clinical scenario
Mr Fallon, a 34-year-old man with a moderate to severe stutter, has previously attended your clinic and has returned today for advice. He has previously received individual speech therapy for 6 months, but was disappointed when this did not ‘cure’ his fluency difficulties. He would like to discuss some other options for therapy, particularly a ‘prolonged speech’/'smooth speech' group program which he has heard about. From this program, he wants to see a difference for himself, is concerned at being able to maintain and transfer the techniques in everyday life after the program, and wants to ‘sound normal’. You wonder what the evidence is for patient-perceived benefit from prolonged-speech approaches to therapy for dysfluency.
How do adults with moderate to severe chronic dysfluency experience the outcomes of prolonged-speech approaches?
Search terms and databases used to find the evidence
Database: PubMed (although CINAHL or Embase would also be appropriate to search).
Search terms: ‘stutter*’ AND (prolonged speech) AND ‘experience*
This returns three articles, one of which was relevant to the question. This article was chosen because a scan of the title and abstract revealed that it was potentially the most appropriate to the question, as it investigated the experiences of adults who had undergone prolonged speech therapy.
Cream A, Onslow M, Packman A, et al. Protection from harm: the experience of adults after therapy with prolonged-speech. Int J Lang Communi Disord 2003;38:379–95.
Structured abstract (adapted from the above)
Study design: Qualitative investigation using phenomenology as the method of inquiry.
Study question: What are the experiences of adults who use prolonged speech to control stuttering?
Context: Sydney, Australia. Therapy based on prolonged speech had been available through the speech pathology departments of public hospitals in Sydney for some time.
Participants: 10 adults (aged 24–54 years; 9 male) who had received prolonged-speech treatment. Participants were purposively sampled, on the basis of representing different viewpoints or experience. However, all had objectively experienced success with the prolonged-speech treatment, as a criterion for inclusion was that ‘they had experienced zero stuttering at the end of treatment’.
Data collection method: Face-to-face interviews and informal discussions (face-to-face or over the telephone) were held with participants over a 2-year period. At the end of the study, participants attended group interviews for further discussion. Interviews were ‘conversational’ in nature, with an open-ended rather than a structured format. A total of 34 interviews/discussions were included in the data.
Analysis: Content analysis was used to determine emerging themes. Transcripts were analysed using, consecutively, a line-by-line approach (to identify all emergent themes), a holistic approach (to understand the overall meaning of the interviews/discussions) and a selective approach (aiming to discover the primary themes). Additional data were sought and analysed to gain further understanding of experiences of people who have undertaken prolonged speech treatment. Analysis occurred concurrently to data collection and emerging themes were discussed with participants.
Key findings and conclusion: Even after prolonged-speech treatment, there is an ongoing risk of stuttering occurring. Although patients may have experience of being able to control stuttering, they also continue to feel different from people who do not stutter and these feelings may be exacerbated after treatment. The maximum benefits of prolonged speech are achieved when patients use a strategic approach to control stuttering and everyday communication.
Is the evidence rigorous and sufficiently reported?
Detailed questions from CASP
3 Was the research design appropriate to address the aims of the research?
Consider:
• Whether the researcher has justified the research design.
The authors have used phenomenology as the methodology for the research. The methods used for this study were clearly justified by the authors and addressed the study aims. The authors collected data over a period of 2 years, which supports their understanding of how patients' experiences with prolonged speech change over time.
4 Was the recruitment strategy appropriate to the aims of the research?
Consider:
• Whether the researcher has explained how the participants were selected.
The article has clear detail about the purposive sampling methods in which each participant was recruited because of their potential representation of different viewpoints.
• Whether they explained why the participants they selected were the most appropriate to provide access to the type of knowledge sought by the study.
The article contains a list of reasons for selecting participants (for example, having had therapy experience outside of the research centre, or having had experience of a non-intensive model of therapy). Recruitment decisions were based on the need to explore themes more fully.
5 Were the data collected in a way that addressed the research issue?
Consider:
• Whether the setting for data collection was justified.
Participants were recruited in the context of ongoing treatment and participation in support groups in Sydney, but justification of the setting is not provided.
• Whether it is clear how data were collected.
The study used formal and informal individual interviews, as well as group interviews, and the authors describe and justify the interview format (a ‘conversational’ approach). Authors explain the strategies used within the interviews (for example, encouraging patients to reflect on their own experiences rather than generalising by suggesting what ‘people who stutter’ might feel).
• Whether the researcher has justified the methods chosen.
The authors explained the importance of helping participants focus on specific experiences for phenomenological research so that their own experiences are captured.
• Whether the researcher has made the methods explicit.
A conversational approach was used, in which the interviewer followed the participant's lead. Hermeneutic interviewing was used, in which analysis occurred concurrently with interviews and emerging themes were discussed with participants.
• Whether methods were modified during the study. If so, has the researcher explained how and why?
There is no mention that the research methods were changed during the study period.
• Whether the form of data is clear.
It is clearly stated that interviews were tape-recorded.
6 Has the relationship between researcher and participants been adequately considered?
Consider:
• Whether the researcher critically examined their own role, potential bias and influence during:
– Formulation of the research questions.
– Data collection, including sample recruitment and choice of location.
The article states that the ‘interviewer (the first author) attempted to bracket assumptions about therapy that were based on clinical experience as a speech pathologist, professional reading and contact with persons who stutter’.
• How the researcher responded to events during the study and whether they considered the implications of any changes in the research design.
7 Have ethical issues been taken into consideration?
Consider:
• Whether there are sufficient details of how the research was explained to participants for the reader to assess whether ethical standards were maintained.
There was only one comment in the article referring to maintaining ethical standards, indicating the voluntary nature of the interviews and ability to withdraw data: ‘Participants were able to turn off the tape recorder at any time or request some parts to be omitted.’
• Whether the researcher has discussed issues raised by the study.
• Whether approval has been sought from the ethics committee.
Approval from an ethics committee was not reported. However, it is unlikely that this did not occur given that this paper was published in 2003. It is more likely to be a reporting omission.
8 Was the data analysis sufficiently rigorous?
Consider:
• Whether there is an in-depth description of the analysis process. Whether thematic analysis is used. If so, is it clear how the categories/themes were derived from the data?
There is a clear description of data analysis. To identify themes, three approaches were used: transcripts were analysed line-by-line and by ‘holistic’ and ‘selective’ approaches.
• Whether the researcher explains how the data presented were selected from the original sample, to demonstrate the analysis process.
This is not clearly explained.
• Whether sufficient data are presented to support the findings.
Sufficient data are provided to illustrate the themes. There were no contradictory data as such, but it was clear that participants had positive and negative perspectives on the use of prolonged speech that were described in the findings.
• Whether the researcher critically examined their own role, potential bias and influence during analysis and selection of data for presentation.
This was not done explicitly for this aspect of the study; however, evidence of reflexivity in earlier phases of this study (through bracketing) may potentially have carried over to the analysis phase.
9 Is there a clear statement of findings?
Consider:
• Whether the findings are explicit. Whether there is adequate discussion of the evidence both for and against the researcher's arguments.
A balanced discussion of the findings for and against therapy with prolonged speech is evident.
• Whether the researcher has discussed the credibility of their findings
The study incorporated member checking/discussion of interpretations with participants, prolonged engagement with the participants and triangulation of data with other data sources (such as websites). For example, the authors stated that they tested their developing insights against ‘descriptions of the period after PS contained in the newsletters of the Speak Easy Association and the Internet Stuttering Home Page’.
• Whether the findings are discussed in relation to the original research question.
The findings were very clearly discussed in relation to the original objective of the research.
10 How valuable is the research?
Consider:
• Whether the researcher discusses the contribution the study makes to existing knowledge or understanding.
The study discusses the findings in relation to informing prospective patients of the experiential consequences of prolonged speech, selecting patients who might benefit and in terms of assisting patients to achieve optimal benefits from it. They also consider these findings in relation to existing research.
• Whether they identify new areas where research is necessary.
The authors also consider implications of the research in relation to the structure of prolonged-speech treatments overall, as well as in relation to future prolonged-speech outcome research.
• Whether the researchers have discussed whether or how the findings can be transferred to other populations or considered other ways the research may be used.
The researchers raise the need to study this same issue in other age groups to see whether similar perspectives are evident. No further discussion of transferability is provided, but researchers do suggest that this research might inform existing and future developments in prolonged-speech approaches overall.
Additional comments from the QARI instrument
The authors have used a constructivist paradigm for their research, a theoretical standpoint where reality and truth are understood to be ‘constructed’ differently by every individual, rather than something ‘objective’ that can be described or understood (and generalised). The authors have used phenomenology as the methodology for the research. Phenomenology is appropriate to discovering what makes up others' experiences, and attempting to get to the ‘essence’ of these experiences. The research aims, methodology, data collection and analysis are all congruent with the philosophical approach selected and chosen methodology.
The themes that emerged describe interwoven experiences of adults after prolonged-speech therapy. As prolonged-speech therapy does not ‘cure’ stuttering, participants reported that their feelings about their stuttering, and the need to ‘protect themselves’ from the negative consequences that they associate with stuttering, does not diminish even after success with prolonged-speech therapy. Being fluent speakers ‘has a price’ for participants, as prolonged speech feels unnatural to produce and may also sound unnatural. Thus, participants were bound to be ‘different’ whether they used prolonged speech or stuttered. Prolonged speech became a ‘tool’ that participants could apply, along with their own existing resources, to protect themselves from harm.
The authors concluded that successful use of prolonged speech from a patient's perspective is the strategic application of the technique in order to reduce harm from stuttering. In some situations, patients may choose the risk of sounding different by stuttering rather than the risk of sounding different through prolonged speech. The findings suggest that practising the technique (either in therapy or in booster/maintenance programs) is not likely to increase use of the technique in the long term. Clinical programs to teach prolonged speech should focus not just on the behavioural changes, but also on the integration of the technique alongside existing resources that the patient has for reducing harm. Clinicians should consider whether patients have the resources to benefit from prolonged speech. For example, patients who are already considerably anxious about being different may not be suitable for this type of treatment.
How might we use this evidence to inform practice?
In the case of Mr Fallon, caution should be used in recommending a prolonged-speech therapy course to him, given his existing anxiety about ‘sounding normal’ and his emphasis on maintaining any therapy technique that he tries. Mr Fallon should be counselled about the realistic benefits that he might perceive from using prolonged speech. For example, although he might be able to use the technique well, he may find it unnatural and will have to weigh up the benefits of applying the technique versus stuttering. It should be emphasised that prolonged speech is not a ‘cure’ for stuttering. If Mr Fallon does choose to begin prolonged-speech therapy, he should be directed towards groups/clinicians who emphasise integration of prolonged speech in everyday life, rather than simply ‘teaching’ patients how to be good users of the technique.
Clinical scenario
As a new general practitioner working in a practice with a largely elderly demographic, you have seen a number of older people with chronic back pain whose pain has not been adequately controlled. One lady in her 80s concerns you in particular. She has a 3-year history of non-malignant lower back pain and evidence of spinal stenosis, has received physiotherapy and has been on extended-release paracetamol and paracetamol/codeine tablets for this period of time with little benefit. You want to reduce her pain, as she reports great difficulty carrying out even basic daily activities, has to receive a lot of assistance from her family and her pain is interfering with her sleep. This is confirmed by her daughter who attends the appointment with her. Knowing that anti-inflammatories have disagreed with her in the past, you wonder about the potential for opioid use with this lady but also decide to consider the prescribing experiences of other doctors in this patient group.
What are the experiences of prescribing opioids for elderly people with non-malignant chronic pain?
Search terms and databases used to find the evidence
Database: PubMed—Health Services Research Queries (with ‘qualitative research’ category and ‘narrow scope’ selected), which is accessed through the ‘Topic-specific queries’ link on PubMed.
Search terms: opioid* AND (elderly or “older adult”) AND (“primary care” OR “general practice”)
This search returned seven articles. One looked directly relevant to the question, so you obtain the full text to look at in detail.
Spitz A, Moore A, Papaleontiou M, et al. Primary care providers' perspective on prescribing opioids to older adults with chronic non-cancer pain: a qualitative study. BMC Geriatr 2011;11:35.
Structured abstract (adapted from the above)
Study design: The authors described this as a ‘qualitative cross-sectional study’.
Study question: To describe primary care providers' experiences and attitudes towards prescribing opioids for older adults with chronic pain, and to identify perceived barriers and facilitators to their prescription.
Context: Doctors and nurse practitioners were recruited from two academically affiliated primary care practices with patient populations that are older (mean age in the mid-80s) and predominantly female. They were selected because providers had substantial experience caring for older adults. Additionally, providers were recruited from three community health centres that serve predominantly Latino patients. All settings were located in New York City.
Data collection method: Six focus groups were conducted with a total of 23 doctors and three nurse practitioners.
Analysis: Focus groups were audiotape-recorded and transcribed. The data were analysed using directed content analysis. These were generated from the data and in reference to related literature.
Key findings: Most participants (96%) employed opioids as therapy for some of their older patients with chronic pain, although not as first-line therapy. Providers identified the main barriers to prescribing opioids were fear of causing harm, the subjectivity of pain, stigma, patient/family member reluctance to try opioids and lack of knowledge.
Conclusion: Providers use opioids cautiously and perceive multiple barriers to prescribing them for older adults with chronic pain. There is a need to establish the long-term safety and efficacy of these medications, develop and implement provider and patient education about their prescription and understand more about communicating with patients about opioids in order to improve the management of chronic pain for older adults.
Is the evidence rigorous and sufficiently reported?
Detailed questions from CASP
3 Was the research design appropriate to address the aims of the research?
Consider:
• Whether the researcher has justified the research design.
The authors simply state that this study was a ‘qualitative, cross-sectional study that employed focus groups to generate discussions’. The specific research methodology used was not evident. Their sampling choices supported the research aims of the study. There was no explanation provided about their choice of using focus groups.
4 Was the recruitment strategy appropriate to the aims of the research?
Consider:
• Whether the researcher has explained how the participants were selected.
All doctors and nurse practitioners at the identified centres were invited to participate and were simply recruited based upon their interest and availability (that is, convenience sampling) to attend one of the focus group sessions.
• Whether they explained why the participants they selected were the most appropriate to provide access to the type of knowledge sought by the study.
The only information offered regarding this was that providers at two of the settings had substantial experience caring for older adults.
5 Were the data collected in a way that addressed the research issue?
Consider:
• Whether the setting for data collection was justified.
The settings were justified. The study states that ‘Focus groups were conducted at the Columbia Presbyterian Allen Pavilion Division of Geriatrics and the New York-Presbyterian (NYP) Wright Center on Aging. In addition one focus group was also conducted with ‘providers from three community health centers within the NYP Ambulatory Care Network that serve predominantly Latino patients.’ The first two settings were selected because ‘they include providers with substantial experience caring for older adults’. The ambulatory-care settings were selected to increase the diversity of study sites.
• Whether it is clear how data were collected.
Six focus groups were conducted with a total of 23 doctors and three nurse practitioners.
• Whether the researcher has justified the methods chosen.
No justification was provided for the use of focus groups.
• Whether the researcher has made the methods explicit.
One researcher moderated the semi-structured focus group discussion, using open-ended questions and probes developed following a review of the literature and pilot testing. The specific questions that were asked during the focus groups are provided.
• Whether methods were modified during the study. If so, has the researcher explained how and why?
There is no mention that the research methods were changed during the study period.
• Whether the form of data is clear.
It is stated that interviews were tape-recorded.
• Whether the researcher has discussed saturation of data.
The authors indicate that focus groups were conducted until researchers agreed that no new themes were emerging.
6 Has the relationship between researcher and participants been adequately considered?
Consider:
• Whether the researcher critically examined their own role, potential bias and influence during:
– Formulation of the research questions.
There is no comment about this issue.
– Data collection, including sample recruitment and choice of location.
• How the researcher responded to events during the study and whether they considered the implications of any changes in the research design.
7 Have ethical issues been taken into consideration?
Consider:
• Whether there are sufficient details of how the research was explained to participants for the reader to assess whether ethical standards were maintained.
No details are provided other than the study was passed by an ethics review board.
• Whether the researcher has discussed issues raised by the study.
The authors acknowledge that the focus groups may not have accurately captured the full extent of participants' views on this topic, as participants were colleagues. One can speculate here that the concern may be about revealing true attitudes, beliefs or practices about prescribing opioids. Concerns about legal/regulatory sanctions were not frequently raised in this study, but it is unknown whether this is due to lower likelihood of local administrative sanctions in this setting or to confidentiality concerns between participants.
• Whether approval has been sought from the ethics committee.
8 Was the data analysis sufficiently rigorous?
Consider:
• Whether there is an in-depth description of the analysis process. Whether thematic analysis is used. If so, is it clear how the categories/themes were derived from the data?
Data were analysed via directed content analysis. Transcripts were read and preliminary themes generated. Interestingly, this process took into consideration themes (such as barriers to opioid prescribing) that have been identified in previous studies. It is possible, therefore, that some themes presented were not uniquely derived from the data.
• Whether the researcher explains how the data presented were selected from the original sample, to demonstrate the analysis process.
This was done in a very basic, limited manner with one specific example provided in the paper.
• Whether sufficient data are presented to support the findings. To what extent contradictory data are taken into account.
Perspectives on both barriers and facilitators to opioid prescription were ascertained during focus groups, and there was some indication of differing perspectives arising among focus group participants on a few issues (for example, misuse of opioids).
• Whether the researcher critically examined their own role, potential bias and influence during analysis and selection of data for presentation.
9 Is there a clear statement of findings?
Consider:
• Whether the findings are explicit. Whether there is adequate discussion of the evidence both for and against the researchers' arguments.
This was a descriptive study seeking to describe perceived barriers and facilitators to prescribing opioids and, as such, the author did not have any stated position. Findings were clearly presented.
• Whether the researcher has discussed the credibility of their findings.
Triangulation of themes evident in related research was used and data were analysed independently by more than one researcher. There was no description of respondent validation.
• Whether the findings are discussed in relation to the original research question.
Yes. The findings were very clearly discussed in relation to the original objective of the research.
10 How valuable is the research?
Consider:
• Whether the researcher discusses the contribution the study makes to existing knowledge or understanding:
Yes. The authors discuss the clinical implications of the findings in relation to provider training, communication with patients and development of risk assessment tools and guidelines.
• Whether they identify new areas where research is necessary:
This study identified the need for research about the long-term safety and efficacy of opioids in different subgroups of older patients and how they respond to varying opioid medication regimens.
• Whether the researchers have discussed whether or how the findings can be transferred to other populations or considered other ways the research may be used.
The authors emphasise that most providers cared for predominantly older adults of white or Hispanic origin, and indicate that the findings may not be transferable to other race/ethnicity groups or non-elderly populations.
Additional comments from the QARI instrument
In this study there is no statement regarding the philosophical position taken or the specific research methodology selected, so it is not possible to determine congruence between the underlying philosophical perspective and research methodology of the study. This study aimed to describe primary care providers' experiences and attitudes towards prescribing opioids for older adults with chronic pain, and thus most likely would be considered within the ‘interpretive’ paradigm and would be suited to phenomenological methods, although this was not stated as such.
Opioids were prescribed by most participants for some of their older patients with chronic pain, but not as first-line therapy. This study found that primary care providers use opioids cautiously with the elderly due to a number of different barriers. These included the older patient and/or their family member being reluctant to take an opioid; fear of causing harm; the subjective nature of pain and the associated difficulty of accurately determining pain severity; stigma; and the possibility of opioid abuse by carers (such as family members and home attendants). Facilitators were provider education, specialist support, evidence-based guidelines, and studies demonstrating long-term benefit.
How might we use this evidence to inform practice?
This study broadened your understanding of the barriers (and facilitators) to prescribing opioids in the older adult population. In relation to your specific patient, you weigh up the risks associated with opioids (such as increased risk of fracture, hospitalisations and all-cause mortality) against the fact that your patient has an identifiable cause of the pain, has very restricted participation in everyday activities and is distressed by the pain. You decide to discuss the option of a careful trial of opioids with your patient and her daughter who brings her to see you, to ascertain their perspectives about trialling opioids.
Clinical scenario
As an experienced nurse who works in a wound clinic with an emphasis on treating chronic vascular ulcers, you have noticed that treatment seems to concentrate on the application of therapies such as compression bandaging. Yet you are aware that some of the patients who are treated in the clinic have had their leg ulcer for a long time, have experienced significant pain, lifestyle alteration and discomfort and are reluctant participants in their therapy. The reasons for low concordance with wearing compression bandages, exercising and elevating their leg are unclear. You suspect that improved concordance could be achieved if the treating team had greater insight into the experiences of patients, particularly patients with ulcers of vascular origins, as many of their wounds on initial presentation are very large and poorly self-managed.
What is the lived experience of a person with chronic venous leg ulcers trying to adhere to advice during the transition to healing?
Search terms and databases used to find the evidence
Database: PubMed—Health Services Research Queries (with ‘qualitative research’ category and ‘narrow scope’ selected), which is accessed through the ‘Topic-specific queries’ link on PubMed.
Search terms: (leg ulcer*) AND adher*
This search returned 26 hits. You choose the one which appears to most closely match your question and scenario as it explores the issues of adherence to recommendations.
Van Hecke A, Grypdonck M, Beele H, et al. Adherence to leg ulcer lifestyle advice: qualitative and quantitative outcomes associated with a nurse-led intervention. J Clin Nurs 2011;20:429–43.
Structured abstract*
Study design: A qualitative evaluation approach and pre-/post-test design were used.
Participants: 26 patients with venous ulcers in a community care setting.
Data collection method: Interviews and participant observation. Frequency and duration of wearing compression, leg exercising and leg elevation, activity level, pain and ulcer size were registered at baseline, after the end of the intervention and 3 months later.
Analysis: Inductive content analysis (for the qualitative data).
Key findings: Knowledge about leg ulcer advice increased. The education contributed to patients more consciously following the advice. The rationale of the advice and its association with healing or recurrence often remained unclear. More patients performed exercises after the intervention and at follow-up. Patients often looked out onto a ‘new’ perspective where enhancement of quality of life and even healing might be attainable. Some patients regained independence after learning how to apply and remove compression garments themselves. No significant changes were reported on hours wearing compression.
Conclusions: The perceived changes suggest that the intervention holds promise for current home care. Combining qualitative and quantitative research assisted in determining the possible effects of the intervention, increasing the potential for a meaningful randomised trial in the future.
Is the evidence rigorous and sufficiently reported?
Detailed questions from CASP
3 Was the research design appropriate to address the aims of the research?
Consider:
• Whether the researcher has justified the research design.
In this study, the authors describe using a qualitative evaluation approach (combined with a pre- and post-test design; however, we are focused on the qualitative evaluation for this scenario). The authors gave a limited description of collecting qualitative data using a semi-structured interview process.
4 Was the recruitment strategy appropriate to the aims of the research?
Consider:
• Whether the researcher has explained how the participants were selected.
The study described inclusion criteria. They chose patients with known non-adherence with limited comorbidities. Purposive selection is a typical approach for qualitative research, as it ensures that the participants are able to participate more fully in responding to the research questions.
• Whether they explained why the participants they selected were the most appropriate to provide access to the type of knowledge sought by the study.
Purposive sampling was used to recruit people for the intervention (and thus also for the qualitative component of the study), to include patients with different characteristics (age, gender, education level, duration of leg ulcer, recurrence rate, pain) and in a different context (amount of social support, living alone versus cohabiting), although the authors did not explicitly explain why.
5 Were the data collected in a way that addressed the research issue?
Consider:
• Whether the setting for data collection was justified.
Interviews took place in the participant's home environment (in which they may have felt more comfortable).
• Whether it is clear how data were collected.
25 semi-structured interviews (with some open-ended questions) were conducted.
• Whether the researcher has justified the methods chosen.
This is not clearly reported, although the questions started with an open-ended question in order to ‘encourage patients to tell their experiences and perceptions’.
• Whether the researcher has made the methods explicit.
A topic guide was used with open-ended questions. The questions were derived from themes established in previous studies. To encourage patients to relate their experiences and perceptions, the interview started with a general and open-ended question (‘What was it like for you to receive the intervention?’). Questions regarding the depth of the open-ended questions are unresolved, as there was a lack of description other than this one question.
• Whether methods were modified during the study. If so, has the researcher explained how and why?
There is no mention that the research methods were changed during the study period.
• Whether the form of data is clear.
Yes. It is clearly stated that interviews were tape-recorded. Interview durations ranged between 18 and 81 minutes and were audiotaped, then transcribed verbatim.
• Whether the researcher has discussed saturation of data.
There was no mention of data saturation in this study, and it must be highlighted that while 18 minutes is very short for a qualitative interview, 81 minutes is almost too long, and participants may have been too fatigued to give full and detailed answers by the late stage of the interview.
6 Has the relationship between researcher and participants been adequately considered?
Consider:
• Whether the researcher critically examined their own role, potential bias and influence during:
– Formulation of the research questions.
– Data collection, including sample recruitment and choice of location.
The authors indicate that the researcher was in contact with the patients for a considerable period of time and that this could have increased the positive nature of the qualitative findings, as patients could feel an obligation to help the researcher. However, no other features of the relationship are considered and so we are unable to draw many conclusions as to the extent, nature and context of the influence.
• How the researcher responded to events during the study and whether they considered the implications of any changes in the research design.
7 Have ethical issues been taken into consideration?
Consider:
• Whether there are sufficient details of how the research was explained to participants for the reader to assess whether ethical standards were maintained.
While this study was scant on detail regarding the issues behind informed consent and data protection, there was clear reference to approval by an ethics committee.
• Whether the researcher has discussed issues raised by the study.
• Whether approval has been sought from the ethics committee.
Yes. It is reported that approval from an ethics committee was provided.
8 Was the data analysis sufficiently rigorous?
Consider:
• Whether there is an in-depth description of the analysis process. Whether thematic analysis is used. If so, is it clear how the categories/themes were derived from the data?
The qualitative component of this article included a clear and detailed description of how data analysis was managed (using specified software). The authors reported using a three-step approach to thematic analysis of coding, categorisation and higher-order theme generation. Researcher triangulation was described as the method for establishing trustworthiness of the resultant themes. Emerging themes were confirmed against the transcripts to ensure that the emerging findings were congruent with the participants' voices.
• Whether the researcher explains how the data presented were selected from the original sample, to demonstrate the analysis process.
Quotes from participants were used to inform and illustrate the themes. The level of description about this process was substantive, and gives a clear indication of a detailed and robust process for the methods by which data analysis was undertaken.
• Whether sufficient data are presented to support the findings. To what extent contradictory data are taken into account.
While there are sufficient data to support the findings, only very limited accounts of contradictory findings are presented.
• Whether the researcher critically examined their own role, potential bias and influence during analysis and selection of data for presentation.
9 Is there a clear statement of findings?
Consider:
• Whether the findings are explicit. Whether there is adequate discussion of the evidence both for and against the researcher's arguments.
Discussion of findings from the qualitative component of the study is extremely limited and insufficient. The authors did note, however, that findings from the qualitative component of the study helped to explain why the intervention in the quantitative component of the study did not seem to have an effect for some participants.
• Whether the researcher has discussed the credibility of their findings.
The authors reported using researcher triangulation. To verify the findings against the data, two researchers who were not involved in the analysis read a random selection of transcripts. No respondent validation was reported.
• Whether the findings are discussed in relation to the original research question.
The findings were only briefly discussed in relation to the original objective of the research.
10 How valuable is the research?
Consider:
• Whether the researcher discusses the contribution the study makes to existing knowledge or understanding.
The authors highlighted that the purpose of the study was to use a qualitative, evaluative approach to enhance adherence to leg ulcer treatment. They report a lack of research from the qualitative perspective on home-based interventions. The qualitative data were reported separately to the quantitative data, which segregated the results rather than bringing them together. The findings of the qualitative arm of the study did provide better understanding of the cognitive, behavioural, emotional and physical changes associated with improved adherence.
• Whether they identify new areas where research is necessary.
The authors recommend use of complementary methodologies to evaluate complex nursing interventions and to evaluate how complex nursing interventions influence patients.
• Whether the researchers have discussed whether or how the findings can be transferred to other populations or considered other ways the research may be used.
The main recommendations from the discussion of the qualitative component is for future research to use both qualitative and quantitative methods.
Comments from the QARI instrument
The authors describe using a qualitative evaluation approach, but unfortunately they did not go on to describe a particular qualitative methodology (such as phenomenology or grounded theory) or identify a philosophical perspective. Instead, they gave a limited description of the collection of qualitative data using a semi-structured interview process. This makes it difficult to consider congruence between research methodology and other aspects of the research such as representation of data or interpretation of results. This problem is compounded by the limited interpretation of results provided.
The primary findings of the qualitative component of this study were based around four domains: cognitive changes, behavioural changes, emotional changes, and physical changes. Cognitively, participants variously described the importance of knowing both what to do and why to do it. Behavioural changes were found in the creative strategies implemented by participants to form good habits that would assist them to adopt lifestyle advice into their routine behaviours. Emotional changes were reported as relating to feelings of hope, a sense of independence gained and a return to a quality of life previously thought lost. The authors posit that understanding which outcomes and what aspects of care patients valued would enable nursing staff to match their care interventions to particular patient values.
How might we use this evidence to inform practice?
Qualitative evidence can be used to inform evidence from a range of perspectives. By itself, qualitative evidence informs experiential domains that are not amendable to experimental research, it opens a window into the perspectives, values and meanings that people identify with and may also have an impact on how they participate in their care. Qualitative evidence can also facilitate a deeper understanding of healthcare interventions and the circumstances in which they work or do not work for particular people; or, as this study found, qualitative research may also illuminate a more complete picture of clinical care.
Specifically, this study found that concordance with therapy was not static and that patients varied in how closely they adhered to recommendations and lifestyle changes. This indicates that typifying patients based on whether they are perceived to be ‘compliant’ or ‘non-compliant’ is not useful and may disadvantage patients when the labels are applied. In addition to avoiding labelling patients based on brief contacts, this study indicates that patients can be assisted to better know why there are preferred approaches to how a therapy such as compression should be worn. We know that patients with a chronic venous leg ulcer are used to self-managing at home, and therefore education which explains why a particular intervention will help was found to be as important as indicating which interventions were effective. From this, we can conclude that patients, as autonomous individuals, are better motivated to participate in their care and demonstrate concordance if knowledge transfer is based on their felt needs and values.
*Reproduced and adapted from Van Hecke A et al. 2011.
Clinical scenario
You are a general practitioner. Occasionally patients ask you whether acupuncture might be useful for their unresolved health issues. You are aware that acupuncturists who practise in Australia may be medical doctors or therapists trained in traditional Chinese medicine. There has been an increase in the number of practitioners of Western-style acupuncture, but you are unaware of patient experiences of this. You wonder if there is any research into patients' perceptions of receiving Western-style acupuncture for medical problems.
What are the perceptions and experiences of patients who have received treatment with Western-style acupuncture?
Search terms and databases used to find the evidence
Database: PubMed—Health Services Research Queries (with ‘qualitative research’ category and ‘narrow scope’ selected), which is accessed through the ‘Topic-specific queries’ link on PubMed.
Search terms: acupuncture AND experience AND western
The search with narrow scope selected returns only one article, and it is not relevant. You repeat the search with ‘broad scope’ selected. This gives 30 articles; you exclude quite a few as they involve acupuncture that was done in the context of a research study to evaluate acupuncture effectiveness or were conducted in Thailand, China or Vietnam. Of the remaining articles, you choose the one that most closely matches your question.
Paterson C. Patients' experiences of Western-style acupuncture: the influence of acupuncture ‘dose’, self-care strategies and integration. J Health Serv Res Pol 2007;Suppl 1:S1-39–45.
Structured abstract (adapted from the above)
Study design: Qualitative research design with participants being interviewed twice over a 4-month period.
Study question: To investigate patients' perspectives of the process and outcome of Western-style acupuncture for chronic health problems.
Context: This study was undertaken with people who lived in urban and rural areas of the UK. Participants received treatment in a range of different settings: pain clinics, outpatient clinics and community-based practices.
Participants: A purposive sample of 19 patients who were having Western-style acupuncture for the first time for a health problem of at least 3 months' duration.
Data analysis: Analysis involved a constant comparative method. Data were analysed across cases and within cases over time.
Key findings: Participants complained of chronic pain and moderate or severe disability which was resistant to conventional treatment. Their experience of acupuncture was diverse and varied according to the ‘dosage’ of acupuncture received, the inclusion of self-care strategies and their relationship with the practitioner. These three factors were interlinked and constituted individual styles of practice for each practitioner. The majority of patients benefited in terms of complete or partial relief of pain and disability, and reduction in conventional medication. However, some patients were disappointed by the treatment, and distressed about ‘wasting people’s time’ and about the lack of continuity of care. Patients showed discerning judgment regarding the ‘dosage’ of acupuncture they required, and combined acupuncture with exercises to good effect.
Conclusions: Publicly funded health services should provide an acupuncture service that provides the optimal ‘dosage’ and uses pain relief to promote self-care.
Is the evidence rigorous and sufficiently reported?
Detailed questions from CASP
3 Was the research design appropriate to address the aims of the research?
Consider:
• Whether the researcher has justified the research design.
The research aimed to investigate the experiences of people with long-standing health problems who receive Western-style acupuncture. The specific research methodology was not clear. The research methods involved recruitment of acupuncture practitioners who provided Western-style acupuncture and, through them, invitations to their new participants to take part in data-gathering interviews. For this aim, this design was appropriate.
4 Was the recruitment strategy appropriate to the aims of the research?
Consider:
• Whether the researcher has explained how the participants were selected.
The author aimed to use maximum variation sampling. There were two stages of recruitment in this study, as described previously. The method of practitioner recruitment was not described. Six practitioners from a range of backgrounds (general practitioner, hospital and ‘other’ doctors, physiotherapists) and practice settings (hospital/outpatients/general practice, public/private, general/specialist services, city/town/rural) were recruited. These six practitioners were then asked to invite all new patients who met the inclusion criteria to contact the researcher if they were interested in participating in the study. No data were presented about whether consecutive patients were invited or how many potential participants declined participation. However, once volunteers met the inclusion criteria, all were included in the study.
• Whether they explained why the participants they selected were the most appropriate to provide access to the type of knowledge sought by the study.
This was not explained, but one might assume that as they were patients of practitioners of Western-style acupuncture and had just begun acupuncture within the last 3 weeks that they might have relevant experiences to discuss.
5 Were the data collected in a way that addressed the research issue?
Consider:
• Whether the setting for data collection was justified.
Interviews were conducted in participants' homes, which is appropriate to address the research aims of exploring people's experiences.
• Whether it is clear how data were collected.
Two semi-structured interviews were carried out: the first within 4 weeks of commencing acupuncture and the second approximately 4 months later.
• Whether the researcher has justified the methods chosen.
Yes. The interviews sought to understand descriptions, views and feelings of the treatment within the wider context of people's lives and experiences of illness across repeated interviews.
• Whether the researcher has made the methods explicit.
The information underpinning the interview schedule, the way the interviews were conducted (such as opening questions for the first and second interviews) and what type of information was sought is well described.
• Whether methods were modified during the study. If so, has the researcher explained how and why?
There is no mention that the research methods were changed during the study period.
• Whether the form of data is clear.
It is stated that interviews were tape-recorded.
6 Has the relationship between researcher and participants been adequately considered?
Consider:
• Whether the researcher critically examined their own role, potential bias and influence during:
– Formulation of the research questions.
– Data collection, including sample recruitment and choice of location.
Yes. The author specifically stated that she kept reflexive notes about the role of her personal and social identities within the data collection and analysis phases. However, no further information is provided as to whether her relationship with the participants had any impact on her results.
• How the researcher responded to events during the study and whether they considered the implications of any changes in the research design.
7 Have ethical issues been taken into consideration?
Consider:
• Whether there are sufficient details of how the research was explained to participants for the reader to assess whether ethical standards were maintained.
There are several ethical issues to consider in this study design. The issue of the dependency relationship between practitioner and patient and the possibility of coercion was addressed by ensuring that the patient was free to contact (or not) the researcher without knowledge of the treating practitioner. All participants were required to provide written informed consent prior to the first interview.
• Whether the researcher has discussed issues raised by the study.
Participants were informed that their practitioner would have no access to their interview data, thereby encouraging participants to feel free to be truthful regarding their experiences.
• Whether approval has been sought from the ethics committee.
8 Was the data analysis sufficiently rigorous?
Consider:
• Whether there is an in-depth description of the analysis process. Whether thematic analysis is used. If so, is it clear how the categories/themes were derived from the data?
The method of data analysis is well described. A constant comparative method was used, whereby data collection and analysis occurred side-by-side, each informing the other. The researcher describes her method of thinking about the data over time, making notes on her thinking, and questions in memos. Data coded over the two interviews for each person were summarised into a story and this became the basis for the thematic analysis.
• Whether the researcher explains how the data presented were selected from the original sample, to demonstrate the analysis process.
• Whether sufficient data are presented to support the findings. To what extent contradictory data are taken into account.
The author sought negative cases to challenge her developing ideas. Contradictory cases were taken into account. For example: ‘While some patients benefited rapidly from small amounts of treatment, others felt short-changed.’
• Whether the researcher critically examined their own role, potential bias and influence during analysis and selection of data for presentation.
Yes. The author specifically stated that she kept reflexive notes about the role of her personal and social identities within the data collection and analysis phases.
9 Is there a clear statement of findings?
Consider:
• Whether the findings are explicit. Whether there is adequate discussion of the evidence both for and against the researcher's arguments.
The main three findings were clearly described as factors that have an impact on patients' satisfaction, namely acupuncture ‘dosage’, integration of the acupuncture with patients' conventional treatment, and the extent to which patients learnt self-help strategies to maintain the improvements in their symptoms.
• Whether the researcher has discussed the credibility of their findings.
This study interviewed each person twice, which provided a longer period of engagement than single interviews. The study would have been strengthened by having a second investigator independently code the data and develop themes and using respondent validation.
• Whether the findings are discussed in relation to the original research question.
Yes. The findings were clearly discussed in relation to the original objective of the research.
10 How valuable is the research?
Consider:
• Whether the researcher discusses the contribution the study makes to existing knowledge or understanding.
The value of the research relates mainly to assisting in the improvement of the provision of acupuncture services by the National Health Service in the UK.
• Whether they identify new areas where research is necessary.
The author indicates that further cross-cultural studies are required.
• Whether the researchers have discussed whether or how the findings can be transferred to other populations or considered other ways the research may be used.
Participants in this study were mainly female, mean age was 59 years, and they were being treated for musculoskeletal problems such as back or neck pain. The author suggests that having 19 participants who encompassed a range of experiences and settings increased the ability to transfer these findings. Readers need to consider whether the setting and type of participants in this study are similar enough to their own patients when considering using these findings.
Comments from the QARI instrument
The author briefly describes the philosophical perspectives underpinning this research as drawing on the theoretical and empirical findings of medical sociology and using a subtle realist position for the analysis. This position assumes that we can only know reality from our own perspective of it and acknowledges multiple perspectives but not multiple realities.
The main finding was that most participants experienced an improvement in their symptoms with Western-style acupuncture and would recommend it to others and use it again themselves. However, participants' experiences with Western-style acupuncture in the UK were quite diverse and largely dependent on the practitioner who treated them. Unlike traditional Chinese medicine practitioners who take a holistic approach and incorporate acupuncture into the treatment of most health problems, most participants in this study were having acupuncture as a stand-alone treatment for one chronic health problem, most commonly some type of musculoskeletal pain. The dose of acupuncture and number, frequency and length of appointments was important to participants, but often severely constrained by the health system. Improvement in symptoms was not infrequently short-lived due to inordinately long periods of time between appointments or too few treatment sessions (as compared with what is used by traditional Chinese medicine practitioners).
Participants were much more likely to be satisfied with their acupuncture if their practitioner involved them in self-care strategies, treated more than just the one main symptom, or if a positive relationship developed with the practitioner. Participants treated with one or two needles for a few minutes by a practitioner running several consultation rooms concurrently were less likely to be satisfied by their treatment. Participants were generally not demanding, but expressed a desire for acupuncture treatment to continue until symptoms had resolved or had improved to the point that people could live with them, and a clear strategy to manage symptoms into the future was achieved.
How might we use this evidence to inform practice?
It is suggested by this small qualitative study that Western-style acupuncture is better suited to treat one-off problems in patients, rather than patients with multiple health problems.
Experiences of Western acupuncture varied with some participants reporting benefit and others not. While some participants were satisfied, others were concerned about ‘wasting people’s time’, and the lack of continuity of care. The diversity in experiences was related to three main factors: dose; the extent to which acupuncture was integrated with their other conventional care and was appropriate in terms of their overall health; and the extent to which they also learnt self-care strategies to maintain improvements gained through acupuncture. Most participants felt they would recommend it to others and use it again themselves. Having read this article, you understand more about the range of issues involved and may consider it for patients who do not have complex health conditions. However, you decide to first look for randomised controlled trials (and systematic reviews) that have explored the effectiveness of acupuncture so that you understand this body of evidence also.
Clinical scenario
You have been employed in a cardiac rehabilitation unit attached to a public hospital. The unit has been staffed by nurses and you will be the first clinical exercise physiologist who works in the team. The manager has stated that one of your first goals is to improve adherence to the Phase 2 (supervised ambulatory outpatient) program. You wish to understand the research that has examined reasons for non-adherence in cardiac rehabilitation programs. This may also provide guidance to develop an approach to investigate this in your unit's own patients.
What reasons are given by patients for non-adherence to a hospital-based cardiac rehabilitation program?
Search terms and databases used to find the evidence
Database: PubMed—Health Services Research Queries (with ‘qualitative research’ category and ‘narrow scope’ selected), which is accessed through the ‘Topic-specific queries’ link on PubMed.
Search terms: (cardiac rehabilitation) AND (adherence OR non-adherence)
This search retrieved 26 results. By reading the titles it could be seen that three were specific to heart failure, one to angina and one to transplant recipients. One was a review article and others investigated service delivery, provision and/or implementation or had very specific populations. One study appears to address non-adherence to cardiac rehabilitation and you retrieve the full text of this one.
Jones M, Jolly K, Raftery J, et al. ‘DNA’ may not mean ‘did not participate’: a qualitative study of reasons for non-adherence at home- and centre-based cardiac rehabilitation. Fam Pract 2007;v24:343–57. Available at: fampra.oxfordjournals.org/content/24/4/343.long
Structured abstract (adapted from the above)
Study design: Qualitative study with individual semi-structured interviews.
Study question: To explore patients' reasons for non-participation in or non-adherence to a home- or hospital-based cardiac rehabilitation program.
Context: Patients participating in the Birmingham Rehabilitation Uptake Maximisation study in the UK. This is a randomised controlled trial of home- versus hospital-based cardiac rehabilitation. This article describes results from a qualitative sub-study. The authors state that they were particularly interested in women, elderly patients and people from ethnic minority groups.
Participants: 49 participants who had not completed their cardiac rehabilitation program. Of the 49 patients, 21 (43%) had been in the home arm and 28 (57%) in the hospital arm.
Data collection method: Sampling was purposive and patients were invited for interview until at least 10 had been interviewed from each of the categories: female, elderly (aged ≥70 years), ethnic minority group and middle-aged males. The semi-structured interviews covered topics related to the patients' cardiac event, including expectations and experience of their rehabilitation program and lifestyle changes. All 49 interviews were conducted in the patient's home by one interviewer.
Analysis: Interview transcripts were analysed using the technique of charting. Transcripts were read independently by three of the authors, and the main themes and subthemes were identified and agreed. The reasons for non-adherence were grouped into categories and repeated reading of the transcripts and field notes enabled the identification of a ‘critical’ reason. The reasons were grouped into four main categories: (1) alternative exercise and activities; (2) other health problems; (3) personal reasons; and (4) program-related reasons.
Key findings: Reasons for not adhering to their program spanned all the categories, with no one overall reason emerging across all patients as a major factor. There were only minor trends by ethnicity, gender, cardiac event or age of the participants. Several patients changed to their own preferred exercise program, but these patients would be labelled as non-adherers. Current methods used to measure adherence to a rehabilitation program may not reflect what people actually do.
Conclusion: Reasons for non-adherence were generally multifactorial and individualistic.
Is the evidence rigorous and sufficiently reported?
Detailed questions from CASP
3 Was the research design appropriate to address the aims of the research?
Consider:
• Whether the researcher has justified the research design.
This study was a part of a large randomised trial of a home- or hospital-based cardiac rehabilitation program. The qualitative study attached to this trial involved semi-structured interviews with 49 participants who had not adhered to the cardiac rehabilitation program. Although the specific choice of methodology is not explicitly justified by the authors, the methods used (such as interviews) were appropriate for the aims of exploring patients' reasons for non-participation in or non-adherence to a home- or hospital-based cardiac rehabilitation program.
4 Was the recruitment strategy appropriate to the aims of the research?
Consider:
• Whether the researcher has explained how the participants were selected.
Participants were recruited from patients taking part in the larger trial. The trial nurses provided the names of patients who had not adhered to a program. Sampling was then purposive and patients were invited for interview until at least 10 had been interviewed from each of the categories: female, elderly (aged ≥70 years), ethnic minority group and middle-aged males. The authors explained numbers who were recruited and why some did not participate as follows: ‘Of the 131 trial patients who did not attend/adhere to their programme, 73 (56%) were approached: of these, 49 patients agreed to be interviewed, 1 had died, 16 could not be contacted and 7 refused.’
• Whether they explained why the participants they selected were the most appropriate to provide access to the type of knowledge sought by the study.
No explanation was provided about why they purposively selected these participants; however, all had not adhered to the rehabilitation program and so would be able to provide information relevant to the aims of the study.
5 Were the data collected in a way that addressed the research issue?
Consider:
• Whether the setting for data collection was justified.
Interviews took place in participants' homes. No specific reason was provided for this choice of setting, but it could be assumed that it might be due to convenience for participants.
• Whether it is clear how data were collected.
Semi-structured interviews were used.
• Whether the researcher has justified the methods chosen.
No specific justification was provided.
• Whether the researcher has made the methods explicit.
The interview schedule was developed to address issues identified from previous studies in the literature and from an earlier study in the same locality of ethnic minority patients. Topics related to the patients' cardiac event, expectations and experience of their rehabilitation program and lifestyle changes.
• Whether methods were modified during the study. If so, has the researcher explained how and why?
There is no mention that any changes occurred to the research methods.
• Whether the form of data is clear.
Yes. It is clearly stated that interviews were tape-recorded.
6 Has the relationship between researcher and participants been adequately considered?
Consider:
• Whether the researcher critically examined their own role, potential bias and influence during:
– Formulation of the research questions or data collection, including sample recruitment and choice of location.
• How the researcher responded to events during the study and whether they considered the implications of any changes in the research design.
7 Have ethical issues been taken into consideration?
Consider:
• Whether there are sufficient details of how the research was explained to participants for the reader to assess whether ethical standards were maintained.
This was not described; however, the study was approved by an ethics committee.
• Whether the researcher has discussed issues raised by the study.
• Whether approval has been sought from the ethics committee.
8 Was the data analysis sufficiently rigorous?
Consider:
• Whether there is an in-depth description of the analysis process. Whether thematic analysis is used. If so, is it clear how the categories/themes were derived from the data?
A detailed description of the analysis is provided. The authors used a technique called charting. Transcripts were read independently by three authors, and the main themes and subthemes were identified. The reasons for non-adherence were grouped into categories and identification of a crucial or ‘critical’ reason among those patients who had given more than one reason for non-adherence confirmed by re-reading transcripts and field notes.
• Whether the researcher explains how the data presented were selected from the original sample, to demonstrate the analysis process.
A worked example from one participant is provided to demonstrate how the authors arrived at the main reason that participants did not adhere to rehabilitation. Examples of quotes from participants are provided to illustrate each of the four categories. Tables which list the main reason provided by each participant for not adhering to the cardiac rehabilitation program are also provided.
• Whether sufficient data are presented to support the findings. To what extent contradictory data are taken into account.
Extensive examples of participant quotes are provided in the article. The research question does not particularly lend itself to considering contradictory data. There were diverse reasons provided for non-adherence, but they could generally be subsumed into the four categories. The reasons for non-adherence were compared by home and hospital, cardiac event, age, gender and ethnicity.
• Whether the researcher critically examined their own role, potential bias and influence during analysis and selection of data for presentation.
9 Is there a clear statement of findings?
Consider:
• Whether the findings are explicit. Whether there is adequate discussion of the evidence both for and against the researcher's arguments.
The findings are very clearly presented and unexpected findings noted. For example, it was noted that many who had not ‘adhered’ to cardiac rehabilitation exercise had actually sought out exercise themselves in other ways.
• Whether the researcher has discussed the credibility of their findings (e.g. triangulation, respondent validation, more than one analyst).
Transcripts were read independently by three of the authors and very simple triangulation between interview multiple readings and field notes was used. However, there was no report of respondent validation or further triangulation.
• Whether the findings are discussed in relation to the original research question.
Yes. The findings were clearly discussed in relation to the original objective of the research.
10 How valuable is the research?
Consider:
• Whether the researcher discusses the contribution the study makes to existing knowledge or understanding.
This study recommends that social characteristics, individual patient needs and preferences, and location of rehabilitation programs need to be considered when designing rehabilitation programs.
• Whether they identify new areas where research is necessary,
The authors do not specifically recommend other research, but suggest that measurement of adherence needs to be reconsidered. This study noted that the way in which adherence is measured (by number of sessions attended) may not be accurate as individuals may find other ways to exercise.
• Whether the researchers have discussed whether or how the findings can be transferred to other populations or considered other ways the research may be used.
The authors acknowledge that the sample was limited to those originally enrolled in a trial and who did not have a choice in the program they were randomised to. Motivations for people not enrolled in a trial might be quite different, so findings should be considered with this in mind.
Additional comments from the QARI instrument
There was no statement regarding either a philosophical perspective or a qualitative research methodology that guided this study. Therefore it is hard to comment on the congruence between these and the representation of the data and interpretation of results, for instance. As the authors indicate that they aimed to explore participants ‘views and experiences’ of cardiac rehabilitation and reasons for non-participation, this could be considered from an interpretivist perspective.
Participants' reasons for not attending varied and were grouped into four categories: (1) undertaking alternative exercise programs or activities; (2) other health problems which interfered with exercise; (3) personal reasons; and (4) factors associated with the individual programs. Overall there were no clear consistent reasons why patients were not adhering to their cardiac rehabilitation program. Patients in the home program cited problems with motivation, and those assigned to the hospital had more issues with communication and access. Individualised approaches need to consider social characteristics, individuals' needs and preferences, and the location of the programs when attempting to maximise participation.
How might we use this evidence to inform practice?
As the exercise physiologist in a cardiac rehabilitation unit, you note that this study highlights the importance of individualising approaches to cardiac rehabilitation. The various reasons given by participants for non-adherence were diverse, and a one-size-fits-all approach would not be conducive to a high level of participation. It was also evident that many participants were willing and able to adopt self-care behaviours to their rehabilitation, with some changing their programs to suit their needs. This also has implications for the way that adherence is measured, with a suggestion that measuring the ‘total number of sessions’ that patients complete may not be appropriate. Indeed, measuring clinical outcomes more objectively may be a more appropriate way of measuring adherence. You decide to look into how to measure adherence and outcomes more thoroughly and to consider ways to provide a more individualised approach to the cardiac rehabilitation that is offered.
Clinical scenario
Hospital adverse events can have negative impacts on patients, extending their hospital stay, and may even lead to death. In the pre-hospital environment, except for death, it is difficult to accurately measure the clinical effects or determine the overall cost of an adverse event. Clinical audit processes are performed in most emergency medical services to determine compliance with practice guidelines. The emergency medical services patient care record audit system does not necessarily examine all patient care records, which means that some substantial errors may go unnoticed. This patient care audit system focuses on the documentation of the patient's management; however, it misses errors that were not documented. This type of adverse event identification fails to detect actual or potential adverse events that may occur at any time while the patient is under the care of the pre-hospital care provider.
There are many negative attitudes towards the reporting of incidents and near-misses in the pre-hospital setting, especially in Australia. As a senior manager in a regional health district, you wonder whether an anonymous incident reporting system as part of a blameless culture might improve patient safety outcomes in the pre-hospital setting.
What are the barriers to the reporting of critical incidents in the pre-hospital setting?
Search terms and databases used to find the evidence
Database: PubMed—Health Services Research Queries (with ‘qualitative research’ category and ‘narrow scope’ selected), which is accessed through the ‘Topic-specific queries’ link on PubMed.
Search terms: (“emergency medical” or prehospital OR ambulance*) AND (“critical incident*” OR adverse)
The search identified 35 articles. Following review of the title and abstract, there was one article that was relevant to your question and specifically focused on barriers.
Jennings P, Stella J. Barriers to incident notification in a regional prehospital setting. Emerg Med J 2011; 28:526–9.
Structured abstract*
Study design: A qualitative approach involving the triangulation of a number of ethnographic methodologies; unscripted focus groups, informal interviews and qualitative aspects of surveys.
Setting: A regional area of Victoria, Australia.
Participants: All operational ambulance personnel and management staff of Ambulance Victoria.
Analysis: Data analysis involved identification of categories and themes.
Key findings: Barriers were identified and categorised into seven themes: burden of reporting, fear of disciplinary action, fear of potential litigation, fear of breaches of confidentiality and fear of embarrassment, concern that ‘nothing would change’ even if the incident was reported, lack of familiarity with process and impact of ‘blame culture’ within the ambulance service.
Conclusion: There are numerous barriers to reporting critical incidents in the pre-hospital setting. One of the main approaches may be shifting to a systems-based patient safety focus rather than an individual ‘shame and blame’ approach.
Is the evidence rigorous and sufficiently reported?
Detailed questions from CASP
3 Was the research design appropriate to address the aims of the research?
Consider:
• Whether the researcher has justified the research design.
This study reports on a small section of a larger study designed to describe the nature and incidence of errors in pre-hospital trauma management in order to design processes to reduce errors. This study specifically indicates taking an ethnographic methodology for this study. As part of this larger study, methods used included interviews, focus groups and open-ended questions within a survey. This article reports on the barriers to critical incident reporting that were reported in the interviews, focus groups and survey. The researchers did not justify the use of these methods in this article, but did briefly explain who the different methods were used with and at what time point in the study they were used.
4 Was the recruitment strategy appropriate to the aims of the research?
Consider:
• Whether the researcher has explained how the participants were selected.
Yes. A purposive sampling was used to recruit pre-hospital ambulance personnel and management staff. All operational staff within the Barwon region of Ambulance Victoria were eligible to participate in the study.
• Whether they explained why the participants they selected were the most appropriate to provide access to the type of knowledge sought by the study.
This is implied, as they were investigating why paramedics did not report critical incidents.
5 Were the data collected in a way that addressed the research issue?
Consider:
• Whether the setting for data collection was justified.
No. There is no comment about the data collection setting.
• Whether it is clear how data were collected.
Yes. Focus groups, informal interviews and open-ended questions in surveys.
• Whether the researcher has justified the methods chosen.
No. There is no explicit reason for the methods chosen.
• Whether the researcher has made the methods explicit.
No. The research has stated that focus groups, interviews and surveys were conducted but not how they were conducted.
• Whether methods were modified during the study. If so, has the researcher explained how and why?
There is no mention that the data collection methods were changed during the study period.
• Whether the form of data is clear.
No. There is no clear statement about the format of the data (for example, whether focus groups were recorded and then transcribed), whether the survey was paper-based or electronic or whether the interviews were recorded and transcribed.
6 Has the relationship between researcher and participants been adequately considered?
Consider:
• Whether the researcher critically examined their own role, potential bias and influence during:
– Formulation of the research questions.
There is no comment about this issue.
– Data collection, including sample recruitment and choice of location.
• How the researcher responded to events during the study and whether they considered the implications of any changes in the research design:
Yes. The authors managed the issue of potential for litigation by de-identifying retained data for the project.
7 Have ethical issues been taken into consideration?
Consider:
• Whether there are sufficient details of how the research was explained to participants for the reader to assess whether ethical standards were maintained.
Yes. All participants in the study signed a written consent statement.
• Whether the researcher has discussed issues raised by the study.
The issue of potential for litigation was resolved by the systematic de-identification of all retained data for the project so that no possible link to a specific patient or specific paramedic was possible.
• Whether approval has been sought from the ethics committee.
Yes. Ethics committee approval was reported to have been provided.
8 Was the data analysis sufficiently rigorous?
Consider:
• Whether there is an in-depth description of the analysis process. Whether thematic analysis is used. If so, is it clear how the categories/themes were derived from the data?
No. Thematic analysis was conducted; however, there is no description of how this process was undertaken or how the themes were arrived at.
• Whether the researcher explains how the data presented were selected from the original sample to demonstrate the analysis process.
No. There is no comment about this process.
• Whether sufficient data are presented to support the findings. To what extent contradictory data are taken into account.
The themes are reported with only very limited supporting information, in the form of participant comments.
• Whether the researcher critically examined their own role, potential bias and influence during analysis and selection of data for presentation.
There was no report that the researchers critically examined their own role, potential biases and influence during analysis and selection of data.
9 Is there a clear statement of findings?
Consider:
• Whether the findings are explicit. Whether there is adequate discussion of the evidence both for and against the researcher's arguments.
There is some comment about the findings and the issues associated with them. There is also reference made to other professions and studies to highlight these points.
• Whether the researcher has discussed the credibility of their finding.:
Respondent validation occurred. Themes were fed back to participants during focus groups to validate the themes and further explore perceptions about this research issue. The authors report that triangulation of data from three methods were used: (1) unstructured discursive focus groups utilising senior, management and operational Rural Ambulance Victoria personnel and senior emergency department staff; (2) informal interviews with operational Rural Ambulance Victoria personnel; and (3) qualitative portions of surveys were examined to elucidate interest, knowledge and free commentary on incident reporting and any perceived barriers.
• Whether the findings are discussed in relation to the original research question.
This part of the larger study was designed to report on barriers to the reporting of critical incidents, and the findings are clearly discussed in relation to this aim.
10 How valuable is the research?
Consider:
• Whether the researcher discusses the contribution the study makes to existing knowledge or understanding.
There is some comment about how this study adds to current knowledge and how it may change policy.
• Whether they identify new areas where research is necessary.
• Whether the researchers have discussed whether or how the findings can be transferred to other populations or considered other ways the research may be used.
The issue of transferability was not considered. One of the main conclusions was to consider approaches to how to shift to a ‘systems-based focus’ rather than an individual ‘shame and blame’ approach.
Additional comment from the QARI instrument
This study specifically indicates taking an ethnographic methodology for this study and, in this sense, the use of multiple sources and methods (interviews, focus groups, information from surveys) is congruent with this methodology. However, there is no specific statement of the underlying philosophical perspective. Overall, there is very limited representation of the data or interpretation of the results, and therefore the ability of the reader to understand, interpret and appraise this paper or use these findings is significantly hampered.
Barriers for reporting critical incidents were identified and were categorised into the following themes:
• Burden of reporting: the need for a process that allowed reporting of incidents in a timely fashion which is quick and simple to complete, and is readily available in a range of locations (that is, ambulance stations and hospital emergency departments).
• Fear of disciplinary action was a significant barrier to reporting.
• Fear of potential litigation: issues surrounding indemnity from prosecution needed to be clarified; concerns about liability if personnel admitted to an incident which may have contributed to an adverse outcome for a patient in their care.
• Fear of breaches of confidentiality and fear of embarrassment: participants were sceptical about the ability of the ‘system’ to maintain their confidentiality; being linked to certain types of critical incidents could be embarrassing within their peer group.
• Concern that ‘nothing would change’ even if the incident was reported: repeated reporting of issues to management with no positive outcome would seem to reinforce this belief.
• Lack of familiarity with process: some participants stated that they had not reported a critical incident as they ‘only started recently and didn’t know I could’.
• Impact of ‘blame culture’: paramedics were frustrated that hospital staff were encouraged to submit reports on critical incidents involving paramedics, while paramedics were not encouraged to report on in-hospital incidents as this was not within the scope of the project.
How might we use this evidence to inform practice?
Despite the very limited methodological information in this article, some information might be drawn from it. The perception of paramedics that the ambulance manager culture of assigning ‘blame’ to someone for an incident or error needs to be changed for the success of an incident monitoring system to be fully realised. For an incident reporting system to be fully functional and effective, the organisation's direction must focus on patient safety, from the upper and middle management to the paramedics. It is not possible to eliminate all pre-hospital errors, even those that cause risk, harm or death to a patient. However, improvements can be assisted by designing systems and tasks so that incidents are minimised. The system as a whole needs to be analysed to determine which components have not met the standard to achieve a good or positive outcome, and this was one of the major recommendations in this article. As a whole, this article has some very useful observations, but you decide to look for additional studies to confirm its findings.
*Reproduced and adapted from the above reference with permission from BMJ Publishing Group Ltd.
Clinical scenario
You are a pharmacist working in primary care where you are accredited to undertake community-based medication reviews in the domiciliary setting. The service is free to patients and is aimed at maximising the benefits of their medications and preventing drug-related problems. The reviews are undertaken in collaboration with patients' general practitioners. During the medication review consultation you interview the patient, preferably in their own home although the interview may also be conducted in another setting (such as the medical practice) based on patient preference. You then provide the patient's general practitioner with a report which outlines the drug-related problems that were identified during the consultation and recommendations for potential solutions.
Although the clinical benefits of medication review services have been recognised nationally and internationally, you note that a number of patients have declined to proceed with the review when you have contacted them to set up a consultation time. You are also aware that the national uptake of the funded program has remained below projected targets, despite showing success in identification of drug-related problems and patient outcomes. There is significant advertising of the program to healthcare providers and specific facilitation activities to increase identification of patients who would benefit from the service. However, you wonder why consumers who would benefit from the service would not want to participate in something that could potentially improve their health, and whether there is something you or the general practitioners could be doing to improve the uptake of this service.
What factors affect consumer participation in a home medication review program?
Search terms and databases used to find the evidence
Database: PubMed—Health Services Research Queries (with ‘qualitative research’ category and ‘narrow scope’ selected), which is accessed through the ‘Topic-specific queries’ link on PubMed.
Search terms: (consumer) AND (home medication review)
Your initial search using the narrow scope does not provide any results. You change to the ‘broad scope’, which gives eight hits—one of which is relevant to your question.
White L, Klinner C, Carter S. Consumer perspectives of the Australian Home Medicines Review Program: benefits and barriers. Res Social Adm Pharm 2012;8:4–16.
Structured abstract (adapted from the above)
Study design: Qualitative, with semi-structured focus group interviews.
Research question: Investigation of the perceived benefits and barriers of the home medication review service by individuals who had used the service and by those who were eligible for it but had not used it.
Participants: 87 patients and carers (55% were female; age range 33–91 years) who were eligible for the home medication review (split into 14 focus groups held across four Australian states) were recruited through participating pharmacies or via a local carer support group.
Data collection: Semi-structured focus group discussions were used. Participants were separated into groups of those who had and had not experienced the review service before the study. Topics covered included: details of the home medication review program, expectations/apprehensions with the service process, influences the service may have on their medications or relationships with healthcare providers, safety concerns, and whether they would actively ask their general practitioner for a home medication review referral.
Analysis: Thematic analysis of interviews using a three-step process: (1) eight codes were generated and collated into tentative themes; (2) codes were connected into categories and subcategories and relationships established between categories to find themes; and (3) themes were mapped and defined and names for each theme generated.
Key findings: The major benefits reported were acquisition of medicine information, reassurance, feeling valued and cared for, and willingness to advocate medication changes to the general practitioner. Perceived barriers were concerns regarding upsetting the general practitioner, pride and independence, confidence issues with an unknown pharmacist, privacy and safety concerns regarding the home visit, and lack of information about the program. Participants agreed that the potential benefits of the service outweighed its potential barriers.
Conclusion: It is expected that direct-to-consumer promotion of home medication reviews would increase the uptake of them. It would be necessary to ensure that the process and benefits of the service are communicated clearly and sensitively to eligible patients and their carers, to remove common consumer misconceptions and/or barriers regarding the home medication review service.
Is the evidence rigorous and sufficiently reported?
Detailed questions from CASP
3 Was the research design appropriate to address the aims of the research?
Consider:
• Whether the researcher has justified the research design.
This study does not use specific qualitative research methodology. The methods include use of semi-structured interviews. The study investigated the perceived benefits and barriers of the patients regarding the home medication review service. The authors discuss the benefits of using focus group methods for ascertaining participants' views.
4 Was the recruitment strategy appropriate to the aims of the research?
Consider:
• Whether the researcher has explained how the participants were selected.
Participants were recruited through voluntarily participating pharmacists throughout Australia. The pharmacists approached potential participants either while dispensing medicines for them or on the occasion of a home medication review visit. In addition, carers were recruited from a support group for family carers.
• Whether they explained why the participants they selected were the most appropriate to provide access to the type of knowledge sought by the study.
The authors sought representation from patients who had had a home medication review in the past 6 months and from those who had never had a home medication review but were eligible for the service, in order to obtain views from these two perspectives. They also sought involvement from carers of people with a disability, mental health problem, chronic condition or who are frail aged.
5 Were the data collected in a way that addressed the research issue?
Consider:
• Whether the setting for data collection was justified.
The place/s in which the interviews were carried out was not identified.
• Whether it is clear how data were collected.
Fourteen focus groups were used with a semi-structured format.
• Whether the researcher has justified the methods chosen.
The authors clearly justify their chosen method.
• Whether the researcher has made the methods explicit.
A semi-structured interview guide was used. The topics discussed were clearly outlined in the article.
• Whether methods were modified during the study. If so, has the researcher explained how and why?
There is no mention that the research methods were changed during the study period.
• Whether the form of data is clear.
Yes. It is clearly stated that interviews were tape-recorded.
6 Has the relationship between researcher and participants been adequately considered?
Consider:
• Whether the researcher critically examined their own role, potential bias and influence during:
– Formulation of the research questions.
There is no comment about this in the article.
– Data collection, including sample recruitment and choice of location.
• How the researcher responded to events during the study and whether they considered the implications of any changes in the research design.
7 Have ethical issues been taken into consideration?
Consider:
• Whether there are sufficient details of how the research was explained to participants for the reader to assess whether ethical standards were maintained.
All participants provided written consent to the research content and process, including audiotaping of the focus groups. Assurances were given regarding anonymity and confidentiality.
• Whether the researcher has discussed issues raised by the study.
The only information provided about this issue is that assurances were given regarding anonymity and confidentiality.
• Whether approval has been sought from the ethics committee.
8 Was the data analysis sufficiently rigorous?
Consider:
• Whether there is an in-depth description of the analysis process. Whether thematic analysis is used. If so, is it clear how the categories/themes were derived from the data?
A detailed description of the analysis is provided and a clear explanation provided as to how the categories and themes were derived.
• Whether the researcher explains how the data presented were selected from the original sample to demonstrate the analysis process.
This was not explicitly explained; however, it can be seen that data presented were used to illustrate the themes and to represent data from both carers and patients.
• Whether sufficient data are presented to support the findings. To what extent contradictory data are taken into account.
There is sufficient data to illustrate the findings. The authors sought to identify common themes and disparate perceptions.
• Whether the researcher critically examined their own role, potential bias and influence during analysis and selection of data for presentation.
9 Is there a clear statement of findings?
Consider:
• Whether the findings are explicit. Whether there is adequate discussion of the evidence both for and against the researcher's arguments.
The findings are very clearly reported. The author aimed to improve understanding of the barriers and benefits to home medication review and, as such, presented a balanced and detailed discussion of these perspectives and their implications.
• Whether the researcher has discussed the credibility of their findings.
There was a brief attempt to validate responses with participants. The focus groups were debriefed by one of the researchers, and agreement or contradicting ideas discussed. It is not stated when this debriefing occurred, but we assume it was probably at the end of the focus group session. If this was the case, then participants were not provided time to carefully reflect on the full discussion prior to commenting. No clear evidence of triangulation or use of independent analysts was provided, although discrepancies in analysis were reported to have been resolved by discussions amongst the research team.
• Whether the findings are discussed in relation to the original research question.
Yes. The findings were clearly discussed in relation to the original objective of the research.
10 How valuable is the research?
Consider:
• Whether the researcher discusses the contribution the study makes to existing knowledge or understanding.
The authors concluded that direct-to-consumer promotion of home medication review services would increase service uptake, and identified a number of issues to consider when communicating the availability of this service.
• Whether they identify new areas where research is necessary.
The authors recommend that quantitative research should be used to confirm the findings of this study with respect to the perceived barriers and benefits.
• Whether the researchers have discussed whether or how the findings can be transferred to other populations or considered other ways the research may be used.
The authors discuss restrictions on the transferability of the data in relation to carers, patients who were offered but refused to have a medication review, and those who were housebound and unable to attend a focus group.
Additional comments from the QARI instrument
This article simply states that semi-structured focus groups were undertaken by people with experience in qualitative research, and does not describe either a philosophical perspective or a specific qualitative research methodology. However, it appears that the authors were taking an interpretivist position, and phenomenological methodology could have been reasonable given that they aimed to understand the ‘perceived benefits and barriers’ of the patients regarding home medication reviews. The interview questions make it clear that the study sought participants' concerns, attitudes and experiences through focus groups and would thus be congruent with phenomenological methodology.
This study identified high consumer satisfaction levels with the home medication review. The major benefits reported were acquisition of medicine information, reassurance, feeling valued and cared for, and willingness to advocate medication changes to their general practitioner. However, specific problems that patients and carers perceived with respect to accepting or asking for a home medication review were identified. In brief, these included concerns regarding upsetting their general practitioner, pride and independence, confidence issues with an unknown pharmacist, privacy and safety concerns regarding the home visit, and lack of information about the program.
How might we use this evidence to inform practice?
These findings indicate that a more thorough explanation of the process of a home medication review is needed for patients, both at the general practitioner level and at the pharmacist level. This might include addressing concerns they may have about ‘upsetting the GP’ and fear they might have of an ‘unknown pharmacist’.
1. Critical Appraisal Skills Program (CASP), 10 questions to help you make sense of qualitative research. CASP, Oxford, 2010. Online Available www.casp-uk.net/wp-content/uploads/2011/11/CASP_Qualitative_Appraisal_Checklist_14oct10.pdf [20 Nov 2012].
2. National Collaborating Centre for Methods and Tools, Critical appraisal tools to make sense of evidence. McMaster University, Hamilton, ON, 2011. Online Available www.nccmt.ca/registry/view/eng/87.html [31 May 2012].
3. Giacomini, M, Cook D for the Evidence-Based Medicine Working Group. Users’ guides to the medical literature, XXIII: qualitative research in health care. Are the results of the study valid? JAMA. 2000; 284:357–362.