reflection paper about survey research

To read this content please select one of the options below:

Please note you do not have access to teaching notes, a reflection on survey research in hospitality.

International Journal of Contemporary Hospitality Management

ISSN : 0959-6119

Article publication date: 25 September 2018

Issue publication date: 20 November 2018

Survey research has developed to become the default empirical approach to answering research questions in the field of hospitality (and many other fields of research within the social sciences). This paper aims to reflect on the use of survey research in hospitality and offers recommendations for improvement.

Design/methodology/approach

First, known dangers to validity associated with survey research are discussed. Next, a sample of studies recently published in leading hospitality journals is assessed in view of these known dangers. Finally, recommendations are offered for editors, reviewers, readers and authors to mitigate the risk of drawing invalid conclusions based on survey research.

Survey research is very common in hospitality research and is used to investigate a wide range of research questions and constructs under study. The nature of constructs studied, the answer scales used and the nature of the samples point to a substantial risk to the validity of conclusions drawn.

Practical implications

A number of risk mitigation measures are proposed that can help authors minimise the risks to validity arising from known dangers associated with survey research. These same risk mitigation measures can be used by editors and reviewers in the assessment of manuscripts and by readers to evaluate the validity of conclusions drawn in already published work.

Originality/value

The value of this study lies in reflecting from a distance on how the survey research is conducted in the social sciences in general and in hospitality research in specific. The paper reveals that some routine approaches particularly prone to undermining the validity of conclusions may have been adopted and offers a few suggestions how this risk can be mitigated.

• Reliability
• Survey research
• Social desirability bias
• Answer scale
• Response style

Dolnicar, S. (2018), "A reflection on survey research in hospitality", International Journal of Contemporary Hospitality Management , Vol. 30 No. 11, pp. 3412-3422. https://doi.org/10.1108/IJCHM-06-2017-0386

Emerald Publishing Limited

Copyright © 2018, Emerald Publishing Limited

Related articles

All feedback is valuable.

Please share your general feedback

Report an issue or find answers to frequently asked questions

Contact Customer Support

• Search Menu
• Sign in through your institution
• Advance articles
• Editor's Choice
• Supplements
• French Abstracts
• Portuguese Abstracts
• Spanish Abstracts
• Author Guidelines
• Submission Site
• Open Access
• About International Journal for Quality in Health Care
• About the International Society for Quality in Health Care
• Editorial Board
• Advertising and Corporate Services
• Journals Career Network
• Self-Archiving Policy
• Dispatch Dates
• Contact ISQua
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

What is survey research, advantages and disadvantages of survey research, essential steps in survey research, research methods, designing the research tool, sample and sampling, data collection, data analysis.

• < Previous

Good practice in the conduct and reporting of survey research

• Article contents
• Figures & tables
• Supplementary Data

KATE KELLEY, BELINDA CLARK, VIVIENNE BROWN, JOHN SITZIA, Good practice in the conduct and reporting of survey research, International Journal for Quality in Health Care , Volume 15, Issue 3, May 2003, Pages 261–266, https://doi.org/10.1093/intqhc/mzg031

• Permissions Icon Permissions

Survey research is sometimes regarded as an easy research approach. However, as with any other research approach and method, it is easy to conduct a survey of poor quality rather than one of high quality and real value. This paper provides a checklist of good practice in the conduct and reporting of survey research. Its purpose is to assist the novice researcher to produce survey work to a high standard, meaning a standard at which the results will be regarded as credible. The paper first provides an overview of the approach and then guides the reader step-by-step through the processes of data collection, data analysis, and reporting. It is not intended to provide a manual of how to conduct a survey, but rather to identify common pitfalls and oversights to be avoided by researchers if their work is to be valid and credible.

Survey research is common in studies of health and health services, although its roots lie in the social surveys conducted in Victorian Britain by social reformers to collect information on poverty and working class life (e.g. Charles Booth [ 1 ] and Joseph Rowntree [ 2 ]), and indeed survey research remains most used in applied social research. The term ‘survey’ is used in a variety of ways, but generally refers to the selection of a relatively large sample of people from a pre-determined population (the ‘population of interest’; this is the wider group of people in whom the researcher is interested in a particular study), followed by the collection of a relatively small amount of data from those individuals. The researcher therefore uses information from a sample of individuals to make some inference about the wider population.

Data are collected in a standardized form. This is usually, but not necessarily, done by means of a questionnaire or interview. Surveys are designed to provide a ‘snapshot of how things are at a specific time’ [ 3 ]. There is no attempt to control conditions or manipulate variables; surveys do not allocate participants into groups or vary the treatment they receive. Surveys are well suited to descriptive studies, but can also be used to explore aspects of a situation, or to seek explanation and provide data for testing hypotheses. It is important to recognize that ‘the survey approach is a research strategy, not a research method’ [ 3 ]. As with any research approach, a choice of methods is available and the one most appropriate to the individual project should be used. This paper will discuss the most popular methods employed in survey research, with an emphasis upon difficulties commonly encountered when using these methods.

Descriptive research

Descriptive research is a most basic type of enquiry that aims to observe (gather information on) certain phenomena, typically at a single point in time: the ‘cross-sectional’ survey. The aim is to examine a situation by describing important factors associated with that situation, such as demographic, socio-economic, and health characteristics, events, behaviours, attitudes, experiences, and knowledge. Descriptive studies are used to estimate specific parameters in a population (e.g. the prevalence of infant breast feeding) and to describe associations (e.g. the association between infant breast feeding and maternal age).

Analytical studies

Analytical studies go beyond simple description; their intention is to illuminate a specific problem through focused data analysis, typically by looking at the effect of one set of variables upon another set. These are longitudinal studies, in which data are collected at more than one point in time with the aim of illuminating the direction of observed associations. Data may be collected from the same sample on each occasion (cohort or panel studies) or from a different sample at each point in time (trend studies).

Evaluation research

This form of research collects data to ascertain the effects of a planned change.

Advantages:

The research produces data based on real-world observations (empirical data).

The breadth of coverage of many people or events means that it is more likely than some other approaches to obtain data based on a representative sample, and can therefore be generalizable to a population.

Surveys can produce a large amount of data in a short time for a fairly low cost. Researchers can therefore set a finite time-span for a project, which can assist in planning and delivering end results.

Disadvantages:

The significance of the data can become neglected if the researcher focuses too much on the range of coverage to the exclusion of an adequate account of the implications of those data for relevant issues, problems, or theories.

The data that are produced are likely to lack details or depth on the topic being investigated.

Securing a high response rate to a survey can be hard to control, particularly when it is carried out by post, but is also difficult when the survey is carried out face-to-face or over the telephone.

Research question

Good research has the characteristic that its purpose is to address a single clear and explicit research question; conversely, the end product of a study that aims to answer a number of diverse questions is often weak. Weakest of all, however, are those studies that have no research question at all and whose design simply is to collect a wide range of data and then to ‘trawl’ the data looking for ‘interesting’ or ‘significant’ associations. This is a trap novice researchers in particular fall into. Therefore, in developing a research question, the following aspects should be considered [ 4 ]:

Be knowledgeable about the area you wish to research.

Widen the base of your experience, explore related areas, and talk to other researchers and practitioners in the field you are surveying.

Consider using techniques for enhancing creativity, for example brainstorming ideas.

Avoid the pitfalls of: allowing a decision regarding methods to decide the questions to be asked; posing research questions that cannot be answered; asking questions that have already been answered satisfactorily.

The survey approach can employ a range of methods to answer the research question. Common survey methods include postal questionnaires, face-to-face interviews, and telephone interviews.

Postal questionnaires

This method involves sending questionnaires to a large sample of people covering a wide geographical area. Postal questionnaires are usually received ‘cold’, without any previous contact between researcher and respondent. The response rate for this type of method is usually low, ∼20%, depending on the content and length of the questionnaire. As response rates are low, a large sample is required when using postal questionnaires, for two main reasons: first, to ensure that the demographic profile of survey respondents reflects that of the survey population; and secondly, to provide a sufficiently large data set for analysis.

Face-to-face interviews

Face-to-face interviews involve the researcher approaching respondents personally, either in the street or by calling at people’s homes. The researcher then asks the respondent a series of questions and notes their responses. The response rate is often higher than that of postal questionnaires as the researcher has the opportunity to sell the research to a potential respondent. Face-to-face interviewing is a more costly and time-consuming method than the postal survey, however the researcher can select the sample of respondents in order to balance the demographic profile of the sample.

Telephone interviews

Telephone surveys, like face-to-face interviews, allow a two-way interaction between researcher and respondent. Telephone surveys are quicker and cheaper than face-to-face interviewing. Whilst resulting in a higher response rate than postal surveys, telephone surveys often attract a higher level of refusals than face-to-face interviews as people feel less inhibited about refusing to take part when approached over the telephone.

Whether using a postal questionnaire or interview method, the questions asked have to be carefully planned and piloted. The design, wording, form, and order of questions can affect the type of responses obtained, and careful design is needed to minimize bias in results. When designing a questionnaire or question route for interviewing, the following issues should be considered: (1) planning the content of a research tool; (2) questionnaire layout; (3) interview questions; (4) piloting; and (5) covering letter.

Planning the content of a research tool

The topics of interest should be carefully planned and relate clearly to the research question. It is often useful to involve experts in the field, colleagues, and members of the target population in question design in order to ensure the validity of the coverage of questions included in the tool (content validity).

Researchers should conduct a literature search to identify existing, psychometrically tested questionnaires. A well designed research tool is simple, appropriate for the intended use, acceptable to respondents, and should include a clear and interpretable scoring system. A research tool must also demonstrate the psychometric properties of reliability (consistency from one measurement to the next), validity (accurate measurement of the concept), and, if a longitudinal study, responsiveness to change [ 5 ]. The development of research tools, such as attitude scales, is a lengthy and costly process. It is important that researchers recognize that the development of the research tool is equal in importance—and deserves equal attention—to data collection. If a research instrument has not undergone a robust process of development and testing, the credibility of the research findings themselves may legitimately be called into question and may even be completely disregarded. Surveys of patient satisfaction and similar are commonly weak in this respect; one review found that only 6% of patient satisfaction studies used an instrument that had undergone even rudimentary testing [ 6 ]. Researchers who are unable or unwilling to undertake this process are strongly advised to consider adopting an existing, robust research tool.

Questionnaire layout

Questionnaires used in survey research should be clear and well presented. The use of capital (upper case) letters only should be avoided, as this format is hard to read. Questions should be numbered and clearly grouped by subject. Clear instructions should be given and headings included to make the questionnaire easier to follow.

The researcher must think about the form of the questions, avoiding ‘double-barrelled’ questions (two or more questions in one, e.g. ‘How satisfied were you with your personal nurse and the nurses in general?’), questions containing double negatives, and leading or ambiguous questions. Questions may be open (where the respondent composes the reply) or closed (where pre-coded response options are available, e.g. multiple-choice questions). Closed questions with pre-coded response options are most suitable for topics where the possible responses are known. Closed questions are quick to administer and can be easily coded and analysed. Open questions should be used where possible replies are unknown or too numerous to pre-code. Open questions are more demanding for respondents but if well answered can provide useful insight into a topic. Open questions, however, can be time consuming to administer and difficult to analyse. Whether using open or closed questions, researchers should plan clearly how answers will be analysed.

Interview questions

Open questions are used more frequently in unstructured interviews, whereas closed questions typically appear in structured interview schedules. A structured interview is like a questionnaire that is administered face to face with the respondent. When designing the questions for a structured interview, the researcher should consider the points highlighted above regarding questionnaires. The interviewer should have a standardized list of questions, each respondent being asked the same questions in the same order. If closed questions are used the interviewer should also have a range of pre-coded responses available.

If carrying out a semi-structured interview, the researcher should have a clear, well thought out set of questions; however, the questions may take an open form and the researcher may vary the order in which topics are considered.

A research tool should be tested on a pilot sample of members of the target population. This process will allow the researcher to identify whether respondents understand the questions and instructions, and whether the meaning of questions is the same for all respondents. Where closed questions are used, piloting will highlight whether sufficient response categories are available, and whether any questions are systematically missed by respondents.

When conducting a pilot, the same procedure as as that to be used in the main survey should be followed; this will highlight potential problems such as poor response.

Covering letter

All participants should be given a covering letter including information such as the organization behind the study, including the contact name and address of the researcher, details of how and why the respondent was selected, the aims of the study, any potential benefits or harm resulting from the study, and what will happen to the information provided. The covering letter should both encourage the respondent to participate in the study and also meet the requirements of informed consent (see below).

The concept of sample is intrinsic to survey research. Usually, it is impractical and uneconomical to collect data from every single person in a given population; a sample of the population has to be selected [ 7 ]. This is illustrated in the following hypothetical example. A hospital wants to conduct a satisfaction survey of the 1000 patients discharged in the previous month; however, as it is too costly to survey each patient, a sample has to be selected. In this example, the researcher will have a list of the population members to be surveyed (sampling frame). It is important to ensure that this list is both up-to date and has been obtained from a reliable source.

The method by which the sample is selected from a sampling frame is integral to the external validity of a survey: the sample has to be representative of the larger population to obtain a composite profile of that population [ 8 ].

There are methodological factors to consider when deciding who will be in a sample: How will the sample be selected? What is the optimal sample size to minimize sampling error? How can response rates be maximized?

The survey methods discussed below influence how a sample is selected and the size of the sample. There are two categories of sampling: random and non-random sampling, with a number of sampling selection techniques contained within the two categories. The principal techniques are described here [ 9 ].

Random sampling

Generally, random sampling is employed when quantitative methods are used to collect data (e.g. questionnaires). Random sampling allows the results to be generalized to the larger population and statistical analysis performed if appropriate. The most stringent technique is simple random sampling. Using this technique, each individual within the chosen population is selected by chance and is equally as likely to be picked as anyone else. Referring back to the hypothetical example, each patient is given a serial identifier and then an appropriate number of the 1000 population members are randomly selected. This is best done using a random number table, which can be generated using computer software (a free on-line randomizer can be found at http://www.randomizer.org/index.htm ).

Alternative random sampling techniques are briefly described. In systematic sampling, individuals to be included in the sample are chosen at equal intervals from the population; using the earlier example, every fifth patient discharged from hospital would be included in the survey. Stratified sampling selects a specific group and then a random sample is selected. Using our example, the hospital may decide only to survey older surgical patients. Bigger surveys may employ cluster sampling, which randomly assigns groups from a large population and then surveys everyone within the groups, a technique often used in national-scale studies.

Non-random sampling

Non-random sampling is commonly applied when qualitative methods (e.g. focus groups and interviews) are used to collect data, and is typically used for exploratory work. Non-random sampling deliberately targets individuals within a population. There are three main techniques. (1) purposive sampling: a specific population is identified and only its members are included in the survey; using our example above, the hospital may decide to survey only patients who had an appendectomy. (2) Convenience sampling: the sample is made up of the individuals who are the easiest to recruit. Finally, (3) snowballing: the sample is identified as the survey progresses; as one individual is surveyed he or she is invited to recommend others to be surveyed.

It is important to use the right method of sampling and to be aware of the limitations and statistical implications of each. The need to ensure that the sample is representative of the larger population was highlighted earlier and, alongside the sampling method, the degree of sampling error should be considered. Sampling error is the probability that any one sample is not completely representative of the population from which it has been drawn [ 9 ]. Although sampling error cannot be eliminated entirely, the sampling technique chosen will influence the extent of the error. Simple random sampling will give a closer estimate of the population than a convenience sample of individuals who just happened to be in the right place at the right time.

Sample size

What sample size is required for a survey? There is no definitive answer to this question: large samples with rigorous selection are more powerful as they will yield more accurate results, but data collection and analysis will be proportionately more time consuming and expensive. Essentially, the target sample size for a survey depends on three main factors: the resources available, the aim of the study, and the statistical quality needed for the survey. For ‘qualitative’ surveys using focus groups or interviews, the sample size needed will be smaller than if quantitative data is collected by questionnaire. If statistical analysis is to be performed on the data then sample size calculations should be conducted. This can be done using computer packages such as G * Power [ 10 ]; however, those with little statistical knowledge should consult a statistician. For practical recommendations on sample size, the set of survey guidelines developed by the UK Department of Health [ 11 ] should be consulted.

Larger samples give a better estimate of the population but it can be difficult to obtain an adequate number of responses. It is rare that everyone asked to participate in the survey will reply. To ensure a sufficient number of responses, include an estimated non-response rate in the sample size calculations.

Response rates are a potential source of bias. The results from a survey with a large non-response rate could be misleading and only representative of those who replied. French [ 12 ] reported that non-responders to patient satisfaction surveys are less likely to be satisfied than people who reply. It is unwise to define a level above which a response rate is acceptable, as this depends on many local factors; however, an achievable and acceptable rate is ∼75% for interviews and 65% for self-completion postal questionnaires [ 9 , 13 ]. In any study, the final response rate should be reported with the results; potential differences between the respondents and non-respondents should be explicitly explored and their implications discussed.

There are techniques to increase response rates. A questionnaire must be concise and easy to understand, reminders should be sent out, and method of recruitment should be carefully considered. Sitzia and Wood [ 13 ] found that participants recruited by mail or who had to respond by mail had a lower mean response rate (67%) than participants who were recruited personally (mean response 76.7%). A most useful review of methods to maximize response rates in postal surveys has recently been published [ 14 ].

Researchers should approach data collection in a rigorous and ethical manner. The following information must be clearly recorded:

How, where, how many times, and by whom potential respondents were contacted.

How many people were approached and how many of those agreed to participate.

How did those who agreed to participate differ from those who refused with regard to characteristics of interest in the study, for example how were they identified, where were they approached, and what was their gender, age, and features of their illness or health care.

How was the survey administered (e.g. telephone interview).

What was the response rate (i.e. the number of usable data sets as a proportion of the number of people approached).

The purpose of all analyses is to summarize data so that it is easily understood and provides the answers to our original questions: ‘In order to do this researchers must carefully examine their data; they should become friends with their data’ [ 15 ]. Researchers must prepare to spend substantial time on the data analysis phase of a survey (and this should be built into the project plan). When analysis is rushed, often important aspects of the data are missed and sometimes the wrong analyses are conducted, leading to both inaccurate results and misleading conclusions [ 16 ]. However, and this point cannot be stressed strongly enough, researchers must not engage in data dredging, a practice that can arise especially in studies in which large numbers of dependent variables can be related to large numbers of independent variables (outcomes). When large numbers of possible associations in a dataset are reviewed at P < 0.05, one in 20 of the associations by chance will appear ‘statistically significant’; in datasets where only a few real associations exist, testing at this significance level will result in the large majority of findings still being false positives [ 17 ].

The method of data analysis will depend on the design of the survey and should have been carefully considered in the planning stages of the survey. Data collected by qualitative methods should be analysed using established methods such as content analysis [ 18 ], and where quantitative methods have been used appropriate statistical tests can be applied. Describing methods of analysis here would be unproductive as a multitude of introductory textbooks and on-line resources are available to help with simple analyses of data (e.g. [ 19 , 20 ]). For advanced analysis a statistician should be consulted.

When reporting survey research, it is essential that a number of key points are covered (though the length and depth of reporting will be dependent upon journal style). These key points are presented as a ‘checklist’ below:

Explain the purpose or aim of the research, with the explicit identification of the research question.

Explain why the research was necessary and place the study in context, drawing upon previous work in relevant fields (the literature review).

State the chosen research method or methods, and justify why this method was chosen.

Describe the research tool. If an existing tool is used, briefly state its psychometric properties and provide references to the original development work. If a new tool is used, you should include an entire section describing the steps undertaken to develop and test the tool, including results of psychometric testing.

Describe how the sample was selected and how data were collected, including:

How were potential subjects identified?

How many and what type of attempts were made to contact subjects?

Who approached potential subjects?

Where were potential subjects approached?

How was informed consent obtained?

How many agreed to participate?

How did those who agreed differ from those who did not agree?

What was the response rate?

Describe and justify the methods and tests used for data analysis.

Present the results of the research. The results section should be clear, factual, and concise.

Interpret and discuss the findings. This ‘discussion’ section should not simply reiterate results; it should provide the author’s critical reflection upon both the results and the processes of data collection. The discussion should assess how well the study met the research question, should describe the problems encountered in the research, and should honestly judge the limitations of the work.

Present conclusions and recommendations.

The researcher needs to tailor the research report to meet:

The expectations of the specific audience for whom the work is being written.

The conventions that operate at a general level with respect to the production of reports on research in the social sciences.

Anyone involved in collecting data from patients has an ethical duty to respect each individual participant’s autonomy. Any survey should be conducted in an ethical manner and one that accords with best research practice. Two important ethical issues to adhere to when conducting a survey are confidentiality and informed consent.

The respondent’s right to confidentiality should always be respected and any legal requirements on data protection adhered to. In the majority of surveys, the patient should be fully informed about the aims of the survey, and the patient’s consent to participate in the survey must be obtained and recorded.

The professional bodies listed below, among many others, provide guidance on the ethical conduct of research and surveys.

American Psychological Association: http://www.apa.org

British Psychological Society: http://www.bps.org.uk

British Medical Association: http://www.bma.org.uk .

UK General Medical Council: http://www.gmc-uk.org

American Medical Association: http://www.ama-assn.org

UK Royal College of Nursing: http://www.rcn.org.uk

UK Department of Health: http://www.doh.gov

Survey research demands the same standards in research practice as any other research approach, and journal editors and the broader research community will judge a report of survey research with the same level of rigour as any other research report. This is not to say that survey research need be particularly difficult or complex; the point to emphasize is that researchers should be aware of the steps required in survey research, and should be systematic and thoughtful in the planning, execution, and reporting of the project. Above all, survey research should not be seen as an easy, ‘quick and dirty’ option; such work may adequately fulfil local needs (e.g. a quick survey of hospital staff satisfaction), but will not stand up to academic scrutiny and will not be regarded as having much value as a contribution to knowledge.

Address reprint requests to John Sitzia, Research Department, Worthing Hospital, Lyndhurst Road, Worthing BN11 2DH, West Sussex, UK. E-mail: [email protected]

London School of Economics, UK. Http://booth.lse.ac.uk/ (accessed 15 January 2003 ).

Vernon A. A Quaker Businessman: Biography of Joseph Rowntree (1836–1925) . London: Allen & Unwin, 1958 .

Denscombe M. The Good Research Guide: For Small-scale Social Research Projects . Buckingham: Open University Press, 1998 .

Robson C. Real World Research: A Resource for Social Scientists and Practitioner-researchers . Oxford: Blackwell Publishers, 1993 .

Streiner DL, Norman GR. Health Measurement Scales: A Practical Guide to their Development and Use . Oxford: Oxford University Press, 1995 .

Sitzia J. How valid and reliable are patient satisfaction data? An analysis of 195 studies. Int J Qual Health Care 1999 ; 11: 319 –328.

Bowling A. Research Methods in Health. Investigating Health and Health Services . Buckingham: Open University Press, 2002 .

American Statistical Association, USA. Http://www.amstat.org (accessed 9 December 2002 ).

Arber S. Designing samples. In: Gilbert N, ed. Researching Social Life . London: SAGE Publications, 2001 .

Heinrich Heine University, Dusseldorf, Germany. Http://www.psycho.uni-duesseldorf.de/aap/projects/gpower/index.html (accessed 12 December 2002 ).

Department of Health, England. Http://www.doh.gov.uk/acutesurvey/index.htm (accessed 12 December 2002 ).

French K. Methodological considerations in hospital patient opinion surveys. Int J Nurs Stud 1981 ; 18: 7 –32.

Sitzia J, Wood N. Response rate in patient satisfaction research: an analysis of 210 published studies. Int J Qual Health Care 1998 ; 10: 311 –317.

Edwards P, Roberts I, Clarke M et al. Increasing response rates to postal questionnaires: systematic review. Br Med J 2002 ; 324: 1183 .

Wright DB. Making friends with our data: improving how statistical results are reported. Br J Educ Psychol 2003 ; in press.

Wright DB, Kelley K. Analysing and reporting data. In: Michie S, Abraham C, eds. Health Psychology in Practice . London: SAGE Publications, 2003 ; in press.

Davey Smith G, Ebrahim S. Data dredging, bias, or confounding. Br Med J 2002 ; 325: 1437 –1438.

Morse JM, Field PA. Nursing Research: The Application of Qualitative Approaches . London: Chapman and Hall, 1996 .

Wright DB. Understanding Statistics: An Introduction for the Social Sciences . London: SAGE Publications, 1997 .

Sportscience, New Zealand. Http://www.sportsci.org/resource/stats/index.html (accessed 12 December 2002 ).

Email alerts

Citing articles via.

• Recommend to your Library

Affiliations

• Online ISSN 1464-3677
• Print ISSN 1353-4505
• Copyright © 2024 International Society for Quality in Health Care and Oxford University Press
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

• DOI: 10.1108/IJCHM-06-2017-0386
• Corpus ID: 158078960

A reflection on survey research in hospitality

• S. Dolnicar
• Published in International Journal of… 25 September 2018
• Sociology, Business

24 Citations

A content analysis of hospitality research's research methods in the 2010s, building “causal realism” in experimental studies within hospitality and tourism, a systematic assessment of gaps between academic research and industry participation in hospitality management discipline, social media attachment: conceptualization and formative index construction, favouritism: a recipe for ostracism how jealousy and self-esteem intervene, attitudes and tourists’ sustainable behavior: an overview of the literature and discussion of some theoretical and methodological issues, investigating the appeal of a visitor guide: a triangulated approach, too afraid to travel development of a pandemic (covid-19) anxiety travel scale (pats), how do consumers select between hotels and airbnb a hierarchy of importance in accommodation choice, expanding theory of tourists’ destination loyalty: the role of sensory impressions, 46 references, assessing analytical robustness in cross‐cultural comparisons, social desirability bias and the validity of indirect questioning, the validity of self-report measures of proenvironmental behavior: a meta-analytic review, adjustment for acquiescence in the assessment of the construct equivalence of likert-type score items, does one size fit all the suitability of answer formats for different constructs measured, the effect of interventions on the environmental behaviour of australian motel guests, extreme response style in cross-cultural research, response style contamination of student evaluation data, development of a scale to measure hotel brand experiences, a new scale of social desirability independent of psychopathology., related papers.

Showing 1 through 3 of 0 Related Papers

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Korean Med Sci
• v.35(45); 2020 Nov 23

Reporting Survey Based Studies – a Primer for Authors

Prithvi sanjeevkumar gaur.

1 Smt. Kashibai Navale Medical College and General Hospital, Pune, India.

Olena Zimba

2 Department of Internal Medicine No. 2, Danylo Halytsky Lviv National Medical University, Lviv, Ukraine.

Vikas Agarwal

3 Department Clinical Immunology and Rheumatology, Sanjay Gandhi Postgraduate Institute of Medical Sciences, Lucknow, India.

Latika Gupta

Associated data.

The coronavirus disease 2019 (COVID-19) pandemic has led to a massive rise in survey-based research. The paucity of perspicuous guidelines for conducting surveys may pose a challenge to the conduct of ethical, valid and meticulous research. The aim of this paper is to guide authors aiming to publish in scholarly journals regarding the methods and means to carry out surveys for valid outcomes. The paper outlines the various aspects, from planning, execution and dissemination of surveys followed by the data analysis and choosing target journals. While providing a comprehensive understanding of the scenarios most conducive to carrying out a survey, the role of ethical approval, survey validation and pilot testing, this brief delves deeper into the survey designs, methods of dissemination, the ways to secure and maintain data anonymity, the various analytical approaches, the reporting techniques and the process of choosing the appropriate journal. Further, the authors analyze retracted survey-based studies and the reasons for the same. This review article intends to guide authors to improve the quality of survey-based research by describing the essential tools and means to do the same with the hope to improve the utility of such studies.

Graphical Abstract

INTRODUCTION

Surveys are the principal method used to address topics that require individual self-report about beliefs, knowledge, attitudes, opinions or satisfaction, which cannot be assessed using other approaches. 1 This research method allows information to be collected by asking a set of questions on a specific topic to a subset of people and generalizing the results to a larger population. Assessment of opinions in a valid and reliable way require clear, structured and precise reporting of results. This is possible with a survey based out of a meticulous design, followed by validation and pilot testing. 2 The aim of this opinion piece is to provide practical advice to conduct survey-based research. It details the ethical and methodological aspects to be undertaken while performing a survey, the online platforms available for distributing survey, and the implications of survey-based research.

Survey-based research is a means to obtain quick data, and such studies are relatively easy to conduct and analyse, and are cost-effective (under a majority of the circumstances). 3 These are also one of the most convenient methods of obtaining data about rare diseases. 4 With major technological advancements and improved global interconnectivity, especially during the coronavirus disease 2019 (COVID-19) pandemic, surveys have surpassed other means of research due to their distinctive advantage of a wider reach, including respondents from various parts of the world having diverse cultures and geographically disparate locations. Moreover, survey-based research allows flexibility to the investigator and respondent alike. 5 While the investigator(s) may tailor the survey dates and duration as per their availability, the respondents are allowed the convenience of responding to the survey at ease, in the comfort of their homes, and at a time when they can answer the questions with greater focus and to the best of their abilities. 6 Respondent biases inherent to environmental stressors can be significantly reduced by this approach. 5 It also allows responses across time-zones, which may be a major impediment to other forms of research or data-collection. This allows distant placement of the investigator from the respondents.

Various digital tools are now available for designing surveys ( Table 1 ). 7 Most of these are free with separate premium paid options. The analysis of data can be made simpler and cleaning process almost obsolete by minimising open-ended answer choices. 8 Close-ended answers makes data collection and analysis efficient, by generating an excel which can be directly accessed and analysed. 9 Minimizing the number of questions and making all questions mandatory can further aid this process by bringing uniformity to the responses and analysis simpler. Surveys are arguably also the most engaging form of research, conditional to the skill of the investigator.

Q/t = questions per typeform, A/m = answers per month, Q/s = questions per survey, A/s = answers per survey, NA = not applicable, NPS = net promoter score.

Data protection laws now mandate anonymity while collecting data for most surveys, particularly when they are exempt from ethical review. 10 , 11 Anonymization has the potential to reduce (or at times even eliminate) social desirability bias which gains particular relevance when targeting responses from socially isolated or vulnerable communities (e.g. LGBTQ and low socio-economic strata communities) or minority groups (religious, ethnic and medical) or controversial topics (drug abuse, using language editing software).

Moreover, surveys could be the primary methodology to explore a hypothesis until it evolves into a more sophisticated and partly validated idea after which it can be probed further in a systematic and structured manner using other research methods.

The aim of this paper is to reduce the incorrect reporting of surveys. The paper also intends to inform researchers of the various aspects of survey-based studies and the multiple points that need to be taken under consideration while conducting survey-based research.

SURVEYS IN THE COVID-19 PANDEMIC

The COVID-19 has led to a distinctive rise in survey-based research. 12 The need to socially distance amid widespread lockdowns reduced patient visits to the hospital and brought most other forms of research to a standstill in the early pandemic period. A large number of level-3 bio-safety laboratories are being engaged for research pertaining to COVID-19, thereby limiting the options to conduct laboratory-based research. 13 , 14 Therefore, surveys appear to be the most viable option for researchers to explore hypotheses related to the situation and its impact in such times. 15

LIMITATIONS WHILE CONDUCTING SURVEY-BASED RESEARCH

Designing a fine survey is an arduous task and requires skill even though clear guidelines are available in regard to the same. Survey design requires extensive thoughtfulness on the core questions (based on the hypothesis or the primary research question), with consideration of all possible answers, and the inclusion of open-ended options to allow recording other possibilities. A survey should be robust, in regard to the questions gathered and the answer choices available, it must be validated, and pilot tested. 16 The survey design may be supplanted with answer choices tailored for the convenience of the responder, to reduce the effort while making it more engaging. Survey dissemination and engagement of respondents also requires experience and skill. 17

Furthermore, the absence of an interviewer prevents us from gaining clarification on responses of open-ended questions if any. Internet surveys are also prone to survey fraud by erroneous reporting. Hence, anonymity of surveys is a boon and a bane. The sample sizes are skewed as it lacks representation of population absent on the Internet like the senile or the underprivileged. The illiterate population also lacks representation in survey-based research.

The “Enhancing the QUAlity and Transparency Of health Research” network (EQUATOR) provides two separate guidelines replete with checklists to ensure valid reporting of e-survey methodology. These include “The Checklist for Reporting Results of Internet E-Surveys” (CHERRIES) statement and “ The Journal of Medical Internet Research ” (JMIR) checklist.

COMMON TYPES OF SURVEY-BASED RESEARCH

From a clinician's standpoint, the common survey types include those centered around problems faced by the patients or physicians. 18 Surveys collecting the opinions of various clinicians on a debated clinical topic or feedback forms typically served after attending medical conferences or prescribing a new drug or trying a new method for a given procedure are also surveys. The formulation of clinical practice guidelines entails Delphi exercises using paper surveys, which are yet another form of survey-mediated research.

Size of the survey depends on its intent. They could be large or small surveys. Therefore, identification of the intent behind the survey is essential to allow the investigator to form a hypothesis and then explore it further. Large population-based or provider-based surveys are often done and generate mammoth data over the years. E.g. The National Health and Nutrition Examination Survey, The National Health Interview Survey and the National Ambulatory Medical Care Survey.

SCENARIOS FOR CONDUCTING SURVEY-BASED RESEARCH

Despite all said and done about the convenience of conducting survey-based research, it is prudent to conduct a feasibility check before embarking on one. Certain scenarios may be the key determinants in determining the fate of survey-based research ( Table 2 ).

ETHICS APPROVAL FOR SURVEY-BASED RESEARCH

Approval from the Institutional Review Board should be taken as per requirement according to the CHERRIES checklist. However, rules for approval are different as per the country or nation and therefore, local rules must be checked and followed. For instance, in India, the Indian Council of Medical Research released an article in 2017, stating that the concept of broad consent has been updated which is defined “consent for an unspecified range of future research subject to a few contents and/or process restrictions.” It talks about “the flexibility of Indian ethics committees to review a multicentric study proposal for research involving low or minimal risk, survey or studies using anonymized samples or data or low or minimal risk public health research.” The reporting of approvals received and applied for and the procedure of written, informed consent followed must be clear and transparent. 10 , 19

The use of incentives in surveys is also an ethical concern. 20 The different of incentives that can be used are monetary or non-monetary. Monetary incentives are usually discouraged as these may attract the wrong population due to the temptation of the monetary benefit. However, monetary incentives have been seen to make survey receive greater traction even though this is yet to proven. Monetary incentives are not only provided in terms of cash or cheque but also in the form of free articles, discount coupons, phone cards, e-money or cashback value. 21 These methods though tempting must be seldom used. If used, their use must be disclosed and justified in the report. The use of non-monetary incentives like a meeting with a famous personality or access to restricted and authorized areas. These can also help pique the interest of the respondents.

DESIGNING A SURVEY

As mentioned earlier, the design of a survey is reflective of the skill of the investigator curating it. 22 Survey builders can be used to design an efficient survey. These offer majority of the basic features needed to construct a survey, free of charge. Therefore, surveys can be designed from scratch, using pre-designed templates or by using previous survey designs as inspiration. Taking surveys could be made convenient by using the various aids available ( Table 1 ). Moreover, even the investigator should be mindful of the unintended response effects of ordering and context of survey questions. 23

Surveys using clear, unambiguous, simple and well-articulated language record precise answers. 24 A well-designed survey accounts for the culture, language and convenience of the target demographic. The age, region, country and occupation of the target population is also considered before constructing a survey. Consistency is maintained in the terms used in the survey and abbreviations are avoided to allow the respondents to have a clear understanding of the question being answered. Universal abbreviations or previously indexed abbreviations maintain the unambiguity of the survey.

Surveys beginning with broad, easy and non-specific questions as compared to sensitive, tedious and non-specific ones receive more accurate and complete answers. 25 Questionnaires designed such that the relatively tedious and long questions requiring the respondent to do some nit-picking are placed at the end improves the response rate of the survey. This prevents the respondent to be discouraged to answer the survey at the beginning itself and motivates the respondent to finish the survey at the end. All questions must provide a non-response option and all questions should be made mandatory to increase completeness of the survey. Questions can be framed in close-ended or open-ended fashion. However, close-ended questions are easier to analyze and are less tedious to answer by the respondent and therefore must be the main component in a survey. Open-ended questions have minimal use as they are tedious, take time to answer and require fine articulation of one's thoughts. Also, their minimal use is advocated because the interpretation of such answers requires dedication in terms of time and energy due to the diverse nature of the responses which is difficult to promise owing to the large sample sizes. 26 However, whenever the closed choices do not cover all probabilities, an open answer choice must be added. 27 , 28

Screening questions to meet certain criteria to gain access to the survey in cases where inclusion criteria need to be established to maintain authenticity of target demographic. Similarly, logic function can be used to apply an exclusion. This allows clean and clear record of responses and makes the job of an investigator easier. The respondents can or cannot have the option to return to the previous page or question to alter their answer as per the investigator's preference.

The range of responses received can be reduced in case of questions directed towards the feelings or opinions of people by using slider scales, or a Likert scale. 29 , 30 In questions having multiple answers, check boxes are efficient. When a large number of answers are possible, dropdown menus reduce the arduousness. 31 Matrix scales can be used to answer questions requiring grading or having a similar range of answers for multiple conditions. Maximum respondent participation and complete survey responses can be ensured by reducing the survey time. Quiz mode or weighted modes allow the respondent to shuffle between questions and allows scoring of quizzes and can be used to complement other weighted scoring systems. 32 A flowchart depicting a survey construct is presented as Fig. 1 .

Survey validation

Validation testing though tedious and meticulous, is worthy effort as the accuracy of a survey is determined by its validity. It is indicative of the of the sample of the survey and the specificity of the questions such that the data acquired is streamlined to answer the questions being posed or to determine a hypothesis. 33 , 34 Face validation determines the mannerism of construction of questions such that necessary data is collected. Content validation determines the relation of the topic being addressed and its related areas with the questions being asked. Internal validation makes sure that the questions being posed are directed towards the outcome of the survey. Finally, Test – retest validation determines the stability of questions over a period of time by testing the questionnaire twice and maintaining a time interval between the two tests. For surveys determining knowledge of respondents pertaining to a certain subject, it is advised to have a panel of experts for undertaking the validation process. 2 , 35

Reliability testing

If the questions in the survey are posed in a manner so as to elicit the same or similar response from the respondents irrespective of the language or construction of the question, the survey is said to be reliable. It is thereby, a marker of the consistency of the survey. This stands to be of considerable importance in knowledge-based researches where recall ability is tested by making the survey available for answering by the same participants at regular intervals. It can also be used to maintain authenticity of the survey, by varying the construction of the questions.

Designing a cover letter

A cover letter is the primary means of communication with the respondent, with the intent to introduce the respondent to the survey. A cover letter should include the purpose of the survey, details of those who are conducting it, including contact details in case clarifications are desired. It should also clearly depict the action required by the respondent. Data anonymization may be crucial to many respondents and is their right. This should be respected in a clear description of the data handling process while disseminating the survey. A good cover letter is the key to building trust with the respondent population and can be the forerunner to better response rates. Imparting a sense of purpose is vital to ideationally incentivize the respondent population. 36 , 37 Adding the credentials of the team conducting the survey may further aid the process. It is seen that an advance intimation of the survey prepares the respondents while improving their compliance.

The design of a cover letter needs much attention. It should be captivating, clear, precise and use a vocabulary and language specific to the target population for the survey. Active voice should be used to make a greater impact. Crowding of the details must be avoided. Using italics, bold fonts or underlining may be used to highlight critical information. the tone ought to be polite, respectful, and grateful in advance. The use of capital letters is at best avoided, as it is surrogate for shouting in verbal speech and may impart a bad taste.

The dates of the survey may be intimated, so the respondents may prepare themselves for taking it at a time conducive to them. While, emailing a closed group in a convenience sampled survey, using the name of the addressee may impart a customized experience and enhance trust building and possibly compliance. Appropriate use of salutations like Mr./Ms./Mrs. may be considered. Various portals such as SurveyMonkey allow the researchers to save an address list on the website. These may then be reached out using an embedded survey link from a verified email address to minimize bouncing back of emails.

The body of the cover letter must be short, crisp and not exceed 2–3 paragraphs under idea circumstances. Ernest efforts to protect confidentiality may go a long way in enhancing response rates. 38 While it is enticing to provide incentives to enhance response, these are best avoided. 38 , 39 In cases when indirect incentives are offered, such as provision of results of the survey, these may be clearly stated in the cover letter. Lastly, a formal closing note with the signatures of the lead investigator are welcome. 38 , 40

Designing questions

Well-constructed questionnaires are essentially the backbone of successful survey-based studies. With this type of research, the primary concern is the adequate promotion and dissemination of the questionnaire to the target population. The careful of selection of sample population, therefore, needs to be with minimal flaws. The method of conducting survey is an essential determinant of the response rate observed. 41 Broadly, surveys are of two types: closed and open. Depending on the sample population the method of conducting the survey must be determined.

Various doctors use their own patients as the target demographic, as it improves compliance. However, this is effective in surveys aiming towards a geographically specific, fairly common disease as the sample size needs to be adequate. Response bias can be identified by the data collected from respondent and non-respondent groups. 42 , 43 Therefore, to choose a target population whose database of baseline characteristics is already known is more efficacious. In cases of surveys focused on patients having a rare group of diseases, online surveys or e-surveys can be conducted. Data can also be gathered from the multiple national organizations and societies all over the world. 44 , 45 Computer generated random selection can be done from this data to choose participants and they can be reached out to using emails or social media platforms like WhatsApp and LinkedIn. In both these scenarios, closed questionnaires can be conducted. These have restricted access either through a URL link or through e-mail.

In surveys targeting an issue faced by a larger demographic (e.g. pandemics like the COVID-19, flu vaccines and socio-political scenarios), open surveys seem like the more viable option as they can be easily accessed by majority of the public and ensures large number of responses, thereby increasing the accuracy of the study. Survey length should be optimal to avoid poor response rates. 25 , 46

SURVEY DISSEMINATION

Uniform distribution of the survey ensures equitable opportunity to the entire target population to access the questionnaire and participate in it. While deciding the target demographic communities should be studied and the process of “lurking” is sometimes practiced. Multiple sampling methods are available ( Fig. 1 ). 47

Distribution of survey to the target demographic could be done using emails. Even though e-mails reach a large proportion of the target population, an unknown sender could be blocked, making the use of personal or a previously used email preferable for correspondence. Adding a cover letter along with the invite adds a personal touch and is hence, advisable. Some platforms allow the sender to link the survey portal with the sender's email after verifying it. Noteworthily, despite repeated email reminders, personal communication over the phone or instant messaging improved responses in the authors' experience. 48 , 49

Distribution of the survey over other social media platforms (SMPs, namely WhatsApp, Facebook, Instagram, Twitter, LinkedIn etc.) is also practiced. 50 , 51 , 52 Surveys distributed on every available platform ensures maximal outreach. 53 Other smartphone apps can also be used for wider survey dissemination. 50 , 54 It is important to be mindful of the target population while choosing the platform for dissemination of the survey as some SMPs such as WhatsApp are more popular in India, while others like WeChat are used more widely in China, and similarly Facebook among the European population. Professional accounts or popular social accounts can be used to promote and increase the outreach for a survey. 55 Incentives such as internet giveaways or meet and greets with their favorite social media influencer have been used to motivate people to participate.

However, social-media platforms do not allow calculation of the denominator of the target population, resulting in inability to gather the accurate response rate. Moreover, this method of collecting data may result in a respondent bias inherent to a community that has a greater online presence. 43 The inability to gather the demographics of the non-respondents (in a bid to identify and prove that they were no different from respondents) can be another challenge in convenience sampling, unlike in cohort-based studies.

Lastly, manually filling of surveys, over the telephone, by narrating the questions and answer choices to the respondents is used as the last-ditch resort to achieve a high desired response rate. 56 Studies reveal that surveys released on Mondays, Fridays, and Sundays receive more traction. Also, reminders set at regular intervals of time help receive more responses. Data collection can be improved in collaborative research by syncing surveys to fill out electronic case record forms. 57 , 58 , 59

Data anonymity refers to the protection of data received as a part of the survey. This data must be stored and handled in accordance with the patient privacy rights/privacy protection laws in reference to surveys. Ethically, the data must be received on a single source file handled by one individual. Sharing or publishing this data on any public platform is considered a breach of the patient's privacy. 11 In convenience sampled surveys conducted by e-mailing a predesignated group, the emails shall remain confidential, as inadvertent sharing of these as supplementary data in the manuscript may amount to a violation of the ethical standards. 60 A completely anonymized e-survey discourages collection of Internet protocol addresses in addition to other patient details such as names and emails.

Data anonymity gives the respondent the confidence to be candid and answer the survey without inhibitions. This is especially apparent in minority groups or communities facing societal bias (sex workers, transgenders, lower caste communities, women). Data anonymity aids in giving the respondents/participants respite regarding their privacy. As the respondents play a primary role in data collection, data anonymity plays a vital role in survey-based research.

DATA HANDLING OF SURVEYS

The data collected from the survey responses are compiled in a .xls, .csv or .xlxs format by the survey tool itself. The data can be viewed during the survey duration or after its completion. To ensure data anonymity, minimal number of people should have access to these results. The data should then be sifted through to invalidate false, incorrect or incomplete data. The relevant and complete data should then be analyzed qualitatively and quantitatively, as per the aim of the study. Statistical aids like pie charts, graphs and data tables can be used to report relative data.

ANALYSIS OF SURVEY DATA

Analysis of the responses recorded is done after the time made available to answer the survey is complete. This ensures that statistical and hypothetical conclusions are established after careful study of the entire database. Incomplete and complete answers can be used to make analysis conditional on the study. Survey-based studies require careful consideration of various aspects of the survey such as the time required to complete the survey. 61 Cut-off points in the time frame allow authentic answers to be recorded and analyzed as compared to disingenuous completed questionnaires. Methods of handling incomplete questionnaires and atypical timestamps must be pre-decided to maintain consistency. Since, surveys are the only way to reach people especially during the COVID-19 pandemic, disingenuous survey practices must not be followed as these will later be used to form a preliminary hypothesis.

REPORTING SURVEY-BASED RESEARCH

Reporting the survey-based research is by far the most challenging part of this method. A well-reported survey-based study is a comprehensive report covering all the aspects of conducting a survey-based research.

The design of the survey mentioning the target demographic, sample size, language, type, methodology of the survey and the inclusion-exclusion criteria followed comprises a descriptive report of a survey-based study. Details regarding the conduction of pilot-testing, validation testing, reliability testing and user-interface testing add value to the report and supports the data and analysis. Measures taken to prevent bias and ensure consistency and precision are key inclusions in a report. The report usually mentions approvals received, if any, along with the written, informed, consent taken from the participants to use the data received for research purposes. It also gives detailed accounts of the different distribution and promotional methods followed.

A detailed account of the data input and collection methods along with tools used to maintain the anonymity of the participants and the steps taken to ensure singular participation from individual respondents indicate a well-structured report. Descriptive information of the website used, visitors received and the externally influencing factors of the survey is included. Detailed reporting of the post-survey analysis including the number of analysts involved, data cleaning required, if any, statistical analysis done and the probable hypothesis concluded is a key feature of a well-reported survey-based research. Methods used to do statistical corrections, if used, should be included in the report. The EQUATOR network has two checklists, “The Checklist for Reporting Results of Internet E-Surveys” (CHERRIES) statement and “ The Journal of Medical Internet Research ” (JMIR) checklist, that can be utilized to construct a well-framed report. 62 , 63 Importantly, self-reporting of biases and errors avoids the carrying forward of false hypothesis as a basis of more advanced research. References should be cited using standard recommendations, and guided by the journal specifications. 64

CHOOSING A TARGET JOURNAL FOR SURVEY-BASED RESEARCH

Surveys can be published as original articles, brief reports or as a letter to the editor. Interestingly, most modern journals do not actively make mention of surveys in the instructions to the author. Thus, depending on the study design, the authors may choose the article category, cohort or case-control interview or survey-based study. It is prudent to mention the type of study in the title. Titles albeit not too long, should not exceed 10–12 words, and may feature the type of study design for clarity after a semicolon for greater citation potential.

While the choice of journal is largely based on the study subject and left to the authors discretion, it may be worthwhile exploring trends in a journal archive before proceeding with submission. 65 Although the article format is similar across most journals, specific rules relevant to the target journal may be followed for drafting the article structure before submission.

RETRACTION OF ARTICLES

Articles that are removed from the publication after being released are retracted articles. These are usually retracted when new discrepancies come to light regarding, the methodology followed, plagiarism, incorrect statistical analysis, inappropriate authorship, fake peer review, fake reporting and such. 66 A sufficient increase in such papers has been noticed. 67

We carried out a search of “surveys” on Retraction Watch on 31st August 2020 and received 81 search results published between November 2006 to June 2020, out of which 3 were repeated. Out of the 78 results, 37 (47.4%) articles were surveys, 23 (29.4%) showed as unknown types and 18 (23.2%) reported other types of research. ( Supplementary Table 1 ). Fig. 2 gives a detailed description of the causes of retraction of the surveys we found and its geographic distribution.

A good survey ought to be designed with a clear objective, the design being precise and focused with close-ended questions and all probabilities included. Use of rating scales, multiple choice questions and checkboxes and maintaining a logical question sequence engages the respondent while simplifying data entry and analysis for the investigator. Conducting pilot-testing is vital to identify and rectify deficiencies in the survey design and answer choices. The target demographic should be defined well, and invitations sent accordingly, with periodic reminders as appropriate. While reporting the survey, maintaining transparency in the methods employed and clearly stating the shortcomings and biases to prevent advocating an invalid hypothesis.

Disclosure: The authors have no potential conflicts of interest to disclose.

Author Contributions:

• Conceptualization: Gaur PS, Zimba O, Agarwal V, Gupta L.
• Visualization: Gaur PS, Zimba O, Agarwal V, Gupta L.
• Writing - original draft: Gaur PS, Gupta L.

SUPPLEMENTARY MATERIAL

Reporting survey based research

Organizing Your Social Sciences Research Assignments

• Annotated Bibliography
• Analyzing a Scholarly Journal Article
• Group Presentations
• Dealing with Nervousness
• Using Visual Aids
• Grading Someone Else's Paper
• Types of Structured Group Activities
• Group Project Survival Skills
• Leading a Class Discussion
• Multiple Book Review Essay
• Reviewing Collected Works
• Writing a Case Analysis Paper
• Writing a Case Study
• About Informed Consent
• Writing Field Notes
• Writing a Policy Memo
• Writing a Reflective Paper
• Writing a Research Proposal
• Generative AI and Writing
• Acknowledgments

Reflective writing is a process of identifying, questioning, and critically evaluating course-based learning opportunities, integrated with your own observations, experiences, impressions, beliefs, assumptions, or biases, and which describes how this process stimulated new or creative understanding about the content of the course.

A reflective paper describes and explains in an introspective, first person narrative, your reactions and feelings about either a specific element of the class [e.g., a required reading; a film shown in class] or more generally how you experienced learning throughout the course. Reflective writing assignments can be in the form of a single paper, essays, portfolios, journals, diaries, or blogs. In some cases, your professor may include a reflective writing assignment as a way to obtain student feedback that helps improve the course, either in the moment or for when the class is taught again.

How to Write a Reflection Paper . Academic Skills, Trent University; Writing a Reflection Paper . Writing Center, Lewis University; Critical Reflection . Writing and Communication Centre, University of Waterloo; Tsingos-Lucas et al. "Using Reflective Writing as a Predictor of Academic Success in Different Assessment Formats." American Journal of Pharmaceutical Education 81 (2017): Article 8.

Benefits of Reflective Writing Assignments

As the term implies, a reflective paper involves looking inward at oneself in contemplating and bringing meaning to the relationship between course content and the acquisition of new knowledge . Educational research [Bolton, 2010; Ryan, 2011; Tsingos-Lucas et al., 2017] demonstrates that assigning reflective writing tasks enhances learning because it challenges students to confront their own assumptions, biases, and belief systems around what is being taught in class and, in so doing, stimulate student’s decisions, actions, attitudes, and understanding about themselves as learners and in relation to having mastery over their learning. Reflection assignments are also an opportunity to write in a first person narrative about elements of the course, such as the required readings, separate from the exegetic and analytical prose of academic research papers.

Reflection writing often serves multiple purposes simultaneously. In no particular order, here are some of reasons why professors assign reflection papers:

• Enhances learning from previous knowledge and experience in order to improve future decision-making and reasoning in practice . Reflective writing in the applied social sciences enhances decision-making skills and academic performance in ways that can inform professional practice. The act of reflective writing creates self-awareness and understanding of others. This is particularly important in clinical and service-oriented professional settings.
• Allows students to make sense of classroom content and overall learning experiences in relation to oneself, others, and the conditions that shaped the content and classroom experiences . Reflective writing places you within the course content in ways that can deepen your understanding of the material. Because reflective thinking can help reveal hidden biases, it can help you critically interrogate moments when you do not like or agree with discussions, readings, or other aspects of the course.
• Increases awareness of one’s cognitive abilities and the evidence for these attributes . Reflective writing can break down personal doubts about yourself as a learner and highlight specific abilities that may have been hidden or suppressed due to prior assumptions about the strength of your academic abilities [e.g., reading comprehension; problem-solving skills]. Reflective writing, therefore, can have a positive affective [i.e., emotional] impact on your sense of self-worth.
• Applying theoretical knowledge and frameworks to real experiences . Reflective writing can help build a bridge of relevancy between theoretical knowledge and the real world. In so doing, this form of writing can lead to a better understanding of underlying theories and their analytical properties applied to professional practice.
• Reveals shortcomings that the reader will identify . Evidence suggests that reflective writing can uncover your own shortcomings as a learner, thereby, creating opportunities to anticipate the responses of your professor may have about the quality of your coursework. This can be particularly productive if the reflective paper is written before final submission of an assignment.
• Helps students identify their tacit [a.k.a., implicit] knowledge and possible gaps in that knowledge . Tacit knowledge refers to ways of knowing rooted in lived experience, insight, and intuition rather than formal, codified, categorical, or explicit knowledge. In so doing, reflective writing can stimulate students to question their beliefs about a research problem or an element of the course content beyond positivist modes of understanding and representation.
• Encourages students to actively monitor their learning processes over a period of time . On-going reflective writing in journals or blogs, for example, can help you maintain or adapt learning strategies in other contexts. The regular, purposeful act of reflection can facilitate continuous deep thinking about the course content as it evolves and changes throughout the term. This, in turn, can increase your overall confidence as a learner.
• Relates a student’s personal experience to a wider perspective . Reflection papers can help you see the big picture associated with the content of a course by forcing you to think about the connections between scholarly content and your lived experiences outside of school. It can provide a macro-level understanding of one’s own experiences in relation to the specifics of what is being taught.
• If reflective writing is shared, students can exchange stories about their learning experiences, thereby, creating an opportunity to reevaluate their original assumptions or perspectives . In most cases, reflective writing is only viewed by your professor in order to ensure candid feedback from students. However, occasionally, reflective writing is shared and openly discussed in class. During these discussions, new or different perspectives and alternative approaches to solving problems can be generated that would otherwise be hidden. Sharing student's reflections can also reveal collective patterns of thought and emotions about a particular element of the course.

Bolton, Gillie. Reflective Practice: Writing and Professional Development . London: Sage, 2010; Chang, Bo. "Reflection in Learning." Online Learning 23 (2019), 95-110; Cavilla, Derek. "The Effects of Student Reflection on Academic Performance and Motivation." Sage Open 7 (July-September 2017): 1–13; Culbert, Patrick. “Better Teaching? You Can Write On It “ Liberal Education (February 2022); McCabe, Gavin and Tobias Thejll-Madsen. The Reflection Toolkit . University of Edinburgh; The Purpose of Reflection . Introductory Composition at Purdue University; Practice-based and Reflective Learning . Study Advice Study Guides, University of Reading; Ryan, Mary. "Improving Reflective Writing in Higher Education: A Social Semiotic Perspective." Teaching in Higher Education 16 (2011): 99-111; Tsingos-Lucas et al. "Using Reflective Writing as a Predictor of Academic Success in Different Assessment Formats." American Journal of Pharmaceutical Education 81 (2017): Article 8; What Benefits Might Reflective Writing Have for My Students? Writing Across the Curriculum Clearinghouse; Rykkje, Linda. "The Tacit Care Knowledge in Reflective Writing: A Practical Wisdom." International Practice Development Journal 7 (September 2017): Article 5; Using Reflective Writing to Deepen Student Learning . Center for Writing, University of Minnesota.

How to Approach Writing a Reflection Paper

Thinking About Reflective Thinking

Educational theorists have developed numerous models of reflective thinking that your professor may use to frame a reflective writing assignment. These models can help you systematically interpret your learning experiences, thereby ensuring that you ask the right questions and have a clear understanding of what should be covered. A model can also represent the overall structure of a reflective paper. Each model establishes a different approach to reflection and will require you to think about your writing differently. If you are unclear how to fit your writing within a particular reflective model, seek clarification from your professor. There are generally two types of reflective writing assignments, each approached in slightly different ways.

1.  Reflective Thinking about Course Readings

This type of reflective writing focuses on thoughtfully thinking about the course readings that underpin how most students acquire new knowledge and understanding about the subject of a course. Reflecting on course readings is often assigned in freshmen-level, interdisciplinary courses where the required readings examine topics viewed from multiple perspectives and, as such, provide different ways of analyzing a topic, issue, event, or phenomenon. The purpose of reflective thinking about course readings in the social and behavioral sciences is to elicit your opinions, beliefs, and feelings about the research and its significance. This type of writing can provide an opportunity to break down key assumptions you may have and, in so doing, reveal potential biases in how you interpret the scholarship.

If you are assigned to reflect on course readings, consider the following methods of analysis as prompts that can help you get started :

• Examine carefully the main introductory elements of the reading, including the purpose of the study, the theoretical framework being used to test assumptions, and the research questions being addressed. Think about what ideas stood out to you. Why did they? Were these ideas new to you or familiar in some way based on your own lived experiences or prior knowledge?
• Develop your ideas around the readings by asking yourself, what do I know about this topic? Where does my existing knowledge about this topic come from? What are the observations or experiences in my life that influence my understanding of the topic? Do I agree or disagree with the main arguments, recommended course of actions, or conclusions made by the author(s)? Why do I feel this way and what is the basis of these feelings?
• Make connections between the text and your own beliefs, opinions, or feelings by considering questions like, how do the readings reinforce my existing ideas or assumptions? How the readings challenge these ideas or assumptions? How does this text help me to better understand this topic or research in ways that motivate me to learn more about this area of study?

2.  Reflective Thinking about Course Experiences

This type of reflective writing asks you to critically reflect on locating yourself at the conceptual intersection of theory and practice. The purpose of experiential reflection is to evaluate theories or disciplinary-based analytical models based on your introspective assessment of the relationship between hypothetical thinking and practical reality; it offers a way to consider how your own knowledge and skills fit within professional practice. This type of writing also provides an opportunity to evaluate your decisions and actions, as well as how you managed your subsequent successes and failures, within a specific theoretical framework. As a result, abstract concepts can crystallize and become more relevant to you when considered within your own experiences. This can help you formulate plans for self-improvement as you learn.

If you are assigned to reflect on your experiences, consider the following questions as prompts to help you get started :

• Contextualize your reflection in relation to the overarching purpose of the course by asking yourself, what did you hope to learn from this course? What were the learning objectives for the course and how did I fit within each of them? How did these goals relate to the main themes or concepts of the course?
• Analyze how you experienced the course by asking yourself, what did I learn from this experience? What did I learn about myself? About working in this area of research and study? About how the course relates to my place in society? What assumptions about the course were supported or refuted?
• Think introspectively about the ways you experienced learning during the course by asking yourself, did your learning experiences align with the goals or concepts of the course? Why or why do you not feel this way? What was successful and why do you believe this? What would you do differently and why is this important? How will you prepare for a future experience in this area of study?

NOTE: If you are assigned to write a journal or other type of on-going reflection exercise, a helpful approach is to reflect on your reflections by re-reading what you have already written. In other words, review your previous entries as a way to contextualize your feelings, opinions, or beliefs regarding your overall learning experiences. Over time, this can also help reveal hidden patterns or themes related to how you processed your learning experiences. Consider concluding your reflective journal with a summary of how you felt about your learning experiences at critical junctures throughout the course, then use these to write about how you grew as a student learner and how the act of reflecting helped you gain new understanding about the subject of the course and its content.

ANOTHER NOTE: Regardless of whether you write a reflection paper or a journal, do not focus your writing on the past. The act of reflection is intended to think introspectively about previous learning experiences. However, reflective thinking should document the ways in which you progressed in obtaining new insights and understandings about your growth as a learner that can be carried forward in subsequent coursework or in future professional practice. Your writing should reflect a furtherance of increasing personal autonomy and confidence gained from understanding more about yourself as a learner.

Structure and Writing Style

There are no strict academic rules for writing a reflective paper. Reflective writing may be assigned in any class taught in the social and behavioral sciences and, therefore, requirements for the assignment can vary depending on disciplinary-based models of inquiry and learning. The organization of content can also depend on what your professor wants you to write about or based on the type of reflective model used to frame the writing assignment. Despite these possible variations, below is a basic approach to organizing and writing a good reflective paper, followed by a list of problems to avoid.

Pre-flection

In most cases, it's helpful to begin by thinking about your learning experiences and outline what you want to focus on before you begin to write the paper. This can help you organize your thoughts around what was most important to you and what experiences [good or bad] had the most impact on your learning. As described by the University of Waterloo Writing and Communication Centre, preparing to write a reflective paper involves a process of self-analysis that can help organize your thoughts around significant moments of in-class knowledge discovery.

• Using a thesis statement as a guide, note what experiences or course content stood out to you , then place these within the context of your observations, reactions, feelings, and opinions. This will help you develop a rough outline of key moments during the course that reflect your growth as a learner. To identify these moments, pose these questions to yourself: What happened? What was my reaction? What were my expectations and how were they different from what transpired? What did I learn?
• Critically think about your learning experiences and the course content . This will help you develop a deeper, more nuanced understanding about why these moments were significant or relevant to you. Use the ideas you formulated during the first stage of reflecting to help you think through these moments from both an academic and personal perspective. From an academic perspective, contemplate how the experience enhanced your understanding of a concept, theory, or skill. Ask yourself, did the experience confirm my previous understanding or challenge it in some way. As a result, did this highlight strengths or gaps in your current knowledge? From a personal perspective, think introspectively about why these experiences mattered, if previous expectations or assumptions were confirmed or refuted, and if this surprised, confused, or unnerved you in some way.
• Analyze how these experiences and your reactions to them will shape your future thinking and behavior . Reflection implies looking back, but the most important act of reflective writing is considering how beliefs, assumptions, opinions, and feelings were transformed in ways that better prepare you as a learner in the future. Note how this reflective analysis can lead to actions you will take as a result of your experiences, what you will do differently, and how you will apply what you learned in other courses or in professional practice.

Basic Structure and Writing Style

Reflective Background and Context

The first part of your reflection paper should briefly provide background and context in relation to the content or experiences that stood out to you. Highlight the settings, summarize the key readings, or narrate the experiences in relation to the course objectives. Provide background that sets the stage for your reflection. You do not need to go into great detail, but you should provide enough information for the reader to understand what sources of learning you are writing about [e.g., course readings, field experience, guest lecture, class discussions] and why they were important. This section should end with an explanatory thesis statement that expresses the central ideas of your paper and what you want the readers to know, believe, or understand after they finish reading your paper.

Reflective Interpretation

Drawing from your reflective analysis, this is where you can be personal, critical, and creative in expressing how you felt about the course content and learning experiences and how they influenced or altered your feelings, beliefs, assumptions, or biases about the subject of the course. This section is also where you explore the meaning of these experiences in the context of the course and how you gained an awareness of the connections between these moments and your own prior knowledge.

Guided by your thesis statement, a helpful approach is to interpret your learning throughout the course with a series of specific examples drawn from the course content and your learning experiences. These examples should be arranged in sequential order that illustrate your growth as a learner. Reflecting on each example can be done by: 1)  introducing a theme or moment that was meaningful to you, 2) describing your previous position about the learning moment and what you thought about it, 3) explaining how your perspective was challenged and/or changed and why, and 4) introspectively stating your current or new feelings, opinions, or beliefs about that experience in class.

It is important to include specific examples drawn from the course and placed within the context of your assumptions, thoughts, opinions, and feelings. A reflective narrative without specific examples does not provide an effective way for the reader to understand the relationship between the course content and how you grew as a learner.

Reflective Conclusions

The conclusion of your reflective paper should provide a summary of your thoughts, feelings, or opinions regarding what you learned about yourself as a result of taking the course. Here are several ways you can frame your conclusions based on the examples you interpreted and reflected on what they meant to you. Each example would need to be tied to the basic theme [thesis statement] of your reflective background section.

• Your reflective conclusions can be described in relation to any expectations you had before taking the class [e.g., “I expected the readings to not be relevant to my own experiences growing up in a rural community, but the research actually helped me see that the challenges of developing my identity as a child of immigrants was not that unusual...”].
• Your reflective conclusions can explain how what you learned about yourself will change your actions in the future [e.g., “During a discussion in class about the challenges of helping homeless people, I realized that many of these people hate living on the street but lack the ability to see a way out. This made me realize that I wanted to take more classes in psychology...”].
• Your reflective conclusions can describe major insights you experienced a critical junctures during the course and how these moments enhanced how you see yourself as a student learner [e.g., "The guest speaker from the Head Start program made me realize why I wanted to pursue a career in elementary education..."].
• Your reflective conclusions can reconfigure or reframe how you will approach professional practice and your understanding of your future career aspirations [e.g.,, "The course changed my perceptions about seeking a career in business finance because it made me realize I want to be more engaged in customer service..."]
• Your reflective conclusions can explore any learning you derived from the act of reflecting itself [e.g., “Reflecting on the course readings that described how minority students perceive campus activities helped me identify my own biases about the benefits of those activities in acclimating to campus life...”].

NOTE: The length of a reflective paper in the social sciences is usually less than a traditional research paper. However, don’t assume that writing a reflective paper is easier than writing a research paper. A well-conceived critical reflection paper often requires as much time and effort as a research paper because you must purposeful engage in thinking about your learning in ways that you may not be comfortable with or used to. This is particular true while preparing to write because reflective papers are not as structured as a traditional research paper and, therefore, you have to think deliberately about how you want to organize the paper and what elements of the course you want to reflect upon.

ANOTHER NOTE: Do not limit yourself to using only text in reflecting on your learning. If you believe it would be helpful, consider using creative modes of thought or expression such as, illustrations, photographs, or material objects that reflects an experience related to the subject of the course that was important to you [e.g., like a ticket stub to a renowned speaker on campus]. Whatever non-textual element you include, be sure to describe the object's relevance to your personal relationship to the course content.

Problems to Avoid

A reflective paper is not a “mind dump” . Reflective papers document your personal and emotional experiences and, therefore, they do not conform to rigid structures, or schema, to organize information. However, the paper should not be a disjointed, stream-of-consciousness narrative. Reflective papers are still academic pieces of writing that require organized thought, that use academic language and tone , and that apply intellectually-driven critical thinking to the course content and your learning experiences and their significance.

A reflective paper is not a research paper . If you are asked to reflect on a course reading, the reflection will obviously include some description of the research. However, the goal of reflective writing is not to present extraneous ideas to the reader or to "educate" them about the course. The goal is to share a story about your relationship with the learning objectives of the course. Therefore, unlike research papers, you are expected to write from a first person point of view which includes an introspective examination of your own opinions, feelings, and personal assumptions.

A reflection paper is not a book review . Descriptions of the course readings using your own words is not a reflective paper. Reflective writing should focus on how you understood the implications of and were challenged by the course in relation to your own lived experiences or personal assumptions, combined with explanations of how you grew as a student learner based on this internal dialogue. Remember that you are the central object of the paper, not the research materials.

A reflective paper is not an all-inclusive meditation. Do not try to cover everything. The scope of your paper should be well-defined and limited to your specific opinions, feelings, and beliefs about what you determine to be the most significant content of the course and in relation to the learning that took place. Reflections should be detailed enough to covey what you think is important, but your thoughts should be expressed concisely and coherently [as is true for any academic writing assignment].

Critical Reflection . Writing and Communication Centre, University of Waterloo; Critical Reflection: Journals, Opinions, & Reactions . University Writing Center, Texas A&M University; Connor-Greene, Patricia A. “Making Connections: Evaluating the Effectiveness of Journal Writing in Enhancing Student Learning.” Teaching of Psychology 27 (2000): 44-46; Good vs. Bad Reflection Papers , Franklin University; Dyment, Janet E. and Timothy S. O’Connell. "The Quality of Reflection in Student Journals: A Review of Limiting and Enabling Factors." Innovative Higher Education 35 (2010): 233-244: How to Write a Reflection Paper . Academic Skills, Trent University; Amelia TaraJane House. Reflection Paper . Cordia Harrington Center for Excellence, University of Arkansas; Ramlal, Alana, and Désirée S. Augustin. “Engaging Students in Reflective Writing: An Action Research Project.” Educational Action Research 28 (2020): 518-533; Writing a Reflection Paper . Writing Center, Lewis University; McGuire, Lisa, Kathy Lay, and Jon Peters. “Pedagogy of Reflective Writing in Professional Education.” Journal of the Scholarship of Teaching and Learning (2009): 93-107; Critical Reflection . Writing and Communication Centre, University of Waterloo; How Do I Write Reflectively? Academic Skills Toolkit, University of New South Wales Sydney; Reflective Writing . Skills@Library. University of Leeds; Walling, Anne, Johanna Shapiro, and Terry Ast. “What Makes a Good Reflective Paper?” Family Medicine 45 (2013): 7-12; Williams, Kate, Mary Woolliams, and Jane Spiro. Reflective Writing . 2nd edition. London: Red Globe Press, 2020; Yeh, Hui-Chin, Shih-hsien Yang, Jo Shan Fu, and Yen-Chen Shih. “Developing College Students’ Critical Thinking through Reflective Writing.” Higher Education Research and Development (2022): 1-16.

Writing Tip

Focus on Reflecting, Not on Describing

Minimal time and effort should be spent describing the course content you are asked to reflect upon. The purpose of a reflection assignment is to introspectively contemplate your reactions to and feeling about an element of the course. D eflecting the focus away from your own feelings by concentrating on describing the course content can happen particularly if "talking about yourself" [i.e., reflecting] makes you uncomfortable or it is intimidating. However, the intent of reflective writing is to overcome these inhibitions so as to maximize the benefits of introspectively assessing your learning experiences. Keep in mind that, if it is relevant, your feelings of discomfort could be a part of how you critically reflect on any challenges you had during the course [e.g., you realize this discomfort inhibited your willingness to ask questions during class, it fed into your propensity to procrastinate, or it made it difficult participating in groups].

Writing a Reflection Paper . Writing Center, Lewis University; Reflection Paper . Cordia Harrington Center for Excellence, University of Arkansas.

Another Writing Tip

Helpful Videos about Reflective Writing

These two short videos succinctly describe how to approach a reflective writing assignment. They are produced by the Academic Skills department at the University of Melbourne and the Skills Team of the University of Hull, respectively.

• << Previous: Writing a Policy Memo
• Next: Writing a Research Proposal >>
• Last Updated: Jun 3, 2024 9:44 AM
• URL: https://libguides.usc.edu/writingguide/assignments

• Student life
• Accommodation
• About Royal Holloway
• Social Sciences, Arts and Humanities
• Management and Economics
• Business and Management
• Computer Science
• Electronic Engineering
• Computer Science and Electronic Engineering
• Arts, Humanities, Economics and Sciences
• Online study
• English Language Preparation
• Prepare for Success
• Enhanced Induction
• International Foundation Year and International Year One
• Pre-Masters
• Fees, accommodation and living costs
• Visa information
• Preparing to arrive
• Enquire now

How to write a reflection paper

• Link copied!

What is a reflection paper?

A reflection paper is an essay that requires you to express your opinion on a topic. In the paper, you will analyse and reflect upon how a book, experience or academic lecture shaped your thoughts and opinions on a subject.   

It is one of the only academic essays where you get to discuss your own opinion and, the best part is, there is no wrong answer.  

Types of reflection papers

There are two main types of reflection papers. The first type is experiential – which is an analysis of a personal experience or observation. In the paper, you will summarise and highlight underlying principles that support your analysis of the experience.  

The second type is textual. This type of paper analyses a written text, which can be an article, essay or book. Your opinion, analysis and interpretation of the material will be backed up using specific quotations. 

Tips on writing a reflection paper

Choose an interesting topic.

Having an interesting topic is important for both you and the reader. Having an interest in what you are writing about will make the paper more enjoyable for you to write. An interesting topic will also make the paper more engaging for the reader.  

An example of a reflection paper theme is: how my views on pop music have changed over time.  

Keep information relevant

Typically, reflection papers are between 250 and 800 words long, and because of the short length it is important to only include relevant information. Avoid tangents and lengthy summaries to ensure you get your points across in the allotted word count.

Keep your tone professional

Although you will be talking about your thoughts and opinions, it is important to remember that writing a reflection paper requires you to use a professional tone suitable for an academic environment.  

Before you can begin writing, it is important to understand how to start a reflection paper. These simple steps will take you from beginning to end through careful planning and thorough analysis.  

Select a topic and summarise the material or experience

To begin your reflection paper, you must first decide on the topic you want to write about. Once you have done this, write a short summary about what you have learned from your experience with the topic. This can include memorable information or specific quotes, pre-existing thoughts and feelings and ways in which it has influenced you.  

Review and analyse the material

Once you have chosen a topic, you need to review and analyse the material. This will allow you to form coherent themes by looking deeper into the meaning and thought behind the text.  

Some good questions to ask yourself when reviewing and analysing material are: 

• How has the material affected me? 
• What have I learned? 
• How does the material catch the reader’s attention? 
• Are there unsolved questions or critical issues? 
• How will the material affect my future thinking? 

These questions will help to streamline your thoughts and opinions of your subject. They will also ensure that your reflection paper flows and is well organised. 

Select a theme

After analysing the material, you can use what you found to select a main theme for your paper. To do so, you should find common points and arguments that incited strong opinions from your analysis.  

Choosing an interesting main theme is important as it will set the tone for your paper and will also make it more engaging for the reader. 

Make connections between your opinions

To give your reflection paper a coherent structure, you should make connections between our opinions. Doing so will give you a clear outline of what to include in the body paragraphs and will ensure your paper flows well.  

Write the paper 

The last thing you will need to do is write the paper. This should be done in three sections: an introduction, the body paragraphs and a conclusion.  

Express your opinions in a concise and academic manner and remember to proofread multiple times to avoid factual and grammatical errors.  

Reflection paper format

Despite unlimited subject options, the typical format of a reflection paper is the same for every essay. 

Introduction

The introduction of your reflection paper will contain the purpose and topic of the essay. You will state the thesis of the paper and give the reader an insight into the reasoning behind your choice of topic. 

The main purpose of a reflection paper is to discuss your thoughts and opinions, so make sure you clearly state your feelings towards your subject. This must be done in a professional manner. 

In the introduction you can include a brief summary of the book, article or experience you are analysing and the themes and topics you are going to explore.

Body paragraphs 

The body paragraphs are where you will present an in-depth analysis of your thesis statement. In them you can include direct quotations and references, examples and supporting arguments.  

If you are reflecting on an experience, use the body paragraphs to introduce the experience, talk about how it has influenced you and what you have learned from it.   

Each body paragraph should introduce a new idea. It is best practice to start each paragraph with a topic sentence. This ensures your paper will have good flow and organisation.  

Conclusion 

Your conclusion should summarise the ideas and opinions you have described in the body paragraphs. You should describe what you have learned through your analysis of the experience or text and areas for further learning.   

It is important to never introduce new ideas in the conclusion of a reflection paper. This section should only be used to restate your original thesis statement.   

If you include quotes or information from secondary sources, you will need to correctly reference them at the end of your reflection paper. This list will need to be formatted accurately to your university/organisation standard.   

Writing a reflection paper can be a common part of your university experience. If you are an international student looking to study a degree in the UK,  Royal Holloway International Study Centre  has a range of pathway programmes to prepare you for university study. As well as pathway programmes, you can also study our introductory  Prepare for Success programme  to further develop your study skills.  

How can you include references in a reflection paper?

Including references in a reflection paper is not mandatory. However, if you want to support your arguments using secondary sources you can do so by citing them correctly within the reflection paper text and including a reference list at the end of the essay.   

How long should a reflection paper be?

Typically, a reflection paper is around 250-800 words. However, the length can vary depending on your level of study and what you are studying.  

What does a reflection paper consist of? 

The format of a reflection paper consists of three parts. These are: 

• The introduction – this is where you will state your subject of reflection and thesis 
• The body paragraphs – here you will describe your subject including your thoughts, feelings and how the material has affected you 
• The conclusion – a summary of what you have found out, bringing your arguments and opinions together to restate your original thesis.   

Personal Reflection Sample: preparing a Research Report for ACCA

The skill and learning statement includes the implications of interactions with mentor, an analysis of the extent to which research questions have been answered, a brief analysis of interpersonal and communication skills and their relevance to the research, as well as the contribution of the research experience to my professional and personal development.

1.      Experiences of interactions with mentor

I had chances of meeting my project mentor three times and obtained practical support regarding various aspects of the work during these meetings. Our first meeting was mainly dedicated to clarifying our expectations from the research experience and the discussions took place related to the issues of selection of the research approach and formulation of research questions and objectives.

By the time I had a meeting with my mentor for the second time Introduction and Information gathering chapters of the work have been completed and I received detailed feedback for these chapters of the research. Also, discussions were held about data analysis and presentation associated with the project.

During the final meeting with my mentor the overall work has been scrutinised and a set of specific points have been mentioned by my mentor. Specifically, my mentor raised a point that my discussions of research findings lacked depth and scale. Then, these points have been addressed and the final draft of the Research Report was completed.

I found advices given by my mentor very helpful in terms of increasing the quality of my Research Report and equipping me with knowledge of effectively conducting similar studies in the future in general. Moreover, my Project Mentor was not only highlighting the shortages that were associated with my project, but also was giving detailed explanations why these changes were desirable in a passionate manner.

Furthermore, I found these three sessions with my mentor to be highly motivational and informative experience because they have increased the level of my personal interest in conducting businesses studies. Prior to conducting the Research Report and having discussions with my mentor I was assuming conducting analytical business studies to be a rather boring experience.

However, thanks to my mentor I learned to appreciate the importance of analysing a business case in terms of identifying a current strategic and financial position of a business, and formulating the ways of identifying further strategic options available to the business.

2.      The extent to which research questions have been answered

Answering the research questions in my Research Report were directly related to the quality of secondary data, and the choice of methodology. Therefore, these issues were approached effectively by critically assessing the validity of the sources of secondary data and assessing alternative choices of methodology. Moreover, my first meeting with my Project mentor was mainly devoted to the discussion of the same issues.

As a result of comprehensive analysis the most reliable sources of secondary data in order to be used in Research Report were found to include published financial statements and annual reports, textbooks on financial and business analysis, information published in official company website, information available from ACCA website, as well as, various business journals an newspapers.

The choice of methods for conducting the study, on the other hand, was guided by the reliability of the data analysis methods and their relevance to the research issues. After spending additional amount of time for the choice of appropriate methodology and taking into account advises of my mentor, financial ratios and analytic tools have been chosen to be employed in my Research Report.

Purposely, financial and accounting ratios that were used in the study include profitability, liquidity, financial position and investor ratios, whereas, the choice of analytic tools consist of SWOT, PESTLE, and Porter’s five forces analysis.

To summarise this part, it is fair to state that all of the research questions in my Research Report have been effectively addressed, because the secondary data have been obtained from reliable sources, relevant methodology has been used to conduct the study, and the research findings have been critically discussed.

3.      Interpersonal and communication skills and their relevance to the research

I have demonstrated my interpersonal and communication skills at various stages of doing Research Report and preparing for and making the presentation. Moreover, without my interpersonal and communication skills completing the Research Report and doing the presentation would have proved to be highly challenging.

For example, my listening skills have proved to be highly valuable in terms of understanding vital information given by my mentor about increasing the quality of my Research Report, because these advises were fully understood and implemented into the practice.

My interpersonal skills have also played a positive role when I asked some of my trusted colleagues to be an audience when I was rehearsing my presentation. I was making presentations in front of my colleagues and was asking for their opinions about the quality of my presentation. This practice took place many times in different settings and I believe that following this strategy has enhanced the quality of my presentation and my marks.

However, my communication skills have played a crucial role in terms of succeeding in making the presentation effectively. I have learned from my experiences within and outside of academic settings that communication skills play the most crucial role in terms of succeeding in personal and professional lives.

For instance, an individual may possess a deep knowledge about a certain area. However, if the individual lacks competency of communicating his or her ideas, knowledge and feelings in an effective manner, the overall competency of the individual and the level of his or her contribution to the organisation will always remain compromised.

Therefore, in my opinion, regardless of the field, industry or type of organisation, communication skills can be specified as a compulsory attribute for an employee in order to be considered an a competent. In my case in particular, my advanced level of communication skills have enabled me to do my Research Report presentation effectively which has resulted in positive acclaim from my peers and mentor.

4.      The potential contribution of Research Report to the level of professional development

Conducting the Research Report and doing the presentation has increased the level of my professional competency in several ways. First of all, I have to mention the fact that I have developed a critical mindset towards solving business issues as a result of conducting the Research Report.

My mentor made it clear that it was important to critically analyse related issues in Research Report rather than just offering description of the issues and supplying calculations. The mentor had stressed many times that critical analysis and discussions are the elements of the work that increase its value. For the same reason I had to revise my Research Report several times until my mentor was satisfied with the level of critical analysis the work had included.

Although, such an approach to work seemed to be very challenging and confusing during the research process, I appreciated the value of critical analysis once the final work was completed. The skills of critical analysis that I have developed and applied in Research Report can easily be applied when real business issues would need to be resolved by me in the future in my professional capacity.

Completing the Research Report was similar to project management in real businesses environment in terms of strict deadlines, scarcity of resources, organising and planning, scheduling meetings, doing presentations etc. Therefore, the skills I developed during the process of completing Research Report can be used in order to successfully manage business projects in the future.

Moreover, my writing skills have also been greatly improved as a result of engaging in Research Report. Despite the popular opinion that with the increasing importance of information technology the practice of writing letters and reports are being replaced by alternative means of business communications, the importance of writing will always remain significant for business managers.

From this point of view engaging in Research Report was a very beneficial experience for me on a personal level. Specifically, writing the paper of almost ten thousand words in total, including this personal reflection, has made me better prepared to join the full-time workforce once my studies are completed.

Lastly, as a result of preparing the Research Report my professional interest on the issues associated with corporate strategy has been enhanced. Moreover, I am planning to continue studying the issues of corporate strategy and that knowledge would benefit me in the future as a corporate leader.

5.      Gains derived from conducting Research Report experience on a personal level

On a personal level I benefited from conducting the Research Report and doing the presentation in a number of ways. The research experience with Oxford Brookes has increased the level of my motivation for studying, making bold plans for my future career and implements necessary measures and initiatives in order to accomplish these plans. My mentor deserves to be mentioned here specifically for all encouragements and practical tips that can be applied in various alternative settings apart from academic life.

The level of my self-confidence has also been increased because I could complete the Research Report in time. Moreover, the presentation experience has increased the level of my self-confidence dramatically, because I understood that if I could do a successful presentation in front of my mentor and colleagues, doing the presentations of multi-million projects in front of top executives was just a matter of time.

The paramount importance of self-confidence for an individual is an undisputable matter. Self-confidence allows us to set ambitious plans and utilise all the available resources efficiently in order to achieve these plans.

My time-management skills have also been improved by the end of the Research Report. This is because there was a specific deadline for both, the Research Report and presentation and I had to adopt some principles related to time management in order to be able to submit my work on time.

These principles included setting specific deadlines for each chapter of the work, and above all, dramatically cutting the amount of time I used to browse social networking sites on the internet. I can highlight this fact as one of the most substantial gains in a personal level. This is because prior to the research experience I used to spend several hours a day browsing a set of social networking sites with no real benefit whatsoever. However, once the priority was given to the Research Project, this bad habit was dealt with effectively and irreversibly.

6.      Conclusions

To summarise, completing the Research Report and making presentation with Oxford Brookes University following my ACCA course has increased the level of my preparedness to join the full-time workforce and successfully utilise my energy and knowledge. In my opinion the biggest benefit I received from enrolling to this course of study is that the course of study, the Research Report and doing the presentation have made me to believe in my skills and capabilities and they have also awoke my desire to approach studying as a lifelong process.

Moreover, I have obtained a set of professional and personal gains as a result of completing the Research Report and making presentation that include the development of a critical mindset, improvement my writing and time management skills and enhancement of the level of my self-confidence.

• Open access
• Published: 22 August 2024

Using the behaviour change wheel to develop a tailored intervention to overcome general practitioners’ perceived barriers to referring insomnia patients to digital therapeutic sleepio

• Ohoud Alkhaldi 1 , 2 ,
• Brian McMillan 3 &
• John Ainsworth 1 , 4  

BMC Health Services Research volume  24 , Article number:  967 ( 2024 ) Cite this article

20 Accesses

Metrics details

Digital therapeutic Sleepio has proven effective in improving sleep quality and decreasing symptoms of anxiety. The National Institute for Health and Care Excellence (NICE) guidance recommends Sleepio as an alternative treatment to usual sleep hygiene education and hypnotic medications. General practitioners (GPs) play a critical role in the adoption of digital therapeutics in patient care. Previous interventions did not adopt theoretical frameworks to systematically understand GPs behaviour toward referring patients to digital therapeutics.

This study aimed to report the systematic and comprehensive development of an intervention to encourage GPs to refer insomnia patients to Sleepio, using the Behaviour Change Wheel (BCW).

The eight steps outlined in the BCW were followed to develop an intervention. The Capability Opportunity Motivation-Behaviour Self-Evaluation Questionnaire (COM-B-Qv1) was adopted to understand GPs perceived facilitators and barriers to refer insomnia patients to Sleepio. The Behaviour Change Technique Taxonomy Version 1 (BCTv1) was thereafter used to identify possible strategies that could be used to facilitate changes in GPs’ behaviour in relation to Sleepio.

The BCW design process resulted in the identification of five intervention functions, three policy categories and five behaviour change techniques (BCTs) as potential active components for an intervention. The intervention includes providing GPs with an orientation about using Sleepio to improve their knowledge and confidence, sending visual reminders to GPs to recommend Sleepio to their patients, providing ongoing technical support.

The BCW can be successfully applied through a systematic process to understand the drivers of GPs’ behaviour and to develop an intervention that can encourage them to refer insomnia patients to Sleepio.

Peer Review reports

Introduction

Insomnia is a health condition that causes difficulty to sleep or a total lack of sleep [ 1 ]. It is estimated to affect one in three adults in the UK, and higher rates are associated with being female, being older and the presence of comorbidities [ 2 , 3 ]. Insomnia is associated with a significant economic burden due to absenteeism, reduced productivity and impaired cognition and mood [ 4 ].

Guidelines recommend cognitive behavioural therapy for insomnia (CBT-I) for the treatment of insomnia rather than pharmacologic and drug therapy due to reported side effects, such as the possibility of developing long-term tolerance and addiction [ 5 ]. Studies have shown that benzodiazepine use is a significant risk factor for fall-related accidents among older adults [ 6 , 7 ].

CBT-I is a psychological treatment that guides patients to change their sleep-related behaviour through a series of techniques in weekly courses that last for five weeks [ 8 ]. However, there are difficulties surrounding CBT-I that limit its prescription to patients, such as issues with accessibility and costs, and poor response [ 8 ].

Sleepio is a digital therapeutic that uses CBT-I and can be accessed through self-referral or general practitioner (GP) referrals. The programme has proven effective in improving sleep quality and decreasing symptoms of anxiety and depression [ 9 ]. In an ambitious attempt to integrate digital therapeutics into patient care, NHS Scotland made Sleepio free of charge to all residents of Scotland in October 2020 [ 10 ].

For GPs, recommending digital therapeutics differs from the standard practice of prescribing medications. Evidence of interventions to improve GPs’ adoption of mobile health (mHealth) apps and/ or digital therapeutics is limited. One interrupted time series study aimed to evaluate the cost-effectiveness of Sleepio in primary care settings in England. As part of the study, attention was given to selected GP practices that involved implementing training and digital prompts for GPs and distributing patient-centred resources and awareness material to practices, along with tailored support [ 11 ]. Some of these implementation strategies targeted patients, while others were tailored to healthcare professionals (HCPs). The study found that a lower level of uptake was observed in areas that lacked involvement in implementation strategies, which suggests that for future work, the impact of resources on Sleepio referrals should be assessed. In another study, the intervention included 40 min of Sleepio training for clinical staff, a protocol for all clinical assessments and a website with logistical tools to assist staff [ 12 ]. There were no measures of the intervention’s impact on GPs’ referral behaviour. Therefore, this paper describes the development of an intervention to encourage GPs to refer insomnia patients to Sleepio.

To understand the current behaviour and select the best interventions that will most likely be beneficial, the behaviour change wheel (BCW) will be used [ 13 ]. This framework provides a comprehensive approach to identifying sources of behaviour and classifying them into the capability, opportunity, motivation, and behaviour (COM-B) model. This model helps to understand the behaviour of interest and select the behaviour change techniques (BCTs) most likely to be effective. The COM-B model is designed to capture factors that affect the target behaviour (physical capability, psychological capability, physical opportunity, social opportunity, reflective motivation and automatic motivation).

The theoretical domain framework (TDF) is a comprehensive framework used in behaviour science and implementation research to determine the key areas that affect behaviour change [ 14 ]. The TDF is used in conjunction with the COM-B model if a more detailed understanding of the behaviour is required.

Several studies in the literature have used the BCW framework to design interventions targeting GPs. One study aimed to assess the effectiveness of a COM-B-based intervention in promoting physical activity among GPs and their patients [ 15 ]. Another study explored the role of GPs in facilitating behaviour change using the TDF and BCW [ 16 ]. While no studies in the literature focused on the use of BCW with GPs in the context of mHealth, the findings of other studies provided insights that BCW could be applied to other disciplines. Overall, the use of the BCW in conjunction with GPs has proven to be valuable in understanding and addressing barriers to behaviour change.

The intervention development followed the BCW developed by Michie et al. [ 13 ]. The process consists of eight steps, as follows:

Step 1: Define the problem

We reviewed the literature regarding barriers to and facilitators of prescribing mHealth apps in general from the point of view of HCPs. By doing this, we aimed to cover all factors that affect behaviour, including environmental, physical and social contexts.

Step 2: Select the target behaviour

In this step, all possible factors that affect behaviour and could be targeted in the intervention were investigated. To determine the target behaviour, the literature on GPs’ use of evidence-based digital therapeutics in the UK was reviewed.

Step 3: Specify the target behaviour

After selecting the target behaviour, specific details were identified, such as who would perform it, what must be done to achieve the desired change and when, where and how often it needed to be done.

Step 4: Identify what needs to change

This step involved behavioural analysis using the COM-B model (capability, opportunity and motivation to recommend a digital therapeutic) to identify which component of the model needed to change to achieve the desired results. The COM-B self-evaluation questionnaire (COM-B-Qv1) was used to understand what it would take for participants to change their behaviour.

Survey development and analysis

GPs in Scotland were invited to take part in the survey because Sleepio is only available for GP referrals in Scotland. The survey was prepared using Qualtrics ( www.qualtrics.com ) and distributed online, mainly through the primary care research network in Scotland. The NHS Research Scoltand (NRS) Primary Care Network is a unit that supports researchers in recruiting participants using electronic databases [ 17 ].

The survey (Supplementary file 1 ) consisted of 27 items, including questions relating to demographics, GPs’ behaviour in terms of recommending Sleepio, and questions based on the COM-B self-evaluation questionnaire (COM-B-Qv1), which is recommended for collecting data during the BCW intervention development process [ 13 ].

The (COM-B-Qv1)18-item scale had a Cronbach’s alpha of 0.910, with 6-item subscale alphas of 0.793 for capabilities, 0.853 for opportunities and 0.812 for motivations. The full scale is available in the supplementary material (Supplementary File 1 includes the survey questions).

The survey was pilot-tested among 12 GPs and 5 health informatics PhD students at the University of Manchester, and feedback was sought about survey items, the format and the time taken for completion before finalising the survey.

GPs were provided with information about the survey (e.g. background and importance of the study, purpose of the study, potential benefits of taking part, and how their personal information would be stored and processed) via an information sheet on the online survey platform. GPs provided consent by selecting the checkbox to confirm that they agreed with the information provided and were happy to participate in the study. Participants received no compensation for completing the questionnaire. The estimated time to complete the survey 4–7 min and was available between available between February and April 2023 (75 days).

Questionnaire data were analysed using IBM SPSS Statistics V.25, including the descriptive data analysis of participant characteristics. The responses from the 5-point Likert scales were combined to create a 3-point scale by combining ‘agree’ and ‘strongly agree’ and ‘disagree’ and ‘strongly disagree’. The frequency and percentage of each COM-B statement response were calculated.

Step 5 & 6: Identify intervention functions and select policy categories

Based on the findings of the questionnaire, we used the BCW to select the most appropriate intervention functions to design the intervention.

Step 7: Select behaviour change techniques

Michie et al. identified 93 possible BCTs, each linked to intervention functions [ 18 ]. In this step, we selected the most effective techniques to produce a successful change in GPs’ prescribing behaviour.

Step 8: Determine the mode of delivery

After selecting BCTs, it is important to consider the mode or modes of delivery most appropriate for the target behaviour. In this step, we considered the difficulty of engaging GPs in research for reasons such as workload and lack of time.

Several studies have revealed that HCPs’ lack of knowledge and awareness of available apps are major barriers to incorporating them into patient care [ 19 , 20 , 21 ]. To overcome these barriers, several studies have emphasised the need to design training for GPs and other allied health professionals to improve their knowledge of the importance of prescribing mHealth apps to patients with long-term conditions [ 20 , 21 , 22 , 23 ]. The cost of using mobile health apps is also a concern for some GPs [ 24 ].

By reviewing the literature in the UK, we found that Sleepio is the first digital therapeutics to receive NICE guidance as an effective digital treatment for insomnia before prescribing sleeping drugs or sleep hygiene. Many studies concluded Sleepio is more effective than usual treatment in reducing symptoms of insomnia in adults [ 25 , 26 ]. However, GPs behaviour toward Sleepio remains unknown. We decided that the intervention should target GPs [ 27 ]. The study is expected to help GPs incorporate digital therapeutics into patient care and improve their confidence in recommending evidence-based apps.

After selecting the target behaviour, further details were determined by answering the questions in Table  1 . GPs can refer insomnia patients to Sleepio if Sleepio is deemed the right treatment option for them.

Step 4: Identify what needs to change

To identify what needed to change, we surveyed GPs in Scotland about their attitudes towards referring insomnia patients to Sleepio. Seventy participants responded to the questionnaire. Five questionnaires were incomplete, leaving sixty-five participants with a full set of data. Table  2 presents the participants’ demographic data.

Participants were asked to rate the extent to which they agreed with each statement. GPs’ ratings on questionnaire statements in each COM-B domain about what would make them recommend Sleepio to their patients are illustrated in Figs.  1 , 2 and 3 .

GPs’ Responses to Capability Statements

GPs’ Responses to Opportunity Statements

GPs’ Responses to Motivation Statements

From the survey, it was determined that the intervention needed to target most components of the COM-B model, with a strong focus on psychological capability, physical opportunity and automatic and reflective motivation.

Psychological capability

About 57% (37/65) of participants reported that knowing the clinical evidence behind the digital therapeutic Sleepio would encourage them to offer it to their patients. Around 56% (36/65) of respondents reported knowing how to determine whether patients would benefit from Sleepio by assessing its clinical suitability and patients’ ability to engage with it. This highlights knowledge, memory, attention and decision processes as important in the TDF to be addressed in the intervention.

Physical opportunity

The cost of digital therapeutics was significantly associated with a reduced likelihood of referring insomnia patients to Sleepio. Participants reported that they would refer patients to Sleepio if it was made freely available to them. Linking that with TDF domains, it was found that environmental context and resources were important in encouraging GPs to recommend Sleepio.

Automatic motivation

GPs reported that making changes to their prescribing habits would facilitate more frequent referrals to Sleepio. GPs need to discuss and recommend Sleepio for any patients who complain about their sleep patterns before prescribing medications. Reinforcement was found to be a crucial TDF domain for inclusion in the intervention.

Reflective motivation

GPs who believe that Sleepio can assist insomnia patients in regaining normal sleeping patterns are more likely to refer them to Sleepio. They need to believe that recommending Sleepio is the best practice. Therefore, targeting beliefs about the outcomes of the intervention would work as a facilitator for changing the target behaviour.

The COM-B behavioural analysis identified five intervention functions: education, training, environmental restructuring, enablement and persuasion. Policy categories that matched our intervention functions included communication/marketing (for instance, using verbal, electronic communication or flyers to improve knowledge of referring patients to Sleepio and health consequences of using Sleepio), guidelines (examples of which include informing GPs of steps for offering Sleepio) and environmental/social planning (e.g., sending a visual reminder to GPs to recommend Sleepio through emails) (Table  3 ).

In total, six behaviour change techniques were selected. The main BCTs selected for encouraging GPs to refer insomnia patients to Sleepio were information about health consequences, instruction on how to perform behaviour, prompts/cues, adding objects to the environment, self-monitoring of behaviour and credible sources.

GPs reported concerns that changing their work habits might cause an extra burden on their daily work schedules. Therefore, to ensure GPs’ engagement, the intervention is designed to be delivered online on an individual level and can be accessed via computers at a convenient time.

This study provides a structured and detailed example of how to design an intervention to target GPs in primary care using the BCW. The BCW framework was used to systematically understand the target behaviour before the intervention was designed in terms of changes to capability, opportunity and motivation (the COM-B system).

GPs reported that they would recommend Sleepio if they had greater capabilities especially in psychological capability domain. This included having better knowledge of convincing evidence of the benefits of Sleepio and knowing how to determine if someone would benefit from Sleepio. This is in line with previous recommendations regarding the need to design training for GPs and other allied health professionals for prescribing mHealth apps to patients with long-term conditions [ 19 , 20 , 21 ].

However, the findings of this study showed that providing GPs with information about Sleepio may not be enough to produce a change in the target behaviour. The survey results indicated that in relation to opportunity, GPs would recommend Sleepio more often if it was freely available to patients. This indicates a potential lack of awareness among GPs regarding Sleepio’s existing availability for GP referral to all adults in Scotland. It was found that this factor was the overarching barrier to referring patients to digital therapeutic Sleepio. In line with the findings of the current study, a previous study reported that the cost of apps was significantly associated with the likelihood of prescribing digital health technologies, suggesting that as cost increases, the rate of digital health technology prescriptions falls [ 24 ]. Moreover, a review of an intervention study found that while providing education and skills training is likely to improve nutritionists’ self-efficacy, having the app easily and freely integrated into dietetic care is essential to influence the prescribing of apps [ 28 ].

In relation to motivation to recommend Sleepio, GPs reported needing to develop a pattern of doing it routinely and have a stronger sense that it is best practice. GPs’ responses concerning motivation reflected that while they had a strong motivation to incorporate digital therapeutics into patient care or develop habits of recommending Sleepio to their patients, they may not have done so primarily due to the perceived difficulty of accessing Sleepio (opportunity) or a lack of knowledge (capability). Therefore, addressing barriers related to opportunity and capability is likely to produce changes in motivation [ 29 ].

In a future intervention, a number of BCTs will be included to maximise successful changes in the target behaviour, such as the inclusion of evidence and scientific rationale for using a digital therapeutic (Sleepio) to treat insomnia, providing GPs with clear steps for offering Sleepio and making sure that GPs are aware that Sleepio is made free to all adults in Scotland. Additionally, to address GPs concerns about the increased workload and time demands when apps are integrated into daily work activities [ 30 , 31 , 32 ], interventions should be delivered online. This will allow them to access the training materials at their convenience. We are in the process of designing an intervention and piloting it to improve GP referrals to Sleepio to treat insomnia patients as an alternative to usual treatments.

Strengths and limitations

This study has a number of strengths. To the authors’ knowledge, this is the first study to investigate the influence of both behavioural and environmental determinants on GPs’ referral attitudes towards the digital therapeutic Sleepio.

The BCW provided a systematic approach to achieving a better understanding of GPs’ perceived barriers to incorporating this digital therapeutic in routine care, and it was found effective in designing an intervention to target GPs’ needs. Furthermore, the COM-B self-evaluation questionnaire (COM-B-Qv1) provided information on self-reported behavioural determinants of Sleepio referrals, which enabled participants to consider a wide spectrum of factors relating to these (i.e. the capability, opportunity and motivation subscales).

With regard to study limitations, we recruited GPs in Scotland only, which reduces the study’s generalisability. Generalisability was also affected by the limited sample size, although the study was advertised using a research network in Scotland. To enhance generalisability, future studies should combine multiple approaches to increase GP participation, such as using social media and targeting GPs in conferences. A possible source of bias is that GPs who were interested in mobile health and cognitive behaviour therapy were more likely to be motivated to complete the survey, leading to self-selection bias.

Conclusions

This study identified a number of intervention components that can be applied to encourage GPs to recommend Sleepio for CBT-I treatment as an alternative to medications for insomnia. This study highlighted the importance of interventions targeting multiple levels of behaviour to produce change. Six BCTs were identified as core methods that affect psychological capability, physical opportunity, automatic motivation and reflective motivation of GPs’ behaviour with regard to referring patients to Sleepio. Future studies should evaluate the feasibility of an intervention based on the findings reported here.

Data availability

The authors confirm that the data supporting the findings of this study are available within the article and its supplementary materials.

Abbreviations

Behaviour Change Techniques

Behaviour Change Technique Taxonomy Version 1

Behaviour Change Wheel

Cognitive Behavioural Therapy for Insomnia

Capability, Opportunity, Motivation, and Behaviour model

Capability Opportunity Motivation-Behaviour Self-Evaluation Questionnaire

General practitioners

Healthcare Professionals

National Institute for Health and Care Excellence

The NHS Research Scotland

The theoretical domain framework

NICE, Insomnia. What is it? 2024. https://cks.nice.org.uk/topics/insomnia/background-information/definition/ . Accessed 27 Feb 2024.

Saddichha S. Diagnosis and treatment of chronic insomnia. 2010. https://doi.org/https://doi.org/10.4103%2F0972-2327.64628

Insomnia NHS, Inform. 2023. https://www.nhsinform.scot/illnesses-and-conditions/mental-health/insomnia/#introduction . Accessed 18 Dec 2023.

Daley M, Morin CM, LeBlanc M, Grégoire J-P, Savard J. The economic burden of insomnia: direct and indirect costs for individuals with insomnia syndrome, insomnia symptoms, and good sleepers. Sleep. 2009;32:55–64.

PubMed   PubMed Central   Google Scholar  

Curran HV, Collins RH, Fletcher SL, Kee S, Woods B, Iliffe S. Older adults and withdrawal from benzodiazepine hypnotics in general practice: effects on cognitive function, sleep, mood and quality of life. Psychol Med. 2003;33:1223–37.

Article   CAS   PubMed   Google Scholar  

Panneman MJM, Goettsch WG, Kramarz P, Herings RMC. The costs of benzodiazepine-associated hospital-treated fall injuries in the EU: a Pharmo study. Drugs Aging. 2003;20:833–9.

Article   PubMed   Google Scholar  

Neutel CI, Perry S, Maxwell C. Medication use and risk of falls. Pharmacoepidemiol Drug Saf. 2002;11:97–104.

Madari S, Golebiowski R, Mansukhani MP, Kolla BP. Pharmacological management of Insomnia. Neurotherapeutics. 2021;18:44–52.

Article   PubMed   PubMed Central   Google Scholar  

Espie CA, Emsley R, Kyle SD, Gordon C, Drake CL, Siriwardena AN, et al. Effect of Digital Cognitive Behavioral Therapy for Insomnia on Health, Psychological Well-being, and sleep-related quality of life: a Randomized Clinical Trial. JAMA Psychiatry. 2019;76:21–30.

Downey A. Digital therapeutics part of NHS Scotland services in ‘world-first’ deal. Digital health. 2021. https://www.digitalhealth.net/2021/10/digital-therapeutics-part-of-nhs-scotland-services-in-world-first-deal/ . Accessed 27 Feb 2024.

Sampson C, Bell E, Cole A, Miller CB, Marriott T, Williams M, et al. Digital cognitive behavioural therapy for insomnia and primary care costs in England: an interrupted time series analysis. BJGP Open. 2022. https://doi.org/10.3399/BJGPO.2021.0146 .

Stott R, Pimm J, Emsley R, Miller CB, Espie CA. Does adjunctive digital CBT for Insomnia improve clinical outcomes in an improving access to psychological therapies service? Behav Res Ther. 2021;144:103922.

Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011. https://doi.org/10.1186/1748-5908-6-42 .

Atkins L, Francis J, Islam R, O’Connor D, Patey A, Ivers N, et al. A guide to using the theoretical domains Framework of behaviour change to investigate implementation problems. Implement Sci. 2017;12:77.

Reid H, Smith R, Williamson W, Baldock J, Caterson J, Kluzek S, et al. Use of the behaviour change wheel to improve everyday person-centred conversations on physical activity across healthcare. BMC Public Health. 2022;22:1784.

Mather M, Pettigrew LM, Navaratnam S. Inf Syst Rev. 2022;11:180.

Article   Google Scholar  

NHS. Primary Care. The NRS Primary Care Network. 2023. https://www.nhsresearchscotland.org.uk/research-areas/primary-care . Accessed 18 Dec 2023.

Michie S, Johnston M, Francis J, Hardeman W, Eccles M. From theory to intervention: Mapping theoretically derived behavioural determinants to Behaviour Change techniques. Appl Psychol. 2008;57:660–80.

Byambasuren O, Beller E, Glasziou P. Current knowledge and Adoption of Mobile Health apps among Australian General practitioners: Survey Study. JMIR Mhealth Uhealth. 2019;7:e13199.

Byambasuren O, Beller E, Hoffmann T, Glasziou P. mHealth App prescription in Australian General Practice: Pre-post Study. JMIR Mhealth Uhealth. 2020;8:e16497.

Kayyali R, Peletidi A, Ismail M, Hashim Z, Bandeira P, Bonnah J. Awareness and use of mHealth apps: a study from England. Pharmacy. 2017;5.

Zhang Y, Li X, Luo S, Liu C, Xie Y, Guo J, et al. Use, perspectives, and attitudes regarding Diabetes Management Mobile apps among diabetes patients and diabetologists in China: National web-based survey. JMIR Mhealth Uhealth. 2019;7:e12658.

Slevin P, Kessie T, Cullen J, Butler MW, Donnelly SC, Caulfield B. A qualitative study of chronic obstructive pulmonary disease patient perceptions of the barriers and facilitators to adopting digital health technology. Digit Health. 2019;5:2055207619871729–2055207619871729.

Leigh S, Ashall-Payne L, Andrews T. Barriers and facilitators to the Adoption of Mobile Health among Health Care professionals from the United Kingdom: Discrete Choice Experiment. JMIR Mhealth Uhealth. 2020. https://doi.org/10.2196/17704

Espie CA, Kyle SD, Williams C, Ong JC, Douglas NJ, Hames P, et al. A randomized, placebo-controlled trial of online cognitive behavioral therapy for chronic insomnia disorder delivered via an automated media-rich web application. Sleep. 2012;35:769–81.

Henry AL, Miller CB, Emsley R, Sheaves B, Freeman D, Luik AI, et al. Does treating insomnia with digital cognitive behavioural therapy (Sleepio) mediate improvements in anxiety for those with insomnia and comorbid anxiety? An analysis using individual participant data from two large randomised controlled trials. J Affect Disord. 2023;339:58–63.

NICE. NICE recommends offering app-based treatment for people with insomnia instead of sleeping pills. 2022. https://www.nice.org.uk/news/article/nice-recommends-offering-app-based-treatment-for-people-with-insomnia-instead-of-sleeping-pills

Chen J, Allman-Farinelli M. Impact of training and Integration of Apps into Dietetic Practice on dietitians’ self-efficacy with using Mobile Health apps and patient satisfaction. JMIR Mhealth Uhealth. 2019;7:e12349.

Michie S, Atkins LWR. The Behaviour Change Wheel: a Guide to Designing interventions. London: Silverback Publishing; 2014.

Google Scholar  

Makhni S, Zlatopolsky R, Fasihuddin F, Aponte M, Rogers J, Atreja A. Usability and Learnability of RxUniverse, an Enterprise-Wide App Prescribing Platform Used in an Academic Tertiary Care Hospital. AMIA Annu Symp Proc. 2017;2017:1225–32.

Lopez Segui F, Pratdepadua Bufill C, Abdon Gimenez N, Martinez Roldan J, Garcia Cuyas F. The prescription of mobile apps by primary care teams: a Pilot Project in Catalonia. JMIR Mhealth Uhealth. 2018;6:e10701.

Sarradon-Eck A, Bouchez T, Auroy L, Schuers M, Darmon D. Attitudes of General Practitioners toward Prescription of Mobile Health apps: qualitative study. JMIR Mhealth Uhealth. 2021;9:e21795.

Download references

Acknowledgements

This research is part of a PhD study sponsored by the Ministry of Education in Saudi Arabia. Professor Ainsworth is funded by the National Institute for Health and Care Research (NIHR) Manchester Biomedical Research Centre. Brian McMillan is funded by an NIHR Advanced Fellowship (reference: NIHR300887). The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.

Not applicable.

Author information

Authors and affiliations.

Division of Informatics, Imaging and Data Sciences, School of Health Sciences, Faculty of Biology, Medicine and Health, The University of Manchester, Oxford Road, Manchester, M13 9PM, UK

Ohoud Alkhaldi & John Ainsworth

Health Information Management and Technology Department, College of Public Health, Imam Abdulrahman Bin Faisal University, Dammam, Saudi Arabia

Ohoud Alkhaldi

Centre for Primary Care and Health Services Research, Division of Population Health, Health Services Research and Primary Care, School of Health Sciences, Faculty of Biology, Medicine and Health, The University of Manchester, Manchester, M13 9PM, UK

Brian McMillan

NIHR Manchester Biomedical Research Centre, Manchester University Hospitals NHS Foundation Trust, Manchester Academic Health Science Centre, Manchester, UK

John Ainsworth

You can also search for this author in PubMed   Google Scholar

Contributions

The authors confirm contribution to the paper as follows: Study conception and design: OA, BM, JA; Data collection: OA; Analysis and interpretation of results: OA; Draft manuscript preparation: OA, BM, JA; All authors reviewed the results and approved the final version of the manuscript.

Corresponding author

Correspondence to Ohoud Alkhaldi .

Ethics declarations

Ethics approval and consent to participate.

Ethics approval was not required because this study involves healthcare staff by virtue of their professional role and presents no material ethical issues. All subjects gave their informed consent for inclusion before they participated in the study.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Alkhaldi, O., McMillan, B. & Ainsworth, J. Using the behaviour change wheel to develop a tailored intervention to overcome general practitioners’ perceived barriers to referring insomnia patients to digital therapeutic sleepio. BMC Health Serv Res 24 , 967 (2024). https://doi.org/10.1186/s12913-024-11384-3

Download citation

Received : 14 May 2024

Accepted : 01 August 2024

Published : 22 August 2024

DOI : https://doi.org/10.1186/s12913-024-11384-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Behaviour change
• Digital therapeutic

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]