|Articles|May 13, 2022
Improving Your Survey Data
Identifying careless responding to improve survey construction and results.
Advertisement
Most people are pretty careful about what to believe on their social media feed, but do they apply the same care to their internet surveys at work? It’s an important issue. The clinical trials industry uses internet surveys for competitor analysis, customer satisfaction/feedback, or even to measure patient outcomes. Internet surveys are ubiquitous and overwhelming so it is easy to respond carelessly.
The purpose of this blog is to describe what careless responding is and how we can avoid and identify careless responses.
Careless responding
Careless responding refers to survey responses that are random or do not reflect the degree to which the construct is present in the respondent. Carelessness is increased when there is a lot of surveys, or when surveys are forced on us, not relevant to us, or force us to respond to items for which we have no opinion. (Boto et al. 12/21 ACT;Duamis & Raymond, 12/21 ACT; Galwicki ACT 11/11). Careless responding can be due to inattention (either partial or complete), fatigue, time pressure, or socially desirable responding. Careless responding is common, varying between 3% to 50%, depending on the context. Measurement problems begin to occur with as little as 10% careless responding (Meade and Craig 2012). The technical consequences of careless responding includes decreasing the reliability of measures, obscure validity, and increase the risk of Type 2 error.
Recommendations: Survey construction and sampling
- Make your surveys short (Dillman et al. 2009).
- Instructions have a small but significant impact
- Reverse coded items
- Instructed response items. (e.g. Fill in a 3 for response.’ Ward 2015)
- Self-reported items - directly asking respondents at the end of the survey whether they think their survey responses are of adequate quality for use in the study Ward (2015). “... self-report alone is not sufficient to identify careless responses.” (Meade & Craig p.8)
- Sample only those potential respondents for whom the survey will be relevant. Not only will this increase your response rate but also reduce careless responding, although you still may get people who want to peruse the survey to get information on your project.
- Don’t use forced responses unless you are sure the question is relevant and the respondent has the information. Still, if they don’t want to give you that information, you should let them progress through the survey.
Recommendations: Analytic procedures to detect careless responding
- response times—either long or short. Meade and Craig 2012 did not find an association between time and careless responding.
- consistent responding
- patterned responding
- abandonment
Conclusion—don’t believe everything you read on the internet
- The clinical trials must rely on internet if it is to remain responsive to industry changes and patient needs.
- The clinical trials industry is being impeded by noisy data from careless responding on internet surveys
Michael Howley PA-C, MBA, PhD, and Peter Malamis MBA
Newsletter
Stay current in clinical research with Applied Clinical Trials, providing expert insights, regulatory updates, and practical strategies for successful clinical trial design and execution.
Advertisement
Related Articles
- Tips to Rescue a Clinical Trial Before It’s Too Late
May 3rd 2024
- Is Trust the Secret Ingredient for Digital Product Success?
December 12th 2023
- FDA Inspections Face Continued Overhaul and Changes
October 13th 2023
- Reaching for the ‘White Spots’ in Rare Disease
September 27th 2023
- The EU Regulations Catching US Pharma Companies Off Guard
September 25th 2023
Advertisement
Advertisement
Trending on Applied Clinical Trials Online
1
Phase III BRUIN CLL-313 Trial Finds Jaypirca Extends Progression-Free Survival in Treatment-Naïve CLL/SLL
2
Q&A: Strategies for Successful Global Clinical Trial Delivery
3
ACT Brief Episode 8: Expert Insights on the Future of Obesity Drug Trials
4
Trial Data Leads to NCCN Recommendation for Modeyso in Recurrent High Grade Glioma Patients
5