Not only are traditional methods of collecting data inferior to SMS text messaging in terms of being able to collect comprehensive interval data, they are significantly impaired because treatment teams cannot interpret or respond to research data in real time. This research will attempt to demonstrate the efficacy of utilizing SMS text messages as a data collection method for interval based diary data. Furthermore, we will attempt to demonstrate that real time responses to participant data are more effective in driving behavioral change than approaches that simply collect data for later synthesis. This investigation represents an evolutionary step in communication and data collection.
Teen Smoking Cessation – Pushing & Pulling SMS Text Messages to Youth
PREVALENCE AND SIGNFICANCE OF WIRELESS TECHNOLOGY
The widespread use of wireless technology, including cell phones and text messaging, has “changed the way that billions of people communicate, make purchases, gather information, learn, meet, socialize, date, and form and sustain intimate relationships.” (Zur, 2007, p. 133) In fact, we would be hard pressed to find an aspect of social life that is not affected by this technology. The proportion of cell phone owners is as high as 85-96% in developed countries. (Kuntsche & Robert, 2009) This universe of potential study participants is as close to inclusive as any researcher could possibly hope for.
DATA ACCURACY – COMPLIANCE
Historically, the industry darling of real time data collection has been the Interactive Voice Response (IVR) system. Essentially, the participant phone rings and IVR prompts participants to press corresponding buttons based on a “menu” of possible conditions. For the latter half of the 21st century, and into the early 22nd, technologies like the IVR represented the only tenable solution to collect raw behavioral data across time and situation. For example, one investigation utilized an IVR system to collect alcohol consumption data. The investigators concluded that the IVR system provided the both the researcher and the participant with ease of use benefits when compared with the more traditional pencil-and-paper log or diary. Real time data was being produced and as a result compliance could not be “faked.” (Collins, Kashdan, & Gollnisch, 2003) SMS responses can’t be “faked” either. Comparatively speaking, when investigators need to ensure responses are delivered in a reasonably short time interval, SMS is the only tenable alternative to IVR. Furthermore, SMS text message prompts (like the IVR) remind participants to respond, thus giving rise to more frequent and timely diary entries.
DATA ACCURACY – DATA DELIVERY DELAY & VERIFICATION
Although it does not amount to a significant limitation in this particular research effort, it should be noted that SMS delivery is subject to negligible delay. We can know the precise time when the initial prompt was sent to the participant via Outlook, and we can know the precise time the response was received to the Outlook Inbox. However, we not know the precise time the initial prompt was actually received by the participant, nor can we infer when the response was actually sent by the participant. Typical delays are less than a minute. This may present difficulties for future research efforts requiring more accurate measurement of ‘mean time to response.’ Furthermore, verification that the intended participants actually receive the SMS text message is impossible. That said, SMS delivery rates with all major carriers in the United States meet or exceed a 99.999% expectation. (Ramsay, 2010) We consider this to be an acceptable completion rate and do not consider it to be a significant confounding variable.
DATA ACCURACY – RECALL BIAS
Ideally participants would complete a diary entry shortly after the relevant behavior occurrence, or, immediately when prompted. However, evidence suggests that conventional hand written methods of data collection are poor at best. In conventional hand written diaries, participants could conceivably fill out the entire diary in one sitting rather than on a day-by-day or event-by-event basis. While this may not appear to be a significant drawback, evidence suggests that recalls of past experiences are often biased because they are not encoded into memory for later retrieval. A persistent lack of consistency in data entry results in excess amounts of missing data and data-entry errors. The lure the IVR is enhanced accuracy because participants need not rely on retrospective memory. SMS text messages, like the IVR, effectively prompt participants to answer and hold them accountable for the general time frame of the response since mean time to response can be tracked. Any method of data collection that reduces or eliminates retrospective recall error and capture the experiences as they occur have the potential to advance our understanding of behavior. Both the IVR and the SMS meet this requirement. (Kuntsche & Robert, 2009; Ravert et al., 2010; Reid et al., 2009)
COST – BENEFIT ANALYSIS OF SMS IMPLEMENTATION
SMS text messaging presents an opportunity collect “real time data” at an exceptionally low cost. Comparatively speaking, research suggests that Short Message Service (SMS) text messaging is a much less expensive option for data collection when compared with IVR systems or hand written logs/diaries. Previous efforts to collect data via SMS have been described as being less labor intensive, in part because minimal time was required for training participants, and in part because the data is digital and does not need to be rekeyed by investigators. (Gleerup, Larsen, Leth-Petersen, & Togeby, 2010; Kuntsche & Robert, 2009) Costs and potential security risks are further lessened if participants use their own wireless phones. (Ravert, Calix, & Sullivan, 2010) Unlike stereotypical IVR implementations, SMS implementation is comparatively easy because SMS utilizes software that is commonly found on most business PCs (Microsoft Outlook) and requires no significant technical knowledge beyond the ability to send an email.
COMPLIANCE & PARTICIPATION CONSIDERATIONS – AVAILABILITY, REINFORCEMENT, AND FATIGUE
The most common reasons for poor compliance in previous research involving SMS text messages were sleep, work, class, or having phone set on “silent mode.” (Ravert et al., 2010) Our participants were advised to not use ‘silent mode’ unless absolutely necessary. Participants were required to fill out web-based survey designating appropriate times to send text messages to maximize response rate and timeliness of participant response. In effort to maximize compliance, we implemented a minimum “availability level” required to participate. We implemented cash prizes for compliance… every response delivered within 5 minutes gives the participant one entry into a drawing for a $10 Visa gift card. One gift card is presented daily, and all participants received a text messaging notifying them of the initials and the participant ID # of the winning participant. Some research suggests evidence of reduced participation and respondent fatigue as study length approached 7 days. (Reid et al., 2009) We are cognizant of the possibility and will continually monitor for participant fatigue. In an effort to preserve high compliance and reduce the effect of fatigue we have taken measures to simplify the response procedure, reduce the response burden on participants, and provide incentives for timely responses.
FEEDBACK vs. BASIC DATA COLLECTION
Providing timely feedback to participants is the hallmark of this investigation. SMS feedback is most likely to have an effect if it is given frequently, over a long period, and at level specific enough to produce change in the target behavior. We put forward the suggestion that the true benefit of real time data has been overlooked in previous research efforts that attempt to harness SMS text messages as a data source. We believe real time data should be utilized by the treatment team by actively interpreting and responding to the data in real time. Our intent is not just to improve data quality and participant compliance, but to improve the overall effectiveness with which we interpret that data and feed it back to participants. Treatment teams need to know if a specific intervention is ineffective, and ineffective interventions should be rapidly modified to meet the needs of individual participants.
The power to deliver nearly instant feedback is coupled with a second agent, reactivity, to produce change. Regular prompting via text messages will inevitably increase participant awareness of both mood and behavior, with or without timely feedback. Regardless of medium, regular prompts have demonstrated positive effects on self-regulatory activity, time-on-task, and retention of content. (Sitzmann & Ely, 2010) The resulting change in behavior produced by increased awareness is referred to as ‘‘reactivity.’’ One study suggests that when people monitor their moods, stressors, and coping responses the increased awareness alone is capable of producing positive results with regard to problem recognition and positive problem solving strategies. (Reid, Kauer, Dudgeon, & Sanci, 2009)
EFFECTIVENESS FOR COGNITIVE IMPAIRMENT
SMS text messages have demonstrated limited effectiveness in compensating for cognitive impairment (i.e., memory and/or planning problems) in men with schizophrenia. (Pijnenborg, Withaar, Evans, van den Bosch, & Brouwer, 2007) It has been suggested that teenagers suffer from a form of “temporary insanity” that is not entirely unlike schizophrenia — so much so that “our rapidly evolving offspring do things to make us wonder whether common sense was ever a human attribute.” (Berger, 2008) The condition of adolescence lends certain limitations to cognitive facilities like memory, attention, psycho-motor speed, mental flexibility, and planning abilities. These underlying deficits associated with adolescence are exacerbated in our sample population for a variety of reasons, although it is certainly worth mention that few demonstrate the level of impairment that someone with full blown schizophrenia might demonstrate.
EFFECTIVENESS WITH ELEVATED RECALL OF INTENTIONS AND GOALS
SMS text messages have also demonstrated efficacy in promoting healthy pro-social behavior. One investigation suggests text messages could enhance interventions that target implementation intentions and goals by elevating recall of both goals and plans. (Prestwich, Perugini, & Hurling, 2010)
4. Problem and its key terms
Although teen smoking rates decreased significantly (15%) from 1997 to 2003, teen smoking rates only dropped 2% from 2003 to 2009. Roughly one in five teenagers say they are current cigarette users. (NBC13.com, 2010) How do we effectively assist teenagers with kicking the tobacco habit?
We intend to demonstrate the effectiveness of behavior prompting among adolescent smokers.
We intend to demonstrate that SMS text messages are a more effective method of prompt delivery when compared with Interactive Voice Response (IVR systems).
We intend to compare and contrast the reliability of the data between and among SMS and IVR alternatives.
Questions of the study/ hypothesis
Hypothesis #1: Prompting via both IVR and SMS text messaging will result in an overall decrease in tobacco use among adolescent youth.
Hypothesis #2: SMS text messages are a more effective method of behavioral prompting when compared with IVR systems.
Hypothesis #3: SMS Text messages are a more reliable source of real time data when compared with IVR systems.
Hypothesis #4: SMS text message interventions are more effective if standard prompts are promptly followed with customized motivational text messages.
We propose an “interval based diary design” in which participants are asked to respond whenever prompted (random). Benefits include the ability to collect data on participants in their natural environment while differentiating change over time and across situations. SMS prompts can be delivered by and answered at any moment and at participant convenience. (Ravert et al., 2010)
We compiled a comprehensive list of schools in the United States. From that list, we randomized the list and began to contact school guidance counselors for referrals of adolescent students who were known to be smokers. Furthermore, we utilized the web to drive referrals from teenage smokers. From the total pool of referrals, we utilized a stratified random sampling method to balance the 1200 participants in the study. Exactly half of the study was male, and the other half was female. Furthermore, each group was further stratified by race. Utilizing the 2000 US Census, load balanced the racial profile of both groups. We would have preferred to use the 2010 US Census, however, that data is not current available. If we were to actually conduct this study, we would probably wait for that data.
The end sample, selected from the total pool of potential participants, represented 300 members of each “high school class.” The sample includes 300 freshman, 300 sophomores, 300 juniors, and 300 seniors; respectively. The socioeconomic status of the population was predominantly “lower middle-class” as it appears as though this particular socioeconomic group is particularly prone to succumb to tobacco at an early age. The size of the sample gives us a significant advantage of previous studies. Each sub-population is randomly separated into one of three conditions…
- Control group
- Experimental group 1 which receives random prompts regarding smoking cessation
- Experimental group 2 which receives random prompts regarding smoking cessation AND “reactive implication feedback”.
Standard SMS text question prompt is a simple question.
How many cigarettes have you smoked today? Participant is require to respond with an appropriate number. In experimental group 2, the participant will receive one of the following randomly selected prompts in response to their initial reply.
- Smoking causes bad breath!
- Smoking causes yellow teeth!
- Smoking makes your clothes smell!
- Smoking makes you cough more!
- Smoking may cause you difficulty keeping up with friends when playing sports!
- Smoking is expensive!
- Utilize web based survey to screen participants for present smoking frequency and characteristics. Determine participant expectations of cessation, timeline, and self-reported ability to quit. (via web survey)
- Explain purpose and voluntary nature of participation to prospective participant. Explain that participant will be randomly receiving text messages requesting them to respond appropriately. (via web survey) I.E. How many cigarettes have you smoked today?
- Deliver, explain, and give opportunity for participant to ask questions with regard to informed consent document. (web based)
IF PARTICIPANT IS ACCEPTED
- Instruct participant to complete participant data sheet and add participant data sheet (to include gender, age, and race/ethnic affiliation, wireless phone number, ) for purposes of measuring relevant sample demographic information
- Physically test the delivery of text messages during intake by delivering a secure code that must be entered into the website. Require participant to log into website with secure code to verify that the wireless number belongs to the respective participant.
- Train participants to properly respond to prompts.
8. Population and sample
WHY USE YOUTH?
At present time the average adolescent is so engaged with text messaging that it has become an integral part of a typical teenagers day-to-day activities. This level of engagement was believed to lead to increased response rates among youth in one study. Familiarity with SMS text messaging technology is so pervasive among this cohort that one study suggested that face-to-face contact was probably unnecessary. “Apparently, a simple instruction given in an e-mail or via an Internet advertisement was sufficient for the participants to complete the study successfully.” (Kuntsche & Robert, 2009; Ravert et al., 2010; Reid et al., 2009)
Populations that are under-penetrated by wireless technology are prone to be under-represented in seemingly “random samples.” There may be a propensity for random samples to over-represent populations who from disproportionately high socioeconomic backgrounds. It should be recognized and acknowledged that some populations may be under-represented in samples that require preexisting access to wireless services. Furthermore, the universe of available participants may be limited in geographical regions that are under-penetrated by the wireless industry.
Website for recruiting.
Review of hypotheses
Hypothesis #1: Prompting via both IVR and SMS text messaging will result in an overall decrease in tobacco use among adolescent youth. We expect to find support among targeted groups, and we may expect to see a negative correlation with age. (i.e. The older the participant, the less likely they are to decrease or quit smoking)
Hypothesis #2: SMS text messages are a more effective method of behavioral prompting when compared with IVR systems. We expect overwhelming support for this… especially since our target group is so attuned to mobile phones and text messaging. It’s just more convenient!
Hypothesis #3: SMS Text messages are a more reliable source of real time data when compared with IVR systems. We expect overwhelming support of this hypothesis.
Hypothesis #4: SMS text message interventions are more effective if standard prompts are promptly followed with customized motivational text messages. This is the wildcard; there is no way of knowing if this will play out. I’d be interested in seeing if it does.
Summary of main findings (as they relate to the hypotheses)
Because we are suitably unable to actually conduct the research, a summary will be suitably difficult to provide. However, I believe we will find that this is an extremely easy and cost effective way to obtain data across time. Furthermore, I believe we will find that participants are “comfortable” with the research. The only possible issue I could see is finding participants… since finding kids to “pony up” and admit they smoke might be quite a challenge. I considered searching police records and court databases for tobacco use and/or possession as a way to find a larger pool of potential recipients.
Limitations of the research
I chose adolescent youth for a very specific reason… I believe that they would be more likely to comply to SMS text message requests. Also, I think they would bite at the chance to win 10 bucks every day (despite the fact that, realistically, that is a tiny amount of cash). Using youth may limit generalization to other age groups. Generalization may be difficult outside of the “lower middle class” socio-economic group.
TEXT MESSAGE LENGTH LIMITATIONS
Any future efforts to replicate or modify this research should take into consideration that the maximum length of a single SMS text message is 160 characters (including spaces). While this does not present a significant confounding variable in this particular research effort, future research should anticipate and plan for the limitation.
IMPLICATIONS – POST STUDY DATA ENTRY
Data entry is not necessary at the end of the study because all digital data can be readily formatted in a standard database format and downloaded in a standard database format. In cases where critical data is absent, researchers can utilize text messages to illicit elaboration on data that is incomplete. (Ravert et al., 2010)
IMPLICATIONS – IDEAS FOR MEASURING TEXT MESSAGE CONTENT
- Number of cigarettes smoked (via self report, SMS text message response)
- Promptness of reply (in minutes)
- Goal obtainment as defined by individual participants… do they actually do what they intended to do, or, did they forget?
This particular research effort was confined by relative cost and available technology. We would have gladly seized an opportunity to create a mobile friendly website that could be accessed via 3G/4G wireless networks. This technology would allow participants to respond to a customized web-based survey at any given time and location via wireless phone. While this technology is readily available, it is very expensive comparatively speaking. Adding unlimited data to each participant handset would have resulted in an approximate $30 USD increase in cost (per month, per participant). Furthermore, development of the website and associated web content would have been both costly and time consuming. The strength of this research effort is in the simplicity of its implementation. Given the size of this study, adding unlimited 3G data to each wireless handset was not a viable option. The utilization of wireless 3G networks may be viable in a smaller analysis, or perhaps as the cost of unlimited data continues to trend downward and become more common on consumer wireless plans this may be a viable option for future research efforts.
Berger, J. (2008, May 4). Understanding the temporary insanity of adolescence. The New York Times. Retrieved from http://www.nytimes.com/2008/05/04/nyregion/nyregionspecial2/04colwe.html
Collins, L. R., Kashdan, T. B., & Gollnisch, G. (2003, Feb). The feasibility of using cellular phones to collect ecological momentary assessment data: Application to alcohol consumption. Experimental and Clinical Psychopharmacology, 11(1), 73-78. doi: 10.1037/1064-12220.127.116.11
Gleerup, M., Larsen, A., Leth-Petersen, S., & Togeby, M. (2010). The effect of feedback by text message (SMS) and email on household electricity consumption: Experimental evidence. Energy Journal, 31(3), 113-132. Retrieved from http://vnweb.hwwilsonweb.com.ezproxy.bellevue.edu/hww/jumpstart.jhtml?recid=0bc05f7a67b1790e01bd7c7a5ed0962d0980d6aed164fe635b88352196c3404f61c2e1fe69c6247a&fmt=P
Kuntsche, E., & Robert, B. (2009, May). Short message service (SMS) technology in alcohol research– A feasibility study. Alcohol and Alcoholism : International Journal of the Medical Council on Alcoholism, 44(4), 423-428. doi: 10.1093/alcalc/agp033
NBC13.com. (2010). Teen smoking rates level off. Retrieved from http://www2.nbc13.com/vtm/news/local/article/teen_smoking_rates_level_off/165829/
Pijnenborg, G. M., Withaar, F. K., Evans, J. J., Van den Bosch, R. J., & Brouwer, W. H. (2007, May). SMS text messages as a prosthetic aid in the cognitive rehabilitation of schizophrenia. Rehabilitation Psychology, 52(2), 236-240. doi: 10.1037/0090-5518.104.22.168
Prestwich, A., Perugini, M., & Hurling, R. (2010, Jan). Can implementation intentions and text messages promote brisk walking? A randomized trial. Health Psychology, 29(1), 40-49. doi: 10.1037/a0016993
Ravert, R. D., Calix, S. I., & Sullivan, M. J. (2010, May/Jun). Research in brief: Using mobile phones to collect daily experience data from college undergraduates. Journal of College Student Development, 51(3), 343-351. Retrieved from http://ezproxy.bellevue.edu:80/login?url=http://proquest.umi.com.ezproxy.bellevue.edu/pqdweb?did=2055146651&sid=3&Fmt=2&clientId=4683&RQT=309&VName=PQD
Reid, S. C., Kauer, S. D., Dudgeon, P., & Sanci, L. A. (2009, Jun). A mobile phone program to track young people’s experiences of mood, stress and coping; Development and testing of the mobiletype program. Social Psychiatry and Psychiatric Epidemiology, 44(6), 501-507. doi: 10.1007/s00127-008-0455-5
Reyno, S. M., Stewart, S. H., Brown, C. G., Horvath, P., & Wiens, J. (2006, Aug). Anxiety sensitivity and situation-specific drinking in women with alcohol problems. Brief Treatment and Crisis Intervention, 6(3), 268-282. doi: 10.1093/brief-treatment/mhl007
Sitzmann, T., & Ely, K. (2010, Jan). Sometimes you need a reminder: The effects of prompting self-regulation on regulatory processes, learning, and attrition. Journal of Applied Psychology, 65(1), 132-144. doi: 10.1037/a0018080
Zur, O. (2007). Telehealth and the technology for delivering care. In Boundaries in psychotherapy: Ethical and clinical explorations (pp. 133-146). doi: 10.1037/11563-008