3/27/2021
Scholars essay based on Test 1 articles
Scholars essay based on Test 1 articles
Submit Assignment
Due Saturday by 11:59pm
Points 15
Submitting a text entry box or a file upload
File Types doc, docx, pdf, and txt
Available Feb 26 at 12am – Mar 27 at 11:59pm 30 days
Instructions to Scholars’ students regarding articles
Overview
For Test 1 please choose to read ONE of the following articles:
Smile-reduce-stress.pdf – this article most closely relates to the Functional theory of behavior by
William James. It is about how the action of smiling can reduce stress (action can change emotion). This
relates to the history of psychology from Chapter 1.
replication-in-research.pdf – this article is about how often psychologists replicate previously published
research. This relates to chapter 2 on Research Methods.
The article you choose will be in addition to the textbook reading.
You will only need to answer essay questions specifically related to the article that you choose.
Here is a sample set of two questions and student answers that received full credit from a previous test
(with a different article): Sample-questions-with-student-answers.pdf
Here are the essay questions for each article. Remember – you only need to answer the
questions for one of the two articles. Please write your answer in prose, paragraph format – not
bullet points. However, do please number each answer.
Please make sure that you write in your own words – paraphrasing the research – not quoting the
research.
Scholars Essay Test 1
Scholars essay questions for: Kraft, T.L. & Pressman, S.D. (2012). Grin and Bear It: The Influence
of Manipulated Facial Expression on the Stress Response. Psychological Science, 23(11) 1372–
1378.
Three short answer questions, each question states how many points it is worth.
https://online.smc.edu/courses/38542/assignments/804189?return_to=https%3A%2F%2Fonline.smc.edu%2Fcalendar%23view_name%3Dmonth%26view_start%3D…
1/2
3/27/2021
Scholars essay based on Test 1 articles
1. Worth 5 points – approximately four-five sentences should cover this answer.
In the article the authors describe three groups of participants related to what they will be asked to do
with their face. State what the three groups are and which group(s) is the experimental group and which
group(s) is the control group. Did the authors use random assignment?
2. Worth 7 points – five to ten sentences should cover this answer.
This study had more than one hypothesis. In your own words, describe what their hypotheses were.
(This does not have to be an If … then… version of each hypothesis, describe what they were trying to
understand and any predictions they made). Briefly describe what their results were and why their
results support their hypotheses.
3. Worth 3 points – approximately three-four sentences
The study had two different independent variables. What were the two different IVs? What was their
dependent variable?
Scholars essay questions for Makel, M.C., Plucker, J.A., & Hegarty, B. (2013). Replications in
Psychology Research: How Often Do They Really Occur? Perspectives on Psychological
Science, 7(6) 537–542.
Three short answer questions, each question states how many points it is worth.
1. Worth 5 points – approximately four-five sentences should cover this answer.
Describe in your own words the main finding of this study and why their results are important.
2. Worth 7 points – five to ten sentences should cover this answer.
In your own words, describe what their hypothesis or hypotheses were. (This does not have to be an If
… then… version of each hypothesis, describe what they were trying to understand and any predictions
they made). Briefly describe why their results support their hypotheses.
3. Worth 3 points – approximately three-four sentences should cover this answer.
In your own words, why is replication of research important and what reason(s) do the authors give for
why replication is not done as much as it should be?
https://online.smc.edu/courses/38542/assignments/804189?return_to=https%3A%2F%2Fonline.smc.edu%2Fcalendar%23view_name%3Dmonth%26view_start%3D…
2/2
Research Article
Grin and Bear It: The Influence of
Manipulated Facial Expression on the
Stress Response
Psychological Science
23(11) 1372–1378
© The Author(s) 2012
Reprints and permission:
sagepub.com/journalsPermissions.nav
DOI: 10.1177/0956797612445312
http://pss.sagepub.com
Tara L. Kraft and Sarah D. Pressman
University of Kansas
Abstract
In the study reported here, we investigated whether covertly manipulating positive facial expressions would influence
cardiovascular and affective responses to stress. Participants (N = 170) naive to the purpose of the study completed two
different stressful tasks while holding chopsticks in their mouths in a manner that produced a Duchenne smile, a standard
smile, or a neutral expression. Awareness was manipulated by explicitly asking half of all participants in the smiling groups to
smile (and giving the other half no instructions related to smiling). Findings revealed that all smiling participants, regardless
of whether they were aware of smiling, had lower heart rates during stress recovery than the neutral group did, with a slight
advantage for those with Duchenne smiles. Participants in the smiling groups who were not explicitly asked to smile reported
less of a decrease in positive affect during a stressful task than did the neutral group. These findings show that there are both
physiological and psychological benefits from maintaining positive facial expressions during stress.
Keywords
emotions, facial expressions, well-being, health, stress reactions
Received 9/19/11; Revision accepted 3/2/12
In the past decade, scientists have produced a wealth of research
connecting positive affect to physical and psychological wellbeing (see Lyubomirsky, King, & Diener, 2005; Pressman &
Cohen, 2005). Although most studies have focused on selfreported positive affect, observational methods have also shed
light on these associations. For example, facial expressions
indicating smiles of “nonenjoyment” have been shown to
differentiate subjects with and without myocardial ischemia
(Rosenberg et al., 2001). Similarly, smiling in photographs has
been associated with well-being outcomes decades later (Abel
& Kruger, 2010; Harker & Keltner, 2001), which raises the
possibility that facial expression is a health-relevant emotion
indicator.
Following the tradition of James (1890), many researchers
have proposed that emotions are the consequence of facial expressions (and other behaviors) rather than the more commonly
considered reverse direction. For example, self-perception theory states that acting as though one feels a certain way will lead
to that feeling (Bem, 1972; Laird, 1974). Especially relevant to
facial-expression research is the related facial-feedback hypothesis, which states that activating facial muscles leads to the psychological experience of emotion (Tourangeau & Ellsworth,
1979). In a classic study demonstrating the facial-feedback
hypothesis, Strack, Martin, and Stepper (1988) had participants
place pencils in their mouths in ways that activated facial muscles involved in smiling (or not smiling) while rating cartoons
for funniness. Participants whose mouths were manipulated to
smile rated cartoons as funnier than did participants in other
conditions, despite a cover story linking the mouth-pencil to
research on physical impairment.
Facial-expression researchers have long agreed that not all
smiles are equal. Ekman (2001) suggested that as many as 50
kinds of smiles exist; however, most research focuses on the
distinction between “genuine” and “standard” smiles based
on activation of the orbicularis oculi muscle surrounding the
eye. A “standard” smile engages the zygomaticus major muscles around the mouth, but only a “genuine” Duchenne smile
engages both zygomaticus major and orbicularis oculi muscles
(Duchenne, 1862/1990; Ekman & Friesen, 1982). Ekman,
Davidson, and Friesen (1990) found that Duchenne smiling
Corresponding Authors:
Tara L. Kraft, Department of Psychology, University of Kansas, 1415
Jayhawk Blvd., Lawrence, KS 66045
E-mail: kraft.tara@gmail.com
Sarah D. Pressman, Department of Psychology, University of Kansas, 1415
Jayhawk Blvd., Lawrence, KS 66045
E-mail: pressman@ku.edu
Downloaded from pss.sagepub.com at PENNSYLVANIA STATE UNIV on March 3, 2016
1373
Grin and Bear It
was seen more frequently when viewing pleasant films than
when viewing unpleasant films and has been associated with
activity in the left frontal and anterior temporal lobes, areas
previously associated with positive affect (e.g., Davidson,
1992). Furthermore, Ekman and Davidson (1993) found that
voluntarily producing a Duchenne smile activated the same
brain regions responsible for positive affect as did involuntary
Duchenne smiles stimulated by outside sources. This supports
the facial-feedback hypothesis, given that the consequences
associated with voluntary and involuntary activation of facial
muscles were remarkably similar in the brain.
One important outcome that might be related to positive
facial expressions is the cardiovascular stress response, a measure tied to future heart-health outcomes (e.g., Treiber et al.,
2001). Although experimental facial manipulation has not
been tied to this outcome directly, naturally occurring and
manipulated positive affect has been linked to “healthier”
cardiovascular stress recovery (i.e., quicker return toward resting heart function; Fredrickson & Levenson, 1998; Steptoe,
Gibson, Hamer, & Wardle, 2007). It is notable that Fredrickson and Levenson (1998) also found that participants who
spontaneously smiled during stress returned to resting levels
of cardiovascular function more rapidly than did nonsmiling
counterparts. This suggests that smiling may be particularly
helpful in speeding stress recovery by reducing negative aftereffects of stress. This is consistent with Fredrickson, Mancuso,
Branigan, and Tugade’s (2000) “undoing hypothesis” of positive affect, which specifically postulates better recovery as a
pathway connecting positive affect to well-being benefits. It is
also in line with the more general stress-buffering model of
positive affect and health (Pressman & Cohen, 2005), which
asserts that positive feelings may guard against the negative
physiological consequences of stress at a variety of times,
including before, during, or after stress. Because smiling was
not randomly assigned in the Fredrickson and Levenson
(1998) study, it remains unclear whether this finding was due
to facial expression directly or was simply the result of differences in emotional states or traits.
In the current study, we sought to better understand whether
the facial changes that occur in smiling might be partially
responsible for observed benefits connecting positive affect to
improved stress recovery and whether smile types would have
differential effects. Specifically, would “sincere” Duchenne
smiling produce greater stress-recovery benefits than would
standard smiling (or no smile)? Awareness of smiling was also
manipulated to determine whether benefits were present when
cognitive awareness of facial expression was absent. Past
studies of the facial-feedback hypothesis and related selfperception work have purposely avoided awareness to prove
the expression-feeling connection without cognition. In the
case of stress, however, facial-muscle activation may not
have the same power given the conflicting emotion and autonomic arousal signals (i.e., from pain and threat). Furthermore,
studies have shown that purposely “faking” positive facialemotion expression in customer service leads to increased
burnout and employee error (e.g., Goldberg & Grandey, 2007),
which suggests that awareness of artificial smiling may be
harmful. Finally, emotion changes that occurred with condition assignment were assessed. On the basis of the facial-feedback hypothesis, we expected smilers to report greater positive
affect than nonsmilers; however, in the stress context, it was
anticipated that this would instead manifest as a lesser decrease
in positive affect and a smaller increase in negative affect.
Method
Participants and procedure
One hundred seventy healthy participants (age = 18–25 years;
66% female, 34% male; 79% Caucasian, 21% other) were
recruited from a large Midwestern university and screened for
facial muscular disorder, lack of English fluency, and psychological disorder. Participants were randomly assigned to a
neutral-expression control group (n = 58), a standard-smile
group (n = 56), or a Duchenne-smile group (n = 56). Groups
were asked to hold chopsticks in their mouths with their teeth
by mimicking the holding pattern of a research assistant
(who was blind to the study hypotheses) and a photo example
(Fig. 1).
Participants in the Duchenne-smile group were trained to
activate zygomaticus major and orbicularis oculi muscles, participants in the standard-smile group were trained to activate
zygomaticus major muscles, and participants in the neutral
group were simply instructed to hold the chopsticks gently in
their mouths with their faces relaxed. Participants were given
positioning assistance, image examples, experimenter instruction, and verbal correction during the study. To ensure that correct muscles were activated, two research assistants trained
with the Facial Action Coding System (Ekman & Friesen,
1978) coded videos of participants. On a scale from 1 (poor
adherence) to 5 (excellent adherence), the neutral group had
the highest average adherence (3.35), with the standard-smile
group averaging 1.90 and the Duchenne-smile group averaging 2.60.
Participants were given a cover story stating that this was a
“multitasking study” (similar in nature to the cover story used
by Strack et al., 1988) to prevent awareness or reactance to
smiling. Because we were also interested in participants’
awareness of smiling, half the participants in each smiling
group (n = 28 per group) were additionally told to smile during
the instruction period (the other half were given no additional
instructions regarding their facial expressions).
After completing baseline questionnaires, participants’
heart rate was monitored in beats per minute (bpm) using an
automated cuff (Dinamap ProCare Auscultatory 400 Vital
Signs Monitor, Lafayette Instrument Company, Lafayette, IN)
that inflated approximately every 90 s over the course of the
study. Heart rate was used as the primary outcome, as it is one
of the most frequently measured variables in cardiovascular
stress studies (e.g., Turner, 1994, p. 45). It is a reliable measure
Downloaded from pss.sagepub.com at PENNSYLVANIA STATE UNIV on March 3, 2016
1374
Kraft, Pressman
Fig. 1. Examples of photographs shown to participants in the neutral group (left), standard-smile group (middle), and Duchenne-smile group (right)
to help them form the appropriate expressions.
that is easily assessed, and it increases in response to a large
range of stressors. Following a 10-min resting period, participants completed a 2-min star-tracing task. This stress-inducing
task requires participants to place their nondominant hand
inside a box and repeatedly trace a star while viewing only a
mirror image of the star and their hand. If they strayed from
the outline of the star, they received negative auditory feedback. Participants were strongly encouraged to be accurate
and were also given incorrect information about performance
standards to increase stress (i.e., they were told that the task
average was eight tracings with fewer than 25 errors). They
were promised an incentive (chocolate) if they could match
this unattainable goal. Average participants could complete
two tracings in 2 min with over 25 errors. This task was followed by a 5-min recovery period.
Next, participants completed a cold-pressor stress task in
which their hand was submerged in ice water (2–3 °C) for 1
min. Participants then recovered again for 5 min. Tasks were
not counterbalanced because of lasting pain from the cold.
Chopsticks were held in the mouth only during the stress
period (not during the recovery period), and verbal reminders
were given when facial expressions lapsed. State affect, stress,
task difficulty, and facial-muscle fatigue were measured at
baseline and following each task. Participants were probed
during debriefing for knowledge of study hypotheses. Only
one individual identified that the study’s purpose was to examine smiling.
Positive- and negative-affect change scores were calculated
from baseline for each task using 16 items drawn from the
factor-analyzed version of the Profile of Mood States (McNair,
Lorr, & Droppleman, 1971; Usala & Hertzog, 1989). Baseline
stress was measured using the Perceived Stress Scale (Cohen,
Kamarck, & Mermelstein, 1983) and via momentary assessments throughout the study, in which participants rated how
stressful they found each task on a 10-point scale. Behavioral
variables known to correlate with cardiovascular functioning
were measured at baseline, including body mass index, sleep,
smoking, alcohol consumption, exercise, and drug use.
Statistical approach
For cardiovascular-recovery analyses, we used repeated measures analyses of covariance, with five time intervals, including the peak stress point of each task and four subsequent 90-s
intervals following each task. This analysis accounted for differences from the final reactivity point to the end of the recovery period with between-subjects effects being conceptually
similar to change scores from reactivity to recovery. To analyze state affect changes during tasks, we used univariate analyses of covariance. The main analyses of interest were
comparisons between (a) the neutral versus the smiling groups,
(b) the Duchenne-smile versus the standard-smile groups, and
(c) the aware versus the nonaware groups. In all analyses,
covariates significantly associated with the outcome of interest were controlled for. Variables included (when significant)
were age, race, sex, body mass index, baseline stress, sleep,
smoking, alcohol use, exercise, condition adherence, perceived task difficulty, self-reported facial-muscle fatigue, perceived task stress, and stress reactivity. Group differences in
stress reactivity were not found for any contrasts of interest.
Results
An overall uncorrected analysis of variance of all five groups
revealed significant differences during recovery following the
star-tracer task, with aware standard smilers showing the lowest
recovery heart rate levels (M = 65.75 bpm), followed by nonaware Duchenne smilers (M = 66.50 bpm), aware Duchenne
smilers (M = 67.40 bpm), the neutral group (M = 71.36 bpm),
and nonaware standard smilers (M = 72.73 bpm), F(4, 139) =
4.68, p < .01. Uncorrected analyses found marginally significant
differences between groups following the cold-pressor task,
Downloaded from pss.sagepub.com at PENNSYLVANIA STATE UNIV on March 3, 2016
1375
Grin and Bear It
a
Neutral Condition
Standard-Smile Condition
Duchenne-Smile Condition
84
Heart Rate
80
76
72
68
64
60
0
90
180
270
360
270
360
Time (s)
b
Neutral Condition
Nonaware Condition
Aware Condition
84
80
Heart Rate
with aware standard smilers (M = 66.33 bpm) showing the lowest heart rate, followed by aware Duchenne smilers (M = 66.34
bpm), nonaware Duchenne smilers (M = 66.86 bpm), the neutral group (M = 70.91 bpm), and nonaware standard smilers
(M = 71.43 bpm), F(4, 142) = 2.27, p = .06. After accounting for
significant covariates, we found clear and consistent group differences overall and between individual contrasts, with smiling
groups showing lower levels of heart rate during recovery than
the neutral group. Average corrected means during recovery for
individual contrasts are reported when main (between-subjects)
effects are significant.
During recovery from the star-tracer task, the following
groups had significantly lower heart rates than the neutral
group: the two smile groups combined (Msmile = 68.19 bpm vs.
Mneutral = 71.45 bpm), F(1, 117) = 3.95, p = .05, Duchenne
smilers only (MDuchenne = 66.40 bpm vs. Mneutral = 71.69 bpm),
F(1, 74) = 6.71, p < .05, and aware smilers (Maware = 66.60
bpm vs. Mneutral = 71.29 bpm), F(1, 72) = 5.40, p < .05. When
comparing three groups, significant heart rate recovery differences were found between the neutral group, standard smilers,
and Duchenne smilers, as well as between the neutral group,
aware smilers, and nonaware smilers (p < .05; Fig. 2).
Results for recovery following the cold-pressor task were
similar. The following groups had lower heart rates than the
neutral group: the two smile groups combined (Msmile = 67.37
bpm vs. Mneutral = 71.69 bpm), F(1, 109) = 4.34, p < .05, Duchenne smilers only (MDuchenne = 65.37 bpm vs. Mneutral = 72.02
bpm), F(1, 69) = 9.12, p < .05, aware smilers (Maware = 66.66
bpm vs. Mneutral = 71.48 bpm), F(1, 66) = 4.61, p < .05, and nonaware smilers, who showed only marginally lower heart rates
than the neutral group (Mnonaware = 68.34 bpm vs. Mneutral = 72.52
bpm), F(1, 68) = 2.78, p = .10. Duchenne smilers also had marginally lower heart rates than did standard smilers (MDuchenne =
64.98 bpm vs. Mstandard = 69.32 bpm), F(1, 76) = 3.61, p = .06.
When the three groups were compared in one model, significant
differences were again found between the neutral group, standard smilers, and Duchenne smilers, as well as between the neutral group, aware smilers, and nonaware smilers (Fig. 3).
Overall, uncorrected analyses of variance examining selfreported affect changes during tasks did not reveal significant
group differences. However, when corrected individual contrasts were examined, affect changes were in the expected direction, although small, often nonsignificant, and only following
the cold-pressor task. Specifically, following cold stress, nonaware smilers (M = –0.32) showed less of a decrease in positive
affect from baseline than the neutral group did (M = –0.65), F(1,
71) = 4.21, p < .05, all smilers showed marginally less of a
decrease in positive affect (Msmile = –0.36 vs. Mneutral = –0.58),
F(1, 115) = 2.56, p = .1, and Duchenne smilers had marginally
less of a decrease in positive affect (MDuchenne = –0.36 vs. Mneutral =
–0.60), F(1, 75) = 2.47, p = .1. Further, nonaware smilers
showed a marginal decrease in negative affect from baseline; in
contrast, the neutral group showed an increase (Mnonaware =
–0.17 vs. Mneutral = 0.07), F(1, 73) = 2.78, p = .1.
76
72
68
64
60
0
90
180
Time (s)
Fig. 2. Mean heart rate (in beats per minute) during the recovery period
following the star-tracer task as a function of measurement occasion and
condition. Results are shown separately for (a) the three facial-expression
groups and (b) the aware and nonaware subgroups of the two smiling
groups (collapsed across groups). The analysis controlled for sex, condition
adherence, baseline perceived stress, perceived task difficulty, task stress, and
facial-muscle fatigue. Error bars represent standard errors.
Discussion
This is the first study to show that experimentally assigned
positive facial manipulation—with or without awareness of
expression—has a direct impact on cardiovascular stress
recovery. This may be relevant for health given that cardiovascular recovery is an outcome known to predict future disease
(e.g., Steptoe & Marmot, 2005) and mortality (e.g., Cole,
Blackstone, Pashkow, Snader, & Lauer, 1999). Duchenne
smiling was particularly advantageous, which indicates that
Downloaded from pss.sagepub.com at PENNSYLVANIA STATE UNIV on March 3, 2016
1376
Kraft, Pressman
a
Neutral Condition
Standard-Smile Condition
Duchenne-Smile Condition
84
Heart Rate
80
76
72
68
64
60
0
90
180
270
360
270
360
Time (s)
b
Neutral Condition
Nonaware Condition
84
Aware Condition
Heart Rate
80
76
72
68
64
60
0
90
180
Time (s)
Fig. 3. Mean heart rate (in beats per minute) during the recovery period
following the cold-pressor task as a function of measurement occasion and
condition. Results are shown separately for (a) the three facial-expression
groups and (b) the aware and nonaware subgroups of the two smiling groups
(collapsed across groups). The analysis controlled for condition adherence,
exercise, baseline perceived stress, perceived task difficulty, and facial-muscle
fatigue. Error bars represent standard errors.
sincere smiles may be more effective for stress recovery than
standard smiles. To our knowledge, this is the first indication
that Duchenne smiling plays a role in the stress response. Note
that the chopsticks were in place only during stress; thus, our
results indicate that smiling was beneficial for a period of time
after the stressors and facial-muscle activation had ended.
These findings match the “stress undoing” predictions of
Fredrickson et al. (2000) given that effects for smiling were
found only after the stressors had occurred
Aware smiling did produce a small advantage in stress
recovery as compared with nonaware smiling. Although
results of the aware and nonaware groups were not different
from each other, the results from those who heard the word
“smile” during instruction were significantly different from
the results of the neutral condition; however, nonaware individuals had only marginal or nonsignificant recovery advantages. This may mean that even though individuals were
unaware that the study was examining smiling, awareness that
their face was positioned like a smile offered some advantage,
perhaps by priming the idea of positive affect prior to a stressful period. Nonetheless, nonaware smilers had similar but
lesser heart rate benefits without this knowledge.
Small and marginal facial influences on affect were found
for those smiling during cold stress, consistent with initial
hypotheses. This was true for both smiling groups compared
with the neutral group but only for those unaware of smiling.
Unlike in past facial-feedback hypothesis manipulations, smiling did not increase positive affect but instead reduced the detrimental affect influences of stress. These findings are consistent
with emotional blunting and affect-processing changes seen in
participants told to inhibit facial-emotion expression (e.g.
Duclos & Laird, 2001) and participants unable to manipulate
facial muscles because of botulinum-toxin-induced paralysis
(Davis, Senghas, Brandt, & Ochsner, 2010; Havas, Glenberg,
Gutowski, Lucarelli, & Davidson, 2010). It is likely that the prevention of negative expressions during stress in addition to
forced positive expression contributed to these results. It is
interesting to note that, given the lack of affect findings for the
star task, state emotion change may not be the mediator connecting facial expression to heart rate. Post hoc analyses testing
positive affect, negative affect, and their arousal subcomponents
as possible mediators of found effects revealed that reported
changes accounted for a nonsignificant amount of variance
(10% or less). If smiling is altering emotion in an important
way, it is occurring outside the range of self-report awareness. It
is also possible that emotion changes from artificial facial
manipulation during stress are not easily tested by self-report,
because of conflicting feelings. This also raises the intriguing
possibility that there are pathways connecting facial-muscle
activity to autonomic activity that do not require conscious
emotion.
The generalizability of these findings to the real world is
questionable given the artificiality of the setting and manipulation. Also, considering that long-lasting emotion-incongruent
displays have been shown to be harmful (e.g., Goldberg &
Grandey, 2007), it is important to consider other factors,
such as duration, context, and frequency. More likely is
the possibility that “fake” smiling may be useful for brief or
painful stressors, such as receiving an injection (see Lanzetta,
Cartwright-Smith, & Eleck, 1976, for a similar paradigm).
Consistent with these ideas, our results showed affect advantages only in nonaware smilers. This may indicate that
individuals who had some idea that their faces were in an
incongruent emotion position did not receive the affect benefits. This study also looked at heart rate as the only dependent
variable of interest. Blood pressure was assessed but not
reported because of space constraints and lack of consistency
in findings. Although blood pressure was sometimes consistent
Downloaded from pss.sagepub.com at PENNSYLVANIA STATE UNIV on March 3, 2016
1377
Grin and Bear It
with heart rate (e.g., Duchenne smilers showed better results
following the two stress tasks than either standard smilers or
the neutral group did), many of the contrasts were not significant, which indicates that smiling may have less of an impact on
vasculature changes. Future work should examine additional
cardiovascular indicators, use more precise beat-by-beat measures of blood pressure, and explore possible pathways for why
these outcomes have differential associations with smiling.
Overall, these results suggest that the adage “grin and bear
it” does have proven value and that the benefits of smiling
through stress should not be ignored. Given that facial expression is a ubiquitous part of everyday communication, future
studies examining stress responses in individuals with facial
paralysis or impairment are warranted, as are studies investigating the relative benefits of different smile types.
Acknowledgments
The two authors contributed equally to this project and are listed
alphabetically. We express our great gratitude to Dena Rhodes, Emily
Hooker, and Vanessa Rupp for their hard work on this study and our
thanks to the many other undergraduate research assistants who
assisted with this project. Finally, we would like to thank Pete
Gianaros for his comments on this manuscript.
Declaration of Conflicting Interests
The authors declared that they had no conflicts of interest with
respect to their authorship or the publication of this article.
References
Abel, E. L., & Kruger, M. L. (2010). Smile intensity in photographs
predicts longevity. Psychological Science, 21, 542–544.
Bem, D. J. (1972). Self-perception theory. In L. Berkowitz (Ed.),
Advances in experimental social psychology (Vol. 6, pp. 1–62).
New York, NY: Academic Press.
Cohen, S., Kamarck, T., & Mermelstein, R. (1983). A global measure
of perceived stress. Journal of Health and Social Behavior, 24,
385–396.
Cole, C. R., Blackstone, E. H., Pashkow, F. J., Snader, C. E., & Lauer,
M. S. (1999). Heart-rate recovery immediately after exercise as
a predictor of mortality. New England Journal of Medicine, 341,
1351–1357.
Davidson, R. J. (1992). Emotion and affective style: Hemispheric
substrates. Psychological Science, 3, 39–43.
Davis, J. I., Senghas, A., Brandt, F., & Ochsner, K. N. (2010). The
effects of BOTOX injections on emotional experience. Emotion,
10, 433–440.
Duchenne, G. B. (1990). The mechanism of human facial expression
(R. A. Cuthbertson, Ed. & Trans.). Cambridge, England: Cambridge University Press. (Original work published 1862)
Duclos, S. E., & Laird, J. D. (2001). The deliberate control of emotional experience through control of expression. Cognition &
Emotion, 15, 27–56.
Ekman, P. (2001). Telling lies: Clues to deceit in the market place,
politics, and marriage. New York, NY: W. W. Norton.
Ekman, P., & Davidson, R. J. (1993). Voluntary smiling changes
regional brain activity. Psychological Science, 4, 342–345.
Ekman, P., Davidson, R. J., & Friesen, W. V. (1990). The Duchenne
smile: Emotional expression and brain physiology II. Journal of
Personality and Social Psychology, 58, 342–353.
Ekman, P., & Friesen, W. (1978). Facial Action Coding System: A
technique for the measurement of facial movement. Palo Alto,
CA: Consulting Psychologists Press.
Ekman, P., & Friesen, W. V. (1982). Felt, false, and miserable smiles.
Journal of Nonverbal Behavior, 6, 238–252.
Fredrickson, B. L., & Levenson, R. W. (1998). Positive emotions
speed recovery from the cardiovascular sequelae of negative
emotions. Cognition & Emotion, 12, 191–220.
Fredrickson, B. L., Mancuso, R. A., Branigan, C., & Tugade, M. M.
(2000). The undoing effect of positive emotions. Motivation and
Emotion, 24, 237–258.
Goldberg, L. S., & Grandey, A. A. (2007). Display rules versus display autonomy: Emotion regulation, emotional exhaustion, and
task performance in a call center simulation. Journal of Occupational Health Psychology, 12, 301–318.
Harker, L., & Keltner, D. (2001). Expressions of positive emotion
in women’s college yearbook pictures and their relationship to
personality and life outcomes across adulthood. Journal of Personality and Social Psychology, 80, 112–124.
Havas, D. A., Glenberg, A. M., Gutowski, K. A., Lucarelli, M. J.,
& Davidson, R. J. (2010). Cosmetic use of botulinum toxin-A
affects processing of emotional language. Psychological Science,
21, 895–900.
James, W. (1890). The principles of psychology (Vol. 1). New York,
NY: Henry Holt.
Laird, J. D. (1974). Self-attribution of emotion: The effects of expressive behavior on the quality of emotional experience. Journal of
Personality and Social Psychology, 29, 475–486.
Lanzetta, J. T., Cartwright-Smith, J., & Eleck, R. E. (1976). Effects
of nonverbal dissimulation on emotional experience and autonomic arousal. Journal of Personality and Social Psychology,
33, 354–370.
Lyubomirsky, S., King, L., & Diener, E. (2005). The benefits of frequent positive affect: Does happiness lead to success? Psychological Bulletin, 131, 803–855.
McNair, D. M., Lorr, M., & Droppleman, L. F. (1971). Profile of mood
states. San Diego, CA: Educational and Industrial Testing Service.
Pressman, S. D., & Cohen, S. (2005). Does positive affect influence
health? Psychological Bulletin, 131, 925–971.
Rosenberg, E. L., Ekman, P., Jiang, W., Babyak, M., Coleman,
R. E., Hanson, M., & Blumenthal, J. A. (2001). Linkages
between facial expressions of anger and transient myocardial
ischemia in men with coronary artery disease. Emotion, 1, 107–
115.
Steptoe, A., Gibson, E. L., Hamer, M., & Wardle, J. (2007). Neuroendocrine and cardiovascular correlates of positive affect measured
by ecological momentary assessment and by questionnaire. Psychoneuroendocrinology, 32, 56–64.
Steptoe, A., & Marmot, M. (2005). Impaired cardiovascular recovery
following stress predicts 3-year increases in blood pressure. Journal of Hypertension, 23, 529–536.
Strack, F., Martin, L. L., & Stepper, S. (1988). Inhibiting and facilitating conditions of the human smile: A nonobtrusive test of the
Downloaded from pss.sagepub.com at PENNSYLVANIA STATE UNIV on March 3, 2016
1378
Kraft, Pressman
facial feed-back hypothesis. Journal of Personality and Social
Psychology, 54, 768–777.
Tourangeau, R., & Ellsworth, P. C. (1979). The role of facial response
in the experience of emotion. Journal of Personality and Social
Psychology, 37, 1519–1531.
Treiber, F. A., Musante, L., Kapuku, G., Davis, C., Litaker, M., &
Davis, H. (2001). Cardiovascular (CV) responsivity and recovery
to acute stress and future CV functioning in youth with family
histories of CV disease: A 4-year longitudinal study. International Journal of Psychophysiology, 41, 65–74.
Turner, J. R. (1994). Cardiovascular reactivity and stress: Patterns of physiological response (W. J. Ray, Ed.). New York, NY:
Springer-Verlag.
Usala, P. D., & Hertzog, C. (1989). Measurement of affective states
in adults: Evaluation of an adjective rating scale instrument.
Research on Aging, 11, 403–426.
Downloaded from pss.sagepub.com at PENNSYLVANIA STATE UNIV on March 3, 2016
Replications in Psychology Research:
How Often Do They Really Occur?
Perspectives on Psychological Science
7(6) 537–542
© The Author(s) 2012
Reprints and permission:
sagepub.com/journalsPermissions.nav
DOI: 10.1177/1745691612460688
http://pps.sagepub.com
Matthew C. Makel1, Jonathan A. Plucker2, and Boyd Hegarty3
1
Duke University, 2University of Connecticut, and 3University of New Hampshire
Abstract
Recent controversies in psychology have spurred conversations about the nature and quality of psychological research. One
topic receiving substantial attention is the role of replication in psychological science. Using the complete publication history
of the 100 psychology journals with the highest 5-year impact factors, the current article provides an overview of replications
in psychological research since 1900. This investigation revealed that roughly 1.6% of all psychology publications used the
term replication in text. A more thorough analysis of 500 randomly selected articles revealed that only 68% of articles using
the term replication were actual replications, resulting in an overall replication rate of 1.07%. Contrary to previous findings
in other fields, this study found that the majority of replications in psychology journals reported similar findings to their
original studies (i.e., they were successful replications). However, replications were significantly less likely to be successful
when there was no overlap in authorship between the original and replicating articles. Moreover, despite numerous systemic
biases, the rate at which replications are being published has increased in recent decades.
Keywords
replication, research methodology, content analysis
Confirmation comes from repetition. Any attempt to
avoid this statement leads to failure and more probably
to destruction.
John Tukey (1969, p. 84)
The recent publication of a controversial study on extrasensory perception (Bem, 2011) along with a few well-publicized
fraud cases have catalyzed a healthy conversation within the
psychological research community about the process used to
publish research (e.g., Carpenter, 2012; Crocker & Cooper,
2011; Roediger, 2012; Wagenmakers, Wetzels, Borsboom, &
van der Mass, 2011; but cf. Bem, Utts, & Johnson, 2011). One
topic that has received substantial attention is the role of replication in psychological science. When do results require replication? Who should conduct these replications? And where
should they be published?
Such concerns are certainly not unique to psychology, but
they further highlight the importance that replication can play
in psychological research. As the introductory quotation by
Tukey notes, although replication is far from a miracle cureall, it can help identify, diagnose, and minimize many concerns about the integrity and reproducibility of research. Some
have gone so far as to call replication the Supreme Court or
gold standard of science (Collins, 1985; Jasny, Chin, Chong,
& Vignieri, 2011, respectively).
However, despite a general positive regard for replications
for the improvement of psychological science, conducting
replications is viewed as lacking prestige, originality, or
excitement (Lyndsay & Ehrenberg, 1993; Neuliep & Crandall,
1993). In other words, a field that replicates its work is rigorous and scientifically sound, but researchers who conduct
those replications are looked down on as bricklayers and not
advancing knowledge. If the field has truly been set up to deter
replications (or to require authors to bend over backward
to make their work appear not to be a replication), then one
would predict that replications in psychology would be
extremely rare. If replications are common, however, their
presence would suggest that concerns about disincentives are
not warranted. However, to our knowledge, there have been no
systematic investigations of the prevalence of various types of
replication. The current study sought to inform the discussion
of research integrity by investigating replication rates in published psychological research.
Replications in Psychology
The current article provides an overview of replications in
psychological research since 1900. We conducted an exploratory investigation into how the issues reviewed above correspond with the publication of replications in psychological
Corresponding Author:
Matthew C. Makel, Talent Identification Program, Duke University, 1121 W.
Main Street, Durham, NC 27701
E-mail: mmakel@tip.duke.edu
538
research. Two primary questions drove our investigation: How
many replications are being published? And is the number of
replications being published changing over time? We were
also interested in whether they were direct or conceptual replication and whether the original findings were successfully
replicated.
Lykken (1968) noted that, as researchers, “we are interested
in the construct . . . not in the datum” (p. 156). He went on to
propose three types of replications: literal, operational, and
constructive. Schmidt (2009), in a review connecting the discussion of replication theory with replication practice, eliminated Lykken’s (1968) literal replication (because it essentially
requires the original investigator to gather data from additional
participants) and reframed the latter two types as direct and
conceptual replications. In a direct replication, the new
research team essentially seeks to duplicate the sampling and
experimental procedures of the original research by following
the same “experimental recipe” provided in the methods section of the original publication. In a conceptual replication, the
original methods are not copied but rather purposefully altered
to test the rigor of the underlying hypothesis. Whereas direct
replication examines the authenticity of the original data, in
conceptual replication, the replicator tests the construct and
not the datum to which Lykken referred. We use Schmidt’s
classification in this article, as it largely encapsulates recent
conversations within the field.
Article Selection Process
We identified the top 100 journals according to 5-year impact
factor in psychology (all types) by using the online search
engine ISI Web of Knowledge Journal Citation Reports, Social
Sciences Edition (2010). In May 2012, using Web of Knowledge, we searched the entire publication history of each of
these 100 journals to identify the total number of articles published as well as the number of articles that contained the
search term “replicat*” in the text (i.e., any articles containing
words with the stem “replicat”). This method is similar to what
Fanelli (2010, 2011) has used when searching publication histories in large databases.
The replication rate of each journal was calculated (number
of articles containing “replicat*” divided by total number of
articles) to determine the percentage of articles that were replications. This was also calculated by year, to determine
whether the replication rate changed over time. Then, as a reliability check to assess the extent to which the term replication
was actually referring to a new replication being conducted,
we randomly selected and analyzed 500 of the articles containing the term “replicat*.” This analysis assessed whether (a) the
term was used in the context of a new replication being conducted; if so, (b) whether it was a direct or conceptual replication; and (c) whether the replication was considered a success
or failure. The number of times both the replication and original article have been cited was also recorded (if multiple studies were being replicated, the average of the replicated studies
Makel et al.
was calculated; the citation counts of books were not recorded
because they are not calculated by Web of Knowledge).
Finally, authorship of each article was also recorded. If the
original and replicating papers had no overlap in authors, they
were coded as “unique.”
All of the data were collected by the first author. The secondary authors were given a set of written instructions (similar
to the paragraphs above) to score a subset of articles by using
the same procedures as the first author. In 18 out of 20 cases,
the articles were assessed similarly, providing evidence that
the method identifying replications is itself replicable. The
500 randomly selected articles were then divided and coded
independently by the authors.
Analysis
Journal information
The average 5-year impact factor of the top 100 journals in
psychology was 5.42 (range = 3.09 to 24.51). Overall, the term
“replicat*” was used in 1.57% (5,051 of 321,411) of articles
with specific journals ranging from 0% to 6.08% (see the
online supplement at http://pps.sagepub.com/supplemental for
individual journal data). However, after the year 2000, “replicat*” use was 2.17 times higher (95% CI = 2.06, 2.30) than it
was from the 1950s to 1999 (2.39% vs. 1.10%, respectively).1
This suggests that use of “replicat*” is becoming significantly
more common compared with previous decades. In fact, 680
articles used the term “replicat*” in their title (nearly one out
of every seven articles that used the term at all). Although this
number is quite a small percentage of the overall sample, it is
quite high for a field that has supposedly discouraged replications. Using the term in the title is not an indication of trying
to hide the concept.
Replication rates
Table 1 reports the results of the more thorough analysis of
500 randomly selected articles that used the term “replicat*.”
Overall, 68.4% (n = 342) of the analyzed articles that used the
term “replicat*” were actual replications.2 With this correction
factor, the replication rate of psychology journals is 1.07%.
However, the replication rate did not remain constant. Because
of this fluctuation, we added the dashed line to Figure 1, representing the replication rate based on the data analyzed from
each decade. Regardless, even after using the correction factors for each time period, the replication rate after the year
2000 was 1.84 times higher (95% CI = 1.72, 1.96) than it was
from 1950 to 1999. The increase in replication rate is particularly noteworthy given that it coincides with an explosion
in the overall number of articles published. For example,
as shown in the secondary y-axis in Figure 1, there were
more articles published in the 10-year period from 2000
to 2009 (98,920) than in the entire period from 1900 to
1979 (75,036). Thus, a higher replication rate represents a
539
Replications in Psychology Research
Table 1. Replication Rates by Authorship and Success Rates by Replication Type (out of 342 Articles).
Percentage of replications published
Replication type
Overall (N = 342)
In same paper
By same authors
By same journala
34.5%
52.6%
19.0%
1950s–1999 (n = 146)
2000–present (n = 196)
26.0%
47.9%
30.1%
40.8%
56.1%
10.7%
42.7%
74.0%
15.1%
11.0%
13.7%
70.0%
20.0%
10.0%
80.8%
78.0%
12.7%
9.3%
5.5%
25.0%
37.5%
37.5%
57.3%
82.7%
5.6%
11.7%
14.3%
75.0%
10.7%
14.3%
82.7%
86.4%
3.7%
9.9%
3.1%
16.7%
33.3%
50.0%
47.9%
90.0%
2.9%
7.1%
52.1%
59.2%
26.3%
14.5%
56.6%
92.8%
0.9%
6.3%
43.4%
69.4%
11.8%
18.8%
Outcome of replications
All replications
Successful
Failed
Mixed
Direct (14%, N = 48)
Successful
Failed
Mixed
Conceptual (81.9%, N = 280)
Successful
Failed
Mixed
Both (4.1%, N = 14)
Successful
Failed
Mixed
78.9%
9.6%
11.4%
72.9%
14.6%
12.5%
82.8%
7.5%
9.6%
21.4%
35.7%
42.9%
Replication by authors
Same authors (52.9%, N = 181)
Successful
Failed
Mixed
Unique authors (47.1%, N = 161)
Successful
Failed
Mixed
91.7%
1.7%
6.6%
64.6%
18.6%
16.8%
a
This does not count articles that replicated findings within the same article.
multiplicative increase in the overall number of replications
being conducted.
Who publishes replications?
Table 1 also reports the percentage of actual replications that
were published in the same article (usually through a subsequent experiment reported as part of a multistudy article).
Subsequent “follow-up” studies in the same multistudy article
may not be as valuable as independent replications (because of
potential experimenter bias) but are also not lacking in value.
Similarly, an unexpected finding was the high rate of replications being published in the same journal that published the
original study; 19% of all replications were published in the
same journal as the original study (this does not count 34.5%
of replications published in the same article). But perhaps
more important, 52.9% of replications were conducted by the
same research team as had produced the replicated article
(defined as having an overlap of at least one author, including
replications from the same publication).
High authorship overlap is important to note because the
success rates of replications were significantly different based
on whether there was author overlap, with replications from
the same research team more likely to be successful than replication attempts from a unique research team (91.7% vs.
64.6%, respectively), χ2(1, N = 303) = 32.72, p < .001, Cramer’s V = .33. In fact, when at least one author was on both the
original and replicating articles, only three (out of 167) replications failed to replicate any of the initial findings. Such
results may reflect the file-drawer problem (i.e., researchers
may be loath to publish failed replications of their own
work). Although certainly contributing to research knowledge,
540
Makel et al.
Total Articles Published
Use of “replicat*”
Replication Rate
3.0
100,000
75,000
Percent
2.0
1.5
50,000
1.0
25,000
Number of Publications
2.5
0.5
0.0
0
1900s 1910s 1920s 1930s 1940s 1950s 1960s 1970s 1980s 1990s 2000s 2010s
Fig. 1. Replication rate in the top 100 psychology journals. The solid line represents the percentage of publications
(from 100 journals with the highest 2010 5-year impact factor) that used the term “replicat*.” The dashed line
reports the replication rate based on the percentage of articles using the term “replicat*” that were actual
replications. The bars represent the total number of articles published in that decade. The 2010s bar is truncated
because data from only 2.5 years of the current decade were available.
“first-party replications” may not account for potential experimenter bias, whether intentional or unintentional.
Table 1 also reports the specific breakdown of the success
rates of replications, on the basis of whether they were direct,
conceptual, or included both (in multiple studies). Overall,
81.9% were conceptual, 14.0% were direct, and 4.1% included
facets of both. Although conceptual replications appear more
likely to be successful than direct replications (82.8% vs.
72.9%, respectively), this was not statistically significant,
χ2(1, N = 295) = 2.94, p = .09, Cramer’s V = .10. Although this
may seem somewhat counterintuitive (i.e., one would expect
successful direct replication to be more likely than conceptual
replication), failed conceptual replications may be less likely
to be submitted or accepted for publication. Moreover, it may
be that only particularly surprising results inspire researchers
to attempt to replicate directly.
Citation of replications
The median citation count of the articles that were actually
replications was 17 (range = 0–409), whereas the median for
the articles being replicated was 64.5 (range = 1–2,099). Obviously, the original articles have had more time to be cited
because they are all older than their replicating counterparts
are (median publication year of 1992 and 2001, respectively3).
However, being cited 17 times is quite high (for a comparison,
only three of the 100 journals have a 5-year impact factor
higher than 17). These citation statistics somewhat weaken
the argument that replications are not valued by the research
community.
Discussion
The current study sought to provide a comprehensive survey
of published replications in psychological research. By analyzing the publication history of the top 100 psychology journals, the current study found that roughly 1.57% of psychology
publications used the term “replicat*.” A more thorough analysis of 500 randomly selected articles revealed that only 68%
of the articles that used the term were actually replications,
creating an overall replication rate of 1.07%. Contrary to previous findings in other fields (e.g., Ioannidis, 2005), this study
found that the vast majority of both direct and conceptual replications in psychology journals reported similar findings to
their original studies (i.e., successful replications). However,
replications were significantly less likely to be successful
when there was no overlap in authorship between the original
and replicating articles.
As seen in Figure 1, an inflection point appears in the current data in the 1990s, with a significant jump in replication
rate. It is interesting to observe that the growth in replications
over time (i.e., the slope of the dashed line in Fig. 1) flattened
between the 1990s and 2000s but appears to be increasing in
the 2010s. This may be a function of the recent increased
attention to positive bias, the file-drawer problem, and prevention of scientific fraud. The replication rate found in the current study is not dissimilar to replication rates reported in other
fields. Although comprehensive replication rates could not be
found for other domains, individual studies report that replication rates in business, marketing, and communication journals
range from 1% to 3% (Evanschitzky, Baumgarth, Hubbard, &
541
Replications in Psychology Research
Armstrong, 2007; Hubbard & Armstrong, 1994; Kelly, Chase,
& Tucker, 1979). However, unlike the current study, those
investigations reported apparent slowdowns in replication
rates over time.
What merits replication?
Currently, the system used to determine what studies merit replication is, as Hunt (1975) somewhat generously described,
“informal and somewhat haphazard and could be improved”
(p. 588). On an intellectual level, it is a question of optimizing
the relationship between resources devoted to research and the
accuracy of results. On a practical level, replicating important
and relevant findings provides policy makers with important
information needed to create effective policy. However, at the
same time, there is concern over who is responsible for determining what needs replicating (and who should do it). As
shown in the current article, the original research article need
not even have spurred hundreds of citations for a replication to
get published. Nevertheless, as a field, we need to decide what
is “good enough” for replication rates. We are reluctant to provide a recommendation for how many replications should
be published because other recommendations, such as using
p values less than .05 and Cohen’s demarcation of small,
medium, and large effect sizes, are typically misused and interpreted more like laws rather than the cautious suggestions they
represent (e.g., Cohen, 1988; Rosnow & Rosenthal, 1989).
We are not suggesting that every undergraduate thesis requires
replication, but a conversation about the replication of important studies that impact theory, important policies, and/or large
groups of people would provide useful and provocative
insights, particularly via the implementation of modern methods and measures. That being said, as an arbitrary selection, if
a publication is cited 100 times, we think it would be strange if
no attempt at replication had been conducted and published.
Such a guideline would help avoid flawed or fraudulent findings going unquestioned over an extended period of time.
Research findings require replication because of their influence, not despite it.
Caveats
If research articles are not framed as replications, then they
were not categorized as such. A potential limitation to the current study is that if an author actually intended to replicate but
did not explicitly include that term, it was not captured by the
methods used in the current study (cf. Kelly et al., 1979). Of
course, this limitation extends beyond the present study; it also
limits readers in their ability to connect research to its intellectual precedents. To calculate the rate of replications relying
on a cloaking device, the entire library of articles would have
to be analyzed, which is impractical. Similar in-depth investigations would be required to calculate accurate replication
rates of specific journals. Future research delving more deeply
through entire issues and volumes of journals may reveal
higher replication rates (not to mention different success rates)
once hidden replications are unmasked, but we doubt the conclusions of that multiyear study would be much different from
those emanating from the present methods.
Finally, it is important to remember that replications are not
a cure-all. Just as Campbell and Stanley (1963) cautioned
against considering experimental methods as a panacea, all
replications are not of equal value. In particular, conceptual
replications published within multistudy articles do not necessarily satisfy all the goals of replication, including limiting
experimenter or measurement bias. And, of course, failure to
replicate does not necessarily suggest that a research team is in
the wrong.
The proverb “No press is bad press” is certainly difficult to
swallow when the field is receiving so much negative attention
and criticism about the quality and integrity of its research.
But if the field comes out of the current scrutiny with stronger,
more rigorous methods that lead to deeper understanding of
psychological constructs, then this scrutiny is a much needed
and welcomed push forward. Whether it be in initiation of
incentives or the removal of roadblocks, the path to better
understanding psychological science goes through replicating
important research findings.
Declaration of Conflicting Interests
The authors declared that they had no conflicts of interest with
respect to their authorship or the publication of this article.
Notes
1. The first use of the term replication that we found in a psychology
journal was Rosenblith’s article (1949) in the Journal of Abnormal
Psychology, titled “A Replication of Some Roots of Prejudice,”
which successfully replicated the findings of Allport and Kramer
(1946) while relying on college students in South Dakota instead of
those in Harvard, Radcliffe, and Dartmouth.
2. The articles that used the term “replicat*” but were not actual
replications typically used related terms in the context of stating that
the study’s results needed to be replicated, that more replications of
a given study were needed, or that specific genes were replicated, or
a specific database with “Replication” in its name was used in the
given study (e.g., National Comorbidity Survey Replication).
3. Only articles that replicated previously published findings were
included in this comparison; articles that replicated only another
study from the same article (i.e., a multistudy article) were excluded.
References
Allport, G. W., & Kramer, B. M. (1946). Some roots of prejudice.
Journal of Psychology, 22, 9–39.
Bem, D. J. (2011). Feeling the future: Experimental evidence for
anomalous retroactive influences on cognition and affect. Journal of Personality and Social Psychology, 100, 407–425. doi:
10.1037/a0021524
Bem, D. J., Utts, J., & Johnson, W. O. (2011). Must psychologists
change the way they analyze their data? Journal of Personality
and Social Psychology, 101, 716–719. doi:10.1037/a0024777
542
Campbell, D., & Stanley, J. (1963). Experimental and quasiexperimental designs for research. Chicago, IL: Rand-McNally.
Carpenter, S. (2012). Psychology’s bold initiative. Science, 335,
1558–1561.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
Collins, H. M. (1985). Changing order. London, England: SAGE.
Crocker, J., & Cooper, M. L. (2011). Addressing scientific fraud. Science, 334(6060), 1182. doi:10.1126/science.1216775
Evanschitzky, H., Baumgarth, C., Hubbard, R., & Armstrong, J. S.
(2007). Replication research’s disturbing trend. Journal of Business Research, 60, 411–415. doi:10.1016/j.jbusres.2006.12.003
Fanelli, D. (2010). “Positive” results increase down the hierarchy of
the sciences. PloS ONE, 5, doi:10.1371/journal.pone.0010068
Fanelli, D. (2011). Negative results are disappearing from most disciplines and countries. Scientometrics, 90, 891–904. doi:10.1007/
s11192-011-0494-7
Hubbard, R., & Armstrong, J. S. (1994). Replications and extensions
in marketing: Rarely published but quite contrary. International
Journal of Research in Marketing, 11, 233–248. doi:10.1016/
0167-8116(94)90003-5
Hunt, K. (1975). Do we really need more replications? Psychological
Reports, 36, 587–593.
Ioannidis, J. P. A. (2005). Why most published research findings
are false. PloS MEDICINE, 2, 696–701. doi:10.1371/journal.
pmed.0020124
Jasny, B. R., Chin, G., Chong, L., & Vignieri, S. (2011). Again, and
again, and again. Introduction. Science, 334, 1225.
Kelly, C. W., Chase, L. J., & Tucker, R. K. (1979). Replication
in experimental communication research: An analysis. Human
Communication Research, 5, 338–342.
Makel et al.
Lykken, D. T. (1968). Statistical significance in psychological
research. Psychological Bulletin, 70, 151–159. doi:10.1037/
h0026141
Lyndsay, R. M., & Ehrenberg, A. S. C. (1993). The design of replicated
studies. The American Statistician, 47, 217–228. doi:10.2307/
2684982
Neuliep, J. W., & Crandall, R. (1993). Reviewer bias against replication research. Journal of Social Behavior and Personality, 8,
21–29.
Roediger, H. L. (2012). Psychology’s woes and a partial cure: The value
of replication. Academic Observer. Retrieved from http://www
.psychologicalscience.org/index.php/publications/observer/2012/
february-12/psychologys-woes-and-a-partial-cure-the-value-ofreplication.html
Rosenblith, J. F. (1949). A replication of some roots of prejudice.
Journal of Abnormal and Social Psychology, 44, 470–489.
Rosnow, R. L., & Rosenthal, R. (1989). Statistical procedures
and the justification of knowledge in psychological science.
American Psychologist, 44, 1276–1284. doi:10.1037//0003066x.44.10.1276
Schmidt, S. (2009). Shall we really do it again? The powerful concept
of replication is neglected in the social sciences. Review of General Psychology, 13, 90–100. doi:10.1037/a0015108
Tukey, J. W. (1969). Analyzing data: Sanctification or detective
work. American Psychologist, 24, 83–91. doi:10.1037/h0027
108
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., & van der Maas,
H. L. J. (2011). Why psychologists must change the way they
analyze their data: The case of psi: Comment on Bem (2011).
Journal of Personality and Social Psychology, 100, 426–432.
doi:10.1037/a0022790
Order an Essay Now & Get These Features For Free:
Turnitin Report
Formatting
Title Page
Citation
Outline
