Health Administration PressChapter 3: Variation in Medical Practice and
Implications for Quality
Chapter Outline
• Background Variation in Medical Practice
• Variation in Healthcare
• Analyzing Variation
• Using Variation Data to Drive Healthcare Quality Initiatives
• Baylor Scott & White Health Case Study
• Conclusion
• Study Questions
Copyright 2019 Foundation of the American College
of Healthcare Executives. Not for sale.
1
Health Administration Press
Background:
Random Versus Assignable Variation
• Variation is the difference between an observed event
and a standard or norm.
• Random variation is an attribute of the event or
process, adheres to the laws of probability, and cannot
be traced to a root cause.
• Assignable variation arises from a single or small set of
causes that are not part of the event or process and can
be traced and identified and then implemented or
eliminated.
Copyright 2019 Foundation of the American College
of Healthcare Executives. Not for sale.
2
Health Administration Press
Background:
Process, Outcome, and Performance Variation
• Process variation refers to different usage of a therapeutic
or diagnostic procedure in an organization, geographic
area, or other grouping of healthcare providers.
• E.g., fecal occult blood testing, sigmoidoscopy, colonoscopy, or a combination of
these options for screening for colorectal cancer
• Outcome variation occurs when different results follow
from a single process.
• Performance variation is the difference between any given
result and the optimal result.
• It is arguably the most important category of variation.
Copyright 2019 Foundation of the American College
of Healthcare Executives. Not for sale.
3
Health Administration Press
Variation in Medical Practice:
Warranted vs. Unwarranted Variation
• Warranted variation is based on differences in patient
preference, disease prevalence, or other patient-related
factors.
• Unwarranted variation is variation that cannot be
explained by patient preference or condition or
evidence-based medicine.
Copyright 2019 Foundation of the American College
of Healthcare Executives. Not for sale.
4
Health Administration Press
Variation in Medical Practice:
Wennberg’s Three Categories
• Wennberg has identified three categories of care in
which unwarranted variation indicates different
possible problems:
• Effective care
• Preference-sensitive care
• Supply-sensitive care
Copyright 2019 Foundation of the American College
of Healthcare Executives. Not for sale.
5
Health Administration Press
Variation in Medical Practice:
Sources of Unwarranted Variation in Medical Practice
• Potential sources of unwarranted variation:
• Inadequate patient involvement in decision making
• Inequitable access to resources
• Poor communication
• Role confusion
• Misinterpretation or misapplication of clinical evidence
• Clinician uncertainty
• Economic incentives
Copyright 2019 Foundation of the American College
of Healthcare Executives. Not for sale.
6
Health Administration Press
Variation in Medical Care:
Tools for Quality Improvement
• Where best practices have been identified:
• Clinical guidelines
• Benchmarking and report cards
• Academic detailing
• Pay-for-performance
• When care is “preference sensitive”:
• Comparative effectiveness research
• Initiatives to increase patient engagement
• When care is “supply sensitive”:
• Outcomes research and effectiveness research
Copyright 2019 Foundation of the American College
of Healthcare Executives. Not for sale.
7
Health Administration Press
Analyzing Variation:
The Challenge of Attribution
• Particularly in the context of population health, multiple
providers may have had the opportunity to influence the
outcome of interest.
• Where patients are retrospectively attributed to providers for
performance measurement, the rules used for that attribution
can greatly influence how providers appear to perform.
• E.g., compared to a “default” attribution rule, 11 alternative rules tested in
the context of using commercial health plan claims data to assign
Massachusetts physicians to the categories of “low cost,” “average cost,”
“high cost,” and “low sample size” assigned 17% to 61% of physicians to a
different performance category.*
*Mehrotra, A., J. L. Adams, J. W. Thomas, and E. A. McGlynn. “The effect of different attribution rules on individual physician
cost profiles.” Ann Intern Med 152, no. 10 (May 18 2010): 649-54.
Copyright 2019 Foundation of the American College
of Healthcare Executives. Not for sale.
8
Health Administration Press
Analyzing Variation:
The Challenge of Attribution
• Careful thought needs to go into the design/choice of
any retrospective attribution rule to ensure the
resulting performance measurement holds the relevant
provider(s) accountable for quality or cost.
• Different attribution rules may, therefore, need to be
applied to different measures contributing to the overall
evaluation of the quality or cost of care provided to a
population.
Copyright 2019 Foundation of the American College
of Healthcare Executives. Not for sale.
9
Health Administration Press
Analyzing Variation: Tools for Analysis
• League tables and caterpillar charts, which order
providers from lowest to highest performers on a
chosen measure, are frequently used to examine
variation.
• They are vulnerable to misinterpretation, as the instinct is to focus on
numeric ordering, missing the uncertainty around each provider’s point
estimate and thus the fact that much of the ordering reflects random
variation.
• Better tools are forest and funnel plots.
• Both can avoid the ranking issue.
• Funnel plot makes clear allowance for additional variability among providers
with small volume.
Copyright 2019 Foundation of the American College
of Healthcare Executives. Not for sale.
10
Health Administration Press
Analyzing Variation: Tools for Analysis
Forest plot showing
variation in heart
failure 30-day riskstandardized
mortality in
Medicare patients
for hospitals in
Dallas County (July
2013 – June 2016)
Copyright 2019 Foundation of the American College
of Healthcare Executives. Not for sale.
11
Health Administration Press
Analyzing Variation: Tools for Analysis
Funnel plot showing
variation in heart
failure 30-day riskstandardized mortality
in Medicare patients
for hospitals in Dallas
County (July 2013 –
June 2016)
Copyright 2019 Foundation of the American College
of Healthcare Executives. Not for sale.
12
Health Administration Press
Analyzing Variation: Tools for Analysis
• Statistical process control (SPC) is an approach adopted from
industrial manufacturing that appeals widely across healthcare
improvement activities.
• It combines statistical significance tests with graphical analysis of
summary data as the data are produced, most often in control charts,
which plot measured points together with upper and lower reference
thresholds (calculated using historical data) that define the range of
the random variation.
Copyright 2019 Foundation of the American College
of Healthcare Executives. Not for sale.
13
Health Administration Press
Analyzing Variation: Tools for Analysis
• SPC can provide the time sensitivity so important to pragmatic
improvement.
• BUT, having originated in a setting characterized by repetitive
manufacturing of identical products, care must be taken in its
application to healthcare, where individual patient characteristics
vary widely and influence both the appropriateness of delivery of
certain processes of care and the outcomes that can be expected
from that care.
Copyright 2019 Foundation of the American College
of Healthcare Executives. Not for sale.
14
Health Administration Press
Using Variation Data to Drive Healthcare Quality
Initiatives
• National quality improvement efforts applying variation
include:
• Medicare’s Hospital Value-Based Purchasing Program
• Medicare’s Hospital Readmissions Reduction Program
• Individual hospitals and healthcare systems with the
capacity and infrastructure for quality monitoring and
improvement use variation data to identify opportunities
for quality improvement and/or evaluate the effectiveness
of quality initiatives.
Copyright 2019 Foundation of the American College
of Healthcare Executives. Not for sale.
15
Health Administration Press
Case Study: Baylor Scott & White Health
• Increasing use of evidence-based heart failure therapies
Copyright 2019 Foundation of the American College
of Healthcare Executives. Not for sale.
16
Health Administration Press
Conclusion
• Keys to successful management (rather than
elimination) of variation in pursuit of quality healthcare
include the ability to
• identify variation;
• distinguish between random and assignable
variation;
• determine the meaning, importance, or value of the
observed variation relative to some standard; and
• implement methods that will take advantage of or
rectify what the variation reveals.
Copyright 2019 Foundation of the American College
of Healthcare Executives. Not for sale.
17
Health Administration Press
Study Questions
1. While exploring opportunities to improve processes of care for a
group practice, you find no variability in compliance with the US
Preventive Services Task Force’s recommendations for colorectal
cancer screening across the practice’s physician over time. Is this
absence of variation optimal? Why or why not?
2. Distinguish between random and assignable variation. Discuss the
relevance of each of these to measuring quality of care and to the
design and evaluation of quality improvement initiatives.
3. Describe the three categories of care identified by Wennberg
(2011), the possible opportunities for improvement unwarranted
variation might indicate in each of these categories, and the goals of
health services research and quality improvement initiatives in each
category.
Copyright 2019 Foundation of the American College
of Healthcare Executives. Not for sale.
18
Health Administration Press
Chapter 5:
Statistical Tools for Quality Improvement
Chapter Outline
• Intro: Framing “Improvement”
• Process-Oriented Thinking and Statistics
• Variation: Common vs. Special Causes
• Analyzing Data over Time:
• Run Chart
• Quantifying a Process’s Inherent Variation: Control Chart
• IChart: “Swiss Army Knife” for Data over Time
• Common Cause Strategies: Stratification
• “Are We ‘Perfectly Designed’ for ‘Never Events’?”
• “Did We Make a Difference?”
Copyright 2019 Foundation of the American College of
Healthcare Executives. Not for sale.
Health Administration Press
Different Mindset from “Basic”
Statistics Courses
• No mention of normal distribution (assumption not
needed)… or any distribution
• Power of plotting data over time (context of “process”)
• Whether or not you understand statistics, you are
already using statistics.
• You are surrounded with daily opportunities.
• Key: understanding common vs. special causes of variation
• People tend to treat all variation as special.
• Tampering: treating common cause as special
Copyright 2019 Foundation of the American College of
Healthcare Executives. Not for sale.
Health Administration Press
“Safety” Concerns
Year
11
Year
Year
Year
22
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sept
Oct
Nov
Dec
10
14
9
12
8
11
14
11
9
8
9
15
130
11
10
16
6
7
8
8
12
7
14
15
8
122
Total
↓ 6.2 %
Want “tough” 25% reduction: What’s next?
Copyright 2019 Foundation of the American College of
Healthcare Executives. Not for sale.
Everything is a process
Improving quality = Improving processes
Inputs
Conversion
action
Outputs
D
a
t
a
• GAP (VARIATION): How it currently works vs. how it should work
o Reduce inappropriate, unintended variation (more predictable)
• Inputs: people, methods, machines, materials, measurements
(data), environment
o Each input is a source of variation, reflected in output.
• Two types of variation: special (unique) and common (systemic)
ALL variation
as special
o People
Treatingtend
one to
astreat
the other
makes things
worsecause.
• The use of data is a process (4 processes):
definition, collection, analysis, interpretation (each has variation)
Health Administration Press
Reduce Variation in Analysis/Interpretation
Run chart: time-ordered plot with median as a reference line
24 months
safety concern
datacause
No change
in of
2 years:
common
Trend: Six successive increases or decreases (RARE)
Shift: Eight-in-a-row all above or all below median
Copyright 2019 Foundation of the American College of
Healthcare Executives. Not for sale.
Health Administration Press
Point to point variation:
Does it look like this…?
Common cause
context of variation
Used as “yardstick” to
determine special cause
…or this?
Copyright 2019 Foundation of the American College of
Healthcare Executives. Not for sale.
Timeordered
Data
Moving Range:
ABSO(xi-x(i-1))
Moving
Range (MR)
Sorted
MR
10
14
ABSO(14-10)
4
0
9
ABSO(9-14)
5
1
12
ABSO(12-9)
3
1
8
ABSO(8-12)
4
1
11
ABSO(11-8)
3
1
14
ABSO(14-11)
3
1
11
ABSO(11-14)
3
1
9
ABSO(9-11)
2
2
8
ABSO(8-9)
1
3
9
ABSO(9-8)
1
3
15
ABSO(15-9)
6
3
11
ABSO(11-15)
4
3 Median
10
ABSO(10-11)
1
4
16
ABSO(16-10)
6
4
6
ABSO(6-16)
10
4
7
ABSO(7-6)
1
4
8
ABSO(8-7)
1
5
8
ABSO(8-8)
0
5
12
ABSO(12-8)
4
6
7
ABSO(7-12)
5
6
14
ABSO(14-7)
7
7
15
ABSO(15-14)
1
8
ABSO(8-15)
7
Common cause limits:
(Process average) + (3.14* x MRmed )
10.5 + (3.14 x 3) = [1 to 20]
MRmax = (3.865*x MRmed )
MRmax = (3.865 x 3) ~ 12
* From statistical theory
o Used only with MR med
7
Copyright 2019 Foundation of the American College of
10
Healthcare Executives. Not for sale.
Health Administration Press
Data points between red lines are indistinguishable:
(1) from each other
(2) from process average
Copyright 2019 Foundation of the American College of
Healthcare Executives. Not for sale.
Health Administration Press
Transition: “perfectly designed”
for 50% to “perfectly designed” for ~68%
Intervention worked: need another intervention to get to 75%
What part of
“No more trend lines ever!”
don’t you understand?
1
Same data
Only 3 increases
Intervention
2
Intervention
Copyright 2019 Foundation of the American College of
Healthcare Executives. Not for sale.
Health Administration Press
Two Invalid ICharts
Using overall average
and overall standard
deviation
Default of most software
(uses MRavg)
Note reduced width of limits
Copyright 2019 Foundation of the American College of
Healthcare Executives. Not for sale.
Health Administration Press
Correct Assessment of Guideline
Compliance Current “Process”
New process
Intervention worked
Significant performance drop
2 special cause moving ranges > 11
Copyright 2019 Foundation of the American College of
Healthcare Executives. Not for sale.
Suppose GOAL =
75?
Green: >= 75
Yellow: 70 to 74
Red: < 65
Common or special
cause strategy?
Health Administration Press
What is the 20% of the process
causing 80% of the problem?
Too many “good ideas”
First: Localize the problem using data.
Copyright 2019 Foundation of the American College of
Healthcare Executives. Not for sale.
Health Administration Press
Aggregate data from most recent stable period
Copyright 2019 Foundation of the American College of
Healthcare Executives. Not for sale.
50 noncompliances from each hospital (“enough”)
Which element(s) were not followed?
Bundle elements 3 and 5
Hospitals B and E
Copyright 2019 Foundation of the American College of
Healthcare Executives. Not for sale.
Health Administration Press
Class exercise: How might you stratify complaints,
medication errors, pressure sores, falls, bacteraemias?
Copyright 2019 Foundation of the American College of
Healthcare Executives. Not for sale.
Health Administration Press
Intervention? Keep “plotting the dots”
8 in a row can be “relaxed” to 5 in a row.
Copyright 2019 Foundation of the American College of
Healthcare Executives. Not for sale.
Health Administration Press
IChart more powerful
7 to 10 points are sufficient for initial limits
Signals success after only 3 postintervention points
Copyright 2019 Foundation of the American College of
Healthcare Executives. Not for sale.
Health Administration Press
Post-intervention “perfectly designed” for 82%
75% goal had no part in improvement discussion
Copyright 2019 Foundation of the American College of
Healthcare Executives. Not for sale.
Health Administration Press
Don’t panic!
IChart is “Swiss Army Knife”
(for data plotted over time)
Copyright 2019 Foundation of the American College of
Healthcare Executives. Not for sale.
Health Administration Press
“Good enough” (Press-Ganey “top box”)
Less confusing
“The purpose is not to have charts. The purpose is to use the
charts... You get no credit for computing the right number—only
for taking the right action. Without the follow-through of taking
the right action, the computation of the right number is
meaningless.” —Dr. Donald Wheeler
Copyright 2019 Foundation of the American College of
Healthcare Executives. Not for sale.
Health Administration Press
Reacting to latest percentile ranking:
Common or special cause strategy?
Is it working? What’s changed in four years?
“PLOT THE DOTS!”
Copyright 2019 Foundation of the American College of
Healthcare Executives. Not for sale.
Health Administration Press
149 Root Cause Analyses MUST have
made a difference!
?
Try a root cause analysis of the
149 root cause analyses?
Copyright 2019 Foundation of the American College of
Healthcare Executives. Not for sale.
2.5 Years of Facility Infection Data
Run chart: Any improvement?
Moving ranges in column 4:
Find MRmed
Calculate the common cause limits of the process
(average = 10.03)
Any special causes?
How could they improve it?
Copyright 2019 Foundation of the American College of Healthcare Executives. Not
for sale.
Health Administration Press
MRmed = 5.23
MRmax = 3.865 x 5.23 = 20.2 [largest difference: 19.63]
Common cause limits: 10.03 + (3.14 x 5.23) = [0 to 26.47]
Need common cause strategy:
Aggregate all 109 infections and brainstorm ways to stratify.
Use Pareto matrix if possible.
Copyright 2019 Foundation of the American College of
Healthcare Executives. Not for sale.
Health Administration Press
Case Study 2:
A Current Vague
Healthcare Issue
du Jour
Sketch a run chart.
Is there any statistical evidence of either a downward trend or
beneficial process shift?
Calculate the common cause limits (it’s all the data you’ve got!).
Copyright 2019 Foundation of the American College of
Healthcare Executives. Not for sale.
29 Incidents didn’t seem like enough.
Run chart
andtoIChart
Someone
was able
find previous 2 years’ of data:
1, 0, 3, 4, 0, 0, 3, 1
Add to beginning and reconstruct charts.
Health Administration Press
No special causes
Average = 1.53
MRmed = 1
5: tick…tick…tick…
2Aggregate
or more : “action plan”
Good
strategy?
29 “never
events.”
Copyright 2019 Foundation of the American College of
Healthcare Executives. Not for sale.
With 8 additional quarters:
Could this apply to complaints, medication
Health Administration Press
errors, pressure sores, falls, bacteraemias?
Same process?
Discuss possible categories for stratification
Now what?
and Pareto matrix analysis.
Can aggregate
41 never events
Copyright 2019 Foundation of the American College of
Healthcare Executives. Not for sale.
Live Session
Module #3
HCM520
Introduction to Medical Education
Instructor Name
STEEEP
► Safe -
No harm to patients
► Timely – no waits or delays
► Effective – science and evidence
► Efficient – care should be cost efficient
► Equitable – all should have access
► Patient centered – patient should be this
at the center of care
Course Code and Title
Types of Data
►
Quantitative - What happens
Qualitative – Why it happens
Course Code and Title
Quantitative
► What happens – something occurs in
nature and something else happens
► Variables – different pieces of data
► Statistical in nature – numbers don't
lie
► Correlations – how does one
variable affect another variable
Course Code and Title
Qualitative
► If we know something happens, why
does it happen
► Non numerical affects
► Theme based
► Independent of statistical relevance
► Answering the question why not
arguing that something has
happened
Course Code and Title
Quality Improvement Tools
►
Cause Analysis – identify issue using benchmarks
►
Five whys – ask why five times
►
Cause and Effect Fishbone Diagram – layout all issues
►
Scatter Diagram – review correlation of issues
►
Pareto Chart – refuse occurrence and frequency of
issues
►
Flow chart – reviews process
►
Run Charts and Control Charts – Measurements overtime
Course Code and Title
Fishbone Diagram
Course Code and Title
Scatter Diagram
Course Code and Title
Pareto Chart
Course Code and Title
Flow Chart
Course Code and Title
Control Chart
Course Code and Title
Validity of Measurements
►
Structure – Characteristics of the environment
►
Process – steps in the delivery of care
►
Outcome – results
Course Code and Title
Validity of Measurements
►
►
Benchmarking:
►
Compare
►
Contrast
►
Test
Metrics
►
Key indicators
►
Measurements
Course Code and Title
Quality Improvement
Approaches
►
The Plan-Do-Study-Act (PDSA) cycle – Popular in hospitals
►
The Model for Improvement – Tom Nolan created
►
Lean, or the Toyota Production System – Created by
Massachusetts Institute of Technology
►
Six Sigma – Developed HP
►
Human-centered design - Human Centered
Course Code and Title
Quality Improvement
Approaches - PDSA
►
►
►
►
Plan
►
Understand the problem
►
Ask questions and make predictions
►
Plan to carry out cycle
Do
►
Education and training
►
Carry out plan
►
Document problems
Study
►
Assess effectiveness
►
Make changes
Act
►
Act on what you've learned
Course Code and Title
Quality Improvement Approaches
– Model for Improvement
Course Code and Title
Quality Improvement
Approaches - Lean
►
Developed in 1987
►
Align steps to the meaning of value
►
Identify value stream
►
Make value steps from beginning to end
►
Pull products do not push products
►
Create perfection in the process
Course Code and Title
Quality Improvement
Approaches – Six Sigma
► Reduce variation
► Define – define the situation
► Measure – create metrics that
matter
► Analyze – analyze the results
► Improve – use the results to
improve the process
Course Code and Title
Quality Improvement Approaches
– Human-Centered Design
► Emphasize – understand the needs and
motivations
► Define – to find issue design goals
► Ideate – create possible solutions
► Narrow – select best solution
► Prototype – create test product
► Test – does it work
Course Code and Title
Patient Safety at a Glance: Ch.
9
Root Cause Analysis
HCM520
Quality and Performance Improvement
Introduction
What is Root Cause Analysis?
► Method of incident investigation
► Diagnostic tool rather than safety solution
► Allows system approach to investigation
► Aligns well with investigation methods
used in healthcare and other high-risk
industries
Why Investigate?
►
Primary aim is to learn from incidents
►
Next aim is to determine what can be done to reduce chance
of recurrence
►
Aim is NOT to apportion blame
►
Incident Decision Tree (IDT)
►
Used to provide guidance on whether and to whom issues should
be referred
► Concerns of capability
► Recklessness
► Maliciousness
►
Investigation of concerns not part of safety investigation
process
RCA Overview
►
Can be comprehensive or concise
►
Must always include basic elements to ensure
thoroughness, credible and actionable, and represent
value
►
Set clear terms and follow them
►
Secure adequate time and skills
►
Record/report impact of constraints
►
Avoid lots of concise investigations as they can prove
false economy
RCA Process
►
►
Gathering and Mapping Information
►
Have to understand what happened leading up to incident
►
Investigative interviewing focuses more on listening
►
Consult patient/family; they have unique information
Identify care and service delivery problems
►
►
►
►
►
Identify all points at which:
►
Something happened that should not have
►
Something that should have happened but did not
Analyzing problems
►
Use a fishbone diagram; place a care delivery problem or service delivery problem at head of
fish, analyze why that course of action seemed right
►
Few carefully reviewed fishbone delivers more benefit
►
Training can be provided to aid in impartiality and quality analysis
►
Root causes most significant factors
Generating recommendations and solutions
►
Problems need more than applying discipline, training, updated procedures alone
►
Training in improvement science provides more effective selection/implementation
Implementing solutions
►
Separate actions plans from investigations
►
Avoid conducting investigations with similar outcomes
Writing the investigation report
►
Use template for trend analysis, audit, and shared learning
Effective RCA Investigation
►
Components for success in patient investigation are
same for clinical investigation
►
Avoid extremes; triggers or indications must be correctly
identified
►
Data gathering must be conducted by those skilled in
process
►
Findings must be robustly interpreted
►
Expert selection, application, and monitoring of effective
treatment and remedial action required
►
If meaningful learning and improvement are expected,
organization-wide support is a must
Module #3 Assignment Requirements
❑ Discussion Module 3
►For this week’s discussion, respond to the following in your initial post:
►
●
●
Using the Saudi Digital Library, locate and read three scholarly research
articles regarding research, statistical methods, and specific tools
healthcare (e.g., flowcharts, cause-and-effect diagrams, Pareto charts,
and run charts used in quality measurement and improvement in the
MOH’s efforts towards achieving the Saudi Vision 2030 healthcare goals.
Summarize the articles or journals and explain which tools were most
appropriate to the results and findings of the study. Discuss what
additional tools you would have used or recommend an alternate tool
and explain why this tool would have been best to present the results.
This concludes our live session.
Thank you for your
attendance!
Order an Essay Now & Get These Features For Free:
Turnitin Report
Formatting
Title Page
Citation
Outline
