The “Reporting” section contains a large batch of data to analyze the outcomes of interventions. What do these metrics all mean? How can you compare them from intervention to intervention? And most importantly, how do different metrics add clinical value when making decisions?
The table below is broken into the two different types of metrics: within-condition and between-condition, and explains how these metrics in the chart view and Reporting section can enhance clinical decision-making.
Click here for an explanation of the different types of metrics (within-condition vs. between-condition metrics).
Within-Condition Metrics
|
Accel and Decel Celeration
|
Calculation
|
- A linear regression with logarithmic transformations of the correct and incorrect observations over time, within each condition.
- The line represents the minimum amount of variance/bounce of all possible lines that can be drawn through the data.
|
Insights
|
- The line represents the growth rate of behavior over time for each intervention.
- The slope of the line is the celeration and is represented with a multiplication symbol (x = growing behavior), or a division symbol (÷ = reducing behavior).
- E.g. On a daily chart, x1.25 represents a 25% increase in the measured behavior each week, and ÷1.15 represents approximately a 13% decrease in the measured behavior each week (1/1.15 = 0.87, 1 - 0.87 = 13%. Only do this modification for deceleration values).
|
Clinical Value
|
- How quickly is the rate of behavior moving towards your goal? And, does that rate of growth suffice, given the threshold of progress defined for the performer on the given intervention?
|
Accel and Decel Bounce
|
Calculation
|
- The bounce envelope (up-bounce and down-bounce) is calculated on the data around the celeration line, and is calculated using a 90% confidence interval.
- Divide the top of the bounce envelope (up-bounce) with the bottom of the bounce envelope (down-bounce).
- Bounce is always represented with the x symbol and never the ÷ symbol.
|
Insights
|
- This represents the variability of the data within the condition, which a practitioner aims to minimize as much as possible because less variability typically equates to better outcomes.
|
Clinical Value
|
- Shows the magnitude of control a practitioner has over the intervention being run.
|
Accel and Decel Level
|
Calculation
|
- The geometric mean of all correct observations (accel level) and the geometric mean of all incorrect observations (decel level).
|
Insights
|
- One accel and one decel value that provide the average rate of correct and incorrect behavior within an intervention.
- Can be compared across charts and performers to understand how frequently a behavior was exhibited on average over an intervention.
|
Clinical Value
|
- Helps users understand the average rate of behavior over an intervention, which can be compared to expected rates of behavior to make decisions on whether to continue or change the intervention.
- More useful for decision-making as an input value to the Level Multiplier (discussed below in between-condition metrics). If the level has increased or decreased significantly from one intervention to the next, that would provide decisive information to the practitioner on the effect of the change in intervention.
|
Improvement Index
|
Calculation
|
- Divide accel celeration by decel celeration.
- E.g. Accel celeration = x1.25, decel celeration = ÷1.10.
- Convert Precision Teaching values to raw numbers: accel celeration = 1.25/1 = 1.25, decel celeration = 1/1.1 = 0.91
- Divide the accel celeration by the decel celeration: 1.25/0.91 = 1.37.
- If the resulting number is greater than 1, then the learner is improving in their intervention and the value is accelerative (x1.37).
- If the resulting number is less than 1, the learner is not improving in their intervention and the value is decelerative. To get the PT value for this example, convert it back: 1/0.91 = ÷ 1.10.
|
Insights
|
- Because of the way ratio charts work, this value represents an overall state of improvement in the condition and how well the performer was able to both increase correct behavior and reduce incorrect behavior.
- In the example above, the performer had an accel celeration of x1.25 and a decel celeration of ÷1.10. By dividing these two values to get an improvement index of x1.37, you are concluding that in this condition, my performer made a 37% improvement in their performance.
|
Clinical Value
|
- Enables users to put one value to overall improvement, as well as the direction and magnitude of that improvement.
|
Between-Condition Metrics
|
Accel and Decel Frequency Multiplier
|
Calculation
|
- Inputs are the last observation of the previous condition and the first observation of the current condition. Calculate one metric for correct data and one for incorrect data.
- Divide the last data point of the previous condition into the first data point of the current condition.
- E.g. If this value was 0.91, users would calculate the PT value as 1/0.91 = ÷1.10.
- E.g. Last data point of the previous condition = 18, first data point of the current condition = 25. Frequency Multiplier = 25/18 = 1.39.
- If the value is greater than 1, it is accelerative. The example above would be x1.39.
- If the value is less than 1, it is decelerative. It needs to be converted to a decelerative PT value by dividing the value into 1.
|
Insights
|
- This value represents the immediate impact from one condition to another.
- From the example above it is concluded that the performer made an immediate improvement, or the data jumped up 1.39 times (39%) from one condition to the next.
|
Clinical Value
|
- How much immediate growth impact did you have? Or, is this change marginal/worse?
- More useful if you are expecting an immediate improvement from one condition to the next, as the inputs for this equation are just 2 data points (the last frequency of the previous condition and the first frequency of the current condition).
|
Accel and Decel Celeration Multiplier
|
Calculation
|
- Divide the current condition’s celeration by the previous condition’s celeration. Calculate one metric for correct data and one for incorrect data.
- E.g. The previous condition celeration = x1.1 and the current condition celeration = x1.2, celeration multiplier = 1.2/1 = x1.09.
- The same rules apply as detailed above, if the value is greater than 1 or less than 1.
|
Insights
|
- Represents the growth of behavior from one condition to the next. From the example above, it is concluded that the performer’s rate of behavior change has increased 9% from one condition to the next.
|
Clinical Value
|
- Speed change: how much faster or slower did the behavior change as a result of your intervention change?
|
Accel and Decel Level Multiplier
|
Calculation
|
- Divide the previous condition’s level into the current condition’s level. Calculate one metric for correct data and one for incorrect data.
- E.g. The previous condition level = 18, the current condition level = 36. The Level Multiplier = 36/18 = x2.
- The same rules apply as detailed above if the value is greater than 1 or less than 1.
|
Insights
|
- Represents the growth in the average rate of behavior from one condition to the next. In the example above, it is concluded that the performer’s average rate of behavior has increased by 100% from one condition to the next.
|
Clinical Value
|
- How meaningful was the rise or drop in the average rate of responding in the new condition?
|
Accel and Decel Bounce Change
|
Calculation
|
- Divide the previous condition’s bounce into the current condition’s bounce. Calculate one metric for correct data and one for incorrect data.
- E.g. The previous condition’s bounce = x1.1 and the current condition’s bounce = x1.2, Bounce Change = 1.2/1.1 = x.1.09.
- The same rules apply as detailed above if the value is greater than 1 or less than 1.
|
Insights
|
- Represents the growth or decline in the variability of the data being plotted from one condition to the next.
- This is one value that you want to decrease. Lower variance means you have more control over your intervention, and therefore values for bounce change that are decreasing represent better control of your intervention.
- In the example above, it is concluded that the performer’s variance increased by 9% from one condition to the next, representing less control over the intervention.
|
Clinical Value
|
- How much more influence/control are you getting over your trial?
- Is your variance getting better or worse?
|
Improvement Index Change
|
Calculation
|
- Divide the previous condition’s improvement index into the current condition’s improvement index.
- E.g. The previous condition improvement index = x1.1, the current condition improvement index = x1.2, Improvement Index Change = 1.2/1.1 = x1.09.
- The same rules apply as detailed above, if the value is greater than 1 or less than 1.
|
Insights
|
- Represents the degree of improvement the performer generated from one condition to the next. In the example above, it is concluded that the performer improved their performance by 9% from one condition to the next.
|
Clinical Value
|
- How much are we able to improve a performer’s behavior from one condition to the next?
|
There is additional data provided in the “Reporting” section that may be useful for your analysis:
- Beginning Frequency: The first frequency measured in the condition.
- Ending Frequency: The most recent frequency measured in the condition.
- Number of Observations: Represents the number of times you have taken data on each condition.
- Within-Condition: The number of observations in the current condition.
- Between-Condition: The number of observations in the current condition and the previous condition.
- Aim Band Minimum and Maximum: The minimum and maximum aim band, depending on the type of pinpoint being measured. If the pinpoint is accelerative, the minimum and maximum aim band for the acceleration data is provided, and vice versa for decelerative pinpoints. This data will be the same for all conditions on the same chart, since aim bands are currently set on the chart level (something we plan to change soon!)
0 Comments