The Homeschool Effect

Rodger Williams

July 5, 2014

Home educated students have long been recognized by research as scoring substantially higher than expected, on average, on nationally normed achievement tests. We will take a deeper look at the mechanics of how that works out in practice.

The students we consider here are all privately home educated. The family budget supplies the money for that education. Parents control the education process and the curriculum. These parents have willingly chosen to homeschool and are prepared to pay the personal price to do so.

Our data source is state-mandated, nationally-normed tests administered to Oregon homeschool students. Home educating parents are required to have their students tested by qualified neutral testers. It is the legally-required test score results that we analyze here.

Homeschool standardized test results

Test publishers "norm" standardized tests by trying them out on representative samples of students and then grading such that one percent of the students are expected to fall in each of 100 groups, called percentiles. So ten of those percentiles grouped together should have 10% of the students. E.g., 10% of the students would be expected to score in the 1-10 range, 10% should score in the 11-20 range, etc.

homeschool standardized test scores

This chart shows an example of homeschool standardized test scores. The distribution of scores is skewed markedly towards the higher end of scores. There are fewer low scores than expected and more high scores than expected, compared with the normed national averages (labeled Public School in the chart). In fact the average score in this example for homeschool students -- the median -- is the 79th percentile rank, that is, 29 percentiles higher than the national average of 50 for all students.

Looking at the data, one gets the impression that some force is steadily moving the homeschooled students from lower-score buckets into higher-score buckets -- much like a brisk wind blows leaves up against a fence.

The homeschool test score curve is the result of an underlying normal (bell-shaped) curve

To mechanically explain the homeschool shift towards high scores, we need to understand that there are two ways to view this data. The first view is of the normed test scores, themselves. We see that view in the graph above.

That first view is the result of the norming process, which translates an underlying measure of achievement into percentiles. The second view looks at the actual underlying measure of achievement.

public school standardized test score deciles

This graph shows the underlying scale of achievement for public school students. The vertical black dotted lines divide the student population into ten groups, depending on their academic achievement as measured by the tests.

The area under the curve represents the percentage of students, totaling 100%. The areas between vertical lines are all identical, with each group containing 10% of the students. Notice that the black dotted decile markers are spaced closer together where the curve is high and farther appart as the curve gets lower. This is to keep the enclosed area at 10%.

translating homeschool raw achievement measures into standardized test scores

In this graph, the curve on the left shows the distribution of achievement for public school students (median of 50). The curve on the right shows the distribution of achievement for homeschool students (median of 79). Both distributions are normal, with the well-known bell curve.

Standardized achievement tests are normed in terms of the public school curve, as indicated by the vertical black dotted lines. We are interested here in homeschool scores, which the normed standardized tests report in terms of public school percentiles. We show our interest by connecting the black dotted lines to the green homeschool curve. The areas between the black dotted lines and under the green curve represent the percentage of homeschool students scoring in each decile.

Because the homeschool curve is shifted towards higher achievement, the homeschool standardized test scores show up as higher than expected. Only 2% of homeschooled students score in the lowest decile, while 32% score in the highest.

normal curve predicts real-world sample test scores

As this graph shows, the normed view of real-world scores gives decile percentages largely tracking those in the underlying achievement view (overlapping bell curves diagram). There are differences caused by the fact that our data is really just a sample out of the world-wide population of homeschooled students. Statisticians refer to this phenomenon as sampling variability. Smaller samples characteristically demonstrate more variability from the underlying distribution than larger samples.

2013 scores plotted to normal curve

We can also plot the data from the normed view of real-world scores back onto the underlying achievement view. More data points on the left reflect the fact that we are plotting homeschool data normed to the public school curve, with only public school deciles available to plot. The data clearly conforms to a normal curve.

We have Oregon data from 1998 (median of 71) and 1999 (median of 73), which also conform to the normal curve.

1998 scores plotted to normal curve

1999 scores plotted to normal curve

Homeschooling's academic boost overpowers demographic factors like parent education and income

The conventional model used for public school policy decisions portrays the effect that demographic variables -- parent education level, family income, etc -- have on education outcomes.

It would seem natural to utilize the same model to portray homeschool education outcomes. But this approach has a problem dealing with actual homeschool data: The demographic variables so clearly associated with public school outcomes have much less influence in the homeschool setting (parent education level, homeschool family income (Figure 6), public school family income, weak effects of demographic factors on homeschoolers). In other words, homeschooling substantially reduces the effect of these traditional education inputs on student achievement.

The conventional model focuses on who is homeschooling and what their demographic characteristics are. This approach clearly misses the essence of what must be going on to produce the striking education results in homeschooled students.

A more helpful approach is to focus on the act of homeschooling itself. What changes when a family homeschools -- in contrast to what happens with children in public school.

Here are some possibilities:

This list is not intended to claim that these things always happen. Rather, it points to a class of factors which may affect students and their parents because they are homeschooling.

What about factors involved in the decision to homeschool? There are two categories:

For those families who actually commit to homeschooling, we expect the capability-related factors to act like other demographic factors, and thus have similar reduced effects. We consider motivation-related factors which contribute to the conviction to homeschool to be homeschool-specific, since they led to the actual act of homeschooling.

So we see a different model for portraying the effects of both types of factors on homeschool achievement:

demographic factors affect public school students more than homeschool students

In this model, the bulk of the observed boost in homeschool achievement is associated with factors which initiate the act of homeschooling itself, or factors which are started or enhanced by homeschooling.

Individual students have their own innate capacity for learning. Intelligence tests are used to measure this capacity. The homeschool-associated factors, as a whole, boost the homeschooled students towards their potential in learning capacity. Thus there is less scope available for conventional demographic factors to play a part in achievement, explaining the apparent limited effects of those conventional factors on homeschoolers.

This new model explains the data better than the conventional model. We will use the smaller, homeschool-specific ranges of demographic effects for adjustments during the analysis which follows.

Adjusting homeschool achievement for conventional demographic factors

We intend to compare homeschool academic achievement with public school achievement as part of our analysis. In order to make that comparison we will want to level the playing field by adjusting the homeschool data to account for known demographic advantages that homeschoolers may have over the general population when it comes to student achievement.

Homeschool parents typically have more formal education than the broader population. So we need to adjust the homeschool average result at the 79th percentile rank downward to offset this advantage.

Prior research (Figure 5) indicates a seven percentile spread in test scores between homeschool children from families where neither parent has a college degree and those from families where both parents have a degree. Using this range as an indicator of the size of the parent-education effect, we subtract half of the range -- 3.5 percentiles -- from the raw homeschool score average of 79 to produce a partially adjusted homeschool average score equal to the 75.5th percentile rank. Then we subtract an additional 2.5 percentiles to adjust for any other demographic advantages that homeschoolers may have, producing an adjusted homeschool average score equal to the 73rd percentile rank.

Our adjustment choices, reasonable in themselves, are aimed at utilizing a known score distribution which has a median score of the 73rd percentile rank: Oregon 1999 homeschool test scores. We do not need precise numbers for the analysis which follows. We just need ballpark figures.

The Homeschool Effect is so strong that any reasonable adjustments will yield similar analysis results showing how homeschooled students benefit.

Reasonable adjustments would take into account two factors:

reasonable adjustments to median

To check what might be reasonable, let us step through the adjustment process using the hypothetical homeschool-specific demographic effect ranges in the above diagram. Remember that demographic factors have relatively small effects on homeschool achievement. They also overlap in their explanatory power regarding achievement, and thus will have reduced cumulative effect on achievement -- the sum will be less than the parts.

The first hypothetical demographic factor we adjust for will exert its full quota of explanatory power as we apply the adjustment. So we reduce the median by half of the first factor's effect range: A downward adjustment of 3.5 percentile ranks.

The second hypothetical factor is highly correlated with the first. It adds little additional explanatory power, so it contributes only a small part of its single-factor quota of explanatory power to the adjustment: 1.0 percentile ranks downward.

When we get to the third hypothetical factor, much of the available explanatory power has already been applied as adjustments to the median by the first and second factors. However, this third factor is independent enough from the first two that it has additional explanatory power that translates into a significant adjustment of its own: 1.5 percentile ranks downward.

But the pattern is clear: As the available explanatory power of demographic factors is used up, it becomes less and less likely that additional factors can contribute any significant new explanatory power towards adjustments. This fact of diminishing returns limits how low the median can actually be adjusted in the real world. It also reflects the reality of the explanatory power of homeschool-specific factors which are pushing the median upwards.

Given those diminishing returns, we can see by looking at the above diagram that real-world demographic adjustments are unlikely to reduce our adjusted median much past the 73rd percentile rank and almost certainly not as far as the 65th percentile rank.

So to account for demographic factors, we will adjust the 2011-2013 homeschool scores from a median-79 curve down to a median-73 curve. The 1999 curve of Oregon homeschool scores has a median (i.e., average) percentile rank of 73. We will borrow that distribution of scores as a working adjusted distribution for our analysis.

adjusted homeschool scores still skewed towards higher achievement


We have subtracted homeschoolers' demographic advantages out of the 2011-2013 median-79 curve to arrive at this median-73 curve from 1999. The remaining achievement boosts -- fewer low scores than expected, more high scores than expected -- must be due to homeschool-specific factors, as illustrated in our conceptual model of student achievement factors.

We have already confirmed that the underlying 1999 homeschool academic achievement is normally distributed. So we will use a median-73 normal curve to portray homeschool achievement for comparison with public school achievement.

The Homeschool Effect

giving the same boost to students at all ability levels

This graph compares the public school academic achievement normal curve with the homeschool achievement normal curve. Standardized test norming centers the public school curve at the 50th percentile rank and the homeschool curve at the 73rd percentile rank (on the public school curve, i.e., in terms of public school scores).

The red arrows reflect the difference in underlying achievement between public schooling and homeschooling (demographics having been factored out). The arrow endpoints are shown connected to the corresponding normed scores on the public school curve.

The percentile difference measured by each arrow varies according to how far out it is from the center of the curve. Arrows towards the ends, where the curve is lower, cover smaller percentile ranges because there are fewer students in those achievement ranges, as shown by the smaller area under the curve there.

These arrows are all the same length since the homeschool curve is shifted as a whole to the right. This means that homeschooled students traveling from the 1st percentile rank to the 4th percentile (measured on the public school curve) receive just as much of an underlying academic achievement boost as those traveling from the 50th to the 73rd.

So students receive the same benefit from homeschooling (or factors uniquely associated with homeschooling) regardless of their academic ability. We label this consistent academic boost The Homeschool Effect.

Although there may be some homeschooled students who do not conform to the normal curve, they are so infrequent that we cannot detect them in the data. This is so even at the lower achievement levels where we have more-fine-grained data.

The perfection of the homeschool normal curve tells us something else: The reported high homeschool academic performances are not the result of parents withholding low scores. There is no telltale distortion at the low end of the curve to indicate such withholding.

Conclusion

The Homeschool Effect clearly exists. It was boosting Oregon homeschooled students to higher academic achievement levels in 1998 and 1999. In 2011-2013 the effect was even greater.

Homeschooled students benefit across the whole ability spectrum, including those who have the most difficulty learning. The Homeschool Effect is an equal-opportunity aid to upward mobility through excellent education.

Data source: All Oregon data used in this study is available from the Oregon Department of Education.

For more homeschool research results, visit the National Home Education Research Institute -- the definitive source for information about research on homeschooling.

(c) Copyright 2014 Rodger Williams. All rights reserved.