High School Dropout, Graduation, and Completion Rates: Better Data, Better Measures, Better Decisions (2011)

National Academies Press: OpenBook

Chapter: 3 Decisions Required to Compute the Indicators

Visit NAP.edu/10766 to get more information about this book, to buy it in print, or to download it as a free PDF.

Suggested Citation:"3 Decisions Required to Compute the Indicators." National Research Council. 2011. High School Dropout, Graduation, and Completion Rates: Better Data, Better Measures, Better Decisions. Washington, DC: The National Academies Press. doi: 10.17226/13035.

3
Decisions Required to Compute the Indicators

Dropout and completion rates are among the most basic indicators of the effectiveness of a school, a school district, or a state education system. Although calculating the actual rates may seem simple, judgments must be made about who to include in the base group of students being tracked (i.e., the cohort), who to count as a graduate or a dropout, how many years to follow these students, and how to construct the formula. Each decision substantially affects the resulting rates. A number of reports have contrasted popular calculations in terms of their accuracy, bias, and ease of computation (e.g., Mishel and Roy, 2006; National Institute of Statistical Sciences and the Education Statistics Services Institute, 2005; Swanson, 2003; Warren, 2004). The technical definitional issues that are embedded in those calculations are at least as deterministic of bias and accuracy as the computations. 1

Dropout and completion rates are based on a ratio, and one of the first decisions that needs to be made is how to define the numerator. Depending on the type of rate, the numerator represents dropouts, completers, or graduates. Identifying the students who are graduates is somewhat straightforward—they are the diploma earners. Identifying the students who fall into the other categories and distinguishing between graduates and other completers are more difficult. The second key decision is how to define the denominator, the full cohort of students with the potential of graduating or dropping out. Decisions

Kaufman (2001), for example, found that differences in dropout and completion rates between the National Education Longitudinal Study and the Current Population Survey derived more from differences in definitions of populations and dropout/completion than from differences in their methods.

about how to include transfer students and students retained in grade in the cohort can have a major impact on its size and on the rate that is calculated.

At the workshop, Elaine Allensworth, with the Consortium on Chicago School Research, made a presentation about the kinds of decisions that must be made and the ways that they affect the calculation of the rates. This chapter is drawn from her paper (Allensworth, 2008).

DECISIONS AFFECTING THE NUMERATOR

Distinguishing Between Graduation and Completion

There is generally little ambiguity about who has obtained a high school diploma—lists of graduates must be maintained. However, there are multiple means of completing school besides earning a regular diploma, and not all students complete high school in the same time span. Accounting for different forms of completion has become increasingly complicated. In addition to granting diplomas, schools may offer programs that award General Educational Development (GED) credentials, or students may receive GEDs outside a K-12 school system. 2 Some districts offer alternative schools for students who have dropped out or are at risk of dropping out, and these schools may grant diplomas based on easier standards than the regular high school standards. 3 A number of states offer different levels of diplomas or grant certificates of attendance for students who do not pass state-required examinations or meet other criteria (Laird et al., 2008). Growth in home schooling and distance learning further complicates the ways in which completion is defined (National Institute of Statistical Sciences and the Education Statistics Services Institute, 2005).

There are a number of valid reasons for producing graduation rates that include only students who earn a regular high school diploma. There is little evidence that alternative methods of completion benefit most students who pursue these pathways to finishing high school. As noted in Chapter 1, GED recipients do not usually realize the same economic outcomes as high school graduates and are less likely to pursue higher education. Also, there is substantial variability across states and districts in the extent to which other forms of completion are available and recorded in an accessible manner. 4 Constraining

Pallas (1989) provides a good description of the GED, as well as issues around alternative credentials and the different paths through which students may eventually obtain a diploma or credential or drop out.

This is the case in Chicago, where alternative schools require students to complete the requirements set by the state for a diploma, rather than the much more demanding requirements set by the district.

Data on alternative diplomas, GEDs, are not always recorded if provided by entities other than regular K-12 schools. This can be seen in the Common Core of Data, which allows three categories of completers to be reported (diploma recipients—included in the graduation rate, high school

completion rates to diploma recipients provides some degree of consistency across states and districts with different policies, programs, and data-recording procedures. It also helps maintain stability over time; that is, more comprehensive completion rates might increase over time simply because of an expansion in the availability of alternative forms of completion. 5 For purposes of accountability, counting only diploma recipients prevents perverse incentives for schools to push students out of regular programs into alternative or GED programs.

Constraining completion indicators to diploma recipients also has some shortcomings. Diploma requirements are not equivalent across states and districts, and they may change over time, making them not equivalent across different cohorts of students, even within the same school district. 6 In addition, when used for accountability, rates that include only graduates provide no incentive to districts to offer recovery programs for students not likely to obtain a regular diploma. Diploma-based rates also cannot be calculated for schools that offer alternative credentials, and thus, their effectiveness cannot be compared with schools that offer regular diplomas.

Timing of Completion

It may seem desirable to include all completers in the rate, regardless of how long it took them to complete their education. If the rates are being used as a tool to encourage good practices, it makes sense to allow a long time for completion so that schools do not have an incentive to give up on students who do not complete high school in four years. Time limitations for completion are problematic for students with disabilities: those whose Individualized Educational Plans (IEPs) specify that they remain in school beyond age 18 or who are ungraded and whose time in high school is not clearly defined. Time limitations may exclude summer graduates or students who take just one extra term to complete school. Practically, each extra year that is incorporated into the statistic means an additional year’s wait before the completion rate can be reported. Whether the statistics are used for accountability, program evaluation, or assessment of the current state of educational attainment—it is generally critical to get timely information.

equivalency recipients, and other). The variability in defining completers is notable across states (Mishel and Roy, 2006).

For example, based on the Common Core of Data, Warren (2005) noted a rise in regular diplomas in California from 1996 to 1997, but much less of a rise in the total number of high school completers—a fact he attributed to changes in how diplomas were classified.

In recent years, many states have raised their graduation requirements or have developed plans to raise them. Currently, the standards for graduation vary widely across states. There are also different types of diplomas in some states, and some states allow parental waivers from the default curriculum (Achieve, Inc., 2008).

Time limits bring questions about how to count students still enrolled in school. Counting them as noncompleters maintains the definition as a four-year rate. However, this may not accurately represent the true state of high school graduation in the school or region. For instance, at the workshop, Mel Riddile, with the National Association of Secondary School Principals, noted that in some school districts, a student who does not graduate in June but finishes incomplete coursework over the summer, so that the diploma is awarded in July or August, is not always counted as a graduate in four-year rates. An alternative is to exclude students still enrolled from the indicator—removing them from both the numerator and the denominator. This allows the completion rate to be the opposite of the dropout rate, which is conceptually easier for a wide audience to understand. This also avoids any incentives to push out students who can no longer be counted as graduates. However, excluding large numbers of students from the calculation can be difficult to justify if there is concern that all students be counted. Furthermore, rates have the potential to be manipulated by delaying classification of students as dropouts. 7

Distinguishing Between Dropouts and Transfers

Determining which students are dropouts is much more difficult than determining which students are graduates or completers. While students can graduate or complete a program only once, they can drop out of school repeatedly and from different school systems, making it difficult to know how to classify them at any given point in time.

Students who drop out often have high absentee rates. It is not always clear at which date these students should be considered dropouts, since they may have been absent for long periods of time before officially being dropped from the enrollment list. Some students never show up for high school. If a student finished grade 8 but does not officially enroll in high school, should the student be considered a dropout from the high school he or she never attended? The National Center for Education Statistics (NCES) requires schools to define enrollment on October 1. How are students classified if they leave grade 9 during September, after less than a month of school?

It is challenging to track down students who simply stop showing up at school. The student may have transferred to another school system, but verifying the transfer is difficult and requires administrative resources. Schools may miscode dropouts as transfers, even though their status is actually unknown,

It may also, potentially, give a more accurate picture of the percentage of students who eventually graduate—if the same percentage of students remaining in school eventually graduate, something that can be tested empirically. In Chicago, this is the case, and excluding students still enrolled after four years produces rates that are similar to those that would be produced if students were allowed extra years to graduate.

a practice that leads to underestimates of the true dropout rate. Of particular concern is the way that schools code students who leave school to pursue a GED credential. Some may miscode them as transfers, particularly if they leave to enroll in a GED-granting institution, and as a result may eliminate them from both the numerator and the denominator when calculating rates, a practice that also leads to underestimates of the true dropout rate. Generally, students who leave school to pursue a GED credential should only be considered as transfers if they enroll in a program that grants diplomas (and in this case removed from the denominator of the rates). This tendency may be more prevalent when the rates are used for accountability purposes. For this reason, the National Forum on Education Statistics (2005) encourages states to require validation of transfers to verify (usually through a transcript request) that the student has enrolled in another school that grants regular diplomas.

Policies that require unverified student leavers to be coded as dropouts provide a substantial incentive for schools to obtain valid records of transfers. However, validation costs money, 8 and schools that have the greatest demand for validation of transfers—those with large numbers of mobile students—may not have the necessary resources. In general, requiring validation will inflate dropout rates and decrease completion rates, because students will be classified as dropouts until their transfer status can be verified. For instance, at the workshop, Rob Curtin, with the Massachusetts Department of Education, described a policy change in his state that required that all students coded as in-state transfers to be verified or, if unverified, to be coded as dropouts. This policy change produced a half-percent increase in the dropout rate in the first year it was implemented. Statewide data systems with individual student identifiers are useful for in-state validation, but they do not remedy the problem for across-state and out-of-country transfers.

Another issue is how to classify students who are incarcerated or otherwise institutionalized. It may seem logical to classify these students as transfers since the decision to leave their school was not voluntary, and the receiving institutions should provide them schooling. However, there is often no verification that incarcerated students are enrolled in a program leading to an accredited high school diploma or that they plan to re-enroll in a regular high school when released. Therefore, they are sometimes counted as dropouts, unless they reenroll in school within the time frame being studied.

Finally, decisions must be made about how to classify students whose leave status changes over time. There are many potential paths to eventually becoming a dropout or a completer (Pallas, 1989). Most problematic is that students who have already been counted as dropouts may re-enroll and drop

For a discussion of some of the issues around validation and data requirements, see the final report of the National Institute of Statistical Sciences and the Education Statistics Services Institute’s (2005, Chapter 6 and Appendix D) task force on graduation, completion, and dropout indicators.

out a second or third time. For instance, at the workshop, Robin Taylor, with the Delaware Department of Education, reported that in her state 3 percent of the dropouts left school multiple times.

It is also unclear how to count dropouts who have re-enrolled in a different school or entered a school system through an alternative school. Since they enrolled as dropouts from another place, it may not be fair to the receiving school to count them as regular transfers into the new school or system. Problems of re-enrollment overlap with issues of transfer between schools, and the ways in which they can be handled depend on the way that the group or cohort being measured is defined in the first place.

DECISIONS AFFECTING THE DENOMINATOR

Defining the Base Grade of the Cohort

The cohort represented by dropout and completion indicators is almost always defined by grade, such as students entering grade 9 for completion statistics and students in grades 9-12 for dropout statistics. One complication posed by defining the base group by grade level is that students repeat grades. Grade 9, in particular, is frequently repeated, particularly by students who eventually drop out. 9 Thus, statistics defined by grade level often double- or triple-count students in rates representing different cohorts or years, and the students who are double-counted are those who are least likely to complete school. If students who are repeating a grade are not removed from the calculations, the statistics will be biased against schools with high levels of grade repetition. Because these schools are also likely to have high dropout rates, this accentuates the real differences in dropout/completion rates across schools and can make issues of equity appear more extreme than they really are (Mishel and Roy, 2006; Warren, 2004).

Even grade retention prior to high school can introduce problems in calculating the rates because a student’s probability of dropping out is highly correlated with age. 10 If larger numbers of students enter high school at age 16 than in the past, dropout rates will rise simply because a larger percentage of the students are at an age when they are permitted to drop out. These age

This can be seen, for example, in Abrams and Haney (2004), Nield and Farley (2004), and Allensworth and Easton (2005).

For example, dropping out becomes legal at age 16, pregnancy becomes more likely at older ages, and full-time employment becomes more feasible as students get older. Data from Chicago show a positive relationship between grade level and the probability of dropping out, but it exists only because of the relationship between age and grade level. Holding age constant, there is no positive relationship between grade level and dropping out (the relationship becomes negative because of grade retention effects). Holding grade level constant, there is still a strong positive relationship between age and dropping out.

effects occur beyond any effects of grade retention, which itself increases the probability of eventually dropping out of school (Alexander, Entwisle, and Dauber, 2003; Allensworth, 2005a).

Changes in grade retention patterns prior to high school can also affect the comparability of grade 9 cohorts over time by delaying the entry of the academically weakest students. If larger numbers of students are retained in elementary school one year (e.g., because of the implementation of promotion standards), the dropout rate for one freshman cohort may improve simply because many of the cohort’s low-achieving students have been moved into the next cohort. These kinds of policies affect the comparability of grade-based cohort rates over time and over cohorts.

Schools that have unusual grade configurations may not always be included in rates that define cohorts by grade. For example, middle schools that contain grade 9 but not grade 12 would not produce a graduation rate, even though they have a full grade 9 class. Incorporating ungraded special education students into grade-based cohorts is also problematic. 11

One way to deal with these issues is to define cohorts by age instead of grade level. Age-based cohorts are not affected by patterns of grade retention in years prior to or during high school. This makes the rates more sensitive, accurate, stable, and unbiased than when the cohort is defined by grade level. In addition, students who never make it to high school can be included in the rates with their age peers, allowing the statistics to include middle grade dropouts. Special education students can also be included with their age peers, even if they are in ungraded classrooms. For districts and states, age-based rates can be more useful than grade-based rates to gauge trends over time and to assess the current state of education. However, they are not useful as indicators of school performance, since schools, parents, and the public generally want to know how many of the students who attend a school in a given year graduate or drop out later.

Specifying Enrollment Timelines

Defining the cohort also requires consideration of when the student enrolled in school. Because students could enroll in a school after a set date or leave school before that date, it is more inclusive to define the group as including any student enrolled over the period of a year (e.g., from October 1 of one year to September 30 of the next year) than to choose one date in the year for the base enrollment (e.g., enrolled by the 30th day of school). Most often, schools define school year enrollment from either October to October

NCES includes ungraded students by prorating the total number across grades in the denominator proportional to known graded enrollment rates, with ungraded dropouts included in the numerator (Laird et al., 2008).

or June to June. Choosing one over the other has little effect on the resulting rates, but it does affect their interpretation—as students who dropped out in the school year versus students who were enrolled one year and did not return by the second year. 12

Indicators that base their denominator on students enrolled on a specific date are particularly susceptible to undercounting students who transfer between schools. Those that compare the number of dropouts or completers over a set period (e.g., the school year, the next four years) to the enrollment at the set date will include students in the numerator who are not in the denominator. For instance, any student who enrolled in a school after the date that defined the cohort and then dropped out will be included in the numerator but not in the denominator. This leads to the potential for dropout rates that exceed 100 percent. In contrast, students could be included in the denominator who left the school soon after the cohort was defined and have no chance of being in the numerator, deflating the statistic. 13 If in- and out-mobility rates are similar, and if the students who leave have the same probability of completing/dropping out as the students who enter, then this does not result in bias or inaccuracy. However, these latter conditions do not always hold.

Adjustments can be made to account for in- and out-mobility so that the numerator remains a subset of the denominator. However, some of these adjustments require demographic assumptions that, if untrue, can introduce additional inaccuracy into the rates. Warren (2004) demonstrates how inaccuracies can be introduced when assumptions about in- and out-mobility, cohort size, and grade retention do not hold. In addition, adjustments that account for demographic instability must also assume that students who leave a school are similar to students who enter in terms of their probability of completion/dropout. This is often not the case when schools have different enrollment policies. If the students who leave a school are qualitatively different from the students who enter in a way that is systematically related to dropout/completion, the rates will be biased, even after adjustments are made to the denominator.

An alternative to defining the cohort on a specific date is to include all students enrolled at any time during the school year, or at any time over the four years of high school, for completion rates (with students entering at higher

For the Common Core of Data dropout rates, some schools use the official October-to-October definition to define dropouts in the numerator, and some states use a June-to-June definition. Districts that use June-to-June dropout rates tend to report slightly higher numbers of dropouts, but the differences are small (Winglee et al., 2000).

For instance, a student may be counted in the September enrollment but transfer in November. Unless adjustments are made for out-mobility, that student will be in the denominator of the indicator for the first school but will not be included in the numerator of the indicator for the first school. This will be true even if the student drops out in the spring, because the student’s final status is known to the new school.

grades in each year after grade 9). 14 However, the data requirements are higher for this more inclusive definition; that is, schools must maintain individual-level records of students’ dates of enrollment and departure. Furthermore, this definition results in students being counted at multiple institutions; if statistics are aggregated to higher level units (such as the district or state level), mobile students will be double-counted in the aggregate rates.

Decisions also must be made about whether to include students who are enrolled for a fraction of a year. Should schools be held responsible for students who enter and drop out within a week? If no minimum time period is set, including all students enrolled at a school in the denominator will generally increase dropout statistics because mobile students are more likely to drop out than other students. However, if a sizeable minimum time period is used to define who is a student (e.g., all students who enter after October 1 and remain through the end of the year), then dropout rates will be depressed, since, by definition, new students are included in the indicator only if they stay in school. In the end, issues of transfer and mobility are made no easier by including students enrolled over a period of time versus a snapshot; while the indicators are more inclusive, issues of aggregation and data requirements are more problematic.

Handling Transfer Students

When a student transfers between schools, it is not clear which unit should take responsibility for that student. In theory, there are four options: attribution to the receiving school, the sending school, both, or neither. In practice, schools tend to mix classification decisions depending on students’ final status. For example, the graduation rate for the state of Illinois counts transfer students who graduate with their receiving school but those who drop out to neither school. Most of the graduation rates based on aggregate counts of students (including the Cumulative Proportion Index and the graduation rate in the Common Core of Data) attribute transfers who drop out to their sending school, but those who graduate to both schools—as a graduate at their receiving school and a nongraduate at their sending school. Mixing attribution inherently produces bias in the statistics.

Attribution to Both Schools

Counting students at both their sending and receiving schools is the most inclusive method for dealing with transfers. It ensures that all students are

For example, students entering as grade 10 are grouped with students who started in grade 9 the year before; those entering as grade 11 are grouped with students who started grade 9 two years earlier, etc.

counted. However, it also ensures that transfer students will be counted at least twice—and more than twice for highly mobile students. This is problematic when school-level statistics are aggregated to produce district- or state-level rates because the mobile students (who are most at risk of dropping out) will be overcounted. Counting students at both schools also requires that schools share information or that information on graduation/dropout is maintained in a way that students’ final status can be attributed to their first school.

An alternative might be to weight students’ contribution to their school’s rates by their time enrolled. This alternative is even more data intense, since it would require information about students’ dates of entry and departure at each school to be available and shared between schools. It also would require strict guidelines for defining a dropout date for students with many absences to define the percentage of time enrolled at the final school. It is doubtful that such a method could be practically implemented.

Attribution to Neither School

Counting transfer students at neither the sending nor the receiving school has the obvious disadvantage of excluding all mobile students, which would produce an overly optimistic picture of the state of education at all levels of aggregation. This option eases calculation and data requirements, because no knowledge of final status is necessary. There are also political advantages, since schools do not feel they are being held accountable for problems created in other schools. In practice, only nongraduates are at risk of being excluded from the statistics of all schools—schools tend to want to count all of their graduates. This biases graduation rates upward, leading to more bias than if all transfer students were simply excluded.

Attribution to the Receiving School

Counting students with their final/receiving school makes practical sense because the final status is known to the school to which the student is attributed. It also makes intuitive sense to group students with the school from which they graduate or drop out. Schools generally feel that they should get credit for all of the students they graduate.

Although there are many practical advantages to counting transfers with their receiving school, doing so produces rates that are biased in favor of schools that can control their enrollment, and it can provide perverse incentives to transfer students at risk of not completing school. Attributing dropouts to their receiving school, instead of their sending school, assumes the problems that led students to drop out occurred at the final school. Yet, as discussed in Chapter 5, students’ performance in their first year of high school is extremely predictive of eventual graduation (Allensworth and Easton, 2007). If a student

leaves a school because of problems at that sending school, it may not be fair to credit that student’s eventual withdrawal to the receiving school, especially if the student spent little time at the receiving school. This results in bias in the rates when schools have different enrollment policies. Schools that can choose whether to enroll a transfer student can boost their graduation rate by accepting students who have already shown some success in high school (e.g., on-time eleventh or twelfth graders). 15 This can happen, for example, if a school uses a lottery to pre-enroll eighth graders into grade 9 and then uses lotteries for grades 10-12; such a system precludes grade 9 repeaters. Likewise, schools with open enrollment can seem to be doing worse than they should since they accept students who have been unsuccessful in other places. This is a flaw inherent with the National Governors Association (NGA) graduation rate; it is biased in favor of schools that have some control over their enrollment.

Attribution to the Sending School

The final option is to count students with their first (sending) school. This is the method used for graduation rates at the college level in the NCES Graduation Rate Survey. Given the strong relationship between first-year course performance and eventual graduation, it makes sense from a conceptual standpoint to count students with their first school. It also prevents schools from benefiting by encouraging poorly performing students to transfer, particularly to schools that may not best serve their interests. It may seem unfair to count a student who transferred and then dropped out against the graduation rate of their original school. However, unless students are systematically leaving a school for a particular reason (e.g., because of safety concerns at the school that are forcing students out), the dropout and graduation rates of the school’s former students should balance each other and not substantially affect the dropout rate. Using the first school also allows schools without a grade 12 (e.g., a middle school with grade 9) to be compared with other schools.

The problem with this approach is that students’ final outcomes may not be known to the sending school. It can be done if student-level data are available at a district or state level, but there will still be out-of-district or out-of-state transfers that cannot be included in the calculations. The NCES college graduation rate circumvents this problem by counting all transfer students as nongraduates from their original college. This results in statistics that are unbiased and accurately represent the graduation rate of students who have matriculated at a given college, thus circumventing the problem of overcounting transfer students if statistics are aggregated. The disadvantage is that it underrepresents

Students who transfer in after grade 9 tend to boost graduation rates because they have already shown enough success to move on past grade 9 and because they are followed for fewer years than students who enroll in grade 9.

the percentage of students who graduate overall; that is, it indicates only the percentage of students who graduate from their initial institution.

For purposes of accountability, this method does not produce perverse incentives to push students out of a school, but neither does it provide an incentive to ensure that students who transfer into a school are well served by the receiving school. Politically, it is difficult to convince schools that their graduates should not be counted in their completion indicators.

Choosing Among Methods

Each method of defining the numerator and denominator in a dropout/completion statistic has advantages and disadvantages. No method will produce a statistic that is ideal for all purposes, and decisions about who to include or exclude will have an impact on the rate that is produced. Likewise, rules about who is included and excluded can produce incentives for school officials to act in ways that are not always in the best interest of students. For instance, a school can lower its dropout rate if it is allowed to classify students as transfers rather than dropouts when they leave school and enroll in a GED-granting program. A sole focus on the on-time (four-year) graduation rate can produce incentives for schools to push out students who are likely to take longer than four years to finish high school, particularly if they are allowed to remove these students from both the numerator and the denominator of the rate.

There are no optimal ways for handling these decisions that overcome all the potential disadvantages. However, steps can be taken to ensure that the impact of the decisions is understood. For instance, requiring school officials to document the status of transfer students and the decisions made in calculating the rates (i.e., who is included in the numerator and the denominator) is critical both for interpreting the rates and for revealing unintended incentives in the rules. Reporting a variety of rates—such as both on-time and eventual graduation rates, completion rates that include other methods of finishing high school, rates that do not remove transfer students or incorporate new students, and age-based rates—can provide a more comprehensive picture of schools’ effectiveness at graduating students. Transparency with regard to the methodology underlying the statistics that are created and acknowledgment of their strengths and weaknesses is essential for understanding.

COMPUTING DISAGGREGATED RATES

Reporting disaggregated dropout and completion rates requires decisions about who should be included in each group. The No Child Left Behind (NCLB) Act requires schools to provide separate rates for students grouped by race/ethnicity, disability status, and English language proficiency. Yet it is not always clear who should be included in each group. For example, students

can now self-identify their ethnicity and race using multiple categories (U.S. Department of Education, 2008), and students may change their race/ethnicity designation(s) from year to year. This can make classification difficult and inconsistent across schools. Changes in the ways students are classified result in rates that are not comparable over time, which can lead to false conclusions about the rates for particular groups.

English Language Learners

Calculating dropout and completion rates for English language learners (ELLs) is particularly difficult for a number of reasons. First, definitions of English proficiency are not always consistent across districts and states (Abedi and Dietel, 2004; National Research Council, 2004, in press). States have different criteria for initially classifying students as ELL and for reclassifying them (i.e., removing them from the ELL classification because they have acquired sufficient proficiency in English). This means that cross-state and/or cross-district comparisons of dropout and graduation rates for this subgroup may not be valid. More importantly, students’ English proficiency status changes over time as they acquire sufficient English language skills to be removed from the ELL classification. Many students who are initially classified as ELL in the elementary grades are no longer classified as such in high school.

Students who are still classified as ELL in high school—the years when dropout and completion statistics are calculated—are generally a select group. They may be students who have had difficulty learning English and have remained in the classification longer than is typical. Or they may have been placed in the classification because they recently moved to the United States. Both of these factors are related to completing school. That is, students’ rate of acquisition of English is correlated with their academic achievement and their socioeconomic status, 16 factors that are correlated with dropping out and completion. Immigration at older ages is also related to the likelihood of graduation, as children who move to the United States at older ages tend to obtain fewer years of education than children who immigrate when very young (Allensworth, 1997). Highly mobile students are also more likely to remain classified as ELL longer than students who are less mobile because they tend to have long periods of “service breaks” due to absence or transfer; mobility is another characteristic correlated with high school completion. For these reasons, students still in the bilingual or English as a second language program

Hakuta, Butler, and Witt (2000) found that academic English proficiency takes at least four to seven years, and that the rate of acquisition is related to students’ economic status. Abedi and Dietel (2004) note that high-performing English language learners get redesignated as they obtain proficiency. As a result, low-achieving students are increasingly concentrated in the subgroup, together with students newly classified as English language learners.

in high school would be expected to have higher dropout rates than students not in these programs.

In order to obtain the most accurate estimates of schools’ success rates at graduating ELL students, it is essential to know which students have been classified as ELL at any time during their schooling, particularly during the primary grades. This requires data systems to include records on ELL status from the primary grades onward that remain constant as students progress to higher grades—a requirement not easily met, since many states do not have access to primary grade records for these students. These primary grade records would need to accompany students who transfer, so that receiving schools can correctly classify students who were formerly classified as ELL. Otherwise, statistics that are aggregated from school records will fail to include former ELL students who transferred out of their original school.

Students with Disabilities

Unlike ELL status, students’ disability status does not typically change over time. That is, having a hearing, visual, emotional, or cognitive impairment or a learning disability is usually not a temporary condition. However, classification criteria for these students are not always consistent across schools, districts, or time (National Research Council, 2004), and thus their status as a recipient of special education services may change over time. The classification criteria depend, in part, on the resources available in the school and the district for diagnosis, as well as district policies about designating students as disabled. 17 Because the criteria for placing students into the category differ across schools and districts, the dropout and completion rates for this group of students may not be comparable across units.

Furthermore, students who have achievement well below grade level are sometimes classified as learning disabled simply on the basis of their low achievement, even though they may not meet the other criteria for the learning disability classification. These students may be low-achieving because they actually have a learning disability, or their low achievement may be due to other factors that are related to the likelihood of completion, such as years of low engagement in school. Classifying students into the group based on a

Eligibility for special education services depends on referral, evaluation, and decisions of school staff and parents. These decisions may differ across schools. School policies and resources may affect whether students are referred or determined to be eligible. Chicago, for example, placed limits on the percentage of students who could be determined eligible for special education services, in part, to keep schools from excluding too many students from the accountability policy. When grade promotion policies were put into place in grades 3, 6, and 8, there was a substantial rise in the identification of students in those grades. This seemed to be due to uncertainty as to how to handle students whose achievement remained low after repeatedly failing the exams and being held back one or more years (Miller and Gladden, 2002).

characteristic (low achievement) that itself is highly correlated with dropping out can produce misleading results when the indicator is calculated. That is, the dropout/completion indicator may look worse for this group because it includes students who may not belong in the category and are at a higher risk of dropping out due to the characteristic that caused them to be placed in the category.

In addition, the time element in cohort/group definition is also particularly problematic for students with disabilities, as some of them are given more than four years to complete high school in their Individualized Education Plan or are to remain in the school system past age 18.

AGGREGATING RATES

An additional issue to consider when designing indicators is whether they will be aggregated to represent completion or dropout rates at higher levels of analysis. Rates that are accurately defined at the school level are subject to a number of errors when aggregated to the district, state, or federal level. 18 For instance, the ways that transfer students are handled can result in their being omitted or double-counted in aggregated rates. Having individual-level student data can alleviate these problems. However, subgroup definitions (e.g., defining who is an ELL student or a special education student) become more problematic at the state level, where there may be less detailed longitudinal information on individual students.

Students still enrolled after the date they should have graduated are often counted multiple times, particularly if the rates are calculated from aggregate numbers of students instead of longitudinal student records. They are initially counted as noncompleters in their own cohort; they are counted again in the cohort that follows, either as completers or noncompleters. This introduces inaccuracy in the rates, but whether they are inflated or deflated depends on the specific way in which students are double-counted and whether adjustments are made to the denominator along with the numerator. Incorporating students who remain in school an additional year also makes the indicator less sensitive to changes in dropout and completion that may be occurring in the school, which in turn makes them less useful for evaluating the effects of new policies and practices.

For example, dropout rates in a district with two high schools might represent all students enrolled at each school over a period of four years. If there were high student mobility rates between the schools, so that many students transfer from one to the other, the resulting dropout rates for the schools would still be correct estimates of the percentage of students ever enrolled at those schools who dropped out. But the aggregate dropout rate for the district, based on an average of the two schools, would double-count students who had enrolled at each school. This would lead the district statistic to be inaccurate, even though based on accurate school statistics.