REPORT: Internal Review of High School Test Participation

Key Highlights

  • The internal review process was labor intensive and the credibility of the review is validated by the fact that the district independently identified many of the same students that the Tennessee Department of Education (TDOE) inquired about.
  • After four months of research, the district found no indication of systemic “gamesmanship” where students were being moved out of classes to avoid end-of-course (EOC) exams in order to inflate school performance data, which was the primary allegation made against the district.
  • The district did identify a relatively small number of cases where students were removed from EOC courses and placed in credit recovery without having the opportunity to first attempt the second semester of the course in a traditional classroom.  Most of these students did not take the EOC exam. However, these students accounted for only 0.1% of EOC exams administered during that particular school year.
  • In response to the internal review findings, the district has revised and strengthened its course retake policy to give more specific guidance on when credit recovery should be used to help struggling students.

Review Process

  • Interim Director of Schools Chris Henson asked Chief Operating Officer Fred Carr to investigate the allegations being made against the district related to the use of credit recovery and EOC testing.
  • The TDOE separately asked for information on more than 500 students in regard to EOC exam participation.
  • The Chief Operating Officer is responsible for testing and accountability through the Department of Research, Assessment and Evaluation (RAE). The Chief Academic Officer is responsible for principal supervision.
    • This structure is set up deliberately to separate the two and keep accountability independent from instructional leadership. One does not answer to the other.
  • RAE worked for four months carefully analyzing student data from multiple sources to answer TDOE inquiries and internal questions about student class assignments and EOC participation.
  • This was a time-intensive process that required many hundreds of staff  hours. The result is an “Internal Review” report, as well as a spreadsheet and multiple pieces of documentation delivered to TDOE.
  • In addition, RAE did a system-wide analysis to review school practices related to credit recovery and participation that have resulted in changes to policy and practice.

SUMMARY OF ANALYSIS AND FINDINGS

 Credit Recovery

  • There is no system-wide course code to identify credits earned in credit recovery, making it impossible to accurately identify the number of students taking credit recovery.
  • Furthermore, there is little state guidance on use of credit recovery programs and great variation from district to district.
  • In Metro Schools, there are inconsistencies in the way credit recovery is applied and and used.
  • In several cases, it appears district policies and procedures were not followed, allowing 42 students to earn course credit through credit recovery without initially completing the second semester of the course in a traditional classroom.
    • 42 students represent 0.1% of the 35,561 EOC exams administered in 2014-15.

End-of-Course Exams

  • While there is one outlier among the district’s 24 high schools, deep and detailed analysis of EOC participation rates found no evidence of systemic gamesmanship or intentional manipulation of EOC data.
  • With the exception of one school, all high schools had EOC errors in numbers small enough to be attributable to human error or complicated student mobility.  At the outlier high school, the issue is being addressed with administration.
  • The state calculates participation rate in a simple manner that counts the number of attempted tests against the number of answer documents returned. This counts all students who are enrolled in an EOC class on that day, but not all students who were enrolled in that EOC class at some point during the year.
  • To account for this difference, RAE used several different methods to calculate participation rate. One included following a cohort of students through four years of high school and analyzing their EOC participation independent of the state’s calculation.
    • This method helps account for students moving in and out of classes and taking tests at different times.
    • It also takes into account legitimate exclusions, such as absent students, students in Advanced Placement or International Baccalaureate classes, certain English learner students and others.
  • In both calculations, the state’s official calculation and the district’s more thorough calculation, participation rates were very high.
    • State Calculation
      • 98% overall in 2014-15
      • At or above 97% in each subject for each of the past three years
      • All zoned high schools exceed 95% annually
    • District Cohort Calculation (including legitimate exclusions)
      • Algebra I – 98.4%
      • Algebra II – 96.4%
      • English I – 98%
      • English II – 96.8%
      • English III – 93.1%
      • Biology I – 99.3%
    • In the case of English III, some of these students are scheduled to take the EOC this spring.  Others missed the test last spring and were not required to take it over the summer because the state did not offer EOCs last summer.
    • Given this exceptional circumstance, and in order to get a more typical view of Algebra II and English III participation at these schools, RAE applied the district calculation to the 2013-14 cohort of 11th graders.
      • In this analysis, Hunters Lane High School did have an unusually high number of students (21) with no EOC record in Algebra II. All other schools had 7 or fewer.
      • Hunters Lane also had an unusually high number of students (27) with no EOC record in English III. All other schools had 3 or fewer.
      • The majority of these Hunters Lane students were moved out of the course during the second semester and into credit recovery.

ACT Participation

  • Calculating ACT participation rate can also be done in multiple ways. To answer media inquiries, RAE was asked to calculate the rate as a percentage of 11th graders, which is typically the year students take the test.
  • But in fact, TDOE has indicated that official participation rates, soon to be used in district accountability, will likely be based upon high school completers, not tied to a specific grade level. TDOE even encourages the use of vouchers to allow 12th graders to take the ACT if they miss it in the 11th grade.
  • Obviously, this means vastly different rates can be produced depending on the method used. The district believes it should be calculated against high school completers, as the ACT is seen as a culminating activity for students.
  • Using this method, the ACT participation rate for the class of 2014 was 94%. That number dropped to 87.4% in 2015 due to a snow storm that closed schools on the ACT test day. The make-up day was during spring break. This caused many students to miss the test entirely.
  • In addition, a great deal of data quality issues exist in calculating ACT participation. ACT will often generate multiple records for one student or put students in the district file who are not enrolled in Metro Schools.
  • Because of these issues, RAE is hesitant to generate more detailed participation data until TDOE finalizes the rules surrounding accountability for participation.

Tennessee Department of Education Inquiries

  • Having TDOE provide a list of specific student names and cases for review was extremely helpful. Some cases could be easily explained by absences, mobility, discipline or other circumstances. Others could not.
  • After reviewing cases and sending responses back to TDOE, Department officials still had questions about 86 students. Of these 86 cases, only four schools had more than 5 students on the list:
    • Hunters Lane – 30
    • Maplewood – 14
    • Glencliff – 7
    • Pearl-Cohn – 7
  • Another TDOE inquiry centered on a relatively high percentage of 9th graders without English I EOC answer documents (20.2% in 2013-14 and 17.8% in 2014-15).
  • Analysis of student course history shows that many of these students have been classified as 9th graders for more than one year because they have not earned the required number or type of credits to be on track for four-year graduation.
    • In many other districts, students are not listed as “9th grade repeaters” regardless of credits earned. That is not the case in Metro Schools.
  • So even though these students may be classified as 9th graders, most would have taken the English I EOC in a previous year and many were enrolled in English II or III.
  • In addition, many of those 9th graders were English learners who are taking language development courses before English I.
  • Taking these into consideration, course enrollment analysis shows that less than 3% of 9th graders during these years were not enrolled in an English class.

RECOMMENDATIONS AND NEXT STEPS

  • Metro Schools takes test participation and academic intervention very seriously. While we do not feel there have been widespread, systemic issues in either area, it is clear that more monitoring needs to occur and new policies and procedures implemented.

Credit Recovery

  • Credit recovery can provide good reinforcement of core academic concepts, and we stand by our use of it to help students earn credits in classes they previously failed.
  • However, the TDOE’s current four year calculation for graduation rate puts increased pressure on students and teachers to earn credits, particularly in subjects that require four years of classes like math and English.
  • We recommend the state adopt a five year calculation for graduation rate to give students more flexibility in retaking classes and earning credits.
  • The new student information system through Infinite Campus will allow for course codes that specifically identify credit recovery. This will make reporting and oversight more robust.
  • The course retake policy has been revised and strengthened to give greater guidance for principals, counselors and teachers on the best educational practices for when to use credit recovery to help struggling students.

End-of-Course Exams

  • RAE will produce an EOC course enrollment file at the time of testing to help verify which students need to take EOC exams. This will be post-exam verification much more thorough than it is currently.
  • RAE will also generate a new report to monitor EOC exam participation based on the more strict calculation method to share with district leadership.
  • Based on the arduous data analysis work done by RAE staff to answer TDOE inquiries, a new report will be built in the Data Warehouse that allows for automatic generation of the same data analysis. This report will be available to school counselors and district staff for easy monitoring of student credits earned and EOC requirements.

ACT Participation

  • ACT participation is now part of district accountability to TDOE, though districts do not yet know how TDOE will calculate the rate. Once these business rules are finalized, RAE will refine its own reporting.
  • RAE will continue working with school staff to identify students eligible to test.
  • RAE will continue attempts to address ACT data quality concerns prior to calculating participation rate.
  • We recommend that TDOE allow 12th graders to test on the one day each year set aside for statewide ACT testing.

TNReady and TCAP Tests

  • RAE recommends using the same methods to monitor EOC participation that are detailed above and applying them also to TCAP and TNReady exams in grades 3-8.

Downloads

Metro Schools receives four charter school applications in 2016

On Friday, April 1, Metro Schools received four (4) complete applications to operate charter schools.  All four applications seek to open elementary schools. 

Each application will go through a thorough review process with carefully trained reviewers who will examine each application according to a detailed scoring rubric before offering recommendations to the Board of Education.  Formal reports will be given to the Board by May 31.  The Board then votes to approve or deny each application on June 14, 2016. 

School

Grade Range

Proposed Opening

Number of Students Year 1

Number of Students at Capacity

Rocketship Fresh Start

K-4

2017

448

550

Rocketship Conversion

K-4

2017

500

500

Napier Community School

Pre-K – 4

2017

550

550

Jump Start Charter School

K-4

2017

324

524

This is the first formal step in the process of evaluating and approving charter schools.  Applications that do not meet exacting quality standards or which do not advance the best interests of the students and the district will be recommended for denial.  Only those applications that meet the quality standards will be recommended for approval.  The review and authorization processes are strong and have good track records over the years of authorizing only high performing schools that benefit students and families across Nashville.

The number one priority is always program quality.  Each proposal is first examined for its capacity to provide an exemplary academic program.   The review then includes an evaluation of the operational capacity and long-term financial viability that can support and sustain academic excellence.  Strength in one or two areas of the application does not negate weakness in others; therefore, an application must score “meets or exceeds” standard in all major areas in order for the evaluation teams to recommend approval to the Board of Education.  Along with the capacity review of the written application, the evaluation team will interview each applicant group before drawing together their final reports and recommendations. 

Our focus is always on providing high quality educational opportunities to Nashville families and students, and our process is designed to ensure only the most exemplary schools are recommended for approval.

We need your input! Take the director of schools survey now

As you have probably heard, a renewed Director of Schools search is underway. The Board has worked hard to make this an inclusive process by engaging and leveraging leadership from the Mayor’s Office, education leaders, parents, faith leaders, civic leaders, private sector partners, the nonprofit community and others. A unified effort is essential as we work to put the city’s collective best foot forward to recruit and hire a terrific leader who can take our public schools boldly forward.

As a first step, the Board joined hands with the Mayor and the Nashville Public Education Foundation to convene a Search Advisory Committee to provide initial recommendations on the search process. Their job is not to identify candidates per se, but rather provide advice and counsel about what kind of leader we need, where we might find people with the strongest track record of success, and how we can best compete for top-tier candidates. With these recommendations as a starting point, the Board will commence a robust national recruitment effort.

As part of their work, the Search Advisory Committee has reviewed in depth the community feedback we received during the initial search effort earlier this year. For those of you who participated in the focus groups and community survey last summer, we are deeply appreciative as that has provided a strong foundation. The Committee has expressed an interest, however, in seeking some additional, more specific feedback from our educators and parents in particular.

We are hopeful you would take a few minutes to provide some additional insights. Anything you share in the survey below is strictly confidential. It is being hosted, collected and analyzed by the Nashville Public Education Foundation, an independent, non-profit community organization. You will not be asked for any identifying information (i.e. name, school, etc.).

Click here to complete the follow-up survey: www.nashvillepef.org/director-survey

Metro Schools receives 13 charter applications in 2015

Today Metro Schools received 13 complete applications to operate charter schools before the April 1 deadline for the 2015 application cycle. Single applications for two (2) elementary, three (3) middle, and one (1) high school were received. The remaining seven (7) of those are multiple applications from Rocketship (3) and LEAD Public Schools (4). One application was returned because it was incomplete.

School Name Grade Range Proposed Opening Number of Students Year 1 Number of Students at Capacity
D.R.E.A.M. Academy Pre-K – 4 2016 200 600
KIPP Nashville Primary K-4 2017 100 500
Rocketship 3 K-4 2016 448 560
Rocketship 4 K-4 2016 448 560
Rocketship Conversion K-4 2016 448 560
East End Prep Add 6-8 2017 100 300
KIPP Nashville Middle School 5-8 2017 96 350
Knowledge Academies @The Crossings 5-8 2016 200 300
LEAD Academy Conversion 1 5-12 2016 140 1,000
LEAD Academy Conversion 2 5-12 2017 140 1,000
LEAD Academy Conversion 3 5-12 2017 140 1,000
LEAD Academy Conversion 4 5-12 2018 140 1,000
Cameron College Prep HS 9-12 2016 320 620

View the Applications Online

Each application will go through a careful and thorough review process as 3 teams of 11-12 specially trained educators and community members will carefully examine each application according to a detailed scoring rubric before offering recommendations to the Board of Education. Formal reports are given to the Board June 16th. The Board then votes to approve or deny each application by June 23, 2015.

This is the first formal step in the process of evaluating and approving charter schools. Applications that do not meet exacting quality standards or which do not advance the very best interests of the students and the district will be recommended for denial. Only those that meet the quality standards will be recommended for approval.

“Our review and authorization processes are strong, and have good track records going back a number of years,” said Alan Coverstone, who leads the Innovation Office that manages charter school authorizing for Metro Schools. “Our Board authorizes good schools and they do so according to the needs of our students and the priorities of the district as a whole. It is a system that works and benefits families across Nashville.”

The number one priority overall is program quality. Each proposal is first examined for its capacity to provide an exemplary educational program. The review then includes an evaluation of the operational capacity and long-term financial viability that can sustain academic excellence. Review teams then interview each applicant group before drawing together their final reports and recommendations.

“Above all else, we are focused on providing high-quality educational opportunities to Nashville families,” said Coverstone. “That includes working collaboratively with charter operators for the greater good of our students, district and Nashville as a whole.”

The Budget: World Class Music for Every Student

The Board of Education approved the 2014-15 budget request on April 8. It calls for a $32.5 million increase over last year. That increase includes $17.3 million in required spending like inflation and payouts to charter schools. The rest is made up of strategic priorities to improve instruction in our classrooms.

  • Budget Item: Music Makes Us
  • Investment: $723,400

by Laurie Schell, Director of Music Makes Us

Springtime in Nashville – flowering pear trees, red maple, dogwood in bloom. And music in our schools. Lots of it.

A joint effort of Metro Schools, Mayor Karl Dean and music industry and community leaders, the Music Makes Us initiative is quickly becoming a presence in the community because of the community. We are becoming a district known for its world class music education thanks to the leadership of Mayor Karl Dean, Director of Schools Dr. Jesse Register, the Music Makes Us Advisory Council members and the Nashville music and arts community.

Just a sampling of this spring’s musical endeavors shows how far we’ve come:

  • East Nashville Magnet School Choir working a gig with Ben Folds.
  • Meigs Magnet Middle School Wind Ensemble performing at the annual Tennessee Music Education Association Convention.
  • Mariachi Internacional de Nashville from Glencliff High School performing at the GRAMMY Block Party.
  • Disney Musicals in the Schools in partnership with the Tennessee Performing Arts Center in 15 schools serving elementary, middle and special needs students.
  • ASCAP songwriters teaching their craft at Hillwood, Overton and Nashville School of the Arts High Schools.

Music education is part of the big picture. A recent research study called Prelude: Music Makes Us Baseline Research Report (2013) shows music participation in Metro Schools has important direct and indirect effects on school engagement and academic achievement. Music participation makes a significant, positive difference in student attendance, discipline, grades, graduation rates and test scores. Overall, the more a student participates in music, the more positive these benefits become.

Screen Shot 2014-04-25 at 11.38.20 AM

Proof the Music Makes Us is making a difference: Watch the inspiring story of Rex Yape and his incredible talent.

For Metro students, music is the language of opportunity and access. Founding donors and key partners, including Martha Ingram, Gibson Guitars, Big Machine, Curb Records, Warner Music Nashville, Little Kids Rock and KHS America have enabled us to launch signature contemporary programs that engage more students in making music. Of the 40 new contemporary classes added in middle and high schools, the mariachi, world percussion, songwriting and hip hop classes in particular are reaching students outside of traditional music. Some students are recent immigrants, with limited English proficiency. Some see the contemporary classes as more accessible for those with little prior musical experience. All are gaining valuable skills and knowledge.

Programs are still growing. The Country Music Association Foundation’s Keep the Music Playing program in Metro Schools is creating lasting impact, with their $7 million contribution over the past 7 years. The new instruments have enabled our chorus, band and orchestra programs to reach new heights of musical achievement. This is reflected in the 16% increase in the number of ensembles participating in Concert Performance Assessment in 2013 and in their improved rating. 69% improved their overall rating or remained at the superior level for 2 consecutive years.

We aren’t just blowing our own horn. It’s official! The NAMM Foundation recognizes Metro Nashville Public Schools for its outstanding commitment to music education with a Best Communities for Music Education (BCME) designation. Metro Schools joins 376 districts across the country to receive the prestigious distinction in 2014. In its 15th year, Best Communities for Music Education affirms school districts that have demonstrated exceptional efforts toward maintaining music education as part of the schools’ core curriculum.

Music is in the air.

For more information, visit http://musicmakesus.org.

Read More in the Budget Series:
Part 1 – Prekindergarten
Part 2 – Teacher and Staff Pay Raises
Part 3 – Technology and Training
Part 4 – Literacy
Part 5 – World Class Music Education for Every Student

Metro Schools officials have a budget hearing with the Metro Council scheduled for June 4 at 4:15 p.m. in the Council Chambers. Wear blue to show your support for Metro Schools.

The Budget: Literacy

The Board of Education approved the 2014-15 budget request on April 8. It calls for a $32.5 million increase over last year. That increase includes $17.3 million in required spending like inflation and payouts to charter schools. The rest is made up of strategic priorities to improve instruction in our classrooms.

  • Budget Priority: Dramatic Literacy Gains
  • Investment: $1.29 million

We have set very ambitious goals for our district. Our strategic plan, Education 2018: Excellence for Every Student, calls for 71% of all students to score proficient or advanced on state assessments by 2018. That’s going to take a lot of work and some pretty dramatic gains.

How do we make them? To meet ambitious goals, we need an ambitious plan. It has to address overall instruction so we’re improving in every classroom while also giving intensive support to schools with large percentages of struggling students.

To make the gains we need in reading and language arts, we’ve developed a full literacy plan with two parts, both designed to deliver big results:

  1. Highly targeted intervention programs for students reading below grade level.
  2. Expert partnerships for continuous improvement of everyday classroom instruction.

Intervention

The key to intervention is to see where individual students stand academically and meet them at that level. Teachers can focus on their strengths and their weaknesses to catch them up to grade level. That’s what personalized learning is all about.

To make this happen, we want to hire Reading Recovery teachers who are dedicated to helping kids catch up. They will work with small groups and individuals for intensive literacy instruction leading to mastery.

We also want to open reading clinics with one-on-one tutoring and online intervention programs. These can be spread around the city and even move from school to school as needed.

Improved Instruction

To make sure our everyday classroom literacy instruction is and remains top notch, we will train literacy coaches and school literacy teams who can work with all teachers on how to improve their instruction and student outcomes. Through a partnership with Lipscomb University, recently named one of the top four teacher prep programs in the country, these coaches will get year-long literacy training.

To oversee all of this, we have a Director of Literacy with a focused vision for reading instruction from prekindergarten to 12th grade. This is the kind of intensive support our students need and the level of attention they deserve.

Read More in the Budget Series:
Part 1 – Prekindergarten
Part 2 – Teacher and Staff Pay Raises
Part 3 – Technology and Training
Part 4 – Literacy
Part 5 – World Class Music Education for Every Student

Metro Schools officials have a budget hearing with the Metro Council scheduled for June 4 at 4:15 p.m. in the Council Chambers. Wear blue to show your support for Metro Schools.

The Budget: Technology and training

The Board of Education approved the 2014-15 budget request on April 8. It calls for a $32.5 million increase over last year. That increase includes $17.3 million in required spending like inflation and payouts to charter schools. The rest is made up of strategic priorities to improve instruction in our classrooms.

  • Budget Priority: Technology to improve instruction and training teachers to use it
  • Investment: $1.3 million for technology and $1.3 million for training

In five years, Metro Schools can be the highest-performing urban district in the country. Our strategic plan, Education 2018: Excellence for Every Student, lays out specific strategies to get there.

The key to the entire plan is personalized learning for every student. To do that, we must transform the way we teach and manage classrooms through technology. We are already heavily invested in that transformation thanks to the federal Race to the Top grant. By continuing our investment, we will achieve our goals.

The Metro Schools Learning Technology team is second to none in the United States. Dr. Kecia Ray, who leads the team, is the president of the International Society for Technology in Education and was recently named one of “20 to Watch in Education Technology” by the National School Boards Association.

Combined with the very best teaching, our learning technology plan will ensure our students get more instruction targeted to their specific needs and abilities.

Our learning technology plan has three key parts:

  1. Software to make classrooms more efficient
    Rather than making teachers use a half dozen or more different systems, we’ve brought together all of our computer-based instruction into a single program called SchoolNet.SchoolNet puts everything right at a teacher’s fingertips: lesson plans, standards, data, best practices and more. Further, it helps teachers report and analyze data so they can personalize instruction based on individual student needs.By making it easier to access and interpret all this information in one place, teachers save time and have a more complete picture of each student’s performance and needs.
  2. Software to personalize learning for teachers
    Personalization doesn’t stop at students. Teachers will get targeted professional development from a library of lessons called PD360. Just as the teacher analyzes a student’s performance and decides the best approach for instruction, the PD360 library can give targeted professional development aligned with each teacher’s individual needs.
  3. Online student instruction
    Mirroring many colleges and professional work environments, all Metro Schools graduates are expected to have taken at least one class online. The MNPS Virtual School offers a full slate of online courses for full- and part-time students. Teachers can also take professional development courses online. All of these online courses are delivered through software called Blackboard, which was originally funded through Race to the Top.

This technology is great and is vital to personalized learning, but what does that matter if teachers aren’t trained to use it?

Our All-Star Training program, already well underway, is the largest professional development program we’ve ever undertaken. Every single teacher in Metro Schools will finish it by July 1, 2014. It shows teachers how to make best use of the technology we provide, how to align their work with the Common Core State Standards and PARCC assessments, and how they can share their best practices and borrow great ideas from other teachers. Nearly 10 percent of our classroom teachers have already completed the training (as of early April).

When they complete the training, teachers receive a laptop. That way we give them to know-how and the tools to personalize learning for every student.

This training could soon be a national best practice. Public school districts from Memphis, San Diego and New York City, as well as private companies, have approached us about using our All-Star training.

To continue our successes and build more in the future, we must keep investing in the technology that will help get us there.

Read More in the Budget Series:
Part 1 – Prekindergarten
Part 2 – Teacher and Staff Pay Raises
Part 3 – Technology and Training
Part 4 – Literacy
Part 5 – World Class Music Education for Every Student

Metro Schools officials have a budget hearing with the Metro Council scheduled for June 4 at 4:15 p.m. in the Council Chambers. Wear blue to show your support for Metro Schools.