Interpreting Assessment Results CMAS and CoAlt Science and

Interpreting Assessment Results CMAS and CoAlt Science and

Interpreting Assessment Results CMAS and CoAlt Science and Social Studies CMAS Mathematics and English Language Arts, including Colorado Spanish Language Arts CDE Assessment Unit 2018 Topics CMAS Reports Individual Student Performance Reports District and School Reports Disaggregated Reports CoAlt Science and Social Studies Reports Additional Information Resources General Information Many reports contain Personally Identifiable

Information (PII) and are not meant for public distribution Sample reports do not contain real data Generated for training purposes only Disclaimer on reports: This report is NOT for public review. Distribution within your school/district must be in accordance with state and federal privacy laws, and local school board policy. General Information All state assessment results were embargoed and could not be publicly disseminated, distributed, or published by any means until they were publicly released by the state

State level results were presented during the August State Board of Education meeting District and school exceptions to embargo: Confidential individual student performance reports could be shared with parents Aggregated and individual student level results could be used internally for informational and planning purposes before results are publicly released Any data released by a district or school after the State Board meeting must have the appropriate suppressions applied to meet federal and state data privacy requirements

Acronyms CAS Colorado Academic Standards CMAS Colorado Measures of Academic Success CoAlt Colorado Alternate CSLA Colorado Spanish Language Arts ELA English Language Arts GLE Grade Level Expectation PGC Prepared Graduate Competency PLD Performance Level Description SD Standard Deviation SEM Standard Error of Measurement SPR Student Performance Report CMAS Reports Purpose CMAS assessments measure the degree to which students have mastered the CAS in the assessed

content areas at the end of the tested grade level CMAS results are intended to provide one measure of a students academic progress relative to the CAS Track yearly student growth Allow teachers to see how their students are performing against the standards and identify areas they may need to adjust in their practice in the future Provide school and district comparisons and accountability information for parents, students, and the community CMAS Performance Levels Math and ELA (including CSLA) have five performance levels; science and social studies have four performance levels Performance Levels:

Exceeded Expectations1 Met Expectations1 Approached Expectations Partially Met Expectations Does Not Yet Meet Expectations (Math and ELA/CSLA only) Performance Levels are Criterion Referenced 1 Based on meeting the expectations of the Colorado Academic Standards Not dependent on the performance of other students

The top two levels are indicators of being on track for college and career in the content area CMAS Performance Levels Math and ELA/CSLA Science and Social Studies Level 5: Exceeded Expectations Level 4: Exceeded Expectations Level 4: Met Expectations Level 3: Met Expectations Level 3: Approached Expectations

Level 2: Approached Expectations Level 2: Partially Met Expectations Level 1: Partially Met Expectations Level 1: Did Not Yet Meet Expectations CMAS Performance Levels and Policy Claims Students who... Exceeded Expectations Demonstrate a distinguished command of the concepts, skills, and practices embodied by the Colorado Academic Standards assessed at their grade level. They are academically well prepared to engage successfully in further studies in this content area. Met Expectations Demonstrate a strong command of the concepts, skills, and practices embodied by the Colorado Academic Standards assessed at their grade level. They are

academically prepared to engage successfully in further studies in this content area. Students who exceeded or met expectations are considered on track for college and career in the assessed content area CMAS Performance Levels and Policy Claims Students who Approached Expectations Partially Met Expectations Did Not Yet Meet Expectations

Demonstrate a moderate command of the concepts, skills, and practices embodied by the Colorado Academic Standards assessed at their grade level. They will likely need academic support to engage successfully in further studies in this content area. Demonstrate a limited command of the concepts, skills, and practices embodied by the Colorado Academic Standards assessed at their grade level. They will likely need extensive academic support to engage successfully in further studies in this content area. Do not yet meet academic expectations for the concepts, skills, and practices embodied by the Colorado Academic Standards assessed at their grade level. They will need academic support to engage successfully in further studies in this content area. Types of Scores Appearing on Reports

Scale scores put student performance on a common scale Overall scale score Comparisons can be made within the current assessment year and across years Subscale score Comparisons can be made within the current assessment year and across years Percent correct Comparisons can be made within the current assessment year Reporting Levels Organization

State District School Scores Population Subgroup Individual Student

Overall Test Overall Performance Level Math and ELA Reporting Category Subclaim Category Domain Evidence Statement Science and Social Studies Standard Reporting Category Prepared Graduate Competency (PGC) Grade Level Expectation (GLE) Item Type CMAS Files and Reports

Math and ELA/CSLA Science and Social Studies Student Data File Summary Data File District Summary of Schools School Student Roster Content Standards Roster (page 1) Performance Level Summary Content Standards Roster Evidence Statement Analysis Item Analysis Report

Student Performance Report Individual Student Performance Reports CMAS Individual Student Performance Reports Created for each student who takes a CMAS assessment Provided to help parents understand their childs command over the CAS in the assessed grade level and content area(s)

Includes the scale score, performance level, percentile rank, reporting category performance, percent correct performance CMAS Student Performance Report Scale Scores Overall Range is displayed Arrows () around student scores represent the standard error of measurement (SEM) The SEM is the estimated range of scores that the student might receive if they took the test multiple times Reporting Category/Subscale Performance Overall Reading Reporting Category Science and Social Studies Content Standards, including Scientific Inquiry/Nature of Science

Item Type (Science and Social Studies only) Performance on constructed response and selected response questions State, district, and school level information is provided in relevant sections of the report to help understand how the students performance compares to other students Data suppressed if fewer than 16 students in a category CMAS Performance Level Cut Scores Science and Social Studies* Partially Met Expectations Approached Expectations Met Expectations

Exceeded Expectations Grade 4 Social Studies 300-556 557-698 699-792 793-900 Grade 5 Science 300-545

546-649 650-770 771-900 Grade 7 Social Studies 300-591 592-700 701-769 770-900 Grade 8 Science

300-555 556-651 652-784 785-900 High School Science 300-542 543-672 673-773 774-900

Grade Level *Cut scores were set by Colorado educators based on the specific content that students should be able to engage with at each performance level. Elementary and middle school cut scores were adopted by the State Board of Education in August 2014; high school cut scores were adopted in April 2016. CMAS Performance Level Cut Scores Mathematics and English Language Arts (ELA)* Performance Level Start End 783-810*

850 Met Expectations 750 782-809* Approached Expectations 725 749 Partially Met Expectations 700

724 Did Not Yet Meet Expectations 650 699 Exceeded Expectations *Varies by assessment CMAS Performance Level Cut Scores Colorado Spanish Language Arts (CSLA) Grade Level Did Not Yet

Partially Met Approached Met Exceeded Meet Expectations Expectations Expectations Expectations Expectations Grade 3 650-699 700-724 725-749 750-778 779-850

Grade 4 650-699 700-724 725-749 750-771 772-850 CSLA Scale Scores are on the same scale as CMAS ELA (650-850) CMAS Student Performance Report Percentile Rank Shows how well the student performed in comparison to other students who took the same

assessment across the state Example: A student in the 70th percentile performed better than 70 percent of students in the state Percentiles are independent of the performance level A student in the 60th percentile can still be in the Approaching Expectations performance level if most students taking that test are not at the Met Expectations performance level CMAS Student Performance Report Percent Correct Scores Based on the particular items that appeared on the years test Cannot be compared across years

Can be compared to district and state to spot trends* Based on a small number of items Do not necessarily cover the entirety of a topic Should not be the sole basis for instructional adjustments Math and ELA Overall Writing Reporting Category Subclaims Science and Social Studies Prepared Graduate Competency (PGC) Grade Level Expectation (GLE), if there is more than one GLE in a PGC *If a school performed worse than the previous year (percent correct went down by a small amount) but a school goes from being below the district average to above the district average, it could be an indication that there was improvement CMAS Math and ELA/CSLA Student Performance Reports

Math and ELA Overall Scale Score and Performance Level Overall scale score range is 650-850 This student Approached expectations Performed better than 37 percent of students across the state May benefit from additional support to meet expectations at the next grade level Math and ELA Overall Scale Score and Performance Level

This student Performed better than school average, but below the district and state averages Percentage of students within the state at each of the performance levels Math and ELA Performance Level Descriptions The concepts and skills students in each performance level typically demonstrate for the assessed grade and content area

The PLD that corresponds to the students performance level is included at the bottom of page 1 This is a partial list of concepts and skills from the PLD Visit website for full version ELA Reporting Category Performance Reading Reporting Category Provided as a scale score (i.e., 110-190)

Different from the overall scale score This student performed better than the school average, but below the district and state averages: Writing Reporting Category Provided as overall percent of points earned This student performed below the district and state averages Math and ELA Subclaim Performance Subclaims describe specific skill sets

Presented as a percent of points earned ELA subclaims are divided by reporting category (i.e., Reading and Writing) Reading Writing Math and ELA Subclaims 2017 to 2018 2017 Categories CMAS PARCC assessments provided categories based on the

overall Performance Level for subclaims using IRT (not based on standard setting) 2018 Percent Correct 2018 CMAS subclaims are reported as a Percent Correct Easier to interpret, harder to over-interpret Average percent correct for students who just crossed into Met Expectations shows the average performance on that subclaim for students who are proficient without making a specific claim about proficiency on that subclaim Writing Claim 2017 to 2018 Writing

In 2018, the ELA writing claim was reported as a percent correct rather than as a scale score (10 to 60) as in previous years Writing scores are based on two written essay prompts Task types vary by grade all three types still appear on the assessment Rubrics are available on the CDE website (http:// www.cde.state.co.us/assessment/cmas) Zeros

Number of zeros in 2018 are consistent with 2017 when comparing the number of students who received a zero on two out of three PCRs in 2017 To compare individual student performance to 2017, look at the Content Standards Roster from 2017 which provides Percent Correct data by PCR and each Writing subclaim CMAS Science and Social Studies Student Performance Reports Science and Social Studies Overall Scale Score and Performance Level

Overall scale score range is 300-900 This student Approached expectations Performed better than the district and state averages, but below the school average Performed better than 50 percent of students across the state Percentage of students within the state at each performance level Science and Social Studies Subscale Performance Reporting Category scores are displayed as Subscale Scores

On the same scale as the Overall Scale Score (300-900) Split into three Performance Indicators Potential Relative Weakness Typical (shaded area) Potential Relative Strength Performance Indicators are set one standard deviation (a measure of the spread of the scores) above and below the state average

Around 70% of students perform within one standard deviation of the state average Science and Social Studies Subscale Performance 1 SD around State Average Science and Social Studies PGCs and GLEs Prepared Graduate Competencies and Grade Level Expectations

Percent correct for the student, district, and state for the assessed PGCs and GLEs GLE number is taken directly form the standards PGCs are not numbered in the standards and do not always line up in order with the GLEs PGC information is not shown if there is only one GLE in a PGC Standards text for PGCs and GLEs Science and Social Studies Selected Response and Constructed Response Nearly half the points on the science and social studies assessments come from constructed response items

This scale score can be compared across years (300-900) Science and Social Studies Performance Level Descriptions The concepts and skills students in each performance level typically demonstrate for the assessed grade and content area PLDs for each performance level are found on page 4 District and School Reports Data Files

Student Data File District and school level Contains all student records Student records do not have aggregate score information if the school/district indicated an invalidation code (i.e., Not Tested or Void) for the record Data can be used for district and school accountability purposes Summary Data File

District and school level Contains summarized performance data for all valid test attempts Includes data for the district and each school within the district Rows correspond to the disaggregated groups on the Performance Level Summary report Each level of aggregation is reported separately Only students with scores are included in aggregated score field calculations Students without scores and students with suppressed scores are not included in the denominator for average scores or in the percentage of students at each performance level District Summary of Schools District level only Available for each grade and content area assessed at each school Provides data aggregated at the state, district, and school levels by

Math and ELA/CSLA Overall average scale score Reading overall average scale score Writing overall percent of points earned Subclaim percent of points earned Science and social studies Overall average scale score Reporting category performance and average scale score PGC/GLE Percent correct for each GLE If there is more than one GLE within a PGC then percent correct by PGC is also provided

Math and ELA/CSLA Student Roster Report School level only Available for each grade and content area assessed at each school Provides aggregated data at the state, district, and school levels including Overall scale score Overall performance level Reporting category (ELA only)

Reading overall scale score Writing percent of points earned Subclaim category percent of points earned Content Standards Roster School level only Available for each grade assessed at each school Lists every student for whom a test book/answer document or online record was submitted For each student, provides performance on Math and ELA Domain and standard percent correct Science and social studies

Overall performance overall scale score and performance level SEM range Reporting category/content standards performance and scale score PGC/GLE percent correct Provides the same information aggregated at the state, district, and school levels CMAS Math and ELA Content Standards Roster Percent of points earned by domain and standard group

Averages by state, district and school Individual student performance CMAS Science and Social Studies Content Standards Roster (Page 1) Key Lists performance levels with scale score ranges Explains content standards performance symbols School summary Number and percentage of students in the school at each

standards performance indicator CMAS Science and Social Studies Content Standards Roster (Page 1) Average scale scores by state, district, and school Individual student performance Overall test level Content standards CMAS Science and Social Studies Content Standards Roster (Page 2)

PGC and GLE performance Header includes the points possible and the average percentage for state, district, and school PGC and GLE number is the same as on the SPR The language and number on the SPR is included in the Interpretive Guide for reference PGC percent correct provided if more than one GLE within a PGC CMAS Performance Level Summary District and school level Available for each assessed content area at each grade

Contains aggregated performance level information across the school, district, and state (percentage of students meeting each performance level) Contains disaggregated performance level data by student demographic and program categories and subgroups for either the school or the district Gender, Race/Ethnicity, Language Proficiency, Disability, etc. Results for students for whom no demographic or program information was coded are included in the not indicated subgroups No Scores are NOT included in the denominator for the % in each level CMAS Performance Level Summary Includes data for the state on the district level report; data for the state and district on school level reports Average scale scores Performance levels distribution

CMAS Item Analysis Report District and School level Available for each grade assessed at each school Only if the number of students is greater than 15 Science and Social Studies Performance on individual items 1 point items: % of students answering correctly Multi-point items: average points earned divided by the number of possible points Includes item information

CMAS Science and Social Studies Item Analysis Report (Page 1) Graph: Average percent correct by item for State, District, and School Ordered by difficulty from most to least difficult (based on State level data) Label on the bottom Standard and GLE

CMAS Science and Social Studies Item Analysis Report (Page 2) Item Information Ordered by difficulty matches the order on Page 1 Location on the test section and item number Related PGC and GLE Item type SR Selected Response CR Constructed Response (includes point value)

CMAS Math and ELA/CSLA Evidence Statement Analysis (page 1) Summary of district and school performance by evidence statement Evidence statements describe the knowledge and skills an item or task assesses The most specific level of information on student achievement ELA/CSLA only CMAS Math and ELA/CSLA Evidence Statement Analysis (Page 2)

Item Information Ordered by difficulty matches the order on Page 1 Evidence Statement Colorado Academic Standard Domain corresponds to domains on the Content Standards Roster Disaggregated Reports State Level CMAS Performance Level Results Including Participation State results are available by

Overall by grade and content area Gender Race/ethnicity Free-reduced lunch status Special programs Language proficiency School and district results are available by Overall by grade and content area These data are posted publicly to the CDE

Assessment webpage on the Data and Results subcategory under each assessment State Level CMAS Performance Level Results Including Participation CMAS Science, Social Studies, Math, ELA/CSLA http://www.cde.state.co.us/assessment/cmas-dataandresults CoAlt Science and Social Studies http:// www.cde.state.co.us/assessment/coalt-sciencesocial-dataandresul ts

CoAlt Science and Social Studies Reports CoAlt Science and Social Studies Scoring 4 point items (selected response) 4 points answered correctly independently 3 points answered correctly with an additional prompt 2 points answered correctly after being told the correct answer 1 point answered incorrectly after being told the correct answer 0 points did not respond to the question 2 point items (in groups of 3)

2 points answered correctly 1 points answered incorrectly 0 points did not respond to the question Interpretation of Overall CoAlt Scores CoAlt measures both entry skills and content Points available for demonstration of entry skills: 2 points can be earned per selected response item with no evidence of content knowledge These points allow students to demonstrate consistency in entry skills (responding when asked a question and repeating an answer) CoAlt is intended to measure content knowledge All performance levels above Emerging require evidence of some

content knowledge Cut scores for the performance levels are at the higher end of the point/scale range due to the way the items are scored CoAlt Performance Levels CoAlt has four performance levels Advanced At Target Approaching Target Emerging Advanced and At Target are considered ready to continue academic study in the content area

Inconclusive Designation Students who attempt the assessment, but do not respond to any items CoAlt Performance Levels and Policy Claims The Advanced and At Target levels are ready for continuing study in the content area Students performing at this level demonstrate a solid understanding of the concepts and skills represented by the Extended Evidence Outcomes Advanced (EEOs) of the Colorado Academic Standards (CAS). They are academically well prepared to engage in further studies in the content area with appropriate supports. Students performing at this level demonstrate a foundational understanding of concepts and skills represented by the EEOs of the At Target CAS. They are academically prepared to engage in further studies in the content area with appropriate supports.

Students performing at this level demonstrate a limited understanding Approaching of concepts and skills represented by the EEOs of the CAS. They will likely need moderate academic supports to engage successfully in Target further studies in the content area. Students performing at this level demonstrate an initial understanding concepts and skills represented by the EEOs of the CAS. They will Emerging of need extensive academic supports to engage successfully in further studies in the content area. CoAlt Performance Levels Cut Scores Science and Social Studies* Emerging Approaching Target

At Target Advanced Grade 4 Social Studies 0-142 143-162 163-187 188-250 Grade 5 Science 0-134

135-159 160-183 184-250 Grade 7 Social Studies 0-133 134-162 136-190 191-250 Grade 8 Science

0-127 128-163 164-189 190-250 High School Science 0-139 140-163 164-192 193-250

Grade Level *Cut scores were set by Colorado educators based on the specific content that students should be able to engage with at each performance level. Elementary and middle school cut scores were adopted by the State Board of Education in August 2014; high school cut scores were adopted in April 2016. CoAlt Science and Social Studies Reports Student Performance Report Intended for parents and teachers Content Standards Roster School level only

District School Roster District level only Performance Level Summary District level only CoAlt Student Performance Report Scale Score

Overall test score Range of 0 to 250 First 100 are compressed Arrows around student scores are the SEM CoAlt Student Performance Report Compares student percent correct to state average percent correct Points earned and points possible are listed Content standard descriptions are from the CAS and are the same as CMAS Performance level descriptions are found on page 2 The concepts and skills students in each performance level typically

demonstrate for the assessed grade and content area CoAlt Content Standards Roster School level only Overall scale score and content standards percent correct Averages for state, district, and school Student level for all students CoAlt District School Roster District level only Available for each grade assessed at each school Provides data aggregated at the state and district levels by

Average scale score Reporting category performance (average percent of points earned) Information also provided for each school in the district CoAlt Performance Level Summary District level only Describes group achievement by performance levels and average scale scores Includes the inconclusive designation Students with an inconclusive designation are included in the denominator for the performance level percentage

Additional Information Format of CMAS and CoAlt Results Electronic Files available through PearsonAccessnext Student Data File Summary Data File Electronic PDF reports available through PearsonAccessnext

District Summary of Schools Student Roster (Math, ELA/CSLA) Performance Level Summary Content Standards Roster Item Analysis Reports/Evidence Statement Analysis Student Performance Reports Hard copy reports sent to DACs Student Performance Reports (two copies*) One copy for parents/guardians, one copy to be kept on file locally Distribute to parents/guardians as soon as possible *For CSLA, one Spanish and two English SPRs are received

Disclaimers Only scale scores can be compared across years CMAS Science and Social Studies Overall, Content Standards, and Item Type CMAS ELA and Math Overall, Reading Reporting Category CoAlt Science and Social Studies Overall only Scores cannot be compared across grades

No vertical scaling Percent scores cannot be compared across years, grades, subclaims, domains, or PGCs/GLEs Items in a subclaim, domain, or PGC/GLE will change between years Resources Interpretive Guide to Assessment Results http:// www.cde.state.co.us/assessment/cmas-dataandr esults English Full Guide

Spanish Guide for Parents Spanish SPR shells, with standalone PLD document CDE Communications Resources http:// www.cde.state.co.us/assessment/resources Score report resources Parents Guide to Understanding the Score Reports How to Use the Test Results to Support Your Student Available in English and Spanish

Contact Jasmine Carey [email protected] Shangte Shen [email protected] Questions?

Recently Viewed Presentations

  • Let's Use a Dictionary!

    Let's Use a Dictionary!

    Let's Use a Dictionary! What do we do first? What is a dictionary? What is a dictionary? How to use a dictionary How to use a dictionary How to use a dictionary How to use a dictionary How to use...
  • Spelling, Punctuation and Grammar Revision Guide

    Spelling, Punctuation and Grammar Revision Guide

    Used to indicate a pause in speech or at the very end of a sentence so that words trail off into silence (this helps to create suspense). A pause in speech: "The sight was awesome… truly amazing." At end of...
  • isgillette.weebly.com

    isgillette.weebly.com

    "The Veldt" Response Questions. At what line does George start to believe there is a problem? Explain. Who is to blame for the way the Hadley family is now? Explain, using a quote from the story to support your answer....
  • America Becomes A Colonial Power

    America Becomes A Colonial Power

    "Why is it that the American outlook is blacker now than it has been since the beginning of the war? The whole population of the islands sympathizes with the insurgents. The sooner the people of the United States finds out...
  • MLA Format - Easy Literacy

    MLA Format - Easy Literacy

    MLA Format What is MLA Format? MLA stands for Modern Language Association This is an academic society for the study of linguistics and literature MLA has developed a very specific method of formatting a paper. By academic standards, a specific...
  • Introduction to Statistics for the Social Sciences SBS200

    Introduction to Statistics for the Social Sciences SBS200

    Schedule of readings. Lab sessions. Labs continue this week. Project 4- Two Correlations- Two Regression Analyses. This lab builds on the work we did in our very first lab. But now we are using the correlation for prediction. This is...
  • Existing Legislation - Public Contracts Regulations (Scotland ...

    Existing Legislation - Public Contracts Regulations (Scotland ...

    Existing Legislation - Public Contracts Regulations (Scotland) 2012. Conditions for performance of contracts. 39. (1) A contracting authority may stipulate conditions relating to the performance of a public contract, provided that those conditions are compatible with EU law and are...
  • Learning intentions

    Learning intentions

    Negative feedback Conformer Vasoconstriction Regulator Hypothalamus Homeostasis Effector Vasodilation Thermoreceptor Endotherm Ectotherm