Testing Constraints

Testing constraints are conditions that occur during the test. Kubiszyn and Borich list typical test constraints to include time, reference material available, people, equipment, prior knowledge, and scoring criteria (2010).

For both the multiple choice/short answer and essay assessments, there will be a time constraint placed on the exams. For the essay portion, students will be allotted forty minutes for completion, which should be ample time. For the multiple choice/short answer assessment, students will be allotted twenty minutes, since the test is being read to students. For students that require extended time, it will be granted according to their IEP. 

Students will not be allowed to utilize the textbook to search for answers because this would defer from testing their ability to describe characters when they would most likely copy exact words from the text. Pictures cues are available for the multiple choice series, which provides enough information for students to answer the questions.

 For the essay, students already have an understanding of what a paragraph entails, including the appropriate sentence structure, capitalization and punctuation, which is why organization of the paragraph is considered in scoring. However, this test is not measuring spelling accuracy; therefore spelling errors will not have an effect on scoring on either portion of the exam.

Students may ask questions during tests, but clues to correct answers cannot be provided to students. If a student needs further explanation of the task, this is acceptable.

Students are informed of the scoring criteria, and prior to the essay assessment a review of the story and the rules of writing are discussed. Prior to the multiple choice format, students will review the components that are being assessed.

 

References:

Kubiszyn, T., Borich, G. (2010). Educational testing & measurement (9^th ed). Hoboken, NJ: John Wiley & Sons, Inc. 

Rubric for Scoring the Essay

It is difficult to objectively score an essay because the answers are not as obvious as those of objective tests. There is no obvious right or wrong answers that when evaluating essays. For this reason, it is important to have a grading rubric to use as a guide for evaluating an essay. Developing a scoring rubric helps eliminate the potential of judgments and grading errors, especially after reviewing numerous essays which can change opinions of the quality of essays.  A rubric also helps ensure the reliability of the scoring, as it provides a basis for scoring that can be used by different teachers scoring the exam.  An example of a rubric for the essay assessment previously posted is below:

Category	5	4	3	2	1
Recall	Student recalls two characters from the story with significant detail	Student recalls two characters from the story with little detail	Student recalls two characters from the story with no detail	Student recalls only one character from the story with little detail	Student provides little information about any characters from the story
Describing Character Traits	Student accurately describes character traits of two characters with evidence from the story	Student accurately describes character traits, but lacks evidence from the story	Student describes character traits, but descriptions are inaccurate	Students describes character traits for one character	Student provides little information about character traits
Analysis	Student provides adequate and logical comparisons among characters, with evidence to support their analysis	Student provides adequate and logical comparisons among characters, but fails to provide supporting evidence	Student provides comparisons among characters, with no supporting evidence and/or logical comparison	Student provides evidence of character traits but fails to provide comparison with another character	Student does not provide evidence of comparisons among characters
Amount of Details - Similarities	Student accurately provides at least three similarities among two characters	Student accurately provides at least two similarities among characters	Student accurately provides one similarity among characters	Student provides similarities among characters that are inaccurate	Student does not provide any information about similarities of characters
Amount of Details – Differences	Student accurately provides at least three differences among two characters	Student accurately provides at least two differences among characters	Student accurately provides one difference among characters	Student provides differences among character that are inaccurate	Student does not provide any information about character differences
Organization	Student completes the writing of at least two short paragraphs, written in a logical manner	Student completes the writing of two short paragraphs, but lacks organization of thought	Student  completes one paragraph with logical thought	Student provides a few short sentences.	Student provides little to no writing

Rationale Behind Learning Outcomes and Assessment

In creating learning outcomes and test items, it is imperative that the test items are valid measurements of the learning outcomes. This may seem like common sense, but invalid measurements occur more often than one might think, and it can be easy to mistakenly create tests that do not truly measure what they are intended to do. For instance, as I stated in the previous post, my learning outcomes are not intended to measure a student’s reading ability. Although reading ability is highly important, my purposes of the learning outcomes learn toward comprehension of literature. I’m not looking for reading ability; I am looking for understanding of literature components, such as the parts of a book, the main idea, and character differentiation.  At this stage, I want to make sure my students are developing an understanding of these components and what is read, otherwise words are simply words with no real meaning. Because this is my intent, all written tests devised under these learning outcomes shall be read to the students. If students are left to read the tests independently, there is a chance of inaccuracy of test results because students that have mastered the content but not yet mastered the skill of reading would be clearly at a disadvantage. This would reflect negatively on test scores and would inaccurately portray that they have failed to master the content.

With that in consideration, let’s examine the different test items. For learning outcomes one and two, my test items were based on multiple choice and short answer responses. The multiple choice questions have been examined for grammatical cues and unintentional trickery. My goal is to measure student’s ability to identify the parts of the book, not to try to trick them. The pictures provided are intended to provide realistic application. Although this is not a performance assessment, I want to give students the most realistic examples in this written assessment. If possible, all children would be provided a copy of the book and answer the questions by examination of the book, but in reality picture cues are easier to implement. The disadvantage to such format is that children are limited to select pages that are pictured, so the answer choices are limited, but it does allow for measurement of identification of the parts of a book. Short answer responses are also added to gain more insight of learning outcomes. With multiple choice questions there is always a possibility that students will simply guess at the answer. In order to provide a more accurate view of student understanding, short answer questions are added so that students must provide an answer. The questions do not require higher-level thinking in these assessments because the intent is to measure at the knowledge level in identifying the components.

Creating an essay item presented more of a challenge for me. Although there are great benefits in essay items, devising an item for second grade students seemed a little difficult. Typically at this stage, students are beginning to understand the purposes of writing and essay formats are not generally used. It’s not that it is not possible for second grade student to write an essay; it’s simply that they are still in the learning how to write stage. For this reason, it is important to acknowledge that there will be spelling and grammatical errors. Most children in this grade level understand that there must be capital letters at the beginning of a sentence and punctuation at the end, but they are still learning how to spell, how to construct sentences with a subject and predicate, how to capitalize proper nouns, and so forth. Obviously they will not be free from errors. For this reason, although instilling grammatical rules and proper spelling are important, the use of the essay in this assessment is to measure the ability to understand character traits and comparison among characters. In such an assessment, points shall not be lost due to such errors and should focus on the content of the essay rather than the structure. If the essay was intended to measure grammatical skills in writing, then by all means errors should be included in scoring, but for the intent of this essay, it would not be valid to do so.

For the essay item of choosing two characters to compare similarities and differences, higher-level thinking skills are used to analyze and compare and contrast the characters. They essay will reflect student’s understanding of character traits and how characters are alike and different, thus reinforcing comprehension skills. The choice of text does not involve a wide array of characters, so students will not have to divulge too much time in choosing which characters to use. Rather than pick the characters for the students, I prefer to allow students to make the choice. The scoring will not be based on which character they choose; rather scoring will be based on the ability to describe the similarities and differences among the two characters they chose. Scoring would be rubric-based with a rating scale based on at the essay consisting of at least three characteristics described for similarities, at least three characteristics described for differences, the descriptions accurately portraying the character of their choosing, and the analysis of the characters.

The testing measurements I chose are intended to measure the learning outcomes. I chose these methods because I feel that they have more advantages in assessing what I want to measure. Although I considered using matching in identifying book parts, I felt that at this stage there would be too many options and I did not want to overwhelm students with too many choices. Multiple choice assessments provide a smaller sample of choices that allow for measurement of knowledge. On a similar note, I chose to avoid true/false questions because I want to ensure that students are identifying the correct book parts. True/false questions are more susceptible to guessing and I did not feel would be advantageous in deciphering if students have mastered identification of the content.

As for the essay response, because I intend to measure the higher level thinking skill of analyzing the characters, this method seemed to be the best measurement tool. Kubiszyn and Borich (2010) point out that although essays are time consuming and can be difficult to score, the essay is highly beneficial for measuring high-level cognitive skills. In using the essay, the answers are not there for students to quickly identify or pick from a set; students must demonstrate their own knowledge of the topic. For this reason, I felt it would be difficult to utilize most other forms of written assessment to measure analysis of characters.

References:

Kubiszyn, T., Borich, G. (2010). Educational testing & measurement (9^th ed). Hoboken, NJ: John Wiley & Sons, Inc. 

Creating Learning Outcomes and Assessments

One of the first steps in classroom assessment begins with creating learning outcomes. Some teachers may struggle in creating assessments because they are unsure of what it is they truly want to measure. In creating learning outcomes, deciphering what to assess becomes easier because there are guidelines in place of what it is you want to measure. From the learning outcomes, teachers can create learning activities that correspond with the desired outcomes and use the outcomes as a basis in assessment. This process of performing these steps is referred to as the three-stage model of classroom measurement, in which learning outcomes are constructed, instructional activities are designed in alignment with the desired outcomes, and assessments are developed in alignment with the measurable learning outcomes (Kubiszyn & Borich, 2010). This method helps ensure that the assessments are valid measurements of learning.

Most states have learning standards that are categorized by grade level and content area. Teachers can utilize these standards in creating learning outcomes. By utilizing the state’s learning standards in conjunction with learning outcomes, teachers can not only ensure the validity of assessments, they can also ensure that learning content is in alignment with state standards.

In creating learning outcomes for a second grade ELA class, learning outcomes can be matched with core performance indicators. The New York State Department of Education English Language Arts Core Curriculum lists the following as NYS learning standards:

1. Students will read, write, listen, and speak for information and understanding.
2. Students will read, write, listen, and speak for literary response and expression.
3. Students will read, write, listen, and speak for critical analysis and evaluation.
4. Students will read, write, listen, and speak for social interaction.

In keeping these standards in mind to create learning outcomes for grade 2 ELA, I have developed the following learning outcomes:

1. Students will identify and explain the parts of a book to include author, illustrator, title page, table of contents, index, and chapter headings.
2. Students will identify and explain main ideas of a reading passage.
3. Students will describe characters and compare and contrast among different characters.

The learning outcomes I have listed are in alignment with New York State ELA standards, and are developmentally appropriate at the second grade level. These outcomes are measurable outcomes that can be appropriately assessed, and learning activities can be devised from these outcomes to boost literacy skills.

The purpose of these learning outcomes is to boost reading comprehension. Reading comprehension is necessary for reading success because without an understanding of literature, the words become merely words with no purpose. Having an awareness of the parts of the book can help children learn how to acquire information.  Examining the main idea of a work helps build an understanding of the purpose of the literature. Describing and comparing characters helps students understand the character traits and how the character contributes to the story. The learning outcomes help develop comprehension skills as students gain an understanding of the purposes of literary works. 

Devising assessments is much more difficult than it initially sounds. There is much to consider - is the assessment   aligned with the learning outcomes? Is the assessment a true measure of achievement? What forms of assessment are children developmentally ready for? What type of assessment is best for measuring certain skills? Creating assessments is more than making a series of questions for students to answer, and the development should not be hastily created.

For the learning outcomes that I previously mentioned, I have decided to use a variety of assessments to measure achievement. A variety of assessment forms is useful because it allows students to shine in certain areas. Some children perform better on written tests, while others perform better on performance assessments. In order to get a better picture of student achievement, you must consider strengths and weaknesses in regards to assessment forms.

For my first learning outcome, students will identify and explain the parts of a book to include author, illustrator, title page, table of contents, index, and chapter headings, I have devised the following assessment. The assessment uses pictures of a book and children will answer the questions in either multiple choice or short answer format. This is my first attempt at creating an assessment, and I'm sure it isn't perfect. Prior to testing, students will be assessed in small group settings to actively demonstrate their knowledge of the parts of the book. A checklist will be used to monitor student achievement.


During implementation of this assessment, the visuals on the assessment will be displayed either with an overhead projector or Smart Board to ensure that the pictures are visible for all students. In order to test their knowledge rather than reading ability, the test will be read to students so validity is not contaminated.





Name: __________________________________    Date _________________________

Using the pictures below, answer the following questions. The pictures are also available on the board.

Picture 1

Picture 2

Picture 3

Using the pictures above, fill in the blanks.

1. What is the title of this book? _____________________________________________
2. Who is the author of this book? ___________________________________________
3. Who is the illustrator of this book?_________________________________________
4. Looking at the table of contents, what is the chapter heading for Chapter 4? _______________________________________________________________
5. Looking at the index, on which page would you find information about the heart?_______________________________________________________________

Using the pictures above, circle the correct answer:

     6. Which picture shows the title page?
          a. Picture 1                b. Picture 2                  c. Picture 3
     7. On which page would you find Chapter 6: The Laws about Drugs?
          a. Page 8                   b. Page 23                    c. Page 36
     8. Where would I look to find information about lungs?
          a. Page 43                 b. Page 39                    c. Page 33
     9. What is the Table of Contents used for?
          a. To tell you where to find a chapter
          b. To tell who wrote the book
          c. To tell you where the book was made
    10. What is the purpose of the Index?
          a. To show who wrote the book       
          b. To tell you where to find information in the book
          c. To show you the chapter titles

* Pictures are from the book titled Drugs and Our World by Gretchen Super (1990) *

For the second learning outcome, students will identify and explain main ideas of a reading passage, I chose a similar method of assessment. For identifying the main ideas, students will read short stories and decipher the main idea. There will also be a continuous assessment performed in small group settings to ensure that students are understanding the concept of main ideas. A checklist system will be implemented to keep track of which students have mastered identifying main ideas. Once again, because I am not at this time measuring reading skills and instead focusing on comprehension of text, the assessment will be read to students. The assessment will depict a series of short passages with multiple choice and short answer questions in response to each passage. A worksheet similar to the form of assessment I would choose can be found at http://printables.scholastic.com/printables/detail/?id=24573.

For the third learning outcome, students will describe characters and compare and contrast among different characters, I have chosen a different form of written assessment. I chose essay response for this assessment because it entails describing, which is difficult to measure how well children can describe differences and similarities using multiple choice or similar short answer responses. After learning activities about characterization and deciphering character traits, students will learn how to compare the character traits. Although at the second grade level, students are beginning to develop writing skills and are not typically capable of writing elaborate essays, the response of the essay can take the developmental level of students into consideration. I feel that a writing task such as an essay is important for children to develop writing skills, but there cannot be much emphasis on spelling and grammatical errors in such an assessment. This essay is intended to measure the ability to compare characters, not to measure spelling or grammar skills. Because it is the main point of the essay that is of significance and not the structure of the essay, points will not be lost due to such errors. 

A sample of how I would implement an essay format assessment is below:

Name: _______________________________________  Date: ___________________

We have recently read the story Cinderella retold by Fabio Coen. From this story, choose two characters and write a short paragraph to describe how they are similar and another short paragraph to describe how they are different from one another. Please describe at least three similarities and three differences in your writing.

__________________________________________________________________________

__________________________________________________________________________

__________________________________________________________________________

__________________________________________________________________________

__________________________________________________________________________

__________________________________________________________________________

__________________________________________________________________________

__________________________________________________________________________

__________________________________________________________________________

__________________________________________________________________________

__________________________________________________________________________

__________________________________________________________________________

References:
Kubiszyn, T., Borich, G. (2010). Educational testing & measurement (9th ed). Hoboken, NJ: John Wiley & Sons, Inc.
New York State Department of Education (May 2005). English language arts core curriculum (prekindergarten-grade 12). NYSED. Retrieved January 31, 2012 from http://www.p12.nysed.gov/ciai/ela/elacore.htm