Summer Institute
By Charlie | July 29, 2008
There are a large number of teachers and administrators registered for the summer institute classes of “Principles and Practices of Quality Assessment”, “Developing Quality Common Assessments” and “Developing Individual Classroom Assessments”. The BISD Assessment Team is grateful for the opportunity to share ideas on developing and using quality classroom assessments.
If you attended any of these sessions, please consider posting your comments, things learned, or ideas for implementation in your classroom in response to this posting so others can benefit.
We will continue to add new materials to this Assessment Blog once school starts.
Thanks for your interest in improving the assessment practice in your classroom.
Charlie
Topics: Uncategorized | No Comments »
Minutes with Charlie - 18
By Charlie | May 12, 2008
Once understanding the concept or definition of common assessments - collaboratively developed formal measures of essential student learning, standardized across the discipline that are administered in a systematic, timely fashion whose results are disaggregated, analyzed, compared and contrasted for the specific purpose of improving student achievement- the next step is beginning the development process.
In developing any quality assessment, the first step is to clearly determine the purpose of the assessment. Common assessments can either be classed as assessments of learning or assessments for learning. The determination of whether an assessment is of or for learning is not based upon the style of assessment but a function of both the timing of the assessment and how the results are used. An assessment given at the end of a unit of study to gain results to inform someone beyond the classroom of student learning is clearly an assessment of learning. An assessment given during the course of instruction to inform both teacher and student of progress and guide toward the next stages of learning is an assessment for learning. An assessment during the course of instruction to inform only the teacher and others outside of the classroom regarding progress in achievement at that time would be an assessment of learning.
In looking at the purpose(s) of common assessment several questions must be answered. First, what is the timing of the administration of the assessment? Is it given at the end of the unit of instruction to measure student achievement or during the process of instruction where both student and teacher can still alter behavior to improve learning? Second, who is using the information and what decisions are to be made based on the results? The answer to these two relatively simple questions will give the answer to “why?” of common assessments. With a clear grasp of the purpose of the common assessment we can proceed to other important considerations.
Our next consideration is “What are we going to assess in these common assessments?” Based upon our operational definition, we are looking to assess essential student learning. Our district is on a path to identify district power standards, but we have not yet arrived at that point. Until then, grade level/campus level determinations need to be made as to what to assess in common assessments. General guidelines for that determination could be the enduring understandings identified in our district curriculum documents. Similarly the principles of power standards will be that they are: endurance (Are students expected to retain mastery of the skill or knowledge beyond the assessment?), leverage (is the knowledge or skill applicable to other academic areas?), and readiness for the next stages of learning (Does the knowledge or skill prepare the student for success at the next stage of learning?)
These learning targets must be determined prior developing the assessment and prior to instruction. They therefore define what we believe essential to be included in instruction and essential goals of student learning.
It is essential, that by whatever method, we produce clearly defined, articulated, and understood learning targets that are commonly accepted prior to proceeding to the next steps in developing common assessments.
Personal Reflection: Does my team of teachers determine a clear purpose of a common assessment at the outset of planning? Do we have clearly defined targets for each common assessment? Do they represent agreed upon essential student learning?
Topics: Assessment OF Learning, Minutes with Charlie | 1 Comment »
Minutes with Charlie - 17
By Charlie | May 5, 2008
Common assessments have been part of the educational landscape for many years. Many campus, grade level or subject area teams have engaged in developing and using common assessments. Both research and most instructional leaders recognize that common assessments, properly developed and used promote increased student achievement. However, the common understanding regarding common assessments is anything but “common”.
In the next few Minutes we are going to examine the concept, development, and use of common assessments.
In essence common assessments are collaboratively developed formal measures of essential student learning, standardized across the discipline that are administered in a systematic, timely fashion whose results are disaggregated, analyzed, compared and contrasted for the specific purpose of improving student achievement.
Let’s look at the important components of that definition. Collaboratively developed means it is not a test made by one teacher and used by many. It is developed specifically by teachers of that discipline on that grade level. Essential student learning means that not every possible student expectation is assessed on a common assessment. Common assessments address the more important (essential) standards or power standards. Standardized across the discipline refers to standardized across the district. For now, we are going to be really happy to have quality common assessments across a campus. Administered in a systematic timely fashion refers to the administration as guided by the district scope and sequence of learning. Results that are disaggregated, analyzed, compared and contrasted means the results of the common assessments are examined somewhat differently than classroom assessments in one classroom. They provide information useful in various arenas by several stakeholders. Finally, for the specific purpose of improving student achievement refers to the primary purpose of developing and using common assessments - improving student achievement.
There are a couple of prerequisites that need to be in place prior to developing and using common assessments.
First, there must be a commitment on the part of the teachers to collaboratively produce and use an assessment that focuses upon what students need to learn and not upon what teachers may or may not have taught. These assessments, as are most assessments, are developed prior to instruction. This commitment to collaboration extends beyond the development and administration of the assessment. It must also be in place during the examination of the results and the implementation on needed changes as indicated by results. This entire process brings a new dimension to teacher collaboration.
Second, it requires the identification of the essential student learning. This, if not done on the district level, must be done on the campus or discipline level. Doing so is no small process but its completion brings a great clarification for assessment and instruction.
In the next Minutes we will begin an examination of the development of common assessments. As with the development of any assessment, we will look at the purpose of common assessments, clear targets, matching they assessment type to the targets (design), the communication tools of the assessments, and student involvement. If you want a review see Minutes 2, 3, and 4.
Personal Reflection: Do I as a teacher recognize the value and importance of common assessments? Am I committed to work collaboratively with fellow educators to accomplish this?
Topics: Assessment OF Learning, Minutes with Charlie | No Comments »
Minutes with Charlie - 16
By Charlie | May 3, 2008
An important part of harvesting the benefits of a quality assessment program lies in the proper and effective use of data garnered from the assessments. In the last few Minutes we have examined the traits and development of summative assessments of assessment of learning. We will now examine the proper and effective use of data generated from the assessments of learning.
A guiding principle in our discussion will be the difference between the purposes of the data from assessment for and assessment of learning. Data from assessment for learning provide information teachers and students should use, in the course of instruction and learning, to make decision to improve instruction and learning at that point in time. Data from assessment of learning provide information for teachers and administrators, after the instruction is completed, to make decision regarding the instruction and curriculum for the next time the unit of study is taught. In simple terms, data from assessment for learning are used now, and data from assessment of learning will be used later.
Upon the completion of a unit of study, the results of a summative assessment will reveal the level of mastery that students attained over the course of study. Assuming we have a quality summative assessment, this potentially allows us to examine the quality and effectiveness of three areas in the learning progress. Each area requires a considerable amount of openness and honesty as we evaluate. We will look at each one briefly.
It seems that the results of a summative assessment can fall into three broad categories: everyone passed marvelously, everyone failed miserably, or the scores were distributed somewhere between the two extremes.
The results of a summative assessment potentially allow us to examine the quality and effectiveness of our curriculum. As we examine the results, we should ask ourselves, what information do these results give me about the content or pacing of our curriculum? We should examine our curriculum in light of our results.
Also summative assessment results allow us to examine our instruction. As we examine our results we should ask ourselves questions about how effectively our instructional methods met the learning needs of all students. Did I vary learning opportunity to meet varied learning styles? Did I effectively use the results of assessments for learning during the unit to change my instructional decisions or did I just plow along as planned? Examining instruction in light of assessment results is at once a very difficult but beneficial activity.
Finally, summative assessment results allow us to examine our formative assessments or assessment for learning. If our assessment for learning results indicated that students were progressing as they should, and our summative assessment indicated that a considerable group did not perform as expected, there most probably was a problem with the formative assessments. As we look at our formative assessments in light of our summative results, we should examine both the quality of the assessments as well as they way in which results were used to improve instruction and learning.
We must begin to look at the data from summative assessments as more than simply “a grade at the end of the unit”. They must serve as information to improve our curriculum, instruction, and assessment programs.
Personal Reflection: Am I in the habit of using summative results to examine the quality of my curriculum, assessment, and instruction? Do I do this as a member of a team of teachers?
Topics: Data, Minutes with Charlie | No Comments »
Minutes with Charlie - 15
By Charlie | April 27, 2008
The distinction between assessments of learning and assessment for learning is vital if we are to develop and use a balanced assessment program. In simplistic terms, assessments of learning are assessments of the amount of learning accomplished at the end of a unit of study. Assessments for learning are assessments for the purpose of improving learning.
We will continue to examine assessments of learning in this Minutes with Charlie. Last time we examined the traits of quality assessments of learning. This time we are looking at the process of developing quality assessments of learning.
There is a common temptation to develop an assessment of learning in a given unit of study by making it a compilation of the assessments for learning used in that unit. One variation of this is to take the “best of the formative assessments” and combine them to make a “best summative assessment of learning”. In response to this plan, let me just quote the old hymn, “yield not to temptation”.
There are several reasons this plan is counterproductive. One, the assessments for learning used during the unit have a purpose distinct from the purpose of the assessment of learning. The formative assessments are to inform student and teacher about learning to that stage and to help guide decisions about instruction and learning. The summative assessments are to measure the learning accomplished over that time period. Two, the repetition of these assessments becomes simply an assessment of recall ability not of the knowledge and skills demanded by the TEKS. Finally, the nature and content of the assessments for learning used during a unit may change based upon the results, but an assessment of learning should have been developed prior to beginning any instruction.
There are several stages in the development of quality assessments of learning. Let’s examine a few of these:
1. Identify the priority leaning targets and the degree of rigor of each one. Among all the learning targets covered in the unit, determine the desired sampling of the priority targets to assess.
2. Develop a “blueprint” of the assessment. This will indicate the focused learning targets, the number of assessment items addressing each targets (sample), the weight of each items (sort of a relative importance), and the method of assessment to be used for each (selected response, extended response, performance/ project or personal communication)
3. Once you have a blue print that you believe accurately portrays a quality assessment for the unit, set about developing the actual array of questions or items based on the blue print.
4. Upon completing the item development, have the assessment reviewed by peers to check for items that either do not truly assessment the identified target or contain items that could bias the student response.
As the unit of study progresses, periodically review your assessment of learning to insure that both instruction and your assessments for learning are providing opportunities for the students to develop the knowledge and skills to be assessed on the assessment of learning.
Personal Reflection: Do I, as a common practice, invest time to develop a quality assessment of learning prior to planning and starting instruction?
Topics: Assessment OF Learning, Minutes with Charlie | No Comments »
Minutes with Charlie - 14
By Charlie | April 27, 2008
Most of the Minutes with Charlie deal with either topics relevant to both assessment of and assessment for learning or topics that relate specifically to assessment for learning. This is because assessment for learning should be the most frequent type of assessment we give and it produces the greatest benefit in regard to student learning. However, we strive to have a balanced assessment program and assessment of learning certainly plays an important role in that system.
As we draw close to the end of the school year many teachers are starting their last unit of study. Many are already into the last unit and some will start immediately after TAKS week.
Over the summer break many grade levels and subject areas meet to plan and prepare for the next year.
So in light of all these facts, we will be looking at assessment of learning or summative assessment in the next couple of Minutes. We will examine the elements or traits of a quality assessment of learning, developing quality assessments of learning, and how to use the data generated from assessments of learning.
One important trait of assessments of learning is the purpose of the assessments. These assessments are to evaluate student learning at the end of a unit of study on a number of learning standards and provide specific data in that regard.
Another trait of quality assessment of learning is that the assessment was planned and developed prior to planning land delivering the instruction. The nature and content of the final assessment in a unit should not come as a surprise to anyone. It assesses student performance on a given set of standards, not necessarily just upon what is taught.
Another trait of assessment of learning is that it is scored and graded. If the assessment is a quality assessment that accurately assesses student performance against established standards, and if the grading system if proper, assessments of learning provide accurate data on student achievement on an array of standards.
Assessments of learning share many traits with assessments for learning. Both have clearly defined, understood, and articulated learning targets. Both select the assessment method (selected response, extended response, performance) that best matches the learning targets. Both provide data to students and teachers on student performance against learning standards.
As you begin to plan and development assessment of learning or summative assessments a few words of caution are in order:
· We must insure that the degree of rigor in our assessments match the degree of rigor called for in the TEKS.
· While assessments of learning assess an array of learning targets, it is not required that we have a question for each standard. We have to prioritize the targets and provide a good sampling of the targets.
· Traditionally, summative exams have lived shrouded in mystery. However, it is important that through the instruction and assessment process that students know and understand the level of performance expected of them. While the individual questions on an exam are not known, the content of the exam should not be secret.
Personal Reflection :Do I invest time to insure that my assessment of learning are quality? Do I plan and prepare my summataive assessment prior to planning and delivery of instruction? Can I clearly distinguish between assessment of learning and assessment for learning?
Topics: Assessment OF Learning, Minutes with Charlie | No Comments »
Minutes with Charlie 13
By Charlie | April 14, 2008
The fourth type or category of learning target that we will consider is that of the product targets. Products are things created by the student that will give evidence of knowledge and understanding, reasoning, and/or skills. When the standards or TEKS call for the student to create or produce something, it usually calls for a product target.
The number of product targets called for in the TEKS is relatively few compared to the number of knowledge, understanding and reasoning targets. However, it is extremely important to distinguish between the use of a product as a learning target and the use of a product as an assessment tool. The failure to make this distinction results in poor assessment practice and inaccurate assessment results and decisions.
An example might be if we assigned a student to make a poster containing a Venn diagram that illustrated the similarities and differences between chemical and physical changes and giving examples of each. “Create a poster” is not a learning target in TEKS however understanding the difference between chemical and physical change would be an important understanding in TEKS. This is an example of a task that can be used to assess understanding, but it is not a product target.
An example of a product target would be if the TEKS called for the student write a persuasive paper on a current events related science topic. The product would be the writing, the organization of the paper, the use of language, etc. In this case it would be vital that the student have the understanding of the topic and the reasoning skill of persuasion. The target would be to produce a written paper.
We often have students produce a product (a poster, a diorama, a model of an Indian dwelling, a map of a territory, etc) as an assessment instrument. Many times this is, in reality, simply a representation of fact without any real assessment of understanding, reasoning, or skill. Yet we assign a grade based on creativity, neatness, timeliness, etc.
In using products as learning targets we should ask ourselves questions such as:
1. Do the standards (TEKS) call for the student to use the skill required to create this product?
2. Does my assignment give the student opportunity to exhibit his ability?
3. Are there other limitations in this assignment that would prevent it from giving me accurate information on student ability?
In using products as an assessment tools we should ask ourselves questions such as:
1. In order to complete this product, does the student have to possess the skills I am trying to assess?
2. Are there factors in the production that would hide the student’s true level of proficiency in this target?
In the creation of a product, students will use some combination of their knowledge and understanding, reasoning ability, and skills. Some of the TEKS clearly call upon students to be able to and to actually create products. We ignore these demands at the students’ loss.
We must use caution to not confuse products as a learning target and products as an assessment tool if we are to gain the full benefit of products.
Personal Reflection: Do the products my students create provide clear evidence of student learning at the level of complexity demanded of them? Or do the products simply demand a representation of fact without requiring the student to know or understand the subject matter?
Topics: Learning Targets, Minutes with Charlie | 2 Comments »
Minutes with Charlie 12
By Charlie | April 7, 2008
The third type of learning target from an assessment perspective is that of a skill target. Skill targets are in essence performances that must be demonstrated by the student and observed by the teacher and others to be assessed.
In skill targets it is the doing of the skill that holds primary importance. In most cases, there are foundational knowledge and understanding targets that form some prerequisites to quality skill performance. In some cases, the mastery of reasoning targets also form an essential foundation for high skill levels. For example, if the skill target is to give an oral presentation examining the merits and demerits of the Marshall Plan, the student must possess basic knowledge of the Marshall Plan and its role in post-war Europe as well as some reasoning ability to determine the merits and demerits of the Plan. But, the skills target is the oral presentation.
While the most obvious skill targets are found in physical education and the fine arts, skill targets abound in other academic areas. In learning to speak a foreign language a skill target would be the proper pronunciation of words and correct conjugation of verbs. In language arts oral reading fluency and poetry reading are examples of skill targets. In mathematics, many of the measurement TEKS and construction TEKS are skill targets. In the sciences, the TEKS often call upon students to demonstrate certain skills in the lab.
All too often it seems we take the skill targets in the TEKS and assess them as if they were knowledge and understanding. We have students identify or select items rather than demonstrate skills. As we examine the TEKS, it is important that we distinguish the skill targets from knowledge targets and assess them appropriately.
There are several important considerations when we embark upon the assessment of skill targets that will prove useful in avoiding problems.
First, as mentioned earlier, knowledge and understanding targets often underlie the skill targets. If we are attempting to assess skill targets without first insuring that the students possess the necessary knowledge and understanding to use in the skill, we are insuring their failure. If we put them to perform a skill, without an adequate foundation, we often are deceived in their skill ability. For example, a student may have excellent skills in giving oral presentations, but if we have them give an oral presentation on a subject in which they lack adequate knowledge, their lack of knowledge will mask or hide their oral presentation skills.
Second, usually skill targets do not fall into a “they have it or they don’t have it” assessment. More often than not, skills develop along a continuum. There will be a range of proficiency or a scale of competency along which a student may demonstrate skill. As such, it is important that going into the skill assessment, we as teachers have a continuum or scale upon which to assess the skill. This continuum must contain specific criteria to move along the scale. These criteria must be understood by the student prior to the instruction and assessment.
Finally, as with most assessments (knowledge and understanding, reasoning, skills, or products), it is important to have multiple assessments for learning during the learning process, not just one skill performance.
Personal Reflection: Am I attentive to when the TEKS call upon a skill performance for my students? Do I regularly assess students in the skills they are required to develop? What can I do to become more proficient at assessing skill targets?
Topics: Learning Targets, Minutes with Charlie | 1 Comment »
Minutes with Charlie - Eleven
By Charlie | March 31, 2008
The second type of learning target from an assessment perspective is that of a reasoning target. Reasoning targets are in essence related to the mental process of applying the knowledge and understanding in various situations or problems.
We often hear teachers say something along the lines of “these kids just don’t know how to think”. In reality, they know how to think very well, what they lack is reasoning ability. Reasoning ability by and large does not increase just by getting older. Reasoning skills must be taught, practiced, assessed, re-taught, re-practiced, and assessed again and again. However, all too often educators teach and assess the knowledge and understanding and somehow expect students to make the leap to reasoning independently.
In general, in any given discipline, the majority of the TEKS that we address fall into the reasoning category. If this is true, then the importance of understanding these reasoning targets and accurately assessing them cannot be overstated.
Reasoning skills have been defined, analyzed, categorized, and classified in any number of ways. To simplify, we are going to look at some of the more common categories of types of reasoning before we talk about them as learning targets.
A common reasoning skill we see in learning standards (TEKS) is that of inference – both inductive and deductive reasoning. Inductive reasoning requires the student to use specific facts from which he infers general rules or principles. Deductive reasoning takes the general rule and makes application in specific settings.
Another pair of common reasoning skills is that of analysis and synthesis. Analysis consists of examining the parts of a system and considering how they interrelate. Synthesis consists of looking at parts of different things and forming something new.
Another pair of reasoning skills is that of comparing similarities and differences between two items or ideas. We know this is a powerful instructional strategy, but it is also an essential reasoning skill that surfaces often in the TEKS.
Finally, we have the reasoning skill of sorting and classifying. This seems to become more important in some disciplines than in others, but the skill is essential to student success in learning.
In the TEKS we see verbs that represent the reasoning skills such as compare, evaluate, analyze, classify, and infer.
We must recognize that in order to assess reasoning skills in any of the reasoning types, we must allow the student to demonstrate his reasoning ability in a new and unique problem area. For example, if we teach the similarities and differences between the natural resources of the Northern and Southern states in the Civil War, and then assess the students on the similarities and differences between the natural resources of the Northern and Southern states in the Civil War, then we are assessing knowledge and recall. If however, we teach students the skills of evaluating the similarities and differences and then assess that skill in a situation we have not compared, then we are assessing reasoning.
If our learning target consists of a reasoning skill, we must take caution to not assess it as a knowledge and understanding target. Assessing reasoning skills requires a different approach to assessment than does knowledge and understanding.
Personal Reflection: When considering my learning targets, do I distinguish between knowledge and understanding targets and reasoning targets? Can I recognize the different reasoning skills required in the TEKS?
Topics: Learning Targets, Minutes with Charlie | No Comments »
Minutes with Charlie - Ten
By Charlie | March 13, 2008
Perhaps the most recurring theme in discussions on classroom assessment for learning is that of clearly defined, articulated, and understood learning targets. These learning targets come to the forefront in discussions on the district initiatives of Performance Excellence, Understanding by Design, and Classroom Assessment.
From an assessment perspective if we do not begin with a clear target, we will certainly not end with sound assessment or accurate data on learning.
Identifying the type of learning target we have helps us know that we have clear and usable learning targets. Determining the type of learning target can generally be determined by examining precisely what the learning targets asks of the student. Learning targets will demand that a student know and/or be able to do something. This can usually be accomplished by examining the verb(s) in the learning target.
It is essential to understand that while the TEKS compose our learning standards, they do not necessarily represent our learning targets at a given time in the progression of learning. We must determine the requisite knowledge and skills that students need in order to reach those standards. It is these requisite knowledge and skills that become our learning targets. In effect, our learning targets form a ladder or scaffold that allows students to climb up to reach the TEKS.
As we breakdown or deconstruct the TEKS into these requisite skills, an examination of the verbs in the TEKS gives us an indication of the type of learning target which is our focus in this stage of learning.
For assessment purposes, clear learning targets fall into four types. The type of learning target in part helps determine the method of assessment to use. In general, learning targets in the assessment world fall into the categories of: knowledge/understanding targets, reasoning targets, skill targets, and product targets.
Over the next few Minutes with Charlie we are going to examine each of these to understand and to know how to recognize them.
Knowledge and understanding learning targets relate to the factual foundations of each discipline. Without a clear grasp of these foundations, the ability of students to reason, practice skills, and complete products related to that discipline is greatly limited.
Not all knowledge/understanding targets have to be memorized or known outright. Some knowledge targets deal with knowing how to find information.
Verbs that commonly fall into knowledge and understanding learning targets are verbs such as: identify, know, explain, understand, list, define, recognize, and describe. In order to do these things, a student must possess a basic knowledge and understanding of a given topic.
There is a tendency in educators, in an effort to set high expectation to consider this knowledge and understanding level of target as less important than the “higher order thinking” targets. However, it is important to realize that in order for a student to prove proficiency at the higher reasoning skills, they must have a basic knowledge and understanding of concepts with which they reason. These knowledge and understanding learning targets form an essential part of a balanced and complete curriculum and assessment system. By the same token, they are not the sum total of said system.
Personal Reflection: Do the learning targets I set for my student reflect the requisite skills they need to reach the level required in TEKS? Can I categorize my learning targets into the types mentioned above?
Topics: Learning Targets, Minutes with Charlie | No Comments »