Depth of Knowledge
What Does It Mean For Teachers & Students?


By
Pam Lowe


Table of Contents:
Overview
Background
    • The DOK
    • Other States
Impact on Educators & Learners
    • MAP & the Fourth Cycle of MSIP
DOK
    • DOK Levels for Four Content Areas
    • Resources
    • WAT: Web Alignment Tool
Alignment
    • Classroom Content Alignment
    • Alignment History
Themes:
Learning Theory
Focus Content Area:
Professional Development

Secondary Content Area:
Education


Overview:

What is the DOK and Why Do We Need It?

The Depth-of-knowledge (DOK) was created by Norman Webb from the Wisconsin Center for Education Research. 


The Depth of Knowledge is the degree of depth or complexity of knowledge standards and assessments require; this criterion is met if the assessment is as demanding cognitively as the expectations standards are set for students.


Completely aligned standards and assessments requires an assessment system designed to measure in some way the full range of cognitive complexity within each specified content standard. Norman Webb identified four levels for assessing the DOK of content standards and assessment items.


The DOK levels are Recall (Level 1), Skill or Concept (Level 2), Strategic Thinking (Level 3) and Extended Thinking (Level 4). Of course to accurately evaluate the DOK level, each level needs to be defined and examples given of types of student behaviors.


DOK implies the interaction of how deeply a student needs to understand the content with different ways of responding and interacting with the content.

DOK Facts



Background:

The DOK

Norman L. Webb, senior research scientist with the Wisconsin Center for Education Research, is a mathematics educator and evaluator who is co-team leader of the Instituteís Systemic Reform Team, rethinking how we evaluate mathematics and science education, while focusing on the National Science Foundationís Systemic Initiatives reform movement. His own research has focused on assessment of studentsí knowledge of mathematics and science. Webb also directs evaluations of curriculum and professional development projects.

Webb, Norman L. ALIGNMENT OF SCIENCE AND MATHEMATICS STANDARDS AND ASSESSMENTS IN FOUR STATES. National Institute for Science Education, University of Wisconsin-Madison.  August, 1999




Other States

The Webb model has been used in alignment studies with more than 10 states. The model has been used for language arts, mathematics, science, and social studies.  Here are a few of the resources from a couple of these states.

Kentucky: Released Items

Kentucky: Mathematics

Colorado: Reading DOK Descriptors

Oklahoma: DOK Definitions

Pennsylvania: DOK

Vermont: Science



Impact on Educators & Learners:

MAP & the Fourth Cycle of MSIP

The resources listed here demonstrate how the DOK alignment has begun and how teachers and students will be affected and held accountable for DOK levels in the future. 

Now:

In the 2006 MAP results the DOK levels were assigned to standards and questions.  As teachers document their strengths and weaknesses, they can now use DOK levels to plan goals for their school and classroom. The MAP data, the IBD report, in particular, shows how the DOK levels were assigned to the GLEs and Standards assessed on the MAP. 

The MAP Analysis Chart is an important tool as we track the DOK levels, Question Types, and Process Standards that contribute to a classroom, school or districtís lowest scores. 

See examples of this information and itís importance in the following PowerPoint by the Jefferson City School District:

Analyzing MAP Data: Jefferson City School District

Goal Process IBD Report

The Very Near Future:

Soon many districts will be completing the fourth cycle of the MSIP process.  This Observation form reveals what team members will be anticipating during their classroom visits. * Note the DOK levels that will be documented during classroom observations.

The Fourth Cycle MSIP Observation Form

The Fourth Cycle MSIP Writing Report Form also demonstrates the importance of the DOK levels as the team members write their findings in the report as documented on page 8 of the form. 

The Fourth Cycle MSIP Writing Report Form

 



DOK:

DOK Levels for Four Content Areas

Four Levels for Content Areas

 




Resources

 

RF PPT 

DOK Chart

DOK Poster

St. Louis Area Curriculum Coordinators PPT




WAT: Web Alignment Tool

Web Alignment Tool

Alignment:

Classroom Content Alignment

As the alignment process takes place, districts, schools and

teachers will want to think about the degree to which classroom

instruction and assessments are aligned with the demands of

content standards.  In order for learners to reach the cognitive

demands of the content, think strategically and extensively,

solve complex problems, and be able to reason, analyze and

communicate their understandings, they will need

well-constructed standards-based, lessons and assessments. 

 

Classroom instruction and assessments will need to require

students thinking and working at all levels of the DOK. 

As classroom teachers, one important question will be,

"How can we incorporate the DOK into our classroom

assessments?"

 

Math GLEs: Draft Copy

Communication Arts GLEs: Draft Copy

Science GLEs: Draft Copy

 

 

 




Alignment History

The No Child Left Behind Act (NCLB) now requires all states to use an alignment process for validation purposes to show that they are aligning their assessments with the depth of each statesí academic content standards at all grade levels.  The U.S. Department of Education issued guidelines that include six dimensions important for making judgments about the alignment between state standards and assessments.  These dimensions include comprehensiveness, content and performance match, emphasis, depth, consistency with achievement standards and clarity for users. As Missouri aligns standards and assessments, these dimensions are considered in the alignment framework.  Norman Webbís model is one of four models that is being used by states to meet this requirement and has been used in test item development and alignment studies across various content areas.  Webb and his team came to Missouri last fall and conducted an alignment study.  A critical part of the alignment process is that Depth of Knowledge levels be assigned to the standards and assessments items.