Date of Graduation

5-2018

Document Type

Dissertation

Degree Name

Doctor of Education (Ed.D.)

College/School

School of Education

Department

Learning and Instruction

Program

Learning & Instruction EdD

First Advisor

Nicola McClung

Second Advisor

Matthew Mitchell

Third Advisor

Helen Maniates

Abstract

Constructing a cogent argument that addresses real-world problems aids students in the development of critical thinking and requires students to present multiple perspectives in a credible manner. Yet, rubrics do not always measure students’ reasoning. The purpose of this study was to create a valid and reliable instrument to measure cogency in argument. I created a Teacher Designed Rubric Measuring Cogency (TDRMC) based on Toulmin’s model of argument for its emphasis on context-specific warrants, and I used Wilson’s framework for assessment to operationalize the construct of cogency. I compared the TDRMC to the current standardized assessment rubric for the Common Core, the Smarter Balanced Assessment Consortium (SBAC). A mixed methods design with a convenience sample of 73 essays from seventh-grade students from a public middle school in northern California addressed three primary questions about the TDRMC: What is the reliability of the TDRMC, and how does it compare to the reliability of the SBAC? Are scores generated from the TDRMC more variable than scores generated from the SBAC? How does the TDRMC correlate to other established measures of writing and academic ability, such as grades, various SBAC scores, and grade point average (GPA), and how do these correlations compare with those of SBAC Writing and established measures of writing and academic ability? Students’ essays were scored by a committee of four middle school English teachers. Both percent agreement and Cronbach’s alpha showed that the TDRMC was more reliable than the SBAC. The TDRMC did not statistically capture more variability than the SBAC; however, visual inspection of the distribution suggests the TDRMC produced more variability in scores. The TDRMC significantly correlated with a range of external measures of academic ability. These correlations were comparable to those between SBAC and the established measures of writing and academic ability. Several nonsignificant trends suggest that the TDRMC was more highly correlated with sixth- and seventh-grade grade point average and SBAC English language arts than the SBAC. Overall results hint that the TDRMC is better at assessing the construct of cogency and is more highly correlated with academic performance measures than the SBAC.

Download

Share

COinS