Report from Joint K-16 Composition Standards Committee onWriting Proficiency Standards for Admission and Graduation from MUS

Montana K-16 Joint Committee on Composition Standards
Final Report - April 28, 2000

The Montana K-16 Joint Committee on Composition Standards was appointed by Nancy Keenan, Montana Superintendent of Public Instruction, and Richard Crofts, Commissioner of Higher Education, in Fall 1998. The Committee was charged to address standards for student learning and proficiency in writing. The Committee met from December 1998 through October 1999 and in sub-committees several times since. The Committee and sub-committees have responded to the items of the charge and report their recommendations below.

1. To review the draft K-12 writing standards for what students should know and be able to do in writing, suggest revisions that would enhance students' transition from high school to college, and make recommendations for the standards' modification prior to public hearings.

The Committee reviewed the standards and in February 1999 formulated its recommendations. These were sent to Dr. Linda Peterson at OPI in March 1999, and Dr. Peterson reported back on how the Committee's input was used in October.

2. To assist OPI and OCHE staff to complete collecting information about current practices in placement for college composition, including use of Advanced Placement examinations.

The Committee assembled this information. Dr. Jan Clinard of OPI prepared a brochure that was printed and disseminated to high school students, parents and guidance counselors as well as to campus officials across the state in Fall 1999 [Appendix A].

The Committee also advised the Montana University System (MUS) about a Web site to post information about credit by examination in the MUS.

The Writing Standards Subcommittee began a project to delineate the writing content and student proficiencies incorporated in three levels of college compositionremedial/basic/developmental, regular and advanced. This data will be used with other information in formulating responses to items 5 and 6 below.

Recommendation: The Committee recommends that OPI and OCHE continue cooperation in this area and regularly update and republish the informational brochure for high schools, their students and the students and faculty of the colleges and universities.

3. To advise OPI and OCHE about strategies for disseminating to high schools and their students

information about tests used to place college freshmen in composition courses and about policies regarding Advanced Placement at each of the units of the University System.

The brochure cited in 2 above fulfills this purpose, as do the materials organized for posting as Credit by Exam Opportunities on the MUS Web site.

Recommendation: The Committee encourages ongoing collaboration between the K-12 and post-secondary sectors and bridge-building to continue the conversation and provide for regular revision and distribution of the information via brochures and Web sites.

Recommendation: The Committee recommends establishing a small, joint standing committee of K-12 teachers of English and university faculty in English composition to:

         Oversee the implementation and integration of K-12 standards and assessment into the K-U system of teaching writing in public education.

         Facilitate and monitor the necessary communication and serve as a resource to all participants in the process.

         Guide statewide training of teachers and faculty in use of ACT Writing Samples.

         Oversee integration of ACT Writing Sample into teaching of writing in Montana.

4.      To examine criteria used for judging writing at high school and college levels and to consider the feasibility of holding a norming session between the two sectors.

The Committee examined the criteria and conducted experimental norming/scoring sessions at Billings and Great Falls. Based on this work, the Committee recommended a Pilot Project for Proficiency-Based Admissions [Appendix B] which the sponsors did not support at this time. The Committee and others in the profession continue to discuss collaboration in writing assessment and may proceed separately to conduct experiments together. This activity is valuable and could eventually lead to a statewide assessment of writing and, ultimately, to a school-based system of writing proficiency standards for admissions to the MUS.

Recommendation: The Committee recommends that local school districts partner with

MUS campuses to use the ACT Writing Sample as an assessment mechanism for high school students at the end of the junior year. These partnerships would

         immerse college and high school teachers of writing in developing and applying criteria for proficiency for the ACT Writing Sample;

         facilitate communication between the two sectors about the qualities of proficient composition and how they are measured;

         provide an ongoing assessment of the ACT Writing Sample by practitioners; and

         provide local school districts with meaningful assessment data on students' composition skills.

         provide composition faculty with relevant professional development.

Such partnerships would be especially valuable for field-testing of the ACT Writing Sample.

5.      To discuss the utility of current high school transcripts for correctly placing students in composition courses at the college level.

The Committee discussed this item and concluded that the high school transcripts are not now useful for placing students in college-level composition because they do not measure consistently proficiency or skills. Transcripts may be made more useful over time with the advent of the K-12/post-secondary articulation via Tech Prep.

In addition, the Committee thought that it would gain more useful information about high school writing through its proposed assessment project. This analysis would be tied to the writing standards and link K-12 standards with useful information on college composition courses, thereby helping students make better decisions on writing.

Sponsors rejected this reasoning, and Dori Neilson from the Office of Public Instruction chose a sub-committee to complete the course coding project that she had committed to do with funding from the National Center on Education Statistics (NCES). The Committee will advise sponsors when this work is completed.

6.      To develop proficiency-based admissions standards and graduation requirements for the Montana University System.

Without Writing Assessment Pilot Project data or another similar exercise, the Committee cannot set writing proficiency admissions standards derived from implementing the state's writing standards in the secondary schools.

Recommendation: The Committee agreed on principles to guide the state in the selection of commercial, standardized tests for assessing writing proficiency:

         A proficiency-based admissions standard in writing should allow multiple measures to accommodate a diverse student body which includes non-traditional learners, non-resident students and transfer students in addition to full-time, first-time freshmen.

         Any instrument used for this purpose should include an authentic writing assessment of sufficient length and substance to give a reliable sample of a student's ability.

         Such instruments should clearly relate to the Montana Writing Standards.

Admissions

After extensive review of commercial products available, the K-16 Composition Proficiency Admissions/Graduation Subcommittee has concluded that three instruments are acceptable for use in college and university admissions decisions.

Recommendation: The Committee recommends that the Montana University System now adopt three instruments for utilization in determining a student's proficiency in writing for the purpose of admission to the four-year campuses.

  1. ACT Writing Sample - The Committee endorses the adoption of this test for the purpose of proficiency assessment of writing if
    • multiple prompts can be made available for examinees;
    • the validity and reliability of the ACT Writing Sample assessment, as developed and conducted for proficiency-based admissions purposes, can be established through field-tests in Montana;
    • ACT can train graders in the state of Montana; and
    • the ACT Writing Sample assessment can be made readily available to students who submit the general SAT for MUS admissions decisions.

As to score, the Committee believes that that one essential by-product of field-testing the ACT Writing Sample in Montana will be the identification of a cut score for proficiency.

  1. Two additional instruments, traditionally used to award credit by examination, are

recommended for this purpose: the Advanced Placement (AP) examinations in English language and the College Level Examination Program (CLEP) in composition.

It is the Committee's view that a student who writes well enough on these tests to earn six college credits for first and second semester college composition is proficient in writing and admissible to the Montana University System. To meet the writing proficiency standard for admission, the Sub-committee recommends:

         A score of 3 on the Advanced Placement (AP) examination in English Language & CompositionorEnglish Language & Literature.

         A score of 44 on the College Level Examination Program (CLEP) examination in Freshman College Composition that includes the optional essay section.

The scores recommended above are derived from the American Council on Education's Guide to Educational Credit by Examination, 4th Edition. ACE recommended scores for awarding credit are the mean test score of students who participated in a national norming study and earned a grade of C in the respective course.

As the writing standards are implemented and writing proficiency and assessment of same gain currency in Montana, there may be other assessment tools that the campuses will wish to adopt for admissions purposes. The campuses should be encouraged to consider additional options for their students and to draw upon the expertise resident on the MUS campuses and in the secondary schools for advice about the appropriateness of these measures and the scores that could represent "proficiency."

Recommendation: The Committee recommends that the Montana University System create and maintain online a remedial composition course for students who cannot meet the proficiency admissions standards and are not admitted to the MUS.

Students could enroll in this non-credit course to improve their writing and meet proficiency requirements. The course would be self-sustaining financially and probably more expensive than regular tuition charges for a three-credit course. Such a course would offer a real service to students and institutions and would be especially useful to students who do not have easy access to a two-year institution or an adult basic education program.

Recommendation: With respect to implementation of these admissions standards, the Committee recommends that the campuses be allowed the next four years to put them in place so that they become effective with the entering freshman class of Fall 2004.

Phased implementation is necessary for two reasons. First, Montana Writing Standards were only approved by the Board of Public Education in 1999 and are just now being integrated into high school curricula. It is unfair to students to impose proficiency standards for admission when they have not yet been a curricular emphasis in the secondary schools.

Second, the Committee anticipates that most applicants for admission will demonstrate writing proficiency through the ACT Writing Sample. Adaptation of that sample to align with Montana Writing Standards, establishing cut scores for proficiency, field-testing the adapted assessment with traditional and nontraditional samples of students, analyzing field-test results and further revising the assessment and the assessment process, as needed, will take time. That time is well-spent if it ensures that students proficiency in writing is validly and reliably assessed and that unanticipated consequences of proficiency-based admissions are identified and addressed prior to their affecting students higher education pursuits.

Graduation

The Sub-Committee is aware of efforts around the Montana University System to develop exit standards for college and university graduates--notably, the work on 'gating exams' at the University of Montana. We believe that this approach is preferable to mandating a single approach for the Montana University System--hence, our final recommendation.

Recommendation: Given the differences in the campuses, their programs and missions, the Sub-committee believes that a single approach to assessing writing proficiency for graduation would not be appropriate. Instead, the Sub-committee recommends that the design and implementation of writing proficiency graduation standards be left to each MUS campus, with implementation to be established within the next two years.

This means that campuses would form appropriate faculty committees to:

  • Generate criteria that students will need to meet in writing to graduate;
  • Establish performance standards delineating the characteristics or skills that students should be able to display for a proficiency rating;
  • Prepare for wide dissemination models of acceptable writing;
  • Identify means by which students will be allowed to demonstrate proficiency in writing such as written exit exams, scores on the verbal portion of the GRE or other standardized tests, completion of "W" courses, capstone projects, senior seminars with substantial writing assignments, etc.;
  • Define a process for regular review of criteria, standards and activities by which student proficiency in writing is to be documented for graduation; and
  • Determine how students who fail to demonstrate proficiency in writing will be assisted to achieve this standard and graduate.

The Committee recommends that each campus complete these studies in the next 24 months. Written reports on each campuses' policies and activities for graduation proficiency could be submitted to the Office of the Commissioner of Higher Education in time for inclusion in the report to the Board of Regents in May 2002, permitting Board review and campus adoption effective with the rising junior class in Fall 2002.

Beverly Ann Chin, Chair, K-16 Joint Committee on Composition Standards

Professor of English

University of Montana

Enclosures

pc:   Members of the Committee
Jan Clinard
Gail Gray
Linda Peterson
Joyce A. Scott