Assessment: Using Data to Grow Universities with Jamie Morley | Changing Higher Ed 020

Episode Summary

Assessment is critical for ensuring instruction is effective in helping students to learn. This data is even more important as the demographics of students and expectations of what higher education should accomplish change.

These instruments need to be part of a curriculum mapping process. In addition, meaningful assessments can be used to make instructional decisions in ways that are more student-centered.

Changing Views of Assessment

Many years ago, nationally accredited universities defined student learning through data such as retention rates and graduation rates, and used this information to measure institutional success and effectiveness.  However, traditional (regionally) universities used completely different measures for these purposes. These differences created a conundrum r-- What is student learning? Does passing a class means that a student has achieved identified competencies or is proficient in the topic?

The role of assessments in classes is especially important due to the changing demographics of students as well as expectations of higher education institutions. Before, the majority of students were 18-years-old coming to college directly out of high school. Instead, 74% of college students are “non-traditional,” i.e., 24-70 years old, and are in the process of changing their career, looking for a job, or looking for a promotion and/or higher pay.  These students are looking for something different from higher education that their predecessors where.  Additionally, employers’ expectations of colleges and universities have changed; they are looking for graduates to be well prepared to jump into a job with the specific skills that they should need (instead of the job providing OJT).

Curriculum Mapping

Meaningful assessment ensures there is an alignment between the course catalog, the syllabus, the program learning outcomes, course learning objectives, and outcomes, and what is actually being taught (and more importantly, learned).

Assessments should be included in the curriculum mapping process when developing a course. This systemic approach identifies the programmatic competencies to be addressed in a class, the sequence in which they are going to be taught and how course learning objectives align to program learning objectives. 

Meaningful assessment determines whether students are understanding key concepts and are able to successfully utilize new skills. Therefore, it’s important that the assessment is a valid instrument and that grading is consistent both among the students in one class as well as across classes on the same subject.  A faculty-developed rubric that gauges the level of learning based on the student’s answers on the assessment is an important piece of this system.

Assessments in Face-to-Face and Online Education

Student outcomes should not differ between face-to-face classes and online classes; they should learn the exact same thing. The easiest way to achieve that is through the mapping process when a faculty member creates an artifact, which is an assignment that matches the learning outcome of the course.

Additionally, faculty should create quiz questions, exam questions or an essay that are tied to the course objective statement. This data offers proof that the quality of instruction and outcomes are exactly the same for students taking online courses and those taking face-to-face classes.

Competency-based Education vs. Credit-Hour Education

CBE and the credit hour were key topics that were discussed during the recent negotiated rulemaking sessions in Washington, D.C. The Department of Education has a specific definition of a credit and that every single credit needs to accounted for based on the Carnegie Unit, i.e., for every hour of lecture a student has two hours of homework; over a term, there should be 45 student hours of effort for one credit hour. 

Competency-based education is controversial because it is a new way of learning that is not reliant on the Carnegie Unit, and because of that, student financial aid comes into question. The money paid for credit hour is the same as the money paid for competency-based education, so policymakers want to ensure that all students are learning the same information while making the same level of effort. 

CBE considers what skills are necessary and then assesses if students have those skills, instead of the number of hours they need to take to gain those skills. This latter approach would be beneficial for students who want to become a nurse, dietician or physical therapist. However, another approach may be beneficial for students who are majoring in history, liberal arts or education.

Faculty and leaders need to consider whether the 45 hours per credit hour make sense in a particular program. This is one of the discussions at Neg Reg (no decision was made to move away from the Carnegie Unit measurement at this time).

In some ways, competency-based learning is easier to assess than traditional credit education because it involves a checklist, externship or a list of skills that students need to demonstrate that they understand and can do (vs an arbitrary time in/out of the classroom plus examinations plus signature assignments plus etc.).

Assessments as Part of Institutional Improvement

Colleges and universities that have a structured plan in which the curriculum, objectives, and outcomes have been clearly mapped can see that the courses fit into the institutional mission. In large institutions, this can be assessed by the institutional research (IR) department/function. However, the dean or program director can take on this role in smaller institutions using an ongoing process. This process entails (1) looking at what is being assessed during the first quarter; (2) reviewing syllabi, competencies, artifacts, and assessments in the second quarter; (3) gathering data in the third quarter; and (4) completing a report in the fourth quarter that goes to administration and the board.

Most institutions conduct annual program assessment and five-year program reviews to understand what’s being done and accomplished, how a program is performing and what improvements can be made. In addition, the results from the program review can be used as part of the institution’s annual planning processes.

However, many institutions invest much time and money into the institutional planning process, yet forget to use this information to inform campus decision-making. For example, there’s a problem if students who are enrolled in a nursing program are not able to pass the licensure exam.  Having regular meaningful assessments allow faculty and leaders to identify where students are struggling, then make changes in the admissions process (such as requiring certain prerequisite skills), curriculum or budget (for updated equipment that matches what is available in businesses). Therefore, student learning assessment outcomes can also inform admissions, financial aid, enrollment management, staffing and even facilities. 

Some accreditation bodies make assessment challenging. For example, WSCUC requires proof that the faculty were involved in the development of curricula and agreed to the important concepts. Once buy-in is achieved, items need to be mapped out to identify the competencies or skills that students need to learn in a course.

Institutions that use meaningful assessment to ensure that students are learning also see their endowments and enrollment numbers increase, attract the best faculty and save money because they focus their efforts on areas that really need it. This is true not only for top-tier institutions; even small- and medium-sized institutions can be successful through using this type of informed decision-making.

Three Tips for Higher Education Leaders

Three pieces of advice for higher education leaders:

• Meet with the institutional research department. Leaders who have worked in business will have an advantage because they can help streamline the process and focus on improving learning outcomes.
• Read the previous report to see what data has been used. This report can serve as a starting place for identifying gaps and places for improvement.
• Walk around and talk to students and faculty. Find out if there is a general consensus that the students are learning and becoming prepared to be a productive part of society.

Bullet Points

• Institutions differ in the way they measure student success, institutional success and institutional effectiveness.
• Assessments are a critical part of the instructional process because they can ensure faculty remain on topic instead of veering onto tangents, the course’s objectives and outcomes are aligned with what is being taught, and students are learning and can apply new skills.
• Faculty should consider the placement and type of assessment to be used during the curriculum mapping phase of course development. In addition, a rubric should be used to analyze the quality of learning displayed on an assessment.
• Assessments can help ensure that instruction and learning are comparable in both face-to-face and online classes.
• Competency-based education and credit-based education are under the microscope. Each form may require a different type of assessment to be used.
• Assessments also can and should be part of institutional improvement. This data can be used beyond just instructional decisions; it also can inform admissions policy, financial decisions, hiring of staffing, professional development offerings, and facility updates.

Links to Articles, Apps, or websites mentioned during the interview:

• Education Consulting Services

Guests Social Media Links:

• Jamie Morley LinkedIn: https://www.linkedin.com/in/jamie-morley-ph-d-998a273b/

The Change Leader’s Social Media Links:

• LinkedIn: https://www.linkedin.com/in/drdrumm/
• Twitter: @thechangeldr
• Email: podcast@changinghighered.com