Assessment Plan
Assessment Plan: Ph.D. in Engineering Education
After careful considerations and discussions, we have identified two most important milestones that all students in our Ph.D. in Engineering Education program are required to complete. The first milestone is the Qualifying Examination (QE). The second milestone is the Dissertation Research Project (DRP).
Qualifying Examination (QE)
The purpose of the Qualifying Examination is to assess the extent to which students have achieved mastery of knowledge gained from the core courses in the Ph.D. in Engineering Education curriculum and to gauge student readiness for subsequent doctoral study in engineering education. The Qualifying Examination consists of five problems administered over five days. Each day, students are given four hours to complete a problem. The five problems cover knowledge gained from eight core courses in the Ph.D. in Engineering Education curriculum: EED 6090, EED 6150, EED 7010, EED 7040, EED 7230, EED 7460, EDUC/PSY 6570, and STAT 5200 (or EDUC/PSY 6600).
Each section includes a set of exam problems. Each exam problem is associated with at least one learning objective (LO1 through LO7). An Exam Review Committee, which consists of the student’s Faculty Advisor (Major Professor) and other faculty members in the Department, read and review student responses to each exam problem to determine the level of evidence that the student meets the learning objectives. The level of evidence is ranked from 1 to 5, where 1 indicates the lowest level of evidence and 5 the highest level of evidence.
Each committee member assesses student responses to exam problems using the following rubric:
|
Learning Objectives (LO) |
Level of Evidence Students Meet Each Learning Objective |
|
LO1: Synthesize issues (e.g., theories, ideas, research findings, conclusions, and recommendations) in writing |
Low evidence High evidence 1 2 3 4 5 |
|
LO2: Understand (e.g., identify, classify, describe, explain) major issues related to human learning including the nature of expertise, knowledge organization and deployment, transfer of learning, and assessment of cognitive skills |
Low evidence High evidence 1 2 3 4 5 |
|
LO3: Understand (e.g., identify, classify, describe, explain) the historical, philosophical, and psychological foundations of engineering education as the field has evolved within academic, social, political, and economic contexts |
Low evidence High evidence 1 2 3 4 5 |
|
LO4: Apply (e.g., use, implement) the appropriate skills necessary to organize and prepare competitive grant proposals |
Low evidence High evidence 1 2 3 4 5 |
|
LO5: Understand (e.g., identify, classify, describe, explain) basic principles and practices of research methodologies typically used in engineering education research |
Low evidence High evidence 1 2 3 4 5 |
|
LO6: Understand (e.g., identify, classify, describe, explain) various methods to measure and evaluate student achievement within the cognitive, affective, and psychomotor domains |
Low evidence High evidence 1 2 3 4 5 |
|
LO7: Create (e.g., design, develop, formulate) and/or evaluate (examine, interpret, critique) learning objectives and relevant assessments |
Low evidence High evidence 1 2 3 4 5 |
The average score (i.e., the level of evidence) is calculated based on all scores provided by all committee members on each learning objective.
Dissertation Research Project (DRP)
All students in our Ph.D. in Engineering Education program are required to complete a 12-credit Dissertation Research Project that leads to a Ph.D. Dissertation. A Ph.D. Dissertation typically consists of six chapters, including Chapter 1 Introduction, Chapter 2 Review of Literature, Chapter 3 Research Methodology/Design, Chapter 4 Results, Chapter 5 Discussions, and Chapter 6 Conclusions/Significance/Implications.
Each chapter in the Ph.D. Dissertation is associated with at least one learning objective (LO1, LO8, and LO9). The student’s Supervisory Committee, which consists of the student’s Faculty Advisor and four other faculty members inside and outside the Department, read and review the student’s Dissertation to determine the level of evidence that the student meets the learning objectives. The level of evidence is ranked from 1 to 5, where 1 indicates the lowest level of evidence and 5 the highest level of evidence.
Each committee member assesses each chapter of the student’s Ph.D. Dissertation using the following rubric:
|
Learning Objectives (LO) |
Level of Evidence Students Meet Each Learning Objective |
|
LO1: Synthesize issues (e.g., theories, ideas, research findings, conclusions, and recommendations) in writing |
Low evidence High evidence 1 2 3 4 5 |
|
LO8: Apply (in this context: use) principles and practices of research methodologies typically used in engineering education research |
Low evidence High evidence 1 2 3 4 5 |
|
LO9: Apply (e.g., use, implement, interpret) regulations, policies, statutes, ethical issues, and guidelines that govern the conduct of research with human subjects |
Low evidence High evidence 1 2 3 4 5 |
The average score (i.e., the level of evidence) is calculated based on all scores provided by all committee members on each learning objective.
NWCCU Ph.D. Program Student Learning Outcomes on the IDEA Curriculum Assessment Map
In addition to using the two most important milestones (QE and DRP), we have developed the following table to illustrate how NWCCU Ph.D. program student learning outcomes are linked with the IDEA curriculum assessment for important graduate courses our faculty members teach. Utah State University has been using the IDEA system for student course evaluation each semester.
| NWCCU Ph.D. Program Student Learning Outcome (LO) Students will be able to: | Courses Involved | IDEA Learning Objectives | IDEA Curriculum Assessment, Assignment, and Artifact of Student Learning |
|---|---|---|---|
| LO1: Synthesize issues (e.g., theories, ideas, research findings, conclusions, and recommendations) in writing |
|
Developing skills in expressing oneself orally or in writing |
|
| LO2: Understand (e.g., identify, classify, describe, explain) major issues related to human learning including the nature of expertise, knowledge organization and deployment, transfer of learning, and assessment of cognitive skills |
|
Developing specific skills, competencies, and points of view needed by professionals in the field most closely related to this course |
|
| LO3: Understand (e.g., identify, classify, describe, explain) the historical, philosophical, and psychological foundations of engineering education as the field has evolved within academic, social, political, and economic contexts |
|
Gaining a basic understanding of the subject (e.g., factual knowledge, methods, principles, generalizations, theories) |
|
| LO4: Apply (e.g., use, implement) the appropriate skills necessary to organize and prepare competitive grant proposals |
|
Learning to apply course material (to improve thinking, problem solving, and decisions) |
|
| LO5: Understand (e.g., identify, classify, describe, explain) basic principles and practices of research methodologies typically used in engineering education research |
|
Gaining a basic understanding of the subject (e.g., factual knowledge, methods, principles, generalizations, theories) |
|
| LO6: Understand (e.g., identify, classify, describe, explain) various methods to measure and evaluate student achievement within the cognitive, affective, and psychomotor domains |
|
Developing specific skills, competencies, and points of view needed by professionals in the field most closely related to this course |
|
| LO7: Create (e.g., design, develop, formulate) and/or evaluate (examine, interpret, critique) learning objectives and relevant assessments |
|
Developing creative capacities (inventing, designing, writing, performing in art, music, drama, etc.) |
|
| LO8: Apply (e.g., use, implement, interpret) principles and practices of research methodologies typically used in engineering education research |
|
Learning to apply course material (to improve thinking, problem-solving and decisions) |
|
| LO9: Apply (e.g., use, implement, interpret) regulations, policies, statutes, ethical issues, and guidelines that govern the conduct of research with human subjects |
|
Learning to apply course material (to improve thinking, problem-solving and decisions) |
|
Graduate Alumni
Each year, we collect, update, and review relevant data regarding our graduate alumni. Since the inception of our Ph.D. program, 32 students have graduated from our Ph.D. program. The following table shows the percentage of job categories they hold. As can be seen from the table, 63% of our Ph.D. graduate alumni secured faculty positions at institutions of higher education. As competition for faculty positions has been traditionally fierce, the high percentage of faculty positions our graduate alumni hold demonstrates the success of our program.
| Job Categories | Percentage |
|---|---|
| Academic tenure and tenure-track faculty | 47% |
| Academic non-tenure-track faculty | 16% |
| Academic post-doc | 13% |
| Industry | 13% |
| Other | 13% |