Department and Program Review Guidelines
Overview of the Review Process
Bates College strives to conduct once-per-decade reviews of all academic departments and interdisciplinary programs to assess their contributions to the college’s mission. The reviews consider the ways in which departments and programs further the success of the college through their contributions to the curriculum, the scholarship conducted by their faculty and students, and their engagement in the life of the college.
Each review includes a self-study conducted by the members of the department or program, an internal review conducted by faculty from other Bates departments or programs, an external review conducted by a visiting committee comprising of two or three faculty from other institutions, and a response and plan developed by the department or program under review.
The schedule of reviews is established by the dean of the faculty’s office in consultation with the Academic Affairs Council. The self-study, internal review, and external review typically span two semesters, either within a single academic year or across two academic years. Following the completion of the reports from the two review committees, the department or program submits to the dean of the faculty a written response to the reports and a plan.
As a first step in the process, the associate dean and assistant dean meet with the department or program chair to discuss the review process and the schedule. The main steps of the review process are detailed in the pages that follow, but are summarized here for clarity:
Key Steps in the Review of a Department or Program
Step 1: Student Learning Assessment: Thesis and Capstone
Step 2: Website Updates
Step 3: Identify the Key Issues for Consideration during the Review
Step 4: Review of Key Issues by Academic Affairs Council
Step 5: Department/Program Faculty Meet with Associate Dean and Division Chair
Step 6: Nominate Members of the Internal and External Review Committees
Step 7: Conduct Department or Program Self-Study
Step 8: Academic Affairs Council Reviews the Self-Study
Step 9: Internal Committee Reviews and Reports
Step 10: External Committee Reviews and Reports
Step 11: Department or Program Responds and Plans
Step 1: Thesis and/or Capstone Assessment
The department or program faculty undertake a comprehensive assessment of the senior thesis or senior project prior to or concurrent with the self-study step of the review. Through this assessment, the faculty clarify the department’s or program’s learning goals and assess the extent to which the majors are reaching them. Evaluating department/program learning outcomes and the thesis or capstone prior to undertaking the remainder of the review is preferable, though it may not be practical in all cases. Information on the Thesis/Capstone Assessment can be found online. For more information, please contact the dean of the faculty’s office.
Step 2: Website Updates
Departments and programs under review should update their websites prior to the review, as the External Committees consult the websites for information about the department or program before visiting.
Step 3: Identify Key Issues
The department or program faculty develop a list of key self-study issues they plan to address during the review. A written list of those issues is then submitted via email to the dean of the faculty’s office by the department or program.
Step 4: Review of Issues by the Academic Affairs Council
The Academic Affairs Council (AAC) reviews and discusses the issues raised by the department or program under review. The AAC may comment on or add to this list other issues it determines will be important to cover during the review.
Step 5: Department/Program Faculty Meet with Associate Dean and Division Chair
After the AAC has discussed the review issues, the appropriate associate dean and division chair meet with the faculty members in the department or program under review to discuss the AAC’s response and the logistics of the review. This meeting helps the department or program under review finalize the list of issues to be addressed in the self-study and throughout the review. These issues will inform the nomination of internal and external review committee members by the department of program. The goal of this meeting is for all faculty in the department or program to participate in the review and understand the process and schedule. Typically temporary faculty are not involved in reviews, though that decision is made by the department or program.
Step 6: Nominate Faculty for Review Committees
The department or program under review submits to the AAC nominations for an Internal Review Committee, consisting of Bates faculty who do not regularly teach in the department or program under review, and an External Review Committee, consisting of faculty drawn from other institutions. The nomination form must include the department’s or program’s preferred internal and external committee rosters along with several alternates for each position on each committee. It is not unusual for many individuals invited to participate – especially on external committees – to decline, so having a deep pool of alternates is essential. Individuals nominated for the external committee should be sufficiently independent from the department or program so that they are able to evaluate objectively and with a perspective that is unhindered by prior knowledge obtained from association with a member of the department or program that is undergoing review. To maximize the possibility that nominated faculty will be able to serve on the review committees, departments and programs should submit their nominations to the dean of the faculty’s office well in advance of the review.
Internal Review Committee
The Internal Review Committee (“Internal Committee”) consists of three Bates faculty members from outside the department or program under review. Bates professors, associate professors, and senior lecturers are eligible to serve. The Academic Affairs Council selects committee members from a list of nominees submitted by the department or program using the Internal Committee Nomination Form. To inform the committee selection process, it is helpful if the department or program includes a brief description of the desired overall composition of the committee. The description might specify, for example, that a department or program wishes the committee to have representatives from particular departments, programs, or divisions; it might state that it would prefer to have at least two of three reviewers with significant service in interdisciplinary programs; or, it might specify that the chair of the committee be selected from a specific subset of the nominees. The dean of the faculty’s office informs the department or program chair of the final selection of Internal Committee members. These decisions are based on faculty leave status and whether colleagues have recently served on committees. The dean’s office invites faculty to serve on the committee.
External Review Committee
The External Review Committee (“External Committee”) consists of two or three faculty from other institutions who possess pertinent experience and who, in the view of the department or program under review, can offer objective, useful advice from a disciplinary perspective. The External Committee should in most cases consist of two members. However, if justified by the size or complexity of a department or program, the dean may decide to appoint a three-person committee.
The department or program under review submits nominations for this committee using one of the External Review Committee Nomination Forms (there are two forms: one for two-person external committees and one for three-person committees). The department or program should describe the characteristics of each position on the committee (e.g., one member from a liberal arts college and the other, from a Research I university; one member focused on molecular biology and one on ecology). The nominating lists should include the first choice for each committee seat and several alternates for that seat. To aid in the selection process, the department or program under review should submit along with the nomination form concise, relevant background information (e.g., CVs or biographical material from the Web, which may be compiled by their AAA) on each External Committee nominee and highlight those characteristics deemed desirable. The AAC bases its decision on the members’ knowledge of the department or program, the nature of the key issues to be addressed, and the preferences expressed by the department or program. The dean of the faculty’s office informs the department or program chair of the AAC’s final roster and alternates. The dean’s office issues formal invitations to serve.
Appendix, Selecting Committees, contains helpful information for nominating members of the internal and external committees.
Step 7: Conduct Self-Study
At the beginning of the review process, the department or program under review conducts a self-study during which it
- Reviews and considers new directions or innovations within the discipline
- Reviews and considers its curriculum
- Collects and considers information and data pertinent to the department or program and its operation, including but limited to enrollment data, results from an alumni survey, or institutional data like senior surveys or graduate school admission results
- Uses those data to assess the department’s or program’s effectiveness in furthering its goals and objectives and the mission of the college, identifying strengths and weaknesses, and highlighting key issues it wishes to raise during the review
- Develops a preliminary plan for the subsequent five to ten years
- Creates a document that presents the results of the self-study
Contents of the Self-Study
A self-study document includes an overview of the department or program and its curriculum, a description of the issues and concerns, a long-range (5-10 year) plan, and appendices containing supporting documents (see below).
In order to produce the self-study, the department or program faculty should assess and analyze its curriculum several months before beginning the self-study. This critical examination of the curriculum and student learning should form the basis of the self-study. It not only measures the department or program’s effectiveness in meeting its educational goals, but it also informs other questions, including workload and resources. Understanding why we teach the courses that we teach, at what level, and with what goals for student learning, is at the heart of a robust self-study.
- Assessment of student learning for both majors and non-majors. The department or program should use available data or generate new data to measure students’ learning as defined by their stated goals and objectives. The outcomes assessed can include knowledge, methods and theories of inquiry, intellectual or academic skills, use of technology, or integrated/interdisciplinary learning.
Below are examples of strategies for assessing selected learning outcomes at the department or program level:
- A department/program may develop an alumni survey asking about a variety of outcomes from career-related information to retrospective evaluation of learning while in the major. Alumni surveys are most useful when departments/programs ask about strengths and
- A department/program might assess writing proficiency in seniors by cross-reading and collectively evaluating senior theses using a grading system or rubric.
- One of the STEM departments or programs might use a standardized disciplinary measure of reasoning, competency, and skills from the field.
- The languages could assess oral proficiency through senior presentations or portfolio defenses.
- A social science department could use senior poster presentations to evaluate the degree to which the seniors have demonstrated mastery of research design and basic statistics.
- Faculty in a studio art major could evaluate student products/making during a senior show using a standardized form or set of evaluation criteria.
- A department or program could use a standardized test of critical thinking to compare their seniors to national norms for college students.
The dean’s office and the Office of Institutional Research can help with department/program-level assessment of outcomes. The dean’s office strongly advises departments and programs to begin assessment activities well in advance of the review.
- Assessment of the Senior Thesis or Senior Project. The rubrics above should be applied to the senior thesis or other capstone experience, and the effectiveness of these senior experiences measured.
- Assessment of General Education. The department or program should discuss its role in the General Education program and the effectiveness of student learning in General Education courses using the rubrics above. Departments or programs may describe the number or percentage of courses that contribute to GECS, the approach to W, S, L, or Q requirements for nonmajors, and how faculty determine successes or challenges in providing General Education opportunities.
- Evaluation of Enrollment Data. The dean’s office provides course enrollment data, numbers of majors and minors, the number of faculty and staff associated with the department or program, and workload information. The department or program can learn a great deal by analyzing and discussing these data at the beginning of the self-study process. Departments and programs are encouraged to include data from corresponding departments and programs at other institutions as well as other Bates departments and programs. Departments and programs may analyze enrollment trends over time, introductory courses vs. advanced courses, over-enrollments and under-enrollments, and whether the current course offerings meet department or program learning goals, among other questions. Whenever possible, the department or program should provide context or explanation for enrollment areas that are problematic. If department or program faculty are uncertain of the kind of information data analysis can provide, they may confer with the dean of the faculty’s office.
- The Role of Scholarship. The department or program should consider the role of both faculty and student scholarship or artistic production, and describe the successes and/or challenges of a departmental or program culture of scholarship.
- The Department or Program and the College. The department or program should assess:
a) Department or program climate, collegiality, and sense of identity among students and faculty
b) The role of the department or program in interdisciplinary teaching and scholarship
c) The role of the department or program in the extracurricular life of the college (special
events, arts events, collaborations, community engagement, etc.)
- Issues and Concerns. The discussion of issues and concerns should address those developed at the outset of the review (in Step 1), but can also include other issues that emerge as the department or program conducts its self-study.It is understood that the allocation of personnel and financial resources is often a central theme of self-study reports. However, departments and programs, and the college as a whole, derive more benefit from reviews if department and program faculty think deeply about the educational goals of their work rather than focus solely on developing a wish list for faculty net additions and requests for increased space and financial resources.
- Long-Range Plan. The department or program should put forth its current thinking in a plan for at least five to ten years, which will be considered and likely modified as a result of the review. Given the large number of retirements that are projected to occur over the next decade, the department or program should consider possible departures and their impact on the curriculum.
- Appendices. The department or program should include the following appendices:a) A summary of the department or program major and minor (where applicable), its goals and requirements
b) Curricula vitæ of all faculty members regularly teaching in the department or program; AAAs typically compile these materials
c) Syllabi of courses offered in the last two years and others that are critical to the department or program curriculum (interdisciplinary programs may include the syllabi of courses central to the major); AAAs typically compile these materials
d) A summary of course enrollment patterns, and the number of majors and minors, during the ten most recent years; these data are typically provided to the department or program under review by the dean of the faculty’s office
e) information on the current annual operating budget of the department or program, provided to the department or program under review by the dean of the faculty’s office
f) A summary of department or program graduates’ educational and career outcomes during the five most recent years, prepared by the department or program with assistance from the Bates Career Development Center, the Office of Advancement, and the Office of Institutional Research and Assessment Support. Departments and programs often conduct surveys among alumni majors when preparing this report.
The self-study document and any supporting documents should be submitted electronically to a Lyceum site established by the dean of the faculty’s office for the review. All materials must be submitted at least three weeks before the Internal Committee begins its work. The dean of the faculty’s office sends information on the availability of the self-study to the Internal Committee, the External Committee, the president, deans, and division chairs.
Step 8: Academic Affairs Council Reviews Self-Study
The Academic Affairs Council discusses the self-study and identifies issues that require further attention or explanation. The associate dean and the appropriate division chair then convey the Academic Affairs Council response to the self-study to the Internal Committee prior to its work.
Step 9: Internal Committee Reviews Department or Program
The Internal Committee reviews the department’s or program’s self-study as well as the response to the self-study by the Academic Affairs Council. The Internal Committee meets individually with department or program faculty and staff, and meets with representative students. These meetings must occur well before the External Committee’s arrival. If the Internal Committee wishes to address any concerns directly to the dean or the division chairs, it may do so. The Internal Committee also meets with the External Committee during its visit.
After preliminary meetings, the Internal Committee submits electronically (as a pdf file) a report that includes its observations of the issues that are the focus of the review. The dean’s office distributes this report to the department or program faculty, the president, the division chairs, and the External Committee well in advance of the External Committee’s visit.
Internal Committee reports are brief (usually 4 to 7 pages) but are essential to the process. They provide the External Committee with a college-wide perspective, clarify issues, make observations, and raise questions that may not have been adequately addressed in the self-study. In this way they allow the External Committee to be sufficiently well informed before beginning its site visit so that it can begin exploring issues as soon as its visit to Bates begins.
The Internal Committee may determine that it is useful to meet with the Academic Affairs Council prior to or after submitting the internal review report.
Step 10: External Review Committee Reviews Department or Program
The External Committee usually visits the campus for two to three days. The most convenient schedule for External Committee members is to arrive on a Sunday afternoon and depart on a Tuesday afternoon or evening. The chair of the department or program under review, assisted by the academic administrative assistant and in consultation with the dean of the faculty’s office, develops a draft schedule and reserves meeting rooms for the External Committee. The draft schedule is reviewed by the deans and the appropriate division chair, who may call for changes to the schedule in order to maximize the effectiveness of the External Committee’s time on campus.
The draft schedule should follow these guidelines:
- The External Committee usually meets with the department or program members over dinner on the evening before the first full day. Primarily a social gathering, this dinner should orient the External Committee and review the agenda for the following days, and provides a general group discussion on the curriculum.
- The External Committee needs considerable time (one hour per faculty member if possible) to meet individually with members of the department or program and with faculty and staff in other departments, as appropriate. The External Committee meets with students, usually in a group at dinner at the end of the first full day. Time should also be made available for the committee to meet by itself to discuss the review as it progresses.
- The committee meets with the president, the deans, and the divisions chairs early in the visit and again with the dean and division chairs at the end of the visit. The dean’s office schedules these meetings.
- Several hours on the last half day of the visit are set aside for the committee to discuss and draft its report.
- Before departing, the External Committee meets with the faculty in the department or program under review to discuss its findings and outline the general conclusions it will present in its written report.
Within one month of the visit, the chair of the External Committee sends electronically (as a pdf file) a written evaluative report to the dean of the faculty. The dean, in turn, sends the report to the president, the department or program faculty, the Internal Committee, the associate deans, and the division chairs. The College encourages the External Committee to
- Share candidly its best judgments about the quality of undergraduate education and scholarship offered by the department or program
- Identify strengths and weaknesses in the department’s or program’s work
- Compare the Bates department or program with departments at other institutions, if useful
- Enumerate resource needs or curricular challenges that deserve attention
- Provide a fresh perspective on the department or program and a focus for its own response and plan
Upon receipt of the External Committee’s report, the dean’s office sends each member of the committee a modest honorarium in appreciation of the individual’s service.
Step 11: Respond to the Review Committees’ Reports
Within six months of receiving the External Committee’s report, the department or program under review submits electronically (as a pdf file) to the dean its response to the review, a written statement reacting to the Internal Committee’s and External Committee’s discussions and reports. This response should be submitted to the dean as soon as possible, but no later than six months after the review. In its response, the department or program assesses its own strengths and areas of concern. It seeks to reach consensus on ways to enhance the quality of education it provides students and the quality of professional life within the department or program. The department or program also summarizes its plans for the next five to ten years.
After the response to the review is submitted, the department or program faculty members meet with the dean, associate dean, and appropriate division chair to discuss the review. That discussion is intended to identify immediate priorities and the steps to address them. In addition, the discussion should yield a long-range plan, including a structure for monitoring and measuring progress toward identified goals.
APPENDIX–Selecting Committee Members
Departments and programs under review nominate individuals to serve on the Internal Committee and the External Committee. It is in the department’s or program’s best interest to explain why each individual was nominated and what they would bring to the review.
The nomination forms and supporting materials should be submitted electronically via the webform to the associate dean by a date established in initial discussions about the review. The associate dean and the appropriate division chair discuss the nominations and propose committee rosters to the dean. If the division chair is a member of the department or program under review, the department or program chair discusses the nominations with the associate dean.
The Internal Committee: Form 2–Internal Committee Nominations
Usually three members of the Bates faculty serve on the Internal Committee. Any Bates professor, associate professor, and senior lecturer who is not a member of the department or program under review, and who does not teach a large number of courses cross-listed in the department or program, may be nominated.
The External Committee: Form 3–External Committee Nominations
The External Committee usually includes two faculty members from other institutions. These colleagues should be tenured in their home department or program or hold senior rank in their discipline. A department or program seeking an External Committee of three members must justify the request, and should anticipate that a longer lead time will likely be required to find three reviewers who can participate.
The department or program completes the nomination form indicating its preferred External Committee, along with several alternates for each seat, should the dean’s office have to move to alternate invitees. Alternates are essential: many years of experience have shown that several individuals who are invited will not be able to serve for one reason or another.
The department or program provides concise, relevant information (C.V.s, Web materials) on each External Committee nominee. That information should include each nominee’s institutional affiliation, institutional address, telephone number, and e-mail address, academic background, research or teaching fields.
Departments and programs are advised not to nominate the most famous scholars in their field, as they are likely to be too busy to serve. The department or program should remember to include individuals from diverse sub-fields, and from a variety of institutions. Please keep diversity of gender, race, and age in mind. Individuals who have close professional or personal ties to a member or the department of program faculty or to the college, or who are actively seeking employment at Bates, are not eligible to serve on external review committees.
It can be difficult to find the line between friendship and a close, collegial relationship that does not inhibit objectivity between a member of the department or program under review. It may be helpful when trying to determine whether a proposed member of the external committee can meet the requirement for objectivity to consider the following questions.
1) Has the proposed member of the external committee been a guest in the home of a faculty member of the department or committee? Has the proposed member of the external committee socialized frequently with any member of the department or program under review?
2) Is a proposed candidate for the external committee related by blood or marriage to any member of the department or program?
3) Has the proposed member of the committee been an advisor, coauthor with, fellow member of an investigating team, or member of the same department as a member of the department or program under review?
If a department or program seeks a particular External Committee composition (for example, at least one person from a Research I institution; persons from certain sub-fields in the discipline; a person to address issues of diversity), that preference should be made clear in the nomination materials, and the information necessary to satisfy that request should be provided. Specifying a preferred committee member and logical alternates is useful. Identifying individuals who would make good committee chairs is useful as well.