Optimize the Portfolio of Academic Programs Offered

Priority for Current Fiscal Health Review: High
Savings/Revenue: $$
Time to Achieve: Recommendation is to begin implementation immediately and continue to review all degree programs at least every two years.

The AFH Working Group recommends that a group be formally charged to utilize the process developed by the FFRO Implementation Teams and approved by the FFRO Oversight Committee to review the University’s Academic Program Portfolio, including the Honors Program and University Studies Program, at least every two years. University Standing Committees such as the Educational Planning and Policies Committee and University Budget Committee have the capacity and charges that align well with performing such review. For a complete list of University Standing Committees, visit the Faculty Senate Committees page. The inclusion of ex-officio members on those committees promotes communication with multiple University offices (e.g. Office of the Chancellor, Office of the Provost, and others).

EAB’s Academic Performance Solutions data shows that almost 70% of students are concentrated in the top two deciles of the most popular majors at their respective institutions (n=51)¹. At ECU, in terms of undergraduate degrees conferred, the top ten majors produced 12,126 of 24,950 (49%) credentials from 2019-20 to 2023-24; the next ten majors added 5,246 credentials, such that the top twenty majors produced 17,372 of 24,950 (70%) of UG degrees. Enrollment is the most common metric for assessing academic program health. Most institutions begin with an enrollment analysis that is straightforward and that often includes a specific target for enrollment and degree conferred. ECU utilizes dashboards that provide information on degree conferral by program. A visual representation of one of those dashboards is included below.

The AFH Working Group recommends that ECU goes beyond a “quick review” and instead adopts as much of a holistic approach to program review as possible. The Working Group recommends utilizing a process wherein program health is assessed via a two- or three-step process.

Offered as further information for the Oversight Committee to consider, the AFH Working Group recommends the degree program health check include the metrics noted below and in ordered phases of review. In the first phase of review, metrics readily available (provided by IPAR) would be reported and either by a program-specific target for each 1^st level metric or another means of determining which programs would be reviewed beyond Level 1 (e.g., 2^nd level review could occur for only the programs that are in the lower 25% of 1 or more of the 1st level metrics).

Currently the IPAR dashboard related to Low Productivity is utilizing the “old” UNC System Low Productivity; however, the Working Group recommends a more nuanced, ECU-specific approach that would consider more than just the level of the degree if establishing target enrollment, degree conferrals, and SCH at the program level occurs.

1^st Level AP Portfolio Review Metrics Reviewed (in order of priority and/or “performance” target is needed)

1. # of degrees conferred in the degree program over the past 5 years
2. # of students enrolled in the degree program over the past 5 years (intended+declared)
3. # of student credit hours (SCH) generated per Instructional Faculty FTE for courses in the degree program (part-time+full time) over past 5 years

2^nd Level AP Portfolio Review Metrics and Information Reviewed

Items below are in order of priority and/or consideration of “performance” target might be useful. It is recommended that this section of metrics/information be required for programs undergoing 2nd Level Review.

1. Enrollment in courses that are critical to other programs at ECU or operated with other entities (e.g. formal Pathway programs with other institutions, joint degree programs with other institutions, programs operated with community college and industry/community partners that are not offered through continuing education, etc.).
2. Effective Deployment of Resources
   1. Individual Course Enrollments (all courses offered by unit)
   2. Instructional Cost (instructional expenditures/faculty FTE)
3. Student Outcomes / Student Feedback
   1. Post Graduation Success (Lightcast, Stepping Blocks, other)
   2. Feedback via Exit Surveys
4. Faculty Contributions
   1. External Funding (including but not limited to measures of F&A and GCRT generated) (currently available via eTRACS, IPAR)
   2. Research and/or Creative Activity (departmental narrative)
   3. Service/Administrative Roles (departmental narrative)
5. Internal Duplication of Programs
   1. Additional Information to be considered in further discussion of the program:
      It is recommended that units be encouraged/permitted to provide the information below. Units should choose which item(s) and associated metrics are relevant to understand the health of the program being reviewed.
6. Critical Need in Workforce
   1. In accordance with UNC System designations
   2. As measured by other data a program may present as being strong indicators of health for programs of their nature (e.g. Bureau of Labor Statistics – state and national)
7. Effective Operation of the Unit
   1. Potential indicators
      1. “Health” of other academic programs (degrees, certificates, other credentials) offered by the unit
         1. Consider whether a “weight” can be assigned (e.g. if UG programs are not doing well, but graduate programs are, what, if anything, is the impact of that?) 
      2. Efficient use of Graduate School Funding (SCH generated/amount of $ allocated; student-collaborations on publications/presentations/creative activity work products)
      3. Persistent high D, F, W (a separate recommendation is to consider more attention be given to the insights provided by the “Course Barrier” dashboard)
      4. Indicators of Administrator Effectiveness
         1. Needs definition 
         2. Provost/Chancellor review of 5-Year Administrator Reviews was discussed and viewed by some Working Group members as confidential/Personnel File information while others consider that information available to the Provost/Chancellor and reasonable to consider (but of course not shared) as more in-depth review of programs occurs. If administrator effectiveness is consistently rated below what would be expected, it may or may not be a factor in the health of the program, but in cases where it is determined by the review by Provost/Chancellor 
8. Duplication of Programs within UNC System, region, and identified competitors 
9. Indicators of the “right size” of the program (national average or numbers of an appropriate comparison group) a. Would likely have to be measured by completion rates 
10. Effects of eliminating or restructuring the program (impact to other programs at the institution, connections to programs/pathways external to the institution (e.g. early assurance, affiliation agreements, etc.) 
11. Ability to transfer faculty/personnel to another program/unit 
12. Impact to institutional reputation 
13. Unit Input (other)
    1. Metrics and/or information from 7-Year Unit Academic Program Review Reports/Progress Reports 
    2. ROI Studies from UNC System Studies 
    3. Other indicators of ROI for an academic program 

This recommendation was based on review of materials such as ECU’s 2011 Program Prioritization Committee’s “Program Data Template,” metrics used for program assessment at other institutions, and EAB’s “Rightsizing the Program Portfolio” publication.

The Working Group recommends the involvement of appropriate faculty committee(s) in the annual assessment of the metrics/process described above.

A potential issue with adopting the approach above is that it does not take into account the “percentage/ratio” of changes in enrollment in an academic program. For example, a program may be among the top 50 programs in terms of enrollment at ECU but six years ago, it may have been in the top 10. The Working Group contemplated adding this “metric/analysis” to the 1st Level Review if overall program health was being assessed. The focus on fiscal health led the group determination to include this comment in this report to draw attention to it as further consideration of the appropriate process for senior leadership’s goals for the process evolve and are articulated. If this analysis is not part of the Portfolio Review process, the University may consider requiring units to address it in a specific way in their annual reports.