“Developing the Integrated Institutional Effectiveness Office” | April 14, 2015 (with gratitude to Ball State University for hosting)
The Association for Higher Education Effectiveness (AHEE) was formed in 2013 as a network of higher education professionals who lead or staff offices that intentionally integrate multiple functions (strategic planning, institutional research, assessment, accreditation, program review) to promote and support evidence-based planning and improvement. AHEE’s mission is to support and develop leadership that educates, advocates, advises, facilitates, and improves higher education’s capacity to use evidence in decisions, policy, planning, and change for the
purpose of improving postsecondary education. One of its goals is to help develop integrated offices that better serve institutions.
This white paper accompanies a webinar recorded in April 2015 that is designed to provide a primer to institutions interested in developing an integrated institutional effectiveness (IIE) unit. Topics include the rationale for moving to the IIE model, components of IIE units, some institutional examples of IIE units, tactics for establishing the IIE unit, and ideas about staffing and leadership of the IIE unit. References are also provided.
The Rationale for Moving to the IIE Model
Several trends have affected higher education in the last few decades. As noted in Postsecondary Education Opportunity, the United States can been experiencing a human capital economy since the early to mid-1970s. Levels of education are highly correlated to both personal success (e.g., income, health) and the advancement of society (e.g., employment,
taxation, civil participation). A bachelor’s degree, and perhaps, increasingly, a master’s degree, is necessary for individuals to maintain a middle class lifestyle and to fulfill the needs of a highly skilled workforce.
Changing age cohort demographics also affect higher education. As the large Baby Boom cohort is retiring, more and more state funding is being directed to both state pension programs and Medicaid. With state constitutional requirements and political support for funding K-12 education and prisons, support for higher education (which often represents the largest “discretionary” portion of state budgets) becomes increasingly constrained. Regional differences
also come into play with this issue; traditional college-age populations are decreasing in the Northeast and Midwest at the same time that they are increasing in the South and West. Additional information is available from the American Council on Education, Association of Governing Boards of Universities and Colleges, and the Grapevine Project.
Another trend affecting state funding is shifting political ideology. The traditional compact that long recognized higher education as a joint individual-public good appears to be breaking down as politicians more and more narrowly view the purpose of attending college to be job training. As a college or university degree is considered to be an individual good, an expectation follows that funding (both directly to public institutions and also in the form of student financial aid) should come from the individual rather than the state. See Jaschik (2015) for a recent related discussion.
A development affecting institutional research specifically is the widespread availability of Enterprise Resource Planning computer systems, data warehouses, business intelligence, predictive modeling, data visualization capabilities, and the proliferation of “student success” applications. The existence of these technological tools creates an expectation on the part of campus leaders for ubiquitous and easy-to-use analytics.
At the same time that a college degree has become critical, demographic and ideological trends are affecting state funding and financial in the public sector and competition, including competition from for-profit and online institutions is affecting finance at independent institutions, with the result that tuition levels are rising at rates substantially greater than inflation and levels of student loan debt are increasing rapidly.
With higher education now framed as an expensive commodity, it should not be surprising that students, parents, politicians, accreditors, and the media are asking hard questions about cost, quality, and return on investment. The focus on learning outcomes assessment over the last 30 years, the focus on enrollment management and student success, changes in the faculty workforce and in the teaching and learning model, greater interest in strategic planning and resource management, and professed greater interest in evidence-based decision making all derive from concerns over accountability and value. See Featherman (2014) for additional details.
Unfortunately these trends that lead to the need for more effective and agile decision support often are stymied by some difficult realities on many campuses. Institutional research offices are often small, spend nearly all of their time on reporting and ad hoc requests (Volkwein, Liu, & Woodell, 2012), and are often disconnected from critical leadership discussions. Leadership for assessment is often part-time, has high turnover, and is not strongly connected to
institutional research. The process and products of strategic planning often varies with campus leadership. Academic program/unit review is often separated from institutional research, assessment, and planning and often takes place at various levels of rigor. Campus leaders are often vulnerable to sales pitches for various “student success” “solutions” that often have tenuous connections to other efforts. Institutional leaders are looking for decision support that is
proactive, entrepreneurial, transformational, yet are often unsure of how to help these units be more effective.
Some recent critiques of institutional research and assessment highlight the need to improved decision support. Only 25% of chief academic officers responding to an 2015 Inside Higher Education survey (Jaschik & Lederman, 2015) say their institutions are very effective in using data to inform campus decision-making; only 23% say their institutions are very effective in identifying and assessing student learning outcomes; only 19% say their institutions are very effective in data analysis and organizational analytics. A national study of institutional research carried out by the National Association of System Heads (Gagliardi & Wellman, 2014)concluded
. . . the picture that emerges from this study is of a field that is at best unevenly positioned to support change. IR offices are running hard and yet many are still falling behind, deluged by demands for data collection and report writing that blot out time and attention for deeper research, analysis and communication. . . . The overall ability of IR offices to use data to look at issues affecting many of the cross-cutting issues of the day—such as the connections between resource use and student success—is nascent at best. (pp. 2-3)
This confluence of trends has led many campuses to establish integrated institutional effectiveness units. Leimer (2012) noted that the Directory of Higher Education listed 43 such units in 1995 and 375 in 2010. A search of the same sources in 2015 using the key word “effectiveness” yielded 501 unique institutions. Establishing such units, whose leaders often have titles such as assistant or associate vice president or assistant or associate provost, and who sometimes serve as members of the president’s cabinet, follows the tradition of the integrated information technology and enrollment management models that have emerged over the last several years.
Components of IIE Units
The names, reporting relationships, and components of IIE units vary widely across institutions. While the name institutional effectiveness is becoming more common, the terms planning and analysis are also sometimes included. IIE units may be led by vice presidents or vice chancellors and represent divisions in a college or university, or they may be led by assistant or associate provosts and reside within Academic Affairs; other structures are possible, but these appear to be most common. The functional components of IIE units typically include institutional research, assessment of student learning outcomes, academic (and sometimes administrative) program or unit review, strategic planning, and accreditation. Not all of these functions are always present. Less common additions to the IIE portfolio may include
institutional budgeting, analysis and allocation of space, and development of new academic programs. IIE units might include a research and development function where seed funding to promote campus improvements is made available through internal grants.
Examples of IIE Units
A recent brief review of Internet sites following the search term of institutional effectiveness resulted in the listing of IIE units provided below. While none of them are exactly alike, they generally include the components discussed above.
• Public Universities
– Ball State University
– Eastern Kentucky University
– Northern Kentucky University
– University of Alabama at Birmingham
– Western Michigan University
• Community Colleges
– California Community Colleges
– Central Piedmont Community College
– Coastline Community College
– Delaware Technical Community College
– Pamlico Community College
– Portland Community College
• Private Colleges or Universities
– Pepperdine University
– University of Richmond
Establishing the IIE Unit
AHEE is currently carrying out a series of interviews with campus presidents to get their perspectives on the IIE concept. It appears that many IIE units were developed because the president became convinced that this model would help the institution achieve its goals, but details of what guided presidents’ thinking in this direction are unclear. For staff members working in institutional research or assessment or planning or accreditation or program review to successfully make the case for establishing an IIE unit requires successfully practicing “leading from the middle” or “influencing up” (see Knight, 2014); this means convincing campus leaders that this new structure will help the college or university achieve its goals or solve its problems. Those wishing to develop IIE units need to help leaders overcome their natural resistance to change and their reaction that building an IIE unit represents a proliferation of administration at a time of constrained resources.
Tangible institutional benefits to establishing an IIE unit might include higher quality and more evidence-based planning and unit review, improved efficiency resulting from more widely shared definitions and assumptions, more adroit responses to critical requests, and more thoughtful resolution of problems. Overcoming problems in the areas of accreditation or assessment might be reasons why IIE units have been established on some campuses. As a follow-up to this webinar, the AHEE Board of Directors plans to post a series of discussion questions on the AHEE listserv; this would be an excellent place for sharing of institutional narratives about how and why IIE units were developed.
Staffing IIE Units
The number of IIE staff members should at minimum include the number of persons (full- and part-time) historically involved in IR, assessment, planning, accreditation, and unit review. Evidence from Volkwein, Liu & Woodell (2012) suggest that many current operations are greatly understaffed. A minimum of 3-4 full-time staff members seems imperative regardless of institutional size. In additional to traditional capabilities of IR and assessment staff (database
management, quantitative and qualitative analyses, reporting), Terenzini’s (1993, 2011) writing suggests the importance of issues and contextual knowledge and skills, including written and oral communication, project management, and group facilitation (Leimer, 2012). IIE staff members should be expected to be called upon and volunteer to make presentations, provide workshops, participate in discussions, and proactively share knowledge. One specific activity to which IIE staff members are well positioned to contribute is data governance.
Leading IIE Units
Leimer (2012), in her article on organizing for evidence-based decision making and improvement, notes that
– one element . . . necessary to spark and sustain evidence-based decision making and improvement . . . is leadership in making sense of, strategically applying, and communicating data and findings to diverse audiences in ways that prompt organizational learning and stimulate people’s desire to know more and then to act on the information. (p. 45)
The leader of the IIE unit works to actively move the institution towards a culture of evidence. Leimer (2012, p. 46) states
– For culture to change, someone must turn data into information and institutional knowledge through analysis and interpretation. . . . Developing such a culture takes sustained effort over a long period of time at multiple levels of the organization. But someone needs to take the lead—to advocate for, and maintain focus on, this mode of thinking and practice.
Leimer (2012) also notes the importance for the IIE leader to have “the abilities to build consensus, negotiate, communicate in non-technical language, coordinate people and projects, and lead” (p. 49). Other important characteristics of the IIE leader include
– sensitivity, openmindedness, flexibility, a capacity to listen, enthusiasm, a commitment to learning, a sense of humor, the ability to build others’ self-confidence and motivate them, creativity, team-building and problem-solving capacities, a thick skin, a tolerance for ambiguity, . . . patience, [and] . . . the abilities to educate, build trust, and use data to tell a
compelling story. (pp. 49-50)
Given what seems to be reluctance on the part of traditional professional associations, and perhaps graduate preparation programs, in institutional research, assessment, accreditation, and planning in leadership development, AHEE’s webinars and other future activities are designed to fill this void, as is my recent book on leadership and management in institutional research and institutional effectiveness (Knight, 2014).
References
American Council on Education Advocacy News. (n.d.). Retrieved from http://www.acenet.edu/advocacynews/Pages/default.aspx
Association for Higher Education Effectiveness. (n.d.). Retrieved from http://ahee.org.
Association of Governing Boards of Universities and Colleges. (n.d.). Retrieved from http://agb.org
Featherman, S. (2014). Higher education at risk: Strategies to improve outcomes, reduce tuition, and stay competitive in a disruptive environment. Sterling, VA: Stylus Publishing.
Gagliardi, J.S., & Wellman, J. (2014). Meeting demands for improvements in public system institutional research: Progress report on the NASH project in IR. Washington, DC:
National Association of System Heads.
Grapevine. (n.d.). Retrieved from http://education.illinoisstate.edu/grapevine/
Jaschik, S. (2015, March 6). Killing all state support. Washington, DC: Inside Higher Ed. Retrieved from https://www.insidehighered.com/news/2015/03/06/arizona-budget-dealwould-eliminate-all-state-funds-3-large-community-college
Jaschik, S., & Lederman, D. (2015). 2015 Inside Higher Ed survey of college and university chief academic officers. Washington, DC: Inside Higher Ed. Retrieved from https://www.insidehighered.com/news/survey/2015-survey-chief-academic-officers
Knight, W.E. (2014). Leadership and management in institutional research: Enhancing personal and professional effectiveness. Tallahassee, FL: Association for Institutional Research.
Leimer, C. (2012). Organizing for evidence-based decision making and improvement. Change: The Magazine of Higher Learning, 44 (4), 45-51.
Postsecondary Education Opportunity. (n.d.). Retrieved from http://www.postsecondary.org.
Terenzini, P. T. (2013). “On the nature of institutional research” revisited: Plus ça change . . . ? Research in Higher Education, 54(2), 137–148.
Terenzini, P. T. (1993). On the nature of institutional research and the knowledge and skills it requires. Research in Higher Education, 34(1), 1-10.
Volkwein, J.F., Liu, Y, & Woodell, J. (2012). The structure and function of institutional research offices. In R.D. Howard, G.W. McLaughlin, and W.E. Knight (Eds.), The handbook of institutional research (pp. 22-39). San Francisco: Jossey-Bass.