Our study “How is the Use of Performance Information Related to Performance of Public Sector Professionals? Evidence from the Field of Academic Research” has just been published in Public Performance & Management Review. In the article, we assert that there is inconclusive evidence as to how performance management is actually related to performance, particularly in subfields of the public sector where professional work prevails. We propose that the association between the use of performance information and performance of public sector professionals varies with the targets of management control. We test our hypotheses in the field of academic research, a prime example of professionalism in the public sector. The overall results of an online survey with 1,976 observations suggest that performance management is positively related to publication performance when performance information is used for the control of input targets. In contrast, we find negative associations of performance information with performance when used to control output targets. Public managers in professional fields should consider these countervailing relationships when they compose and use control systems.
Access the study here: Vogel, R. & Hattke, F. (2017): How Is the Use of Performance Information Related to Performance of Public Sector Professionals? Evidence from the Field of Academic Research. In: Public Performance & Management Review, 1-26.
The closing conference of the program “Performances de la recherche en sciences humaines et sociales” takes place at the University of Bern from 3 – 4 November 2016.
The conference addresses topics such as impact and quality in research in the humanities and the social sciences, excellence in research and societal usefulness, conveying quality of research in political areas and preserving diversity of research in times of excellence categories and rankings. The closing conference presents the results of the program and opens the discussion. It is the conference’s aim to bring together researchers, project leaders and important stakeholders of the Swiss higher education landscape.
- Prof. Dr. Shalini Randeria, Rector of the Institute for Human Sciences (IWM) in Vienna and Research Director and Professor of Social Anthropology and Sociology at the Graduate Institute of International and Development Studies (IHEID) in Geneva
- Prof. Dr. Peter Dahler-Larsen, Department of Political Science, University of Copenhagen, former President of European Evaluation Society
Please find the full conference program here. Registration is possible via the website.
A quick follow-up on the proliferation of excellent universities: A new piece by Samuel Moore and colleagues critically discusses the pervasive rhetoric of excellence in academia. The full pre-print paper is available here.
We’ve presented our study on “Performance Indicators in Academic Research”, asking “Do they Improve Performance?” at the 20th Annual Conference of the International Research Society for Public Management in Hong Kong. We had the chance to discuss our study in a PMRA-sponsored panel. Our empirical findings’ implications that “after a thorough selection of researchers, the best way to enhance academic performance is to grant them autonomy and to govern them by expertise rather than by performance appraisals” activated great consent. Besides the conference, the journey was an intense and enriching cultural experience. We’re looking forward to next year’s conference in Budapest.
As reported by the Korea Harold, South Korea is facing academic scandal. Prosecutors suspect 200 professors and several employees from academic publishers to be involved in a huge copyright-violation complot. Professors allegedly changed the covers of existing books which were authored by other scholars and published them in their own names. Most original authors seem to have had no idea what was going on, others are accused of having particpated in the fraud for financial compensations. Investigations suggest that scholars tried to boost their academic profiles ahead of rehiring assessments.
If accusations turn out to be true, it would raise serious concerns about the certain quality control mechanisms in scholarly publishing and HRM practices. It also seems to provide a rich case for studying the dynamics of corruption in academia.
On the 15th and 16th of July Alfred Kieser, Jetta Frost, Rick Vogel, Fabian Hattke, Jessica Petersen and me attended the “Governance, Performance & Leadership of Research and Public Organizations” symposium at the Bavarian Academy of the Sciences in Munich.
We heard several keynote addresses, inter alia of Margit Osterloh who gave a lecture about the shortcomings of current assessment procedures in academia and initiated a passionate discussion about this topic and John P.A. Ioannidis who illustrated paradoxes and room for improvement of research evaluations.
The parallel tracks concerned the central topics of the symposium (Governance, Performance & Leadership) and contained several interesting contributions. Rick Vogel, Alfred Kieser, Fabian Hattke and Jessica Petersen introduced the audience to the reasons of journals ranking-success. Jetta Frost, Fabian Hattke and me presented the audience how effects of performance measurement in academia can be captured through a multi-dimensional framework of scholars‘ organizational identification.
On the whole it was a really inspiring symposium with a nice composition of lecturers!
As reported by the Times Higher Education, the pressure on academics to raise external funding may cause a labor dispute. In September, scholars of Birmingham University will decide on whether they go on strike to fight against “the imposition of grant capture as a generic duty and disciplinary necessity”. While third party funding becomes more and more important in German universities, the UK is already more involved in actions related to “New Public Management”. Let’s hope Germany and other European countries can seize a “late mover’s advantage” and do not make the same mistake as the UK by putting too much economic pressures on scholars. Stefan Grimm’s tragic fate should remind us that there’s more to science than “bring your own buck”.
On the 3rd and 4th of June Jetta Frost, Fabian and me attended the 2015 EFMD Higher Education Research Conference in the Said Business School at the University of Oxford, United Kingdom. The Conference dealt with “The Legitimacy and Impact of Business Schools and Universities”.
The first day was opened with a keynote address of Jeffrey Pfeffer who shed light on current student culture in business schools in a slightly critical manner and reflected on the role and responsibility of business schools in society. The first parallel sessions dealt with reputation, rankings and the legitimacy of the business school and competition and change in higher education. They offered interesting perspectives on topics like the relations between states, universities and business schools, institutional change in the higher education system and competition. The second parallel sessions dealt with rankings, markets and performance in higher education and the legitimacy of Chinese universities and business schools. Fabian showed the audience how the effects of performance measurement in academia can be captured through a multiple-dimensional framework of scholars’ organizational identification and gained some positive and constructive feedback. The first day was concluded with a keynote address of Mats Alvesson who took the audience on a really entertaining trip through several paradoxes of the current higher education and business school system.
The second day was introduced with a keynote of Alis Oancea who presented the audience how the impact of research is related to several values in several societal spheres and how this relation can be systematized. The parallel sessions dealt with the meaning and significance of business school impact and the entrepreneurial activities and legitimacy of universities and business schools. They offered interesting perspectives on topics like the strengthening of higher education institutions legitimacy or the complexity of higher education institutions impact. At the end Andrew Pettigrew took leave in the name of the organisational team and named the conference “best EFMD-Conference so far”.
Beside the interesting content of the presentations we could enjoy the impressive atmosphere of Oxford. We visited many of the 38 colleges that constitute the University of Oxford. On wednesday we had our dinner in the hall at Balliol College, which is the oldest Oxford College that has a lot of famous absolvents, inter alia Adam Smith and a few prime ministers. On the whole it was a really nice conference in an inspiring environment. We are looking forward to next year’s EFMD Higher Education Research Conference!
The topic of the symposium is “Institutional Design Frontiers of Publicness and University Performance”.
Guest Editors are Derrick Anderson from Arizona State University and Andrew Whitford from University of Georgia.
“Issues of organizational theory and institutional design increasingly transcend the boundary between theory and practice in complex social enterprises especially higher education. For example, as the federal government advances a program to create a new college rating system, the stated ambition of making access to public resources contingent upon institutional performance a host of important administrative, policy and design considerations are manifest. Similarly, as new organizational forms emerge in the realm of higher education and new public policies aim to protect public investments therein, questions rise relative to the attributes of organizations that promote and stifle public value.
Symposium topics include: Institutional setting and “Dimensional publicness” in higher education; public, private and for-profit sector differences in higher education, public value assessments of university performance, and the evolution and future of institutional design of in higher education.
Insights from perspectives beyond public administration, including (but not limited to) economics, organizational studies, sociology and higher education are welcomed in as much as they sufficiently integrate consideration of relevant public administration and policy themes. A special emphasis is placed on empirical contributions, both qualitative and quantitative, but conceptual papers will be considered.
Manuscripts are due by June 15,2015 to the coordinating guest editor at email@example.com. After initial screening, authors of selected manuscripts will be invited to submit directly to PAR’s Editorial Manager for double-blind review. Final decisions will be made by the journal after full peer review. Authors should follow PAR’s style guidelines.”
On February 21st Fabian and me attended the 16th Workshop for Higher Education Management at the University Duisburg-Essen, Campus Duisburg.
Sadly we could only join the second part of the workshop. Nevertheless we had a nice Saturday with interesting presentations. The lectures dealt with choice for studies, diversity and learning software and performance measurement in science.
Michaela Schaffhauser-Linzatti introduced us to the reasons for university and academic subject choice at the business faculty of the University of Vienna and reflected about the possible managerial consequences of the results. Winfried Reiß and Helge Krusche from the University of Paderborn gave us an entertaining and interactive impression of the interactive learning system OViSS 3.0. Laura Graf held a presentation about an explorative study concerning performance and performance measurement in science which resulted in an inspiring discussion about qualitative and quantitative measurement of scholarly performance. Omar Adam Ayaita presented a fruitful approach of how to identify productivity patterns of researchers beyond the current methods. Christoph Biester presented us some unexpected insights into scholar’s perception of the performance-related payment in academia. Finally, we presented some first results of our current project about the effects of performance measurement on medical scholars’ organizational identification and gained some positive and constructive feedback.
We are looking forward to next years’ workshop!
On Thursday 10th and Friday 11th September 2015 the 8th Annual UK & Ireland Conference
of the higher Education Institutional Research Network takes place at the University of the West of Scotland, Paisley Campus PA1 2BE, Scotland, UK. The topic of the 2015 conference is: “The Stories We Tell: Using institutional research to enhance policy, practice and Engagement.”
Abstracts can be submitted until March 27th 2015.
For further Informations about the speakers, the different tracks dealing inter alia with “Performance and Impact” or “Institutional Management” and the submission formats take a look at http://www.uws.ac.uk/heir2015/#.VOg8ui4l-Wk.
The Institute of empirical and applied sociology (EMPAS) of the university bremen hosts a workshop dealing with the sociology of assessment.
In the light of the increasing relevance of assessment procedures in all societal spheres several sociological approaches analyze processes and practices of assessment and related phenomenons like categorizing, comparison and measurement. These approaches can be subsumed under the umbrella term sociology of assessment.
The Initiators of the Workshop want to bring together assessment-researchers of different social scientifical contexts. They aim at stimulating a discourse between researchers of different disciplines, expanding individual research perspectives and developing a comprehensive research-agenda for a sociology of assessment.
Researchers of different contexts are welcome to submit Proposals ( 2 pages) concerning practices of assessment, organisation and assessment, society and assessment or limitations of assessment. Proposals can be submitted until 15th April 2015. For further Informations take a look at the Call for Paper.
Peer reviewing is a widespread procedure for evaluating the quality of scholarly manuscripts before publication. As such, it is at the heart of academia. Yet, little is known how scholars percieve the peer-review process.
Today, we initiated our survey into the peer-review system from the perspective of those who submit papers to academic journals. The survey asks questions about general attitudes towards your job, your personal experiences with peer reviews, and possible alternatives to the common pre-publication peer review process. We’ll keep you posted on the results!
After finishing the first interviews for our Project “A conceptualization of scholarly performance in university hospitals and its impact on a pluralistic environment” it’s time to offer a brief glance at this project.
In the context of the increased importance of performance measurement in academia we focus on how scholarly performance and impact are constituted. We understand scholarly performance as a relativistic construct that is shaped by different demands placed on certain scholarly activities. Accordingly it can be assessed on different stages: Besides the understanding of scholarly performance and its impact that is shaped through current performance measurement methods stands scholars’ subjective understanding of their performance and its impact.
Higher education institutions where scholarly performance is embedded in a very complex environment are university hospitals. Scholars that are employed in a university hospital have to combine research and teaching activities with clinical activities. In addition they have to cope with demands of multiple stakeholders, like third party funders, ethical committees or their own professions. Due to this complexity the components of scholarly performance in university hospitals and the demands placed at these components are highly diverse.
We analyze how the understanding of scholarly performance and impact in university hospitals is constituted by and related to the current forms of performance measurement and the pluralistic demands that stakeholders place at scholarly activities. For this purpose we are currently conducting semi-structured interviews with scholars that are employed at a university hospital. We assess medical scholar’s subjective understanding of performance and its impact as well as the valuation-driven understanding that is shaped by the demands that current performance measurement methods place at scholarly activities. Central research questions are: How do perceived demands of multiple stakeholders and areas of activity shape the understanding of academic impact and its performance? Do demands of certain stakeholders become suppressed through an economic legitimization of activities? How do scholars cope with certain performance measurement instruments?
Our first interviews suggest interesting insights into unintended effects of performance measurement in academia. We`ll keep you updated!
The Network for Science Management (Netzwerk Wissenschaftsmanagement) invited us to participate in their annual meeting which took place in Munich last week. The network aims to support the professionalization of administration through regular exchange of ideas and experiences and has, thus, a strong practical focus. Together with Isabel Welpe, Jutta Wollersheim, and Stefanie Ringelhan from the Chair of Strategy and Organization at TU Munich, I was invited to talk about the governance of academic cooperation from a scholarly standpoint.
The research group from Munich held an impressive talk about quantitative assessments of research productivity and possible performance paradoxes, the differing intensity of collaboration between PhD students of economics and among PhD students of management, and the possibilities of new forms of collaboration (e.g. crowd research), publication (e.g. open peer review and open access) and scholarly communication (e.g. web 2.0 applications). If I might say so, their research looks very interesting and especially their new ventures promise relevant insights.
After their talk, I presented the results of two studies which are currently under review for publication (fingers crossed). The studies approach the emerging actorhood of universities from a governance- and a resource-perspective. As already reported earlier, governance reforms in the early 2000s have strenthened managerial mechanisms and delegated decision-competencies to the upper echelons of universities, the president, vice-presidents, and chancellors. So we asked ourselves, whether the composition of this “top management team” has any influence on how successful the university is in acquiring competitive funding for large collaborative projects? The results suggest that socio-demographic diversity of decision-makers has positive effects on performance in that regard. The second study investigates the increasing professionalization of administration in specialized central support units. Results indicate that performance is not always enhanced by these developments. In some cases, voluntary collective action seems to be more important than support from such units. However, we’re currently gathering additional data and running further tests on our models, so the final results may be more rubust and provide further clarity.
There was broad consensus about the implications of the first study. Some were reminded of Niklas Luhmann who stated that complexity can only be reduced by complexity – a nice thought, indeed. The results of the second study were, not surprisingly, seen a little bit more controversial. I’m grateful for this opportunity to discuss our research with practitioners in the field (many thanks to Dr. Brauns from the Thuringian Ministry of Education, Science, and Culture).
The body of available management literature has grown considerably during the past years. A search in Thompson Reuters’ ISI Web of Knowledge shows 19,143 articles published in the journals listed in the Journal Citation Reports for Busines and Management in 2013. Meaning 52.4 new articles were published per day – or one every 27.5 minutes. These numbers have almost doubled since 2003 when “only” 30.9 articles were published per day. And they are more than five times as much as 1993 (9.7 articles per day).
But how may we interpret these numbers? Is this increase a sign of scientific advancement, as treated by most rankings and performance-management systems? Or does it indicate a dysfunctional “inflation” of publications without further enlarging the scientific knowledge-base?
Either way, the vast majority of publications are rarely recognized by the scientific community (i.e. cited) at all. Of course, it’s not possible for scholars to keep up with the development of management science by reading all published articles. But considering this trend, we should ask ourselves wether there are more effective ways to communicate our scholarly results and to contribute to the development of new knowledge.
The IndiKon project aims to contribute to one of the most urging topics in the organization of science and higher education: how to conceptualize, ascertain, assess, and, ultimately, measure academic performance? The question gains more and more importance since universities around the globe increasingly apply performance management systems in order to allocate their scarce resources more effectively. At this year’s annual meeting of the Academy of Management, Herman Aguinis, Elena Antonacopoulou, and Debra Shapiro had a panel discussion in which they argued for a pluralistic approach to academic performance (see this article).
The widespread practice to confine scholarly impact to publications in peer-refered journals is a problem, indeed. Since inadequate performance management systems may cause counterproductive academic behavior and, consequently, attentuate scientific innovation, the design of such systems has to be grounded in thourough reasoning based on empirical evidence. We seek to provide some insights in that regard. During the next 3 years, we’ll try to develop novel qualitative and quantitative indicators for scientific innovation and assess possible cognitive and behavioral effects resulting from the deployment of performance management systems in organizations of higher education.
With some delay to Ferdinand’s report, but here they are: our impressions from the EGOS colloquium in Rotterdam, Netherlands. IndiKon’s Alfred Kieser (Zeppelin University) co-chaired the track “Universities in Unsettled Times: Effects of Evaluations, Accreditations and Rankings” together with Richard Whitley (University of Manchester) and Lars Engwall (Uppsala University). The sub-theme’s program covered a broad range of topics relating to current reforms in higher education systems around the world. We presented our paper “What makes journals highly ranked?” and received valuable comments from the other participants. The positive feedback encourages us to continue with this sub-project on a broader database and with new bibliometric indicators. Jessica Petersen (Zeppelin University) and Fabian Hattke (University of Hamburg) will join us in our efforts.
One of the most striking insights I had in the course of the three days was the following: Much is written about the impact of performance management systems in higher education on the professional identities of academics, and many authors argue that the new managerialism in universities is a serious threat to academic identities. However, as Richard Whitley and others argued, performance management systems may also strengthen the commitment of scholars to their professional communities. If performance measurement is based on rankings, elements of peer control within scientific communities become incorporated into output-based control systems of universities. Since rankings reflect the stated or revealed preferences of scholarly groups (i.e., invisible colleges), their professional standards gain in importance for, and are appreciated by, performance appraisals in universities (i.e., visible colleges). This, in turn, may even elevate the professional identities of scholars. For me, this is a nice thought worth examining empirically.
As you might have noticed, our blog has become somewhat quiet recently. But the hot summer is not the (only) reason.. During the past weeks, we’ve been busy crafting our papers for this year’s EGOS conference.
As you’ll know from your own experience, the title of a short paper may change significantly when it is developed towards a full paper. The story gets a new twist, is enlarged or further condensed, theoretical constructs may change, and conceptual frameworks are specified. And sometimes empirical observations just won’t tell the story you’ve expected in the first place..
So here’s the list of our short and full paper titles:
Short: Organizational responses to evaluations, rankings and performance indicators – Evidence from French and German universities. Full: Organizational Responses to Institutional Complexity – Evidence from French and German Universities.
Short: How Rankings Impede Scientific Progress. Full: What Makes Journals Highly Ranked? A Bibliometric Analysis of Management and Organization Studies.
Short: University commons: An empirical analysis of collective resources in German universities. Full: Organizing Collective Action? An Empirical Analysis of Common Goods in German Universities.