| Peer-Reviewed

Case Study Evaluation of Organisational Performance Measurement Implementation in a Non-acute Health Charity

Received: 13 July 2022    Accepted: 6 August 2022    Published: 17 August 2022
Views:       Downloads:
Abstract

There is community and stakeholder expectation that health charities should be well governed and held accountable. Non-acute health charities are both not-for-profit organisations and health service providers. Organisational Performance Measurement (OPM) is recognised as being a key instrument to enable success and even survival in the modern business world, yet it is under-utilised by non-acute health charities. NCPI Framework was developed to encourage OPM uptake by the sector. A case study evaluation using convergent parallel mixed methods research design evaluated the effectiveness of an NCPI Framework informed twelve-month implementation plan to introduce OPM in a non-acute health charity. Measures were put in place to manage risks of consistency, replicability and bias in using a case study method. Pre and post quantitative (74% and 64% response rates) and qualitative research (24% response rate) were utilised as part of the case study evaluation. The qualitative and quantitative findings complimented each other, and the qualitative data provided further insights into participant perspectives. The quantitative data results allow for the study’s hypothesis to be accepted and the null hypothesis to be confidently rejected. The implementation of the twelve-month OPM implementation plan informed by the NCPI Framework positively impacted the introduction of OPM to the case study organisation. Further, the study results conclusively demonstrate a significant improvement in the utility and usability of OPM in the case study organisation. This is the first evaluation of its kind for this sector. The non-acute health charities sector under-utilises or under-reports OPM and prior to this study an evidence-informed method for OPM implementation has not existed for the sector. The NCPI Framework is the first OPM implementation tool for the sector and was found to have positively impacted the introduction of OPM to the case study organisation and was found to be useful in terms of both utility (amount of user satisfaction) and usability (ease of the system’s functionality). The Framework could now be used by Boards and executive leaders in the sector to enhance their organisation’s governance standards, accountability and performance.

Published in Journal of Public Policy and Administration (Volume 6, Issue 3)
DOI 10.11648/j.jppa.20220603.14
Page(s) 139-150
Creative Commons

This is an Open Access article, distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium or format, provided the original work is properly cited.

Copyright

Copyright © The Author(s), 2024. Published by Science Publishing Group

Keywords

Charity, Not-for-Profit, Health, Performance, Organisation Performance

References
[1] Jones, G. R. and Jones, G. R., 2013. Organizational theory, design, and change (pp. 31-33). Boston: Pearson.
[2] Griseri, P., 2013. An introduction to the philosophy of management. Sage.
[3] Archambault, E., 2017. The evolution of public service provision by the third sector in France. The Political Quarterly, 88 (3), pp. 465-472.
[4] Hyndman, N. and McConville, D., 2018. Trust and accountability in UK charities: Exploring the virtuous circle. The British Accounting Review, 50 (2), pp. 227-237.
[5] Schmitz, J., 2021. Is charitable giving a zero-sum game? The effect of competition between charities on giving behavior. Management Science, 67 (10), pp. 6333-6349.
[6] Yang, Y., Brennan, I. and Wilkinson, M., 2016. Value similarity: the key to building public trust in charitable organisations. Voluntary Sector Review, 7 (1), pp. 47-66.
[7] Mueller, J., 2007. When doing good is just the start to being good: a possible tool to improve the organizational effectiveness of non-profit health care organizations. Journal of Hospital Marketing & Public Relations, 17 (2), pp. 45-60.
[8] Lecy, J. D., Schmitz, H. P. and Swedlund, H., 2012. Non-governmental and not-for-profit organizational effectiveness: A modern synthesis. Voluntas: International Journal of Voluntary and Nonprofit Organizations, 23 (2), pp. 434-457.
[9] Kaplan, R. S., 2001. Strategic performance measurement and management in nonprofit organizations. Nonprofit management and Leadership, 11 (3), pp. 353-370.
[10] Dickerson, C., Grills, N., Henwood, N., Jeffreys, S. and Lankester, T., 2012. The World Health Organization Engaging with Civil Society Networks to Promote Primary Health Care: A Case Study. Global Health Governance, 6 (1).
[11] Department of Health and Human Services, State Government of Victoria 19 [DHS]. Maintenance Care. 2020. Retrieved March 15, 2020, from 20 https://www2.health.vic.gov.au/hospitals-and-health-services/patient21care/rehabilitation-complex-care/maintenance-care/22
[12] Colbran, R., Ramsden, R., Stagnitti, K. and Toumbourou, J. W., 2019. Advancing towards contemporary practice: a systematic review of organisational performance measures for non-acute health charities. BMC health services research, 19 (1), pp. 1-12.
[13] Myburgh, C., Brandborg-Olsen, D., Albert, H. and Hestbaek, L., 2013. The Nordic maintenance care program: what is maintenance care? Interview based survey of Danish chiropractors. Chiropractic & manual therapies, 21 (1), pp. 1-9.
[14] Park, Y. J. and Peng, S., 2020. Advancing public health through tax-exempt hospitals: Nonprofits’ revenue streams and provision of collective goods. Nonprofit and Voluntary Sector Quarterly, 49 (2), pp. 357-379.
[15] Hung, C. and Hager, M. A., 2019. The impact of revenue diversification on nonprofit financial health: A meta-analysis. Nonprofit and voluntary sector quarterly, 48 (1), pp. 5-27.
[16] Hunter, J. J., 2017. A Systems Approach to Revenue Fluctuation in Nonprofit Human Service Organizations.
[17] Soysa, I. B., Jayamaha, N. P. and Grigg, N. P., 2016. Operationalising performance measurement dimensions for the Australasian nonprofit healthcare sector. The TQM Journal.
[18] Hardwick, R., Anderson, R. and Cooper, C., 2015. How do third sector organisations use research and other knowledge? A systematic scoping review. Implementation Science, 10 (1), pp. 1-12.
[19] Colbran, R., Ramsden, R., Stagnitti, K., Toumbourou, J. and Pepin, G., 2021. A Framework to Implement Organisational Performance Measurement in Health Charities. Journal of Public Policy and Administration, 5 (1), p. 13.
[20] Jones, T. M., Harrison, J. S. and Felps, W., 2018. How applying instrumental stakeholder theory can provide sustainable competitive advantage. Academy of Management Review, 43 (3), pp. 371-391.
[21] León, C., & Bousquet, B. (2018). A Performance Measurement Framework for NPOs. 2018 IISE Annual Conference. Retrieved October 25, 2019, from https://www.researchgate.net/profile/Hilda_Martinez_Leon/publication/325581735_A_Performance_Measurement_Framework_for_NPOs/links/5bc67a50299bf17a1c55d563/A-Performance-Measurement-Framework-for-NPOs.pdf
[22] Sen, D., Bingol, S. and Vayvay, O., 2017. Strategic enterprise management for innovative companies: The last decade of the balanced scorecard. International Journal of Asian Social Science, 7 (1), pp. 97-109.
[23] Nalwoga, M. M. and Dijk, M. P. V., 2016. Organisational performance measurement models, also for poverty alleviation. International Journal of Water, 10 (2-3), pp. 122-138.
[24] Carneiro-da-Cunha, J. A., Hourneaux Jr, F. and Corrêa, H. L., 2016. Evolution and chronology of the organisational performance measurement field. International Journal of Business Performance Management, 17 (2), pp. 223-240.
[25] Odor, H. O., 2019. A literature review on organisational learning and learning organisations. International Journal of Information, Business and Management, 11 (3), pp. 281-295.
[26] Guta, A. L. (2015). An analysis of factors that influence organisational learning: The case of higher education institutions. 15th International Academic Conference, Rome. doi: 10. 20472/IAC. 2015. 015. 067. https://www.iises.net/proceedings/international-academic-conferencerome/table-of-content?cid=10&iid=067&rid=3387
[27] Argote, L., 2012. Organizational learning: Creating, retaining and transferring knowledge. Springer Science & Business Media.
[28] Jyoti, J. and Rani, A., 2017. High performance work system and organisational performance: Role of knowledge management. Personnel Review.
[29] García-Morales, V. J., Jiménez-Barrionuevo, M. M. and Gutiérrez-Gutiérrez, L., 2012. Transformational leadership influence on organizational performance through organizational learning and innovation. Journal of business research, 65 (7), pp. 1040-1050.
[30] Grigoroudis, E., Orfanoudaki, E. and Zopounidis, C., 2012. Strategic performance measurement in a healthcare organisation: A multiple criteria approach based on balanced scorecard. Omega, 40 (1), pp. 104-119.
[31] Aboramadan, M. and Borgonovi, E., 2016. Strategic management practices as a key determinant of superior non-governmental organizations performance. Problems of management in the 21st century, 11 (2), p. 71.
[32] Richard, P. J., Devinney, T. M., Yip, G. S. and Johnson, G., 2009. Measuring organizational performance: Towards methodological best practice. Journal of management, 35 (3), pp. 718-804.
[33] Bititci, U. S., 2015. Managing business performance: The science and the art. John Wiley & Sons.
[34] Seaman, B. A. and Young, D. R. eds., 2018. Handbook of research on nonprofit economics and management. Edward Elgar Publishing.
[35] Kirk, G. and Beth Nolan, S., 2010. Nonprofit mission statement focus and financial performance. Nonprofit management and Leadership, 20 (4), pp. 473-490.
[36] Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P., & Kyriakidou, O. (2004). Diffusion of innovations in service organizations: systematic review and recommendations. The Milbank Quarterly, 82 (4), 5810–629.
[37] Kareithi, R. N. and Lund, C., 2012. Review of NGO performance research published in academic journals between 1996 and 2008. South African Journal of Science, 108 (11), pp. 1-8.
[38] Creswell, J. W., Klassen, A. C., Plano Clark, V. L. and Smith, K. C., 2011. Best practices for mixed methods research in the health sciences. Bethesda (Maryland): National Institutes of Health, 2013, pp. 541-545.
[39] Kaushik, V. and Walsh, C. A., 2019. Pragmatism as a research paradigm and its implications for social work research. Social sciences, 8 (9), p. 255.
[40] Creswell, J. W. and Clark, V. L. P., 2017. Designing and conducting mixed methods research. Sage publications.
[41] Creswell, J. (2013, November 14). Steps in Conducting a Scholarly Mixed Methods Study. In J. Creswell, Steps in Conducting a Scholarly Mixed Methods Study. DBER Speaker Series. 48. Retrieved October 28, 2017, from http://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1047&context=dber speakers
[42] Yazan, B., 2015. Three approaches to case study methods in education: Yin, Merriam, and Stake. The qualitative report, 20 (2), pp. 134-152.
[43] Yin, R. K., 2018. Case study research and applications. Sage.
[44] Tarkkanen, K., Harkke, V. and Reijonen, P., 2015. Are we testing utility? Analysis of usability problem types. In Design, User Experience, and Usability: Design Discourse (pp. 269-280). Springer, Cham.
[45] Gervais, M., 2008. A journey through five evaluation projects with the same analysis framework. The Canadian Journal of Program Evaluation, 23 (2), p. 165.
[46] Serry, T. and Liamputtong, P., 2013. The in-depth interviewing method in health. Research methods in health: Foundations for evidence-based practice, pp. 39-53.
[47] Giorgi, A., 2009. The descriptive phenomenological method in psychology: A modified Husserlian approach. Duquesne University Press.
[48] Austin, Z. and Sutton, J., 2014. Qualitative research: Getting started. The Canadian journal of hospital pharmacy, 67 (6), p. 436.
[49] Coffey, A. and Atkinson, P., 1996. Making sense of qualitative data: Complementary research strategies. Sage Publications, Inc.
[50] Hycner, R. H., 1985. Some guidelines for the phenomenological analysis of interview data. Human studies, 8 (3), pp. 279-303.
[51] Groenewald, T., 2004. A phenomenological research design illustrated. International journal of qualitative methods, 3 (1), pp. 42-55.
[52] Bowden, C. and Galindo-Gonzalez, S., 2015. Interviewing when you’re not face-to-face: The use of email interviews in a phenomenological study. International Journal of Doctoral Studies, 10 (12), pp. 79-92.
[53] Yüksel, P. and Yıldırım, S., 2015. Theoretical frameworks, methods, and procedures for conducting phenomenological studies in educational settings. Turkish online journal of qualitative inquiry, 6 (1), pp. 1-20.
[54] Elo, S. and Kyngäs, H., 2008. The qualitative content analysis process. Journal of advanced nursing, 62 (1), pp. 107-115.
[55] Tweedie, D., 2016. Not-for-profit accountability: Addressing potential barriers. THE THREE SECTOR SOLUTION, p. 215.
[56] Chelliah, J., Boersma, M. and Klettner, A., 2015, January. Governance challenges for not-for-profit organisations: Empirical evidence in support of a contingency approach. In Australasian Conference on Business and Social Sciences 2015, Sydney.
[57] Niven, P. R., 2008. Balanced scorecard: Step-by-step for government and nonprofit agencies. John Wiley & Sons.
[58] Macmillan, R., Ellis Paine, A., Kara, H., Dayson, C., Sanderson, E., & Wells, P. (2014) Building capabilities in the voluntary sector: What the evidence tells us. TSRC Research Report 125, Birmingham: TSRC. Retrieved October 10, 2018, from https://www.birmingham.ac.uk/Documents/college-socialsciences/social-policy/tsrc/reports/research-report-125-buildingcapabilities.pdf
[59] Chavan, M., 2009. The balanced scorecard: a new challenge. Journal of management development.
[60] Goldman, G. A., Nienaber, H. and Pretorius, M., 2015. The essence of the contemporary business organisation: A critical reflection.
[61] Som, H. M., Saludin, M. N., Shuib, M. S., Keling, M. F., Narsquo, M. and Nam, Y. T., 2010. Learning organization elements as determinants of organizational performance of non-profit organizations (NPOs) in Singapore. International NGO Journal, 5 (5), pp. 116-127.
[62] Lipshitz, R., Friedman, V. and Popper, M., 2006. Demystifying organizational learning. Sage Publications.
[63] Ricciardi, F., Cantino, V. and Rossignoli, C., 2021. Organisational learning for the common good: an emerging model. Knowledge Management Research & Practice, 19 (3), pp. 277-290.
[64] Nilsen, P., 2020. Making sense of implementation theories, models, and frameworks. In Implementation Science 3. 0 (pp. 53-79). Springer, Cham.
[65] Zimmerman, J. 2009. Using a balanced scorecard in a nonprofit organization. Nonprofit World, 27, 10–12.
Cite This Article
  • APA Style

    Richard Colbran, Robyn Ramsden, Genevieve Pepin, John Toumbourou, Karen Stagnitti. (2022). Case Study Evaluation of Organisational Performance Measurement Implementation in a Non-acute Health Charity. Journal of Public Policy and Administration, 6(3), 139-150. https://doi.org/10.11648/j.jppa.20220603.14

    Copy | Download

    ACS Style

    Richard Colbran; Robyn Ramsden; Genevieve Pepin; John Toumbourou; Karen Stagnitti. Case Study Evaluation of Organisational Performance Measurement Implementation in a Non-acute Health Charity. J. Public Policy Adm. 2022, 6(3), 139-150. doi: 10.11648/j.jppa.20220603.14

    Copy | Download

    AMA Style

    Richard Colbran, Robyn Ramsden, Genevieve Pepin, John Toumbourou, Karen Stagnitti. Case Study Evaluation of Organisational Performance Measurement Implementation in a Non-acute Health Charity. J Public Policy Adm. 2022;6(3):139-150. doi: 10.11648/j.jppa.20220603.14

    Copy | Download

  • @article{10.11648/j.jppa.20220603.14,
      author = {Richard Colbran and Robyn Ramsden and Genevieve Pepin and John Toumbourou and Karen Stagnitti},
      title = {Case Study Evaluation of Organisational Performance Measurement Implementation in a Non-acute Health Charity},
      journal = {Journal of Public Policy and Administration},
      volume = {6},
      number = {3},
      pages = {139-150},
      doi = {10.11648/j.jppa.20220603.14},
      url = {https://doi.org/10.11648/j.jppa.20220603.14},
      eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.jppa.20220603.14},
      abstract = {There is community and stakeholder expectation that health charities should be well governed and held accountable. Non-acute health charities are both not-for-profit organisations and health service providers. Organisational Performance Measurement (OPM) is recognised as being a key instrument to enable success and even survival in the modern business world, yet it is under-utilised by non-acute health charities. NCPI Framework was developed to encourage OPM uptake by the sector. A case study evaluation using convergent parallel mixed methods research design evaluated the effectiveness of an NCPI Framework informed twelve-month implementation plan to introduce OPM in a non-acute health charity. Measures were put in place to manage risks of consistency, replicability and bias in using a case study method. Pre and post quantitative (74% and 64% response rates) and qualitative research (24% response rate) were utilised as part of the case study evaluation. The qualitative and quantitative findings complimented each other, and the qualitative data provided further insights into participant perspectives. The quantitative data results allow for the study’s hypothesis to be accepted and the null hypothesis to be confidently rejected. The implementation of the twelve-month OPM implementation plan informed by the NCPI Framework positively impacted the introduction of OPM to the case study organisation. Further, the study results conclusively demonstrate a significant improvement in the utility and usability of OPM in the case study organisation. This is the first evaluation of its kind for this sector. The non-acute health charities sector under-utilises or under-reports OPM and prior to this study an evidence-informed method for OPM implementation has not existed for the sector. The NCPI Framework is the first OPM implementation tool for the sector and was found to have positively impacted the introduction of OPM to the case study organisation and was found to be useful in terms of both utility (amount of user satisfaction) and usability (ease of the system’s functionality). The Framework could now be used by Boards and executive leaders in the sector to enhance their organisation’s governance standards, accountability and performance.},
     year = {2022}
    }
    

    Copy | Download

  • TY  - JOUR
    T1  - Case Study Evaluation of Organisational Performance Measurement Implementation in a Non-acute Health Charity
    AU  - Richard Colbran
    AU  - Robyn Ramsden
    AU  - Genevieve Pepin
    AU  - John Toumbourou
    AU  - Karen Stagnitti
    Y1  - 2022/08/17
    PY  - 2022
    N1  - https://doi.org/10.11648/j.jppa.20220603.14
    DO  - 10.11648/j.jppa.20220603.14
    T2  - Journal of Public Policy and Administration
    JF  - Journal of Public Policy and Administration
    JO  - Journal of Public Policy and Administration
    SP  - 139
    EP  - 150
    PB  - Science Publishing Group
    SN  - 2640-2696
    UR  - https://doi.org/10.11648/j.jppa.20220603.14
    AB  - There is community and stakeholder expectation that health charities should be well governed and held accountable. Non-acute health charities are both not-for-profit organisations and health service providers. Organisational Performance Measurement (OPM) is recognised as being a key instrument to enable success and even survival in the modern business world, yet it is under-utilised by non-acute health charities. NCPI Framework was developed to encourage OPM uptake by the sector. A case study evaluation using convergent parallel mixed methods research design evaluated the effectiveness of an NCPI Framework informed twelve-month implementation plan to introduce OPM in a non-acute health charity. Measures were put in place to manage risks of consistency, replicability and bias in using a case study method. Pre and post quantitative (74% and 64% response rates) and qualitative research (24% response rate) were utilised as part of the case study evaluation. The qualitative and quantitative findings complimented each other, and the qualitative data provided further insights into participant perspectives. The quantitative data results allow for the study’s hypothesis to be accepted and the null hypothesis to be confidently rejected. The implementation of the twelve-month OPM implementation plan informed by the NCPI Framework positively impacted the introduction of OPM to the case study organisation. Further, the study results conclusively demonstrate a significant improvement in the utility and usability of OPM in the case study organisation. This is the first evaluation of its kind for this sector. The non-acute health charities sector under-utilises or under-reports OPM and prior to this study an evidence-informed method for OPM implementation has not existed for the sector. The NCPI Framework is the first OPM implementation tool for the sector and was found to have positively impacted the introduction of OPM to the case study organisation and was found to be useful in terms of both utility (amount of user satisfaction) and usability (ease of the system’s functionality). The Framework could now be used by Boards and executive leaders in the sector to enhance their organisation’s governance standards, accountability and performance.
    VL  - 6
    IS  - 3
    ER  - 

    Copy | Download

Author Information
  • New South Wales Rural Doctors Network, New South Wales (NSW), Australia

  • School of Health and Social Development, Deakin University, Victoria (VIC), Australia

  • School of Health and Social Development, Deakin University, Victoria (VIC), Australia

  • School of Psychology, Deakin University, Victoria (VIC), Australia

  • School of Health and Social Development, Deakin University, Victoria (VIC), Australia

  • Sections