Research Summary
Guidance on Local Performance Indicators
Introduction
AMION Consulting, in association with KPMG and Pion Economics, were appointed by the Department for Transport, Local Government and the Regions (DTLR) to review the use of local Performance Indicators (PIs) in the Best Value Regime. Local PIs are defined as a form of measurement developed/adopted and reported by the local authority in order to assess and guide its performance in undertaking its functions and meeting local objectives.
The purpose of the research was to examine the nature and structure of local PIs developed by local authorities and, in particular, assess the degree to which they address cross-cutting issues and the needs and priorities of the local community. It also highlighted innovative practice and looked at any future lessons for the development of national BVPIs.
The research included detailed interviews with 28 local authorities and two workshop sessions. The case study authorities were in part selected because they had made significant progress in developing and implementing local PIs.
Key findings
The research has identified the critical role of local PIs in building an overall framework for performance management. Significant progress is being made by many of the case study authorities in the use of local PIs, although more needs to be done if their full potential is to be realised. Performance management in general is becoming well established within the fabric of many of the local authorities studied. Local PIs are forming an increasingly important component of the Performance Management Framework. However, consultees identified that further effort is still required in relation to issues, such as, local accountability, higher-level indicators, partner involvement and creating effective linkages between indicators at different levels in the hierarchy.
The findings of the research are as follows:
(i) HIERARCHY OF INDICATORS
- Hierarchy of indicators the bulk of local PIs are at the lower (management information and service indicator) end of the following hierarchy of indicators:
Community/partnership indicators;
Council corporate indicators;
Service indicators; and
Management information indicators.
Most authorities have started to build their hierarchy or framework from the bottom-up, although a few have developed from the top-down. Joining up the indicators within a framework covering all levels is important and may take many authorities up to three years to establish.
(ii) USE AND ROLE OF LOCAL PIS
Number and usage case study authorities had between 25 and 400 local PIs. On average, they had around 150, with most having over 100. In many cases, the number is reducing significantly through consolidation and refinement. The local PIs are developed for a wide range of purposes, not just as part of Best Value, although the Best Value Performance Indicators (BVPIs) and other statutory planning requirements (such as the Education Development Plan and Community Care Plan) play an important role in their development and usage. Amongst the case study authorities, local PIs exist for the vast majority of functions. Those service areas for which there are a significant number of BVPIs tend to have fewer local PIs.
Political and senior staff commitment political and senior staff commitment to performance management is a critical determinant in the use of local PIs. Overall, there was strong evidence of increasing commitment. However, higher priority is accorded to complying with nationally prescribed requirements. Member involvement is principally through scrutiny and reporting structures. The potential of sub-authority level local PIs to empower local members has been recognised by some authorities.
Cultural change organisational culture is another key factor determining the effective use of local PIs. Some resistance still exists in relation to performance management. Effort is required to reinforce the value of local PIs through, for example, departmental co-ordinators and the development of effective feedback arrangements.
Community accountability local PIs offer significant potential to strengthen local accountability. To date this potential has not been fully realised. The existing community engagement structures, including citizens panels, need to be used to develop effective involvement, although some capacity building may be required. Effective community engagement is both resource intensive and time consuming. There was some anecdotal evidence that people are more likely to be interested where local PIs are generated at a neighbourhood level. Few of the authorities were comfortable about the current level of community engagement and were attempting to do more.
Cross-cutting local PIs consultees highlighted the value of cross-cutting local PIs in terms of: informing progress towards corporate objectives, promoting increased partnership working and focusing on those factors that most influence a citizens quality of life. Most of the case study authorities have some cross-cutting local PIs. However, there was widespread recognition that significantly more needs to be done to develop these. Those service areas where partnership delivery structures have already become embedded have seen the greatest use of crosscutting local PIs. Cross-cutting local PIs need careful development, interpretation and analysis if they are to correctly inform decisions.
Setting of targets generally, targets are set out for the vast majority of local PIs. A number of authorities highlighted the need for realistic but challenging targets. In some cases, targets were identified as being inappropriate either because they were too hard, too soft or simply unrealistic. However, there was evidence of increasing scrutiny and review, which should result in more appropriate targets.
(iii) COVERAGE AND NATURE OF LOCAL PIS
Identifying a balanced set of indicators most local authorities are primarily focusing on output and to a lesser extent outcome indicators. Many of the authorities were using the DTLRs balanced set of five types of indicators to develop their suite of indicators. In developing local PIs that effectively measure progress towards achieving corporate and linked service objectives, authorities need to refer to a number of sources about what are the right indicators for a service. These include: members; staff; customers; benchmarking and other partners; and Government or national body standards. As corporate objectives are often not very precisely defined, developing local PIs to assess progress towards achieving these is difficult. Clearer objectives need to be defined and there is evidence that authorities are beginning to do this.
Coverage of specific groups most of the case study authorities have developed local PIs for specific communities of interest, although the extent of development does vary. On the whole, there were extensive differences between authorities in both attitude and practice towards the use of local PIs at the sub-authority level. A fragmented approach to sub-authority local PIs is likely to emerge. This may well be appropriate to local circumstances. However, it could contribute to uneven levels of accountability across authorities.
Verification of targets and indicators there are again differences in the mechanisms and protocols used to verify local PIs. Most systems are primarily reliant upon paper-based activity records, which are entered onto a database before being analysed and reported. Improvements in Management Information Systems (MISs) are being made by many of the case study authorities. Verification procedures vary from the relatively ad-hoc to the use of internal auditors to verify both systems and the robustness of indicators.
(iv) DEVELOPING AND COLLECTING LOCAL PIS
Sources of local PIs most service managers can develop local PIs at their discretion. However, community/partnership and council corporate indicators are being developed centrally. Best Value Reviews are an important driver for the development of local PIs and in monitoring the ongoing impact of the review and improvement plan. As performance management and planning frameworks become more formally established, corporate, business and service planning processes are likely to become the main source of local PIs.
Provision of training all case study authorities have a central team that provides support and assistance but few provide formal training in relation to local PIs. Many have short written guidance within, for example, Best Value Toolkits.
Involvement of partners in the process the need for greater levels of joint working are widely recognised, but significant constraints such as organisational boundaries and structures have severely limited the level of partner involvement in the development and use of local PIs. Partner engagement is greatest in areas where collaborative working is either required (health) or is the standard approach (regeneration).
Frequency of collection and reporting the most common reporting framework was quarterly, although some authorities are seeking to shorten the reporting cycle. The ability to do this is though heavily dependent upon the data collection process and MISs. The frequency of collection of local PIs is influenced by reporting and scrutiny arrangements, annual service planning Best Value Performance Plan (BVPP) requirements and the availability of data. The reporting arrangements will vary at different levels of the hierarchy.
(v) EFFECTIVENESS AND VALUE OF LOCAL PIS
The effectiveness of local PIs the vast majority of case study authorities recognised the potential contribution of local PIs to improving performance and service delivery. In some cases, the use of local PIs is not as effective as it should be due to: a lack of integration between different levels of the hierarchy of indicators, insufficient feedback loops, and occasionally a lack of senior commitment.
Value of local PIs case study interviewees were very positive about the value of good local PIs and believed, in the majority of cases, they warranted the resources necessary to collect and monitor them. The main benefits of local PIs were identified as:
measuring progress towards achieving stated local objectives;
making sure that local priorities are highlighted;
increasing community accountability;
service management and improvement;
increasing internal accountability;
informing policy decisions and resource allocation; and
promoting improved partner working.
Full value, however, will only be realised once the system had had time to bed in.
(vi) DISSEMINATION AND FEEDBACK
Methods of dissemination authorities use a variety of methods to disseminate local PIs internally and externally. Many identified the internet as a future tool. The format of many of the documents, especially the BVPP, are being modified to make them more user-friendly. The proportion of local PIs reported in the BVPPs varied across the authorities.
Feedback the feedback received from partners and the general public relating to local PIs and performance management is generally very limited.
(vii) PERFORMANCE MANAGEMENT FRAMEWORK AND BENCHMARKING
Performance Management Frameworks local PIs form an essential component of the performance management frameworks being used by the authorities.
Benchmarking many of the case study authorities use benchmarking groups/clubs but with variable results. Concerns were expressed about the robustness of some inter-authority local PI comparisons. A significant number of the authorities are to continue using a number of the Audit Commission Performance Indicators (ACPIs) and would like the Audit Commission to continue to collate and publish the information. Many of those interviewed are more interested in seeking out best practice examples rather than exchanging data between family groups.
(viii) FUTURE DEVELOPMENTS, LESSONS AND GUIDANCE
Future developments the importance of local PIs is expected to increase significantly. It is anticipated that there will be more: cross-cutting local PIs, involvement with partners and the community, and sub-authority or neighbourhood level local PIs. However, for many authorities the overall number will not increase due to the refinement of existing local PIs and removal of those which do not add value.
Constraints on their development the main constraints on the development of local PIs have been identified as: time and competing pressures, resources, initiative fatigue, inadequate MISs, lack of political/senior officer commitment, cultural resistance and the lack of stability within the national Best Value framework and, in particular, the BVPIs.
Wider lessons for Central Government greater co-ordination and integration between Government departments and agencies is required and a period of greater stability is needed in the Best Value framework. In addition, Government could help to collate information from partners. A single web-site should be created for PIs.
Guidance and support on local PIs Short and accessible guidance should be provided setting out practical examples. The guidance should be made available and disseminated in a number of ways, including on the internet.
This research has highlighted the substantial potential value of local PIs and found that significant progress is being made in their use. It has identified a number of factors that are important in the effective use of local PIs, as follows:
- political and senior staff commitment;
- cultural and structural change;
- community, partner and service user involvement;
- appropriate systems to record and verify indicators;
- provision of training and support;
- effective benchmarking;
- internal communication and ownership;
- careful interpretation and analysis; and
- an integrated hierarchy or framework that links (local and national) indicators at different levels and that effectively informs policy development, management decisions and resource allocation.
© Crown copyright 2001
Further copies of this document are available from:
DTLR Free Literature
PO Box 236
Wetherby
West Yorkshire
LS23 7NB
Tel: 0870 1226 236
Fax: 0870 1226 237
Textphone: 0870 1207 405
Email: dtlr@twoten.press.net
[ Previous ] [ Contents ] [ Next ]
Published 11 December 2001
Return to Perfomance Indicators Index
Return to Best Value Index
Return to Local Government Index
Return to DTLR Homepage
Web site terms