4. Evaluation Design and Methodology
This Document is Archived
Evaluation of the Industry Capability Network New Zealand: Final Report
Research, Evaluation and Monitoring Team - Industry and Regional Development Branch
[ Last Updated 2 November 2005 ]
Limitations for Evaluation Design Options
Reliance on Recipients' Views of the Added Value
Correlation Analysis
Use of a "Non-User" Supplier Group
Data Collection
Document and File Review
Interviews
Programme Participants in the Government Procurement Market
Programme Participants in New Zealand Major Projects
Knowledgeable Stakeholders
Surveys
Content of the Surveys
Pilot Study of Surveys
Survey Procedure
Limitations of the Data
Interview Data
Survey Data
Correlation Analysis
This section summarises the methods and data sources used by the review team to reach evaluation results. The evaluation design and methodology impact on the confidence
policy makers can have in the conclusions. This section outlines some of the issues to consider when interpreting the results.
The review team employed differing evaluation designs for the intervention of I CN in the government procurement market and in major projects. This was due to the different
lengths of time I CN had been delivering to each target market, the information available, and practical considerations such as time and money.
As set out in the introduction (Section 2), the evaluation focuses on the added value of I CN, over and above what would otherwise have occurred. It focuses on the
intermediate outcomes (as identified in the intervention logic model) for New Zealand programme participants. These focal points for the evaluation steer the choice of
methodology.
Limitations for Evaluation Design Options
Design options to produce an estimate of the added value of I CN were limited by practical considerations of time and money, as well as the availability of information and the
potential burden to programme participants. In...