ORGAPET Section C3:
Generic Indicators

Nic Lampkin and Ian Jeffreys
Aberystwyth University, UK

Version 6, April 2008

C3-1 Introduction

The methods for defining objectives and selecting indicators for specific evaluations are considered in ORGAPET Section C1 and Section C2 respectively. The aim in this section is to identify generic primary indicators and data sources that can be used for all evaluations and to present examples of optional or secondary indicators that might be used in specific contexts, depending on the nature of the action plan.

The list of indicators presented here represents the outcome of previous work in the ORGAP, EU-CEE-OFP and other projects where the selection steps outlined previously, including stakeholder participation, have been followed. As part of the ORGAP project, a series of national workshops were conducted in April 2006 to identify priority objectives and possible indicators to measure progress. These workshops were designed to be analogous to evaluation steering group meetings that might be implemented in order to plan an evaluation (see ORGAPET Section A5). The problem discussed in ORGAPET Section C2 concerning the overloading of indicators is well illustrated by the list of indicators proposed in the workshops, with simplification made more difficult by the very limited overlap in ideas between the different workshops. In addition, some proposals were not sufficiently well defined to be usable as indicators. The outcomes of the national workshops (Annex C3-5), supplemented by other sources, influenced the choice of a draft list generic indicators, which were then subject to further testing in reviews of ORGAPET by national experts in the context of national action plans (Dabbert and Eichert, 2007). A revised list of indicators was then subjected to quality assessment in the context of the EU organic action plan (Annex C2-4) and further revised. To simplify presentation, only the final results are of the development process are presented here.

The generic indicator list is not intended to be applied rigidly to every action plan evaluation, but to support the process of indicator definition outlined in Sections C1 and C2. For any particular action plan situation, some of the indicators listed may not be relevant to the specific action points or objectives, or suitable data may not be available making the indicator unusable. In addition, there are likely to be action points or objectives that are not covered by the generic indicators, so that additional indicators will have to be identified.

C3-2 Generic indicators for action plan evaluation

The classification of indicators used here is that outlined in ORGAPET Section C2, which is an adaptation of the MEANS/Evalsed classification. In the following tables, a distinction is also made between primary indicators, for which data should be readily available and which could be used in every evaluation, and secondary (optional) indicators where supplementary information might need to be collected.

In addition to the indicator identifier and description, each of the following tables contains five columns covering:

Relevance: this column concerns the relevance of the indicator to the specific generic objectives (GO) identified in ORGAPET Section C1, and the individual EU organic action plan (EU) action points.
Data sources: this column identifies the main sources of data with respect to the indicator concerned. Where research is indicated as data source, examples of recent studies may be given. Alternatively, Organic E-prints is a database containing a significant number of research outputs that might be helpful. Where necessary further information on sourcing data, and in some cases examples of available data, can be found in the methodology and data fact sheet (MDFS) identified in the Method column.
Method: this column identifies the method by which the indicator can be measured or calculated. In many cases, a methodology and data fact sheet (MDFS) provides more detailed information on methodological issues, as well as guidance on indicator and data quality, data sources and examples of available data. In some cases methods are still under development as part of current EU-funded research programmes - it is envisaged that relevant MDFS documents will be added in future updates.
Criteria: this column identifies suggested performance criteria that can be used to link the indicator result to whether or not a particular objective has been achieved (pass/fail), or how well an objective has been achieved (score). In some cases, further details on criteria may be included in the methodology and data fact sheet (MDFS).
Context indicators: this column suggests relevant context or programme indicators to assist in interpreting the indicator results. In some cases, further details on criteria may be included in the methodology and data fact sheet (MDFS).

C3-2.1 Primary indicators

A: Primary generic programme design indicators for action plan evaluation

Indicator		Relevance	Data sources	Method	Criteria	Context indicators
A1	Programme content	GO: All EU: All	Programme documents (e.g. Annexes A1-1, A1-2)	Text	N/A	N/A
A2	Programme design scoring (secondary indicators)	GO: All EU: All	A2i-A2v indicators	Sum of scores of secondary indicators	Highest: 15 Lowest: 0	Similar programmes
A3	Programme design qualitative assessment	GO: All EU: All	Documents, interviews	ORGAPET Section B1	High: 3 None: 0	Similar programmes
A4	Stakeholder engagement (qualitative assessment)	GO: All EU: All	Documents, interviews	ORGAPET Section B3	High: 3 None: 0	Similar programmes

B: Primary generic resource/implementation indicators for action plan evaluation

Indicator		Relevance	Data sources	Method	Criteria	Context indicators
B1	Budgeted/planned expenditure for individual actions or plan in total	GO: All EU: All	Documents, interviews	Annual values	Increasing/ target organic share of total agricultural support	Total RDP or departmental expenditure
B2	Number of staff months allocated to implementation for individual actions or plan in total	GO: All EU: All	Documents, interviews	Annual values	Staffing available cf. required	Total RDP or departmental staffing
B3	Legal framework for programme	GO: All EU: All	Documents	Text	Yes/No	Similar programmes
B4	Monitoring/evaluation implemented from start of programme	GO: All EU: All	Documents, interviews	Text	Yes/No	Similar programmes
B5	Stakeholder engagement (qualitative assessment)	GO: All EU: All	Documents, interviews	ORGAPET Section B3	High: 3 None: 0	Similar programmes

C: Primary generic output indicators for action plan evaluation

Indicator		Relevance	Data sources	Method	Criteria	Context indicators
C1a	Number (or proportion) of action points achieved/ completed with description	GO: All EU: All	Documents, interviews	Count, description (e.g. Annex A1-3)	Target 100%	Similar programmes
C1b	Number (or proportion) of action points started/ in progress with description	GO: All EU: All	Documents, interviews	Count, description (e.g. Annex A1-3)	Target 100%	Similar programmes
C2	Actual expenditure on individual actions or plan in total	GO: All EU: All	Monitoring data	Values	≥ planned	Similar programmes
C3	Area/ holdings/ businesses/ people/ animals/ projects/ events supported by individual action points (secondary indicators)	GO: All EU: 1, 3, 4, 6, 7	Monitoring data, Annex C3-4	MDFS C3	≥ specific targets	Total RDP or relevant measure expenditure
C4	Availability of statistical data to meet business and policy evaluation needs by topic/indicator	GO: All EU: All	Monitoring data, research, EISfOM	Scoring	High: 3 None: 0	Availability of similar data for all agriculture

D: Primary generic result indicators for action plan evaluation

Indicator		Relevance	Data sources	Method	Criteria	Context indicators
D1	Number of certified organic and in-conversion holdings (secondary indicators)	GO: All EU: All	EU control system, Eurostat, Annex C3-4	MDFS D1, Annex C3-1	Target % or increasing faster than prior trend	Total number of agricultural holdings
D2	Area of certified organic and in-conversion land (secondary indicators)	GO: All EU: All	EU control system, Eurostat, Annex C3-4	MDFS D2, Annex C3-1	Target % or increasing faster than prior trend	Total utilized agricultural area (UAA)
D3	Organic farm incomes (secondary indicators)	GO: 1-4, 10, 11 EU: All	FADN (EU or national)	MDFS D3a Annex C3-2	Increasing, ≥ non-organic	Overall farm incomes
D4	Number of certified market operators (secondary indicators)	GO: 1-4, 10, 11 EU: All	EU control system, Eurostat	MDFS D4	Increasing, target %	Total number of market operators
D5	Organic market size (retail sales value and/or volume) by region (secondary indicators)	GO: 1-4, 10, 11 EU: All	Market research (e.g. OMIaRD), Annex C3-4	MDFS D5a, Annex C3-3	Increasing share/ target % of overall food markett	Total retail sales value/ volume
D6	Consumer confidence and trust (secondary indicators)	GO: 3-6 EU: 1, 8-21	Consumer survey	MDFS D6	High: 3, None: 0	All consumers
D7	Business confidence (secondary indicators and indicator D11iii)	GO: 1-4, 10, 11 EU: All	Business survey	MDFS D7	High: 3, None: 0	All businesses
D8	Number of control organizations (secondary indicators)	GO: 1-5 EU: 9-21	EU control system, organisation list	MDFS D8	Increasing or decreasing – depends on perspective	Similar non-organic control bodies
D9	Number of inspection visits (secondary indicators)	GO: 2-5 EU: 9-21	EU control system, annual report	MDFS D9	Increasing or decreasing if risk-based perspective	Number of operators
D10	Number/frequency of revisions to key regulations (secondary indicators)	GO: 3-5 EU: 2, 8-21	Documents	EU Reg. 834/2007 (consolidated version)	Fewer changes	Number of revisions to EU Reg. 2092/91
D11	Regulatory burden on businesses (secondary indicators)	GO: 1, 4, 5, 11, 12 EU: 2, 9-21	Indicators D11i-D11iii	Scoring, MDFS D11	High: 3 None: 0	Regulatory burden for non-organic
D12	Number of research and extension organisations supporting organic food and farming (secondary indicators)	GO: All EU: 1, 3, 6, 7, 9, 11, 14	Organisation databases (e.g. organic-europe.net)	Count	Increasing	Total number of similar institutions for all agriculture

E: Primary generic impact indicators for action plan evaluation

Indicator		Relevance	Data sources	Method	Criteria	Context indicators
E1	Overall environmental impact (secondary indicators)	GO: 4-8 EU: 7-9	Experts, Research (e.g. Stolze et al., 2000), Indicators E1i-E1vii	ORGAPET Section C4	++ to -- ≥ non-organic	Similar for non-organic
E2	Overall animal health and welfare impact (secondary indicators)	GO: 4-6, 9, 12 EU: 7-9	Experts, Research (e.g. SAFO), Indicators E2i-E2iv	ORGAPET Section C4	++ to -- ≥ non-organic	Similar for non-organic
E3	Overall social impact (secondary indicators)	GO: 4-6, 10, 12 EU: 6-9	Experts, Research (e.g. FW7 societal impacts), Indicators E3i-E3vii	ORGAPET Section C4	++ to -- ≥ non-organic	Similar for non-organic
E4	Overall economic/ rural development impact (secondary indicators)	GO: 1-7, 10, 11 EU: 1, 3, 4, 6-9	Experts, Research (e.g. OF in Europe), Indicators E4i-E4iv	ORGAPET Section C4	++ to -- ≥ non-organic	Similar for non-organic
E5	Overall food quality/ safety/ security impact (secondary indicators)	GO: 1-6,12 EU: All	Experts, Research (e.g. QLIF), Indicators E5i-E5vi	ORGAPET Section C4	++ to -- ≥ non-organic	Similar for non-organic

C3-2.2 Secondary indicators

A2 Programme design secondary indicators

Indicator		Relevance	Data sources	Method	Criteria	Context indicators
A2i	Clarity of objectives (SMART)	GO: All EU: All	Documents, interviews	ORGAPET Section C1	High: 3 None: 0	Similar programmes
A2ii	Logic	GO: All EU: All	Documents, interviews	ORGAPET Section B2	High: 3 None: 0	Similar programmes
A2iii	Synergy	GO: All EU: All	Documents, interviews	ORGAPET Section B2	High: 3 None: 0	Similar programmes
A2iv	Priority	GO: All EU: All	Documents, interviews	ORGAPET Section B2	High: 3 None: 0	Similar programmes
A2v	Failure risk	GO: All EU: All	Documents, interviews	ORGAPET Section B2	High: 3 None: 0	Similar programmes

C3 Uptake/activity levels supported by individual action points - secondary indicators

Indicator		Relevance	Data sources	Method	Criteria	Context indicators
C3i	Expenditure per unit supported	GO: All EU: 1, 3, 4, 5, 7	Indicators C2, C3, Annex C3-4	MDFS C3	Appropriate relative to undertaking	Similar values for RDP or measures
C3ii	Proportion of organic businesses receiving support	GO: All EU: 1, 3, 4, 6, 7	Indicators C3, D1, D4	MDFS C3	≥ specific target?	Similar values for RDP or measures
C3iii	Proportion of organic land area receiving support.	GO: All EU: 4, 6	Indicators C3, D2	MDFS C3	≥ specific target?	Similar values for RDP or measures

D1 Number of certified organic and in-conversion holdings secondary indicators

Indicator		Relevance	Data sources	Method	Criteria	Context indicators
D1i	Number of new entrants/ withdrawals	GO: All EU: All	EU control system, Eurostat, Annex C3-4	MDFS D1	Entrants > withdrawals, Withdrawals < average	New entrants/ departures for all agriculture
D1ii	D1, D1i differentiated by farm type and size	GO: All EU: All	EU control system, Eurostat, EUCEEOFP	MDFS D1	Relative growth rates	Total number of holdings of each type
D1iii	D1, D1i, D1ii differentiated by region	GO: All EU: All	EU control system, Eurostat, EUCEEOFP	MDFS D1	Relative growth rates	Total number of holdings in each region

D2 Area of certified organic and in-conversion land secondary indicators

Indicator		Relevance	Data sources	Method	Criteria	Context indicators
D2i	D2 differentiated by farm type, land use and livestock numbers	GO : All EU: All	EU control system, Eurostat, Annex C3-4	MDFS D2, Annex C3-1	Relative growth rates	Total UAA by category
D2ii	D2, D2i differentiated by region	GO: All EU: All	EU control system, Eurostat	MDFS D2, Annex C3-1	Relative growth rates	Total UAA by region
D2iii	Output quantity by crop/ livestock type	GO: 1, 2, 3, 11, 12 EU: 3, 6, 7	EU control system, Eurostat, FADN, Research	Survey or multiply indicators D2i, D2ii by regional yields	Relative growth rates	Total output for each crop/ livestock type

D3 Organic farm incomes secondary indicators

Indicator		Relevance	Data sources	Method	Criteria	Context indicators
D3i	D3 differentiated by farm type, size and region	GO: 1-4, 10, 11 EU: All	FADN (EU/ national)	MDFS D3a, Annex C3-2	Relative performance	Overall farm incomes by type etc.
D3ii	Returns to different labour types (family, non-family)	GO: 1-3, 10, 11 EU: 6	FADN (EU/ national)	MDFS D3a, Annex C3-2	Increasing, ≥ non-organic	Returns to labour for all farms
D3iii	Returns to capital invested	GO: 1-4, 10, 11 EU: All	FADN (EU/ national)	MDFS D3a, Annex C3-2	Increasing, ≥ non-organic	Returns to capital for all farms
D3iv	Support payments by type as % of income	GO: 1,2,10, 11 EU: 4,6	FADN (EU/ national), Eurostat	MDFS D3a	Depends on goals	Support payments to all farms
D3v	Producer prices	GO: 1-4, 10, 11 EU: All	FADN (EU/ national), Survey, Eurostat, Annex C3-4	MDFS D3b, Annex C3-3	Increasing, ≥ non-organic	Non-organic producer prices

D4 Certified market operators secondary indicators

Indicator		Relevance	Data sources	Method	Criteria	Context indicators
D4i	Turnover of registered organic businesses	GO: 1-4, 10, 11 EU: All	Certified operators	Survey	Increasing	Turnover of non-organic businesses
D4ii	Number of registered organic producers engaged in processing, tourism, retailing or other consumer-facing activities	GO: 1-4, 10, 11 EU: All	Eurostat FSS, EU control system, Annex C3-4	MDFS D4	Increasing, target %	Proportion of all producers engaged in activities

D5 Organic market size secondary indicators

Indicator		Relevance	Data sources	Method	Criteria	Context indicators
D5i	Expenditure on organic food per inhabitant	GO: 1-4, 10, 11 EU: All	Calculated from D5 and population	MDFS D5a, Annex C3-3	Increasing share of per capita food expenditure	Total per capita food expenditure
D5ii	D5 and D5i differentiated by product	GO: 1-4, 10, 11 EU: All	Market research	MDFS D5a, Annex C3-3	Increasing share/ target % of product market	Total retail sales value for product
D5iii	Proportion of organic production marketed as organic	GO: 1-4, 10, 11 EU: All	Market research Indicators D2iii, D5ii	MDFS D5a	Target 100%	Trend
D5iv	Domestic self-sufficiency	GO: 1-4, 10, 11, 12 EU: All	Indicators D2iii, D5ii	MDFS D5a	Domestic production as % of consumption increasing, Target %	Self-sufficiency of all similar food production
D5v	Import/ export trade levels (supply balance)	GO: 1-4, 10, 11, 12 EU: All	Eurostat	MDFS D5b (see also EISfOM)	Target import/ export ratio	Import/export ratio for all food production
D5vi	Consumer prices	GO: 1-4, 10, 11 EU: All	Market research, Eurostat, Annex C3-4	MDFS D5c	Change relative to non-organic	Consumer prices for non-organic products
D5vii	Value added by organic market	GO: 1-4, 10, 11 EU: All	Indicators D5iv, D3v, Annex C3-4	MDFS D5c	Increasing	Non-organic consumer/ producer prices

D6 Consumer confidence and trust secondary indicators

Indicator		Relevance	Data sources	Method	Criteria	Context indicators
D6i	Consumer purchasing commitment (loyalty/ intensity)	GO: 3, 4, 6 EU: 1, 8-21	Market research	MDFS D6	Increasing % of committed/ occasional consumersrs	Total number of consumers
D6ii	Consumer recognition/ understanding, differentiated by D6i	GO: 3-6 EU: 1, 8-21	Market research	MDFS D6	Increasing % of consumers correctly identifying/ explaining	Total number of intensive, occasional and non-consumers

D7 Business confidence secondary indicators

Indicator		Relevance	Data sources	Method	Criteria	Context indicators
D7i	Level of private sector investment in organic market	GO: 1-4, 10, 11 EU: All	RDP monitoring, FADN	MDFS D7	Increasing absolutely and relative to public support	Public/ private investment in organic and total food sector
D7ii	Business longevity differentiated by business type and region	GO: 1-4, 10, 11 EU: All	Survey, FADN, FSS	MDFS D7	Increasing number/ % of businesses operating after 8 years	Similar values for all business

D8 Control bodies secondary indicators

Indicator		Relevance	Data sources	Method	Criteria	Context indicators
D8i	D8 differentiated by private/public sector status	GO: 1-5 EU: 9-21	EU control system, organisation list	MDFS D8	Increasing or decreasing – depends on perspective	Similar non-organic control bodies
D8ii	Size of control organizations: number of certified operators	GO: 1-5 EU: 9-21	Survey, EU control system	MDFS D8	Increasing	Similar non-organic control bodies
D8iii	Cost of control system	GO: 1-5 EU: 9-21	Research (e.g. FW7 CertCost), EU control system	MDFS D8	Increasing or decreasing – depends on perspective	Similar non-organic control systems

D9 Control system secondary indicators

Indicator		Relevance	Data sources	Method	Criteria	Context indicators
D9i	D9 differentiated by regular/ additional visits	GO: 2-5 EU: 9-21	EU control system, annual report	MDFS D9	Increasing or decreasing – depends on perspective	Similar non-organic control systems
D9ii	D9 differentiated by announced/ unannounced visits	GO: 2-5 EU: 9-21	EU control system, annual report	MDFS D9	Increasing or decreasing – depends on perspective	Similar non-organic control systems
D9iii	Number of samples taken	GO: 2-5 EU: 9-21	EU control system, annual report	MDFS D9	Increasing or decreasing – depends on perspective	Similar non-organic control systems
D9iv	Number of samples indicating breach of regulation	GO: 2-5 EU: 9-21	EU control system, annual report	MDFS D9	Decreasing	Similar non-organic control systems
D9v	Number of minor/ major non-compliances/ infringements	GO: 2-5 EU: 9-21	EU control system, annual report	MDFS D9	Decreasing (but may indicate less control)	Similar non-organic control systems
D9vi	Number/scale of fraud cases	GO: 2-5 EU: 9-21	EU control system, annual report	MDFS D9	Decreasing (but may indicate less control)	Similar non-organic control systems

D10 Regulatory framework secondary indicators

Indicator		Relevance	Data sources	Method	Criteria	Context indicators
D10i	Number of deviations between EU regulations and member state implementation	GO: 3-5 EU: 2, 8-21	Organic Revision, Organic rules database, EU control system	MDFS D10	Fewer references to SCOF	Number of 2092/91 deviations
D10ii	Number of derogations agreed	GO: 3-5 EU: 2, 8-21	EU control system	MDFS D10	Fewer derogations agreed	Number of 2092/91 derogations
D10iii	Number of collaboration activities between member states	GO: 3-5 EU: 2, 8-21	EU control system	MDFS D10	More collaborative activities	Number of 2092/91 collaborations

D11 Regulatory burden secondary indicators

Indicator		Relevance	Data sources	Method	Criteria	Context indicators
D11i	Pages of forms to be completed	GO: 1, 4, 5, 11, 12 EU: 2, 9-21	Business survey, EU control system	MDFS D11	Decreasing	Similar non-organic control system
D11ii	Time spent by businesses on inspection process	GO: 1, 4, 5, 11, 12 EU: 2, 9-21	Business survey, EU control system	MDFS D11	Decreasing	Similar non-organic control system
D11iii	Operator confidence in new logo/ regulation	GO: 1, 4, 5, 11, 12 EU: 2, 9-21	Business survey	MDFS D7	High: 3 None: 0	Similar non-organic control system

D12 Research and extension secondary indicators

Indicator		Relevance	Data sources	Method	Criteria	Context indicators
D12i	Size (staff and financial resources allocated to organic work) of D12 organisations	GO: All EU: 1, 3, 6, 7, 9, 11, 14	Survey	Values	Increasing	Total size of similar institutions for all agriculture
D12ii	D12, D12i differentiated by private/ public sector status	GO: All EU: 1, 3, 6, 7, 9, 11, 14	Survey	Count/ Values	Private sector share of activity	Similar data for all agric. institutions
D12iii	Proportion of D12 organisations receiving programme support	GO: All EU: 1, 3, 6, 7, 9, 11, 14	Survey	Values	Increasing, target %	Similar data for all agric. Institutions
D12iv	Number of researchers/ consultants/ trainers/ other experts specialising in organic farming	GO: All EU: 1, 3, 6, 7, 9, 11, 14	Survey	Values	Increasing	Similar data for all agriculture
D12v	Numbers of businesses benefiting from research/ training/ advice	GO: All EU: 1, 3, 6, 7, 9, 11, 14	RDP monitoring, Survey	Count	Increasing	Total number of organic businesses
D12vi	Research output/ quality	GO: All EU: 7	Various	MDFS D12	Increasing	Similar values for all agricul-tural research

E1 Environmental impact secondary indicators

Indicator		Relevance	Data sources	Method	Criteria	Context indicators
E1i	Number of clauses in new EU regulation directly addressing environmental issues	GO: 4-8 EU: 7-9	EU Reg 834/2007 text	Count	Increasing	EU Reg. 2092/91 text
E1ii	Biodiversity	GO: 4-8 EU: 7-9	Research (e.g. FW7 biodiversity indicators)	MDFS E1a	Increasing, ≥ non-organic	Similar for non-organic
E1iii	Energy balance	GO: 4-8 EU: 7-9	Research (e.g ART LCA)	MDFS E1b	Reducing fossil input, ≤ non-organic	Similar for non-organic
E1iv	Nitrogen, nutrient balances	GO: 4-8 EU: 7-9	Research (e.g. ART LCA)	MDFS E1b	Balanced	Similar for non-organic
E1v	Global warming potential (CO2 equivalent)	GO: 4-8 EU: 7-9	Research (e.g. ART LCA)	MDFS E1b	Decreasing, ≤ non-organic	Similar for non-organic
E1vi	Soil conservation	GO: 4-8 EU: 7-9	Research (e.g. FIBL DOC trial)	MDFS E1c	Increasing, ≥ non-organic	Similar for non-organic
E1vii	Water conservation	GO: 4-8 EU: 7-9	Research	MDFS E1c	Increasing, ≥ non-organic	Similar for non-organic

E2 Animal health and welfare impact secondary indicators

Indicator		Relevance	Data sources	Method	Criteria	Context indicators
E2i	Number of clauses in new EU regulation directly addressing animal health and welfare issues	GO: 4-6, 9, 12 EU: 7-9	EU Reg 834/2007 text	Count	Increasing	EU Reg. 2092/91 text
E2ii	Number of veterinary derogations	GO: 4-6, 9, 12 EU: 7-9	EU control system	MDFS D9, MDFS D10	Decreasing	Similar values for 2092/91
E2iii	Longevity of breeding stock	GO: 4-6, 9, 12 EU: 7-9	Survey	Values	Increasing, ≥ non-organic	Similar values for non-organic
E2iv	Number of farms combining organic with increased welfare standards/ animal welfare support	GO: 4-6, 9, 12 EU: 7-9	Survey, RDP monitoring	Count	Increasing, ≥ non-organic	Similar values for non-organic

E3 Social impact secondary indicators

Indicator		Relevance	Data sources	Method	Criteria	Context indicators
E3i	Number of clauses in new EU regulation directly addressing social issues	GO: 4-6, 10, 12 EU: 6-9	EU Reg 834/2007 text	Count	Increasing	EU Reg. 2092/91 text
E3ii	Occupational health	GO: 4-6, 10, 12 EU: 6-9	Research (e.g. Cross et al., 2008)	MDFS E3a	Increasing, ≥ non-organic	Similar for non-organic
E3iii	Age distribution	GO: 4-6, 10, 12 EU: 6-9	FSS	MDFS E3b	Younger, ≤ non-organic	Similar for non-organic
E3iv	Gender balance	GO: 4-6, 10, 12 EU: 6-9	FSS	MDFS E3b	Increasing/ target % women	Similar for non-organic
E3v	Educational level/ background	GO: 4-6, 10, 12 EU: 6-9	FSS	MDFS E3b	Increasing, ≥ non-organic	Similar for non-organic
E3vi	Migrant labour	GO: 4-6, 10, 12 EU: 6-9	Survey	Values	Dependent on policy perspective	Similar for non-organic
E3vii	Number of business combining organic with additional ethical/ fair trade standards/ support	GO: 4-6, 10, 12 EU: 6-9	Survey, RDP monitoring	Count	Increasing, ≥ non-organic	Similar for non-organic

E4 Economic/rural development impact secondary indicators

Indicator		Relevance	Data sources	Method	Criteria	Context indicators
E4i	Employment levels	GO: 1-7, 10, 11 EU: 1, 3, 4, 6-9	FSS, FADN	MDFS D3a, MDFS E4	Increasing, ≥ non-organic	Similar for non-organic
E4ii	Income levels	GO: 1-7, 10, 11 EU: 1, 3, 4, 6-9	FADN, Indicators D3, D3i, D3ii	MDFS D3a, MDFS E4	Increasing, ≥ non-organic	Similar for non-organic
E4iii	Diversity of income sources	GO: 1-7, 10, 11 EU: 1, 3, 4, 6-9	FADN, Research	MDFS E4	Increasing, ≥ non-organic	Similar for non-organic
E4iv	Local/regional economic impact	GO: 1-7, 10, 11 EU: 1, 3, 4, 6-9	Research	MDFS E4	Increasing, ≥ non-organic	Similar for non-organic

E5 Food policy impact secondary indicators

Indicator		Relevance	Data sources	Method	Criteria	Context indicators
E5i	Food output and relative productivity	GO: 1-6, 10-12 EU: 1-7	Indicator D2iii, FADN, Research	MDFS D2	Increasing absolutely & relative to non-organic	Similar for non-organic
E5ii	Number and severity of food safety incidents	GO: 1-6, 12 EU: 13, 14	Food safety monitoring, Research (e.g. QLIF, JRC)	MDFS E5	Decreasing absolutely & relative to/≤ non-organic	Similar for non-organic
E5iii	Pesticide residue levels	GO: 1-6, 12 EU: 13, 14	Food safety monitoring, Research (e.g. QLIF, JRC)	MDFS E5	Decreasing absolutely & relative to/≤ non-organic	Similar for non-organic
E5iv	Food quality assessments	GO: 1-6, 12 EU: All	Research (e.g. QLIF, JRC)	MDFS E5	Increasing absolutely & relative to/≥ non-organic	Similar for non-organic
E5v	Public health impact	GO: 2-6, 12 EU: All	Research	MDFS E5	Improving absolutely & relative to/≥ non-organic	Similar for non-organic
E5vi	Local food system self-reliance (self-sufficiency, sovereignty)	GO: 1-6, 10, 12 EU: All	Research, Indicator D5iv	MDFS D5a	Improving absolutely & relative to/≥ non-organic	Similar for non-organic

C3-3 Conclusions

The final choice of indicators for an evaluation needs to be directly relevant to the objectives and actions of a specific action plan. The generic indicators can only provide a source of ideas on possible indicators, where to source evidence and how to measure and assess them. The final list of indicators selected should be parsimonious - i.e. kept to a manageable number so that it is a) feasible to assess them in terms of the resources required for monitoring and data collection, and b) possible in the end to reach an overall judgement keeping all the different indicators in mind without being overwhelmed by sheer mass of data. The quality of the indicators, the type of evaluation (formative/summative) and the stage of the policy cycle (ex-ante, mid-term, ex-post) will all be relevant to the selection of a parsimonious list of indicators.

C3-4 Checklist

In the context of a specific action plan evaluation:

1. Identify which of the generic indicators are relevant, and which should be left out

2. Identify additional indicators as described in ORGAPET Section C2

3. Quantify or describe the selected indicators, including start and end positions, direction and scale of change

4. Identify indicators that need assessment by expert judgement (ORGAPET Section C4)

NB The final analysis, interpretation and synthesis of results is covered in ORGAPET Section D1 .

C3-5 References

Dabbert, S. and Eichert, C. (2007) Public synthesis report on the scope of national action plans, their evaluation procedures and the operability and appropriateness of the developed evaluation concept at national level, as well as the impact of conflict/synergies and policy proposals for implementing the EU Action Plan in member states. ORGAP project deliverable. University of Hohenheim, Stuttgart.

C3-6 Annexes

Annex C3-1: Irena 7 Organic Land Area Methodology and Data Factsheet

Annex C3-2: Irena 5.2 Organic Farm Incomes Methodology and Data Factsheet

Annex C3-3: Irena 5.1 Organic Market Shares Methodology and Data Factsheet

Annex C3-4: Baseline data: EU Commission (2005) Facts and figures on organic farming

Annex C3-5: ORGAP (2006) national workshops report

Annex C3-6: Baseline data: EU Commission (2007) Report on the supervision of organic inspection bodies and authorities of the Member States 2005

Annex C3-7: Baseline data: Eurostat (2007) organic farming statistics report

Annex C3-8: Baseline data: EU Commission (2007) List of organic control bodies 2006

ORGAPET Section C3: Generic Indicators