Gapi Approach


This first iteration of GAPI assesses 44 separate species-country pairs, which together comprise 94% of global marine finfish aquaculture production by volume and 91% by value. Ten different categories or “indicators” of environmental impact that experts have identified as important and relevant to finfish aquaculture are assessed. In addition to assessing how each species-country pair performs in these ten impact categories, GAPI also establishes a standard scale so that environmental performance can be measured and compared among species and across countries.

One of the major strengths of GAPI is that it enables aquaculture performance to be judged against a set of targets that would be unrealistic as standards but provide crucial information regarding how close marine finfish aquaculture comes to meeting an ecological ideal. By setting a zero-impact target for each indicator, GAPI permanently sets the environmental performance at the ecological ideal rather than continually recalibrating the goal as the performance of the industry improves or as viewpoints of what is an “acceptable” level of impact shift. As such, GAPI provides a robust tool to assess any real progress or decline in environmental performance over time.

A closer look at deriving the GAPI Scores

  • Step 1
    Identifying
    Key Indicators
  • Step 2
    Indicator
    Formulas
  • Step 3
    Setting
    the Targets
  • Step 4
    Data
    Colleciton
  • Step 5
    Winsorisation
     
  • Step 6
    Proximity
    to Target
  • Step 7
    Applying
    Weightings
  • Step 8
    Performing
    the Calculation
  • Step 9
    Aggregating
    the Scores

Step 1 | Identifying Key Indicators of Environmental Performance

An important sxtep in the development of GAPI was to identify key indicators of environmental performance. It is not an exaggeration to say that one could identify hundreds of indicators to evaluate the environmental performance of an aquaculture production system. However, within GAPI, emphasis is placed on identifying a suite of indicators that sufficiently describes the major ecological impacts of marine finfish aquaculture while using the fewest indicators possible. Each additional indicator increases the complexity of the analysis, the likelihood of significant data gaps, and the effort required to collect data. Therefore, rather than attempting to measure all conceivable impacts from production systems, GAPI evaluates the most significant and measurable environmental effects.

In order to determine the suite of GAPI indicators, the project examined existing aquaculture assessment efforts and pinpointed those environmental impacts that were commonly addressed across these efforts. Those issues that appeared consistently among initiatives were considered to have passed something of a peer review and, as a result, were important enough to include within GAPI. Ten common impact categories were identified and incorporated into GAPI, including, among others, the impacts of: escaped fish; parasites/disease; discharges of organic and inorganic waste; energy use; dependence on wild fish for feed; and dependence on wild fish for broodstock and juveniles.

Step 2 | Constructing Formulas to Calculate Scores for Each Indicator

The second, more challenging step was to determine how best to measure actual performance within each indicator category. The GAPI project developed specific criteria to ensure that GAPI indicators were sufficiently rigorous. Included in these criteria are:

  • Relevance: How well the indicator gauges the environmental impact at hand
  • Performance orientation: Whether the indicator tracks actual, on-the-water performance
  • Transparency of formulas and data
  • Availability of quality data

In the same way that it is valuable information to know a country’s GDP and its GDP per capita or its overall contribution to carbon dioxide emissions versus its per capita contribution, both cumulative and normalised performance are assessed within GAPI. The absolute GAPI score for each species-country pair reflects the overall environmental impact of the production of a species in a particular country. However, because absolute scores take into account the volume of fish produced, they can be greatly affected by differences in production volume (e.g., large producers will tend to have low cumulative scores given their sheer volume of production). In order to level the playing field among the range of performers from small to large and to highlight intrinsic performance differences among species, thereby allowing for direct comparison, performance in each indicator is divided by the production volume (mT, live-weight equivalents). These normalised values for performance within each indicator are used to obtain a normalised GAPI score for each species-country pair. Compared to cumulative scores, the normalized GAPI scores offer decision makers greater insight not only into how players are performing compared to their peers, but also into where they are leading or lagging, and where effective solutions might lie.

Step 3 | Setting Targets for Each Indicator as Zero-Impact Targets

One of the major strengths of GAPI is that it enables aquaculture performance to be judged against a set of targets that would be unrealistic as farming standards but provide crucial information regarding how close marine finfish aquaculture comes to meeting an ecological ideal. By setting a target of zero for each indicator, GAPI permanently sets the environmental performance at the ecological ideal rather than continually recalibrating the goal as the performance of the industry improves or as viewpoints of what is an “acceptable” level of impact shift. As such, GAPI provides a robust tool to assess any real progress or decline in environmental performance over time.

Step 4 | Collecting Data for Each Species-Country in Each Indicator

GAPI uses a wide range of data sets drawn from international organisations, regulatory bodies, conservation organisations, academia, seafood industry groups, and the seafood industry trade press. The data used in GAPI are publicly available and traceable. Data sources are referenced for each Indicator (see Indicator Data in the Downloads section). The GAPI indicator data and reference sheets provides a log of all data and respective sources. All data included within the current GAPI data set are from 2007, unless otherwise indicated.

As with any effort to assess aquaculture performance, GAPI faces challenges related to data availability and quality. Limited data coverage, methodological inconsistencies, low-quality metrics, and poor (or nonexistent) reporting structures pose problems for all assessment efforts. While GAPI is focused at the country level, where most aquaculture data are collected and reported by regulatory authorities, data inaccuracies are still likely. Where questions regarding data accuracy or gaps in data remain, GAPI is transparent about how these potential inaccuracies and gaps are treated. This information is summarised in the Indicators section.

As new and better data become available, the website and analysis will be updated. Lastly, while the preference is to use data that track on-the-water performance, in some cases there is simply a lack of direct empirical data. For instance, there is currently no method that is both feasible and credible for predicting or tracking the full range of actual effects on wild fish of farm-derived disease transmission or the effect of escaped fish from farms. Given that expert opinion suggests that these impacts are important components of environmental performance, GAPI relies on a combination of “measured” performance data and “modelled” indicators of performance and/or risk.

In keeping with the approach of the Environmental Performance Index (EPI), GAPI aims to stimulate discussion on defining the appropriate metrics and methodologies for evaluating environmental performance in addition to highlighting the need for improved data collection.

Step 5 | Winsorisation: A Statistical Approach to Dealing with Data Outliers

Once indicators are defined and the relevant data are collected, the GAPI scores are calculated. The first step of this process is winsorisation. Winsorisation is an accepted statistical approach to dealing with outliers. It allows users to address the small number of extremely high or low values in a data set so that those values do not distort the distribution of the entire data set. The Environmental Performance Index (EPI) suggests that when assessing environmental performance, such extreme values tend to be the result of measurement errors rather than signals of legitimately high or low performance (Esty et al. 2008).

Winsorisation: Treating Outliers Within GAPI

In winsorisation, if any performance lies outside the normal distribution of data for the entire group of performers, that outlier performance value is adjusted so that it lies at the extreme edge of the normal range (two standard deviations from the mean), as demonstrated in the figure above. Since the GAPI target performance is set at zero, however, no performer can overperform (i.e., do better than zero impact). Thus, winsorisation is only used to adjust for extreme underperformance (i.e., performing significantly worse in any one indicator than the data set would suggest is plausible).

Step 6 | Proximity-to-Target Calculation to Standardise the Data

Data come in many different units, scales, and ranges. To be able to compare performance among escapes and the sustainability of feed sources, for example, it is necessary to standardise the data for each indicator. GAPI’s aim is to standardise all data so they can be mapped on the same 0-to-100 scale, where individual scores can be compared in a statistically meaningful way. GAPI uses the proximity-to-target approach to calculate how close each performer is to meeting the established precautionary targets (i.e., zero impact).

Proximity-to-target is calculated for each individual indicator separately, using the following formula:

Proximity-to-target = 100 − [100 — (Actual Performance − Target) *
                                            (Maximum Winsorised Value − Target)]

The proximity-to-target calculation measures the distance between actual performance and the established target for each indicator. In order to provide some context for this value or score, this number is expressed as a proportion of the distance from target of the worst performer (i.e., maximum winsorised value). This distance is converted into a percentage and then transformed into the GAPI scale, so that a high score indicates better performance. This results in an initial, unweighted GAPI score for each individual indicator.

Since the worst performer in the analysis sets the floor for performance, the GAPI score is partially dependent on the pool of performers included in the analysis. Thus, it is important that this pool of performers is representative of the marine finfish aquaculture industry. The performers included within GAPI comprise approximately 93.7% of marine finfish aquaculture by weight and 91.0% by value, which is a solid representation of the global marine finfish aquaculture industry.

Step 7 | Weighing Indicators Using Principal Component Analysis

At this point, the overall GAPI score could be calculated by taking the average of the 10 individual indicator scores. However, doing so would ignore the fact that some indicators are more important than others in explaining the difference in performance among two or more players. Therefore, a data aggregation and weighting scheme needs to be applied to reflect the differential importance of indicators to overall environmental performance.

A recent review of sustainability assessment methodologies (Singh et al. 2009) demonstrated that normalization and weighting of indicators used in sustainability assessments is typically associated with subjective judgements and reveals a high degree of arbitrariness without mentioning or systematically assessing critical assumptions. For instance, an assessment tool may be designed to weight energy use or carbon footprint more heavily than other indicators because of global attention to this issue or evidence of its large-scale effect. But, is this type of subjective weighting sound? Further, can we legitimately say that disease impacts are more important than escape impacts in all geographies or at all times?

GAPI addresses this dilemma by shifting away from weighting based on the assumed magnitude of environmental impact of each indicator. By selecting an indicator to be included within GAPI, it has already been decided that it is a relatively important driver of environmental performance. However, to ensure that GAPI is as rigorous, transparent, and objective as possible, the data and not the investigator determine the degree of weighting for each indicator. A standard statistical procedure for such a task is the principal component analysis (PCA).

Example of Calculating a GAPI Score for Each Species-Country Pair

Within GAPI, the 10 indicators generate a large “cloud” of data. PCA essentially creates a lens through which we can view this complex set of data as simply and clearly as possible. In this case, the purpose of using PCA is to help find trends in the data in order to determine how important each indicator is in describing the difference in performance across many performers. PCA measures how much of the total variation in the data is explained by each indicator, thus providing a measure of each indicator’s relative importance or weight.

PCA-derived weights for each GAPI indicator are listed in the above table (Column D). The larger weights identify those indicators where the largest differences in performance among species-country pairs lie and lower weights indicate proportionally smaller differences in performance. As shown in Column D, indicators, including antibiotics, ecological energy, sustainability of feed, and pathogens, explain 15% of the variation in performance across all performers. The other remaining indicators all add similar but more modest levels of insight (explaining 5% to 8.3% of variation) and so are not weighted as heavily.

Step 8 | Calculating the Final GAPI Score for Each Species-Country Pair

The standard unit of analysis within GAPI is the species-country pair. For each of the 20 farmed species and 22 major producing countries assessed within GAPI, there is a corresponding GAPI score. This results in GAPI scores for 44 species-country pairs such as Atlantic salmon–Norway, Atlantic cod–Iceland, and barramundi–Australia.

Example of Calculating a GAPI Score for Each Species-Country Pair

The above table demonstrates how the final GAPI score for each species-country pair is calculated for a hypothetical performer. First, the performer’s environmental performance within each indicator is determined by calculating the proximity-to-target for each normalised indicator, standardised on a scale of 0 to 100 (Column C). The weight of each of these indicators is then calculated using PCA. Next, the indicator performance values (Column C) are multiplied by the PCA-derived weight assigned to each indicator (Column D) to yield the weighted performance within each indicator (Column E). The final GAPI score (Column F), which describes the performer’s normalized performance within all environmental indicators, is the sum (rounded) of the 10 weighted performance scores in Column E. Within this example, the hypothetical performer’s normalised GAPI score is 64 (out of 100).

Step 9 | Aggregating GAPI Scores by Species and Country

While GAPI assesses performance at the species-country level, it is critical that these individual GAPI scores can be aggregated so that some conclusions can be made about performance across farmed marine finfish species and the countries in which they are produced. While one user might be interested in how Atlantic salmon–Chile scores, another user may be more interested in how Chile’s marine finfish industry is performing overall or how Atlantic salmon scores compare to Atlantic cod in general. The Species Performance and Country Performance provide both the aggregate species and country GAPI scores in addition to individual species-country pair GAPI performance.

The aggregate scores are simply the average of the individual normalised species-country scores related to the particular species or country weighted by the proportion of production assessed by GAPI. For example, the Species GAPI score for “Species A” below would be calculated as:

Species A GAPI Score = (68 X 0.10) + (50 X 0.90) = 51.8

Similarly, the Country GAPI Score of Country 1 would be calculated as:

Country 1 GAPI Score = (68 X 0.65) + (80 X 0.35) = 72.2