Chapter 6 Toolbox Ecosystem

6.1 Overview

Welcome to the OHI Toolbox, the engine behind all OHI assessments. The Toolbox is an ecosystem of R scripts and data files, organized in folders and sub-folders. The information in your repository will interact with an R package called ohicore, which does the core operations to calculate OHI scores.

In this chapter, we will learn how the Toolbox is structured - where you would store data, modify models, make calculations, as well as record decision-makings. The Toolbox ecosystem introduced here is the same in any assessment. So, if you would like to explore your own repository or an existing repository such as the Gulf of Guayaquil assessment you can navigate through those repositories in the same way.

This tutorial will take roughly 30 minutes.

Let’s get started!

6.2 Prerequisites

We will explore these files together on your own computer, but we will not be calculating scores in this chapter. If you will be using the Toolbox, please clone the repository to your computer with the following steps. You can clone either the toolbox-demo or your assessment’s repository. If you will not be calculating scores, you do not need to clone anything; please follow along online at GitHub https://github.com/OHI-Science/toolbox-demo or with your assessment’s repository.

Have up-to-date versions of R and RStudio and have RStudio configured with Git/GitHub
Fork the toolbox-demo repository into your own GitHub account by going to https://github.com/OHI-Science/toolbox-demo, clicking “Fork” in the upper right corner, and selecting your account
Clone the toolbox-demo repo from your GitHub account into RStudio into a folder called “github” in your home directory (filepath “~/github”)
Get comfortable: be set up with two screens if possible. You will be following along in RStudio on your own computer while also watching an instructor’s screen or following this tutorial.

Now you have set up your computer, let’s open the toolbox-demo repository on your computer, and go from there.

You can also watch a video recording of the following, identified in the OHI Onboarding Book: http://ohi-science.org/onboarding/conduct.

6.3 Repository home folder (ex: `toolbox-demo`)

When you first open the repository, you’ll encounter the following files and folders. Note: if you have a starter repository, you’ll have a prep folder but not a scenario folder (which is region2016 in the screenshot below).

Overall, all the main files you will see are either of the two file types:

.csv files contain data inputs or configuration information.
.R or .Rmd scripts are written in the programming language R and use data inputs for processing and calculations.

Among all the files on your screen, the most important files are:

prep is a data preparation folder. This is where you will store raw and/or intermediate data, and the R scripts you create to explore and format those data, and make trial runs of different models. You will spend roughly the first half of your time during your assessment here.
region2016 is the scenario folder. The scenario folder has the architecture of the OHI Toolbox that is needed to calculate scores. It contains all of the inputs needed to calculate OHI scores, and you will modify these inputs when conducting your assessment. By default it is named regionYEAR (e.g. cnc2016 for New Caledonia 2016) to indicate that the assessment is conducted at the region scale (province, state, district, etc.), based on input layers and goal models used in the most recent global assessment (e.g. 2016).
README.md file accompanys most folders or files, as you will discover. It is a place to take notes for yourself, your team, and thinking ahead to anyone interested in the future.

Let’s take a closer look at the prep and region2016 folders.

6.4 Prep folder

Click on prep and let’s see what is inside:

Within prep, there is a sub-folder for each goal and subgoal, as well as pressures and resilience. These folders are meant to help you be organized and promote scripting for data preparation (this is important for reproducibility and repeatability). Note that some goals depend on the same data and thus data can be prepared into layers from one place instead of for each goal. This is the case for goals dependent on habitat data: CP, CS, HAB, and goals dependent on species data: SPP, ICO.

Click on the ‘CW’ folder. Within each goal folder, there are typically a .Rmd script and a sub-folder to contain data.

The CW_data_prep.Rmd document is a script where you will explore your raw data, format the data, try different models, make graphs and maps, as well as document decision making. We will come back to this script in Chapter 7 to learn how to prepare data for calculations.

We use RMarkdown (.Rmd) files because they can be rendered as webpages (for example, CW_data_prep.html in the same CW folder) and can help explain the data processing steps and motivation behind them in written language. Click ‘Knit’ from the RStudio editor or click on the .html file from the file pane and select ‘View in web browser’, and you will see a clean, human-readable webpage that can be shared online and viewed by anyone.

Now we’ve walked through the prep folder, let’s take a look at the scenario folder. Click back to the main toolbox-demo folder, and open the region2016 folder.

6.5 Scenario folder (ex: `region2016`)

Again, this scenario folder contains all the data layers and scripts for you to calculate OHI scores. Let’s look into the sub-folders and see what they do.

6.5.1 layers folder & layers.csv

After you prepared your data layers in the prep folder, you will save them in the layers folder, and register them in layers.csv. (You will learn how to do so in Chapter 7).

The layers folder contains individual data layers as .csv files for each goal and subgoal, named in a specific manner for easy organization and recognition, as shown here.

layers.csv is a data registry for all the data layers in the layers folder. It is also a directory for ohicore to find the exact data layers for calculations. The file itself is spreadsheet that is best to be opened in a spreadsheet editor such as Microsoft Excel.

6.6 conf folder

The conf folder includes important configuration files required to calculate OHI scores, as shown here. These are the model parameters that the Toolbox needs to calculate scores, along with your prepared data layers.

Most of the maneuvering in the Conduct phase of the assessment is done within this folder. There are both .R scripts and .csv files. We will investigate them in the order you will interact with them.

6.6.1 functions.R

functions.R contains models to calculate goal status and trend for all goal and sub-goals. Much of your time will be spent here developing goal models to calculate status and trend with the data layers you prepared.

Each goal and sub-goal equation is stored as a separate function within the script, identified by a 2- or 3- letter code. These functions calculate the status and trend using prepared layers saved in the layers folder and registered in layers.csv.

6.6.2 scenario_data_years.csv

This file organizes year information for each layer. OHI assessments use the most recent data available, but which year is most recent will vary from data set to data set. scenario_data_years.csv acts like a registry to help keep track of all these years. When you calculate OHI scores, you will be explicit about the year your completed assessment will represent, and we call this the “scenario year”.

This file also makes it possible for the Toolbox to calculate repeated assessments.

This file has 3 columns: the layer_name, the year your assessment represents, called the scenario_year, and the most recent year of data, called data_year.

Let’s have a look:

You can see that for each layer, the data_year is associated with a scenario_year. In the example above, the data from the ao_access layer have never been updated so we use the same data year for each scenario year, which is therefore repeated in the data_year column. But for the ao_need layer, many years of data are available, but there is a lag time for which year of data corresponds to which scenario year.

6.6.3 pressures & resilience

There are four .csv files relating to pressures and resilience.

pressures_categories.csv
pressures_matrix.csv
pressures_categories.csv
pressures_matrix.csv

They are spreadsheets that are best opened with Excel.

Pressures and Resilience are two of the four dimensions used to evaluate each goal or sub-goal, along with goal status and trend that you calculate in functions.R. More details on how to operate with these files are explained in future chapters.

pressures_categories.csv

This is a table to record the name of each pressures data layer, its category, and sub-category. Each data layer name is the same name that is saved in the layers folder and is registered in layers.csv. Each layer falls under one of two categories - ecological or social pressures, and one of several sub-categories to further represent the origin of the pressure (e.g. climate change, fishing, etc), which is also indicated by a prefix of each data layer name (for example: po_ for the pollution sub-category).

pressures_matrix.csv

This is a table that indicates which individual pressures (stressors) affect which goal, sub-goals, or components, and weights them from 1-3. A weight of 0 is shown as a blank. A higher weight indicates more negative impacts on that goal or component of the goal. These weights are relative to each row of the matrix (goal, sub-goal, or component). These weights are used in global assessments based on scientific literature and expert opinion, and you can modify the weightings if necessary for your assessment. The pressures matrix is the same as Table S25 in Halpern et al. 2012, with current and detailed information in the online OHI-Global supplemental information document.

Each pressure (column) of the pressures matrix is the layer name of the pressures layer file that is saved in the layers folder and is registered in layers.csv, matching what’s recorded in the pressures_categories.csv.

resilience_categories.csv

Similar to pressures_categories.csv, this file contains information on each resilience data layer, including its name, category, and sub-category. Each resilience layer’s name is the same as the data layer to be saved in the layers folder and is registered in layers.csv. In addition, it also includes information on category type - ecosystem, regulatory, or social, indicating the origin of the resilience layer.

Each resilience layer is also assigned a weight of 0-1, representing the level of resilience against pressures. Different from the values used in pressures matrix, the resilience weights depend on the level of information available.

resilience_matrix.csv

This is a table that indicates which individual resilience measures affect which goal, sub-goals, or components. The resilience matrix is the same as Table S26 from Halpern et al. 2012, with current and detailed information in the online OHI-Global supplemental information document.

6.6.4 goals.csv

goals.csv is another spreadsheet file that is best opened in Excel.

goals.csv has information about goals and sub-goals, and controls which goals are calculated, and how they are represented in figures. If a goal exists in this list, it will be calculated, and needs an accompanying function in functions.r. You will likely only modify this file if you are removing or adding goals or sub-goals, or changing the weighting for how goals are combined.

Here are the columns within goals.csv:

order_color & order_hierarchy: the order to display in flower plots
order_calculate: the order in which the goals and sub-goals are calculated for the overall Index scores
goal & parent: indicates the relationship between sub-goals and supra-goals (i.e. goals - with sub-goals)
weight: the relative weight of each goal to calculate overall Index scores
preindex_function: indicate what parameters are called to calculate scores for goals and - sub-goals in functions.R
postindex_function: indicate what parameters are called to calculate scores for supra-goals in functions.R

6.6.5 config.R

config.R configures labeling and constants appropriately. You will only need to modify this file when working with goals that have categories (example: habitat types or economy sectors) that are affected differently by pressures and resilience measures.

6.6.6 conf folder recap

We have just explored the files in the conf folder. To recap, the conf folder contains important configuration files required to calculate OHI scores, including functions.R, scenario_data_years.csv, and pressures and resilience files. Now let’s take a step back to the region2016 folder, and check out what else is there.

6.7 calculate_scores.Rmd

Note: calculate_scores.Rmd replaces three files that existed in previous versions of the Toolbox: calculate_scores.R, configure_toolbox.R, and install_ohicore.R. We collapsed those three files into one .Rmd file to simplify the workflow, although they still function correctly as individuals if your repository has them as three separate files. You can find information about these three files below.

calculate_scores.Rmd is the code and workflow to calculate OHI scores. It enables the information in your repository to interact with an R package called ohicore, which does the core operations to calculate OHI scores. This Rmd will install the ohicore package at the beginning of your assessment, and then call it as a library after that.

Whenever you change any data in layers, modify models in functions.R, and make changes to pressures and resilience, you could run code from this file and see how the OHI scores change. Scores will be saved in scores.csv.

6.7.1 calculate_scores.R

Note: depreciated, see calculate_scores.Rmd section above. This script used to be inside the repository’s scenario folder.

calculate_scores.R is a script you’ll come to use often. As the name tells us, it is a script to calculate OHI scores. Whenever you change any data inlayers, modify models in functions.R, and make changes to pressures and resilience, you could run this script and see how the OHI scores change.

This script runs everything required to calculate OHI scores using the prepared layers the layers folder that are registered in layers.csv. Scores will be saved in scores.csv.

6.7.2 configure_toolbox.R

Note: depreciated, see calculate_scores.Rmd section above. This script used to be inside the repository’s scenario folder.

This script does the pre-checks before running goal models and calculate scores. It loads ohicore, calls all goal functions and data layers in the “conf” folder, and check that all data layers are registered properly. You are encouraged to use this script when you’re working on individual goal models. After you register data layers for a goal, or make any changes to the data layers, source this script before running model-specific functions in functions.R.

6.7.3 install_ohicore.R

Note: depreciated, see calculate_scores.Rmd section above. This script used to be inside the repository’s home folder.

ohicore is the backbone software package of the Toolbox. It is a R package of functions that contain all the core operations for the data and models that you provide for your assessment, and it will calculate OHI scores. You don’t need to interact with or see the inside of ohicore yourself during the assessment, but you do need to install this software package once at the start of your assessment by running this script.

6.7.4 Recap

The OHI Toolbox is a ecosystem of data layers and scripts, contained in folders and their sub-folders.

The information in your repository will interact with an R package called ohicore, which does the core operations to calculate OHI scores. You will need to install this package at the beginning of your assessment, and then call it as a library after that.

As you begin your assessment, you will spend most of your time in prep folder, where you upload, explore, and format your data layers to be saved in the scenario folder region2016 for the next step of calculations.

region2016 is a folder with all the data layers, files, and scripts you’ll need to calculate scores. There you will register finished data layers, modify goal models, modify pressures and resilience, calculate OHI scores, and make flowerplots.

Now we have walked thorugh the files, how do they fit into the OHI process? Below is a figure that summarizes how each file will be used in relation to the steps you would take in an assessment for future reference.