View on GitHub

Biodiversity Specimen Data

a collection of specific use cases of biodiversity specimen data, documented conceptually and, where possible, linked to technical solutions

Code here written by [Erica Krimmel](https://orcid.org/0000-0003-3192-0080). Please see **Use Case: [Download media based on specimen record search](https://biodiversity-specimen-data.github.io/specimen-data-use-case/use-case/download-media-from-specimens)** for context. _If you are running this code on your own computer, you may wish to create a new folder for the working directory and save this file to it. This code will create two new subdirectories and save media files to them._ ```{r message=FALSE} # Load core libraries; install these packages if you have not already library(ridigbio) library(tidyverse) # Load library for making nice HTML output library(kableExtra) ``` ## Write a query to search for specimen records First, you need to find all the media records for which you are interested in downloading media files. Do this using the `idig_search_media` function from the ridigbio package, which allows you to search for media records based on data contained in linked specimen records, like species or collecting locality. You can learn more about this function from the [iDigBio API documentation](https://github.com/iDigBio/idigbio-search-api/wiki) and [ridigbio documentation](https://cran.r-project.org/web/packages/ridigbio/ridigbio.pdf). In this example, we want to search for images of herbarium specimens of species in the genus _Acer_ that were collected in the United States. ```{r} # Edit the fields (e.g. `genus`) and values (e.g. "manis") in `list()` # to adjust your query and the fields (e.g. `uuid`) in `fields` to adjust the # columns returned in your results; edit the number after `limit` to adjust the # number of records you will retrieve images for records <- idig_search_media(rq = list(genus = "acer", country = "united states"), fields = c("uuid", "accessuri", "rights", "format", "records"), limit = 10) ``` The result of the code above is a data frame called `records`: ```{r echo = FALSE, results = 'asis'} knitr::kable(records) %>% kable_styling(bootstrap_options = c("striped", "hover", "condensed", "responsive")) %>% scroll_box(width = "100%") ``` ## Generate a list of URLs to download media from Now that we know what media records are of interest to us, we need to isolate the URLs that link to the actual media files so that we can download them. In this example, we will demonstrate how to download files that are cached on the iDigBio server, as well as the original files hosted externally by the data provider. You likely do not need to download two sets of images, so can choose to comment out the steps related to either "_iDigBio" or "_external" depending on your preference. ``` {r} # Assemble a vector of iDigBio server download URLs from `records` mediaurl_idigbio <- records %>% mutate(mediaURL = paste("https://api.idigbio.org/v2/media/", uuid, sep = "")) %>% select(mediaURL) %>% pull() # Assemble a vector of external server download URLs from `records` mediaurl_external <- records$accessuri %>% str_replace("\\?size=fullsize", "") ``` These vectors look like this: ```{r} mediaurl_idigbio ``` ```{r} mediaurl_external ``` ## Download media We can use the download URLs that we assembled in the step above to go and download each media file. For clarity, we will place files in two different folders, based on whether we downloaded them from the iDigBio server or an external server. We will name each file based on its unique identifier. ``` {r message=FALSE} # Create new directories to save media files in dir.create("jpgs_idigbio") dir.create("jpgs_external") # Assemble another vector of file paths to use when saving media downloaded # from iDigBio mediapath_idigbio <- paste("jpgs_idigbio/", records$uuid, ".jpg", sep = "") # Assemble another vector of file paths to use when saving media downloaded # from external servers; please note that it's probably not a great idea to # assume these files are all jpgs, as we're doing here... mediapath_external <- paste("jpgs_external/", records$uuid, ".jpg", sep = "") # Add a check to deal with URLs that are broken links possibly_download.file = purrr::possibly(download.file, otherwise = "cannot download") "mode" argument (="wb") in the walk function to download.file. # Iterate through the action of downloading whatever file is at each # iDigBio URL purrr::walk2(.x = mediaurl_idigbio, .y = mediapath_idigbio, possibly_download.file) # Iterate through the action of downloading whatever file is at each # external URL purrr::walk2(.x = mediaurl_external, .y = mediapath_external, possibly_download.file) ``` You should now have two folders, each with ten images downloaded from iDigBio and external servers, respectively. Note that we only downloaded ten images here for brevity's sake, but you can increase this using the `limit` argument in the first step. Here is an example of one of the images we downloaded: ```{r include=FALSE} exampleimgpath <- paste("jpgs_idigbio/",records$uuid[1],".jpg", sep = "") ``` ![Herbarium specimen of an _Acer_ species collected in the United States](`r as.character(exampleimgpath)`)