With etn you can query metadata (animals, tags, deployments, receivers, projects) and data (acoustic detections, sensor data) from the European Tracking Network (ETN) database and use these in your analyses. Data access requires user credentials and is subject to a data policy.

Because of connection restrictions, previous versions of the package could only be used on the VLIZ RStudio server. That is now no longer the case! 🎉

Use etn on your computer

• etn now connects to the ETN database with an API provided by the etnservice package. This means you can use the package from your own computer. Note that this will be slower than running it from the ETN RStudio server.
• etn will automatically switch to a local database connection when available (e.g. the ETN RStudio server). Use Sys.setenv(ETN_PROTOCOL = "opencpu") to override this behaviour and force the package to use the API.
• Queries via the API and the ETN RStudio Server will return the same results.
• When using a local database connection, etn will check if the installed helper package etnservice that is used to place these queries is up to date with the one deployed via the API. This is to ensure that queries placed via the API and via the local database connection always result in consistent results. If the installed version of etnservice is older, you will be prompted to install a newer version.

Credentials

Breaking change! Action required

Your credentials (username and password) to connect to the ETN database are no longer passed via the connection argument. They are asked or retrieved from your .Renviron file every time you run a function.

• New authentication mechanism.
• New vignette vignette("authentication").
• connection argument is deprecated in all functions.
• connect_to_etn() is deprecated.

Here is how you can migrate:

1. Use the new ETN RStudio server (https://rstudio.europeantrackingnetwork.org). The LifeWatch RStudio server (https://rstudio.lifewatch.be) won’t work with this version of etn and will be discontinued.
2. Follow the steps in vignette("authentication") to look up and store your credentials.

3. Update your scripts:

   # Good
   get_animals(animal_id = 305)
   
   # Bad
   connect_to_etn()
   get_animals(con, animal_id = 305)
   get_animals(connection = con, animal_id = 305)
   

Accessing data

• get_acoustic_detections() now uses a different protocol to retrieve data. It can now reliably return 10M+ detections without timeouts.
• get_acoustic_detections() now returns a progress bar for large queries.
• get_acoustic_detections() now has a deployment_id filter argument.
• get_acoustic_detections() now has a tag_serial_number argument, which is more reliable than acoustic_tag_id (which is still supported). Thanks @lottepohl for the suggestion.
• get_acoustic_detections() may return fewer (erroneous) detections than before, due to fixes in the database.
• get_animals() now includes type_type = "archival" data.

Changes for developers

• New vignette vignette("options") describes some developer options.
• Contributors can change the default domain of the API to the URL of a test server via the environmental variable ETN_TEST_API.
• Tests make use of {vcr} to record and replay HTTP requests to the API. These results are stored in /tests/fixtures.
• Tests have new helper functions, including skip_if_not_localdb(), skip_if_http_error() and expect_protocol_agnostic(). The latter is used to compare SQL vs API calls in test-protocol_agnostic.R.
• Tests for download_acoustic_datasets() are updated for archival tags and SQL vs API calls and makes use of markdown snapshots.
• pkgdown website is now automatically build by a GitHub Action and is served from the gh-pages branch. The docs/ directory has been removed.
• vignette("acoustic-telemetry") is precompiled with vignettes/precompile.R, so it doesn’t run for every build.

Miscellaneous

• etn now relies on R >= 4.1.0 (because of {arrow} and {vcr} dependencies) and uses base pipes (|> rather than %>%).
• write_dwc() now invisibly returns the transformed data as a list of data frames (rather than a data frame).
• Previously deprecated functions get_deployments(), get_detections(), get_projects(), get_receivers(), list_network_project_codes() are now removed.
• vignette("etn_fields") was outdated and has been removed.

How to install etn?

Want to use etn in your work? The package is available on GitHub and can be installed with:

# install.packages("pak")
pak::pak("inbo/etn")

For more information, see the package documentation. Found a bug? Please report an issue.

With movepub you prepare animal tracking data from Movebank for publication in a research repository. You can document data with metadata following the Data Package standard and transform these to Darwin Core and Ecological Metadata Language (EML) for publication to the Global Biodiversity Information Facility (GBIF) and the Ocean Biodiversity Information System (OBIS).

What has changed?

We added two new functions:

• write_eml() is now a separate function from write_dwc(). write_eml() now formats the derived_paragraph as DocBook (rather than HTML), a format supported by EML and the GBIF IPT.
• html_to_docbook() allows to convert a string or character vector from HTML to DocBook. You can use this to convert descriptions in order to have valid EML.

We also updated write_dwc():

• write_dwc() can now be used for an unpublished dataset (i.e. without metadata on DataCite, which was required for the previously build-in write_eml() functionality). Some record-level terms (e.g. dwc:datasetName) can be provided as arguments.
• write_dwc() (and write_eml()) no longer writes to "." by default, since this is not allowed by CRAN policies. The user needs to explicitly define a directory.
• write_dwc() now writes the output file as occurrence.csv (previously dwc_occurrence.csv) and adds a meta.xml file. The sex and life stage of the animal are - in addition to dwc:sex and dwc:lifeStage in occurrence.csv - expressed in an extended measurement or facts file (emof.csv), for better support with OBIS.
• write_dwc() provides a message regarding the matching of scientific names with WoRMS Aphia IDs. These IDs are now clickable URLs, making it easier to verify the match.
• write_dwc() is now more modular, facilitating extension for non-GPS tracking data.

And there are other changes:

• movepub now relies on R >= 4.1.0 (because of move2 dependency) and uses base pipes (|> rather than %>%) (#98).
• Many functions of frictionless are now reexported by movepub, so you no longer have to load that package to create Data Packages (#54).
• get_mvb_term() is deprecated in favour of move2::movebank_get_vocabulary()

For a full overview of all the changes, see the CHANGELOG.

How to install movepub?

Want to use movepub in your work? The package is available on GitHub and can be installed with:

devtools::install_github("inbo/movepub")

For more information, see the package documentation. Found a bug? Please report an issue.

With camtrapdp you can read and manipulate Camera Trap Data Packages (Camtrap DP). Camtrap DP is a data exchange format for camera trap data. The R package supports reading, filtering and transforming data (including to Darwin Core) before further analysis in e.g. camtraptor or camtrapR.

What has changed?

We added two long-requested functions:

• write_camtrapdp() to write a Camtrap DP dataset to disk.
• merge_camtrapdp() to merge two datasets, allowing you to combine data from multiple studies.

We also added new functions to:

• Update (or clean) data: round_coordinates(), shift_time(), update_taxon().
• Extract data: contributors(), individuals().

The Darwin Core processing was also improved, allowing GBIF to ingest Camtrap DP datasets. This includes a new function write_eml() to better populate dataset landing pages with metadata.

For a full overview of all the changes, see the CHANGELOG.

How to install camtrapdp?

Want to use camtrapdp in your work? The package is available on CRAN and can be installed with:

install.packages("camtrapdp")

For more information, see the package documentation. Found a bug? Please report an issue.

Billions of flying animals move through the air, day and night. Due to the scale, volume and high altitudes, the scope of these movements are hard to study through observations or biologgers. Weather radars however continuously monitor the sky across large areas, registering not only weather, but also the movements of animals in the air. Archives of weather radar data are therefore goldmines for biological monitoring, providing an overview of the year-round activity of animals in the air in a way no other method can. Even though weather radar networks exist in many countries, they are usually operated on a national basis so combining biological data from several such networks across Europe has been a decade-long challenge.

We have now processed and published the first large-scale datasets of biological activity in the air, derived from weather radars across a large part of Europe. The radar data were obtained from the Operational Programme for the Exchange of Weather Radar Information (OPERA) at EUMETNET and three national meteorological services. The resulting datasets are a major milestone for the aeroecology community and the result of long-term interdisciplinary collaborations in the projects ENRAM, GloBAM and HiRAD.

Visualization of biological activity in the air, derived from radar SEANG in Ängelholm, Sweden (see blue dot on map). The charts show data from October 14 to 16, 2024. The top chart depicts the number of birds flying over the location per hour. The bottom chart shows at what altitudes the birds are flying (yellow = high density). Explore the data yourself at crow.aloftdata.eu.

The data consist of measures of all biological activity in the air surrounding the radar stations, stored as vertical profiles that describe the amount, speed and directions of animals at different altitudes. These were obtained by processing weather radar data with dedicated and validated methods for detecting birds and especially bird migration, but the data in itself do not offer any distinction between birds, insects or bats, so it must be interpreted through biological knowledge of what species groups are expected to be moving during different periods and places. Likewise, careful consideration of the variation in data coverage and quality is necessary. For more information on the intricacies of this exciting new datasets, we have provided usage notes in our paper.

The paper describes two public datasets that collectively contain biological data from 141 radar stations in 18 European countries, from the years 2008 to 2023. You can download these data from Zenodo or explore and visualize them in our Aloft data portal at aloftdata.eu. The portal is also updated daily with new data.

Weather radar data gives us a glimpse into life in the air, shedding light on the movements that take place above our heads. It has great potential to be used for conservation efforts, such as mapping where and how high birds fly in relation to anthropogenic threats, mapping diseases that spread along flyways, and highlighting periods of high bird activity for flight safety applications. These new datasets offer tremendous value for researchers studying flying animal movements and for enthusiasts simply witnessing these aerial spectacles. We look forward to seeing the creative, important and urgent applications that emerge as these biological data reach the wider scientific and conservation community in the coming years.

Geolocators are essential tools for studying bird movement, enabling researchers to gain insights into migration and behavior. However, managing and sharing geolocator data consistently remains challenging. To address this, I developed the Geolocator Data Package (Geolocator DP), an extension of the Data Package standard, designed specifically for geolocator data.

The development of Geolocator DP was greatly facilitated by the Camera Trap Data Package (Camtrap DP), which is also build upon the Data Package standard. Camtrap DP provided a robust foundation that I could adapt to the needs of geolocator data. I was also inspired by the Camtrap DP website to create a Geolocator DP website, reusing the same free Petridish theme.

To further enhance usability, I developed the R package GeoLocatoR. Just like the R package camtrapdp, it makes use of the R package frictionless to read and manipulate data. GeoLocatoR allows researchers to seamlessly create, manage, and validate geolocator data packages. It is designed to work in tandem with the geolocator analysis R package GeoPressureR, which retrieves bird locations by combining light, activity, and pressure measurements. Together, GeoPressureR and GeoLocatoR provide a cohesive pipeline for analyzing geolocator data and publishing findings in movement ecology research.

I extend my gratitude to the individuals and teams behind the tools that made this work possible. In particular, I want to thanks the Research Institute for Nature and Forest (INBO), Frictionless Data, and Biodiversity Information Standards (TDWG) for releasing their software open source. Diving into source code is a really useful way to learn how functionality can be implemented and build upon. I also wish to express my gratitude to Peter Desmet, whose advice and contributions have been instrumental.

These efforts exemplify the power of open science and collaboration, enabling researchers to address data challenges effectively and advance movement ecology research.

Tools used

We developed the R package frictionless to facilitate our work, but it has since been picked up by other users and developers. Much of the feedback we gathered is now incorportated in a new release of the package: version 1.1.0.

How to install frictionless?

The package is available on CRAN and can be installed with:

install.packages("frictionless")

For more information, see the package documentation.

What has changed?

Quite a lot. See the CHANGELOG.

What is next?

The Data Package standard is currently undergoing changes to be released as a finalized product (Data Package v2), a process we are involved in. We plan to support these changes (and new features!) in an upcoming version of frictionless, so there’s more to come!

GPS trackers are used by ecologists to follow the movements of a wide range of animal species and gain unique insights into their behaviour. Thousands of researchers manage and share these data on Movebank, a free data platform hosting more than 7,500 animal tracking studies, many of which have made their data available for anyone to view and download. These same data offer evidence of when and where species occur, information widely used for biodiversity research and assessments and found through the open data portals Global Biodiversity Information Facility (GBIF) and Ocean Biodiversity Information Facility (OBIS). However, these platforms are not linked, and with few tracking datasets available through GBIF and OBIS, there is unmet potential for them to contribute to studies of biodiversity. To build connections between these data and disciplines, a team of researchers from Radboud University, Research Institute for Nature and Forest, Sovon, the Dutch Bryological and Lichenological Society, and the Max Planck Institute of Animal Behavior have developed an open-source workflow to publish animal tracking data stored on Movebank to GBIF and OBIS.

Oystercatchers as a test case

The researchers demonstrated the new workflow using tracking data from Eurasian oystercatchers (Haematopus ostralegus), collected since 2008 as part of multiple projects, primarily based in the Wadden Sea. Across more than a decade, ecologists equipped over 200 oystercatchers with GPS trackers at their breeding and wintering grounds in the Netherlands and Belgium, resulting in an improved understanding of this species’ behaviour. “We always thought oystercatchers returned, almost stubbornly, to the exact same breeding and wintering location year after year,” says Henk-Jan van der Kolk, who was involved the study. “By following individuals over time with GPS, we discovered that although this was indeed true for many individuals, there were also others that visited a variety of different locations. This exploratory behaviour was especially clear in young individuals.” The tracking data, collected using the University of Amsterdam Bird Tracking System, consist of a total of 6 million GPS locations and document oystercatchers migrating from the Netherlands to Scandinavian countries.

Oystercatchers on Vlieland, the Netherlands. The left individual is equipped with a GPS tracker and colour rings.

Technical challenge

The key challenge to sharing data between platforms is not scientific, but technical. “They store slightly different information, in different formats, with many possible ways to ‘translate’ between them,” explains the project’s lead researcher, Peter Desmet. To make it easier for researchers to prepare data from Movebank for publication in GBIF and OBIS, the researchers developed an open-source software package, “movepub”, that automatically reads data in the Movebank format and converts it to files that can be imported to GBIF and OBIS. To offer one data transformation that can be applied to many studies and species, the researchers considered which information is most commonly collected in tracking studies, and which is most likely to be important for biodiversity research. For example, “our workflow reduces high-resolution GPS data to hourly positions per animal, in line with expected applications of those using GBIF and OBIS for their work,” describes Desmet. Using the new workflow, the oystercatcher tracking data are now publicly available on Movebank, GBIF and OBIS, and are described in a scientific article published in the journal ZooKeys.

Oystercatcher GPS tracks from Vlieland are available via Movebank, GBIF and OBIS.

Wider use

The researchers hope that this approach will be adopted by other research groups, and eventually, build new connections between movement ecologists and biodiversity experts. Already, data for several other bird species have been published using the new workflow. “Increased access to animal tracking data offers the possibility to improve our understanding of biodiversity and contribute to assessments, policy decisions and conservation efforts,” says Sarah Davidson, the curator for Movebank. GBIF and OBIS have been used for thousands of biodiversity studies worldwide: as global efforts increase to protect biodiversity and animal populations, so does the value of this underrecognized source of information about species distributions and behavior.

Learn more

The MOVE2GBIF project is funded by the Netherlands Biodiversity Information Facility. Further descriptions of the data and software published through this project are published at Zenodo (https://doi.org/10.5281/zenodo.7408643) and ZooKeys (https://doi.org/10.3897/zookeys.1123.90623).

Text: Henk-Jan van der Kolk (BLWG), Peter Desmet (INBO), Sarah Davidson (Movebank), Bruno Ens (Sovon) & Eelke Jongejans (Radboud University)

Pictures: Henk-Jan van der Kolk

LifeWatch and Elixir Belgium invite you to a hands-on workshop to manage and publish research data. The workshop is organized in the context of the Empowering Biodiversity Research conference and will cover data management plans, data cleaning and GitHub. If this is jargon you always wanted to know about (but were afraid to ask), then this workshop is for you!

Who

This workshop is aimed at environmental and life scientists that manage research data, from PhD students to senior scientists. It is an introductory workshop to data management, data cleaning and data publishing. We recommend participants to have a basic understanding of R and its data types. We can send resources for people who need an introduction to R before this workshop. The level of this workshop is from beginner to intermediate, we will not cover advanced materials.

When

Tuesday September 27, 2022

Where

Herman Teirlinck building, Havenlaan 88, 1000 Brussels (room 01.70, Ferdinand Peeters)

Requirements

Participants should bring a laptop with a Mac or Windows operating system (not a tablet, Chromebook, etc.) and administrative privileges. Please install the following programs beforehand:

• a text editor (such as atom or notepad++)
• a spreadsheet editor (excel or libreoffice)
• RStudio (open source desktop edition)
• GitHub Desktop

Programme

Today the Open science lab for biodiversity at the INBO and the Royal Meteorological Institute of Belgium (RMI) are launching a web application allowing anyone to view this migration in real time across the Benelux. It shows that bird migration started exceptionally early this year, caused by the unusually warm weather in the second half of February. The development of the website was co-supervised by the Royal Belgian Institute of Natural Sciences (RBINS) and financially supported by the Federal Science Policy (BelSPO valorisation project CROW).

The new web application, showing the Helchteren radar for a three days period around February 21, 2021. Due to the exceptionally warm weather during that period, spring migration started earlier than normal. From the figure it is also clear that the most intense migration always takes place in the first half of the night. The birds reach heights of up to three kilometers (and sometimes even more).

What does meteorology have to do with bird migration? A lot, it turns out, and in both directions. Birds wait for the ideal weather conditions to start their long journey to their wintering grounds in the south and back to their breeding grounds in the other direction. But equally intriguing is the fact that one of the most important measuring instruments in meteorology today, the weather radar, can measure this migration in detail. Indeed: weather radars do not only detect water droplets in the atmosphere, but also birds moving through the air.

The website launched today offers a unique overview of bird migration across the Benelux, based on ten weather radars located in Belgium, the Netherlands, France and Germany. The graph that is shown first when loading the website is the so-called “MTR” (Migration Traffic Rate). This parameter expresses the number of birds that pass through the vicinity of the radar at a given moment, through an imaginary screen that is 1 km wide and “infinitely” high (in practice, 5 km is taken as maximum height). It can be compared with the “stream” or the “flux” of birds that “flows” through the air.

The colourful figure underneath shows how these birds are distributed across the vertical layers of the atmosphere. The parameter shown in this figure is the bird density, expressed as the number of birds per cubic kilometer. However, certain assumptions have to be made to arrive at this number (e.g. about the typical size of a migratory bird). It also excludes bird movements close to the ground, as radars are not able to detect those. More explanation about the correct interpretation is provided on the website itself.

As usual for our team, we developed this tool as open source software. We hope to extend it to visualize even more open bird migration data for the GloBAM project.

• INBO press release (in Dutch)
• Short video on radar aeroecology

The vast majority of species distribution records come from citizen scientists. These data complement records from professional researchers and help global efforts to conserve biodiversity and understand the natural world. Invasive alien species policies heavily depend on citizen science data. Many invasive alien species related research and policy activities require pooling data. Therefore, open data publication, standardization and interoperability are particularly important. Without good data publishing practises, these data get lost or become useless. With time, the link between the data and the methods that were used to collect them gets lost. The FAIR Principles have become the gold standard for publishing scientific data. They ensure that the data are Findable, Accessible, Interoperable and Reusable, and can be used for science indefinitely. Nevertheless, these are generic principles for any kind of data and there is a need to further refine and tailor them specifically for biodiversity data collected by citizen scientists.

The primary place for publishing open biodiversity observation data is the Global Biodiversity Information Facility (GBIF). By publishing data through GBIF, many of the FAIR principles are met automatically. For example, a dataset published through GBIF is accompanied by a Digital Object Identifier (DOI) which makes the dataset findable using a searchable resource such as Google Dataset Search. GBIF also obliges users to conform to community standards, such as Darwin Core and Ecological Metadata Language. This makes the data interoperable. Yet despite this, the richness of the metadata and attributes (principles F2 and R1, https://go-fair.org/fair-principles) associated with the dataset really depends on how well publishers implement these standards. Rich metadata should provide information about the context, quality and characteristics of the data and should allow a computer to accomplish routine sorting. However, the richness of metadata associated with citizen science datasets varies wildly, and it is not that we need to set the bar higher, it is that the bar is not visible. We need clear, achievable guidelines for these datasets. These guidelines should be simple and available in a diversity of languages, so that every researcher involved in alien species citizen science projects can understand and apply them.

Within Working Group 3 (Data Management and Standards) of the COST Action 17122 “Increasing understanding of alien species through citizen science” (Alien CSI), we aim to develop metadata guidelines for publishing citizen scientist datasets on alien species. However, these guidelines should be tailored to the needs of any citizen scientist and researcher. With this blogpost, we are looking for collaboration with citizen science researchers, project coordinators or citizen science system owners, although everyone is welcome to join us. More specifically, we are looking for contributions to (1) formulate metadata guidelines for publishing citizen science datasets, (2) generate an informative leaflet to be distributed amongst citizen scientists and (3) translate these recommendations to a multitude of languages. Discussions will be organized on a monthly basis via conference call. Knowledge exchange and acquisition can also be facilitated through a Short Time Scientific Mission (STSM). Anyone willing to contribute to this specific work or to engage in an STSM is highly welcome to join our working group. Please contact lien.reyserhove@inbo.be for further information.