etn 0.3.0

We just released a new major version (3.0.0) of our R package etn.

With etn you can query metadata (animals, tags, deployments, receivers, projects) and data (acoustic detections, sensor data) from the European Tracking Network (ETN) database and use these in your analyses. Data access requires user credentials and is subject to a data policy.

Because of connection restrictions, previous versions of the package could only be used on the VLIZ RStudio server. That is now no longer the case! 🎉

Use etn on your computer

• etn now connects to the ETN database with an API provided by the etnservice package. This means you can use the package from your own computer. Note that this will be slower than running it from the VLIZ RStudio server.
• etn will automatically switch to a local database connection when available (e.g. the VLIZ RStudio server). Use Sys.setenv(ETN_PROTOCOL = "opencpu") to override this behaviour and force the package to use the API.
• Queries via the API and the VLIZ RStudio Server will return the same results.
• When using a local database connection, etn will check if the installed helper package etnservice that is used to place these queries is up to date with the one deployed via the API. This is to ensure that queries placed via the API and via the local database connection always result in consistent results. If the installed version of etnservice is older, you will be prompted to install a newer version.

Credentials

Breaking change! Action required

Your credentials (username and password) to connect to the ETN database are no longer passed via the connection argument. They are asked or retrieved from your .Renviron file every time you run a function.

• New authentication mechanism.
• New vignette vignette("authentication").
• connection argument is deprecated in all functions.
• connect_to_etn() is deprecated.

Here is how you can migrate:

1. Use the new RStudio server (https://rstudio4.vliz.be/). The LifeWatch RStudio server (https://rstudio.lifewatch.be) won’t work with this version of etn and will be discontinued.
2. Follow the steps in vignette("authentication") to look up and store your credentials.

3. Update your scripts:

   # Good
   get_animals(animal_id = 305)
   
   # Bad
   connect_to_etn()
   get_animals(con, animal_id = 305)
   get_animals(connection = con, animal_id = 305)
   

Accessing data

• get_acoustic_detections() now uses a different protocol to retrieve data. It can now reliably return 10M+ detections without timeouts.
• get_acoustic_detections() now returns a progress bar for large queries.
• get_acoustic_detections() now has a deployment_id filter argument.
• get_acoustic_detections() now has a tag_serial_number argument, which is more reliable than acoustic_tag_id (which is still supported). Thanks @lottepohl for the suggestion.
• get_acoustic_detections() may return fewer (erroneous) detections than before, due to fixes in the database.
• get_animals() now includes type_type = "archival" data.

Changes for developers

• New vignette vignette("options") describes some developer options.
• Contributors can change the default domain of the API to the URL of a test server via the environmental variable ETN_TEST_API.
• Tests make use of {vcr} to record and replay HTTP requests to the API. These results are stored in /tests/fixtures.
• Tests have new helper functions, including skip_if_not_localdb(), skip_if_http_error() and expect_protocol_agnostic(). The latter is used to compare SQL vs API calls in test-protocol_agnostic.R.
• Tests for download_acoustic_datasets() are updated for archival tags and SQL vs API calls and makes use of markdown snapshots.
• pkgdown website is now automatically build by a GitHub Action and is served from the gh-pages branch. The docs/ directory has been removed.
• vignette("acoustic-telemetry") is precompiled with vignettes/precompile.R, so it doesn’t run for every build.

Miscellaneous

• etn now relies on R >= 4.1.0 (because of {arrow} and {vcr} dependencies) and uses base pipes (|> rather than %>%).
• write_dwc() now invisibly returns the transformed data as a list of data frames (rather than a data frame).
• Previously deprecated functions get_deployments(), get_detections(), get_projects(), get_receivers(), list_network_project_codes() are now removed.
• vignette("etn_fields") was outdated and has been removed.

How to install etn?

Want to use etn in your work? The package is available on GitHub and can be installed with:

# install.packages("pak")
pak::pak("inbo/etn")

For more information, see the package documentation. Found a bug? Please report an issue.