Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

284 add support for camtrap dp v1.0 #286

Merged
merged 39 commits into from
Nov 8, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
9e231ef
change supported version to v1.0 only
PietrH Nov 3, 2023
22cc1b3
refer to v1 camtrap-dp example instead of old rc
PietrH Nov 3, 2023
1261a5c
use release instead of commit
PietrH Nov 3, 2023
03cfcbb
check for supporting v1.0 not rc.1
PietrH Nov 3, 2023
a3710b0
no longer use taxonID for join as per camtrap-dp v1.0
PietrH Nov 3, 2023
1c4b612
fail earlier on missing scientificName, no longer use taxonID
PietrH Nov 3, 2023
08809e4
update to test for v1.0 `datapackage.json`
PietrH Nov 3, 2023
d75885a
set error expectation to v1.0
PietrH Nov 3, 2023
bbcb3fd
Downconvert taxonID to NA
PietrH Nov 3, 2023
5fda125
`dp_v1` instead of `dp_v1_rc1`
PietrH Nov 3, 2023
89c0202
refer to `v1.0` not release candidate
PietrH Nov 3, 2023
0615366
downconvert `deployments.cameraDepth` by removing it
PietrH Nov 3, 2023
61fcd0d
remove reference to release candidate
PietrH Nov 3, 2023
5f210ef
remove references to release candidate
PietrH Nov 3, 2023
7761d29
also add taxonIDReference if missing when downconverting
PietrH Nov 3, 2023
391c027
remove code from #282
PietrH Nov 3, 2023
8a31e59
remove references to release candidate
PietrH Nov 3, 2023
62c980d
recode captureMethod factor in downconversion
PietrH Nov 3, 2023
0899146
don't depend on forcats
PietrH Nov 3, 2023
d8e89c5
add test for replacing values in media.captureMethod
PietrH Nov 3, 2023
2ea4878
add test for downconverting captureMethod values in package$project
PietrH Nov 3, 2023
dc42fec
convert captureMethod to v0.1.6 values in `package$project`
PietrH Nov 3, 2023
395506d
remove references to release candidate
PietrH Nov 3, 2023
0813325
end file on linebreak
PietrH Nov 3, 2023
fb477ef
split off test, skip if no NA in deployments.csv
PietrH Nov 6, 2023
d703dfb
always keep deploymentGroup info, see comment in #286
PietrH Nov 6, 2023
9d8361a
avoid writeLines so message can be silenced
PietrH Nov 6, 2023
706592e
skip test if there are no NA in deploymentGroups, pass if one row is …
PietrH Nov 6, 2023
6878f41
Update Camtrap DP section in README
peterdesmet Nov 8, 2023
3f2b840
Update URL of Camtrap DP
peterdesmet Nov 8, 2023
48d5d71
Update supported versions message
peterdesmet Nov 8, 2023
b149c32
Change taxonID to character
peterdesmet Nov 8, 2023
ee4cea4
Simplify glue in messages
peterdesmet Nov 8, 2023
1c77116
Simplify check_value(), don't use custom transformer
peterdesmet Nov 8, 2023
868c307
Don't name regexp parameter
peterdesmet Nov 8, 2023
46e40d7
Bump version
peterdesmet Nov 8, 2023
3472788
Correct version
peterdesmet Nov 8, 2023
48f5626
Create NEWS.md
peterdesmet Nov 8, 2023
b6a0310
Remove check on windows-latest + R 3.6.0
damianooldoni Nov 8, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/workflows/R-CMD-check.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,6 @@ jobs:
- {os: ubuntu-latest, r: 'release'}
- {os: ubuntu-latest, r: 'oldrel-1'}
- {os: ubuntu-latest, r: '3.6.3'}
- {os: windows-latest, r: '3.6.0'}

env:
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
Expand Down
4 changes: 2 additions & 2 deletions DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Package: camtraptor
Title: Read, Explore and Visualize Camera Trap Data Packages
Version: 0.20.1
Version: 0.21.0
Authors@R: c(
person("Damiano", "Oldoni", email = "damiano.oldoni@inbo.be",
role = c("aut", "cre"), comment = c(ORCID = "0000-0003-3445-7562")),
Expand All @@ -22,7 +22,7 @@ Authors@R: c(
role = "fnd", comment = "https://lifewatch.be")
)
Description: Read, explore and visualize Camera Trap Data Packages (Camtrap DP).
'Camtrap DP' (<https://tdwg.github.io/camtrap-dp/>) is a community developed
'Camtrap DP' (<https://camtrap-dp.tdwg.org>) is a community developed
data exchange format for this type of data. With camtraptor you can read and
filter data, create overviews of observed species, relative abundance or
effort, and plot these data on a map.
Expand Down
8 changes: 8 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# camtraptor 0.21.0

- `read_camtrap_dp()` supports Camtrap DP 1.0 (upcoming Agouti export format)
in favour of Camtrap DP 1.0-rc.1 (#284).
To avoid breaking changes to users, it will down-convert Camtrap DP 1.0 to
0.1.6 which is currently used as internal data model for camtraptor.
- `get_custom_effort()` now calculates per calendar month/week (#219).
- `write_dwc()` has an updated mapping for dwc_audubon.csv (#274).
7 changes: 5 additions & 2 deletions R/calc_animal_pos.R
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,11 @@ calc_animal_pos <- function(animal_pos,
assertthat::assert_that(
length(not_found_cols) == 0,
msg = glue::glue(
"Columns `{not_found_cols}` not found in `animal_pos`.",
.transformer = collapse_transformer(sep = "`, `", last = "` and `")
"Columns ",
glue::glue_collapse(
glue::backtick(not_found_cols), sep = ", ", last = " and "
),
" not found in `animal_pos`."
)
)

Expand Down
4 changes: 2 additions & 2 deletions R/check_package.R
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,8 @@ check_package <- function(package = NULL,
assertthat::assert_that(
length(tables_absent) == 0,
msg = glue::glue(
"Can't find {length(tables_absent)} elements in data package: {tables_absent*}",
.transformer = collapse_transformer(sep = ", ", last = " and ")
"Can't find {length(tables_absent)} elements in data package: ",
glue::glue_collapse(tables_absent, sep = ", ", last = " and ")
)
)
if (media) {
Expand Down
2 changes: 1 addition & 1 deletion R/data.R
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#' Sample of Camtrap DP formatted data
#'
#' A sample [Camera Trap Data Package](https://tdwg.github.io/camtrap-dp) as
#' A sample [Camera Trap Data Package](https://camtrap-dp.tdwg.org) as
#' read by [read_camtrap_dp()].
#' The source data are derived from the [Camtrap DP example dataset](
#' https://github.com/tdwg/camtrap-dp/tree/ad0278ef86ef518dacfb306c598dce97667cfb81/example)
Expand Down
5 changes: 2 additions & 3 deletions R/get_n_individuals.R
Original file line number Diff line number Diff line change
Expand Up @@ -87,9 +87,8 @@ get_n_individuals <- function(package = NULL,
if (length(species) > 1) {
ignored_species <- species[!species == "all"]
warning(glue::glue(
"Value `all` found in `species`.",
"All other values are ignored: {ignored_species*}.",
.transformer = collapse_transformer(sep = ", ", last = " and ")
"Value `all` found in `species`. All other values are ignored: ",
glue::glue_collapse(ignored_species, sep = ", ", last = " and ")
))
}
species <- get_species(package)$scientificName
Expand Down
5 changes: 2 additions & 3 deletions R/get_n_obs.R
Original file line number Diff line number Diff line change
Expand Up @@ -84,9 +84,8 @@ get_n_obs <- function(package = NULL,
if (length(species) > 1) {
ignored_species <- species[!species == "all"]
warning(glue::glue(
"Value `all` found in `species`.",
"All other values are ignored: {ignored_species*}.",
.transformer = collapse_transformer(sep = ", ", last = " and ")
"Value `all` found in `species`. All other values are ignored: ",
glue::glue_collapse(ignored_species, sep = ", ", last = " and ")
))
}
species <- get_species(package)$scientificName
Expand Down
2 changes: 1 addition & 1 deletion R/get_prefixes.R
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
#' Stores prefixes for info shown while hovering over a deployment with the
#' mouse.
#' List fields in deployments as in
#' <https://tdwg.github.io/camtrap-dp/data/#deployments>.
#' <https://camtrap-dp.tdwg.org/data/#deployments>.
#'
#' Returns a data.frame of all prefixes with the following columns:
#' - `info`: Deployment info.
Expand Down
6 changes: 3 additions & 3 deletions R/map_dep.R
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@
#' - `latitude`
#' - `longitude`
#'
#' See the [Deployment](https://tdwg.github.io/camtrap-dp/data/#deployments)
#' See the [Deployment](https://camtrap-dp.tdwg.org/data/#deployments)
#' section of Camtrap DP for the full list of columns you can use.
#' @param palette The palette name or the colour function that values will be
#' mapped to.
Expand Down Expand Up @@ -545,8 +545,8 @@ map_dep <- function(package = NULL,
if (n_not_found_cols > 0) {
warning(glue::glue(
"Can't find {n_not_found_cols} columns defined in `hover_columns` in ",
"deployments: {not_found_cols*}",
.transformer = collapse_transformer(sep = ", ", last = " and ")
"deployments: ",
glue::glue_collapse(not_found_col, sep = ", ", last = " and ")
))
}
}
Expand Down
21 changes: 9 additions & 12 deletions R/read_camtrap_dp.R
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#' Read a Camtrap DP
#'
#' Reads files from a [Camera Trap Data Package](
#' https://tdwg.github.io/camtrap-dp) into memory.
#' https://camtrap-dp.tdwg.org) into memory.
#' All datetime information is automatically transformed to Coordinated
#' Universal Time (UTC).
#' Vernacular names found in the metadata (`package$taxonomic`) are added to the
Expand Down Expand Up @@ -80,12 +80,12 @@ read_camtrap_dp <- function(file = NULL,
package <- frictionless::read_package(file)

# supported versions
supported_versions <- c("0.1.6", "1.0-rc.1")
supported_versions <- c("0.1.6", "1.0")

# get package version
profile <- package$profile
if (profile == "https://raw.githubusercontent.com/tdwg/camtrap-dp/1.0-rc.1/camtrap-dp-profile.json") {
version <- "1.0-rc.1"
if (profile == "https://raw.githubusercontent.com/tdwg/camtrap-dp/1.0/camtrap-dp-profile.json") {
version <- "1.0"
} else {
if (profile == "https://raw.githubusercontent.com/tdwg/camtrap-dp/0.1.6/camtrap-dp-profile.json") {
version <- "0.1.6"
Expand All @@ -97,13 +97,10 @@ read_camtrap_dp <- function(file = NULL,
# check version is supported
assertthat::assert_that(
version %in% supported_versions,
msg = paste0(
glue::glue("Version {version} "),
"is not supported. Supported versions: ",
glue::glue_collapse(glue::glue("{supported_versions}"),
sep = " ",
last = " and "),
".")
msg = glue::glue(
"Version `{version}` is not supported. Supported versions: ",
glue::glue_collapse(supported_versions, sep = " ", last = " and ")
)
)

# get resource names
Expand Down Expand Up @@ -152,7 +149,7 @@ read_camtrap_dp <- function(file = NULL,
package <- add_taxonomic_info(package)

# convert to 0.1.6
if (version == "1.0-rc.1") {
if (version == "1.0") {
package <- convert_to_0.1.6(package, version, media = media)
}

Expand Down
10 changes: 5 additions & 5 deletions R/read_wi.R
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
#' [private](https://www.wildlifeinsights.org/get-started/download/private)
#' download.
#' The function transforms data and metadata to a [Camera Trap Data Package](
#' https://tdwg.github.io/camtrap-dp) which can be written to file with
#' https://camtrap-dp.tdwg.org) which can be written to file with
#' [frictionless::write_package()].
#'
#' **The function has only been tested on image-based projects.**
Expand Down Expand Up @@ -61,7 +61,7 @@ read_wi <- function(directory = ".") {
# Create package
package <- frictionless::create_package() # Also sets profile, resources

# Set metadata properties, see https://tdwg.github.io/camtrap-dp/metadata
# Set metadata properties, see https://camtrap-dp.tdwg.org/metadata
package$name <- basename(directory) # Unique name if unchanged from WI export zip
package$id <- wi_project$ark_id # (e.g. http://n2t.net/ark:/63614/w12001317)
package$created <- lubridate::format_ISO8601(lubridate::now())
Expand Down Expand Up @@ -229,7 +229,7 @@ read_wi <- function(directory = ".") {
# packageID = ""
)

# Create deployments, see https://tdwg.github.io/camtrap-dp/data/#deployments
# Create deployments, see https://camtrap-dp.tdwg.org/data/#deployments
deployments <-
wi_deployments %>%
dplyr::left_join(wi_cameras, by = c("project_id", "camera_id")) %>%
Expand Down Expand Up @@ -298,7 +298,7 @@ read_wi <- function(directory = ".") {
`_id` = NA_character_
)

# Create media, see https://tdwg.github.io/camtrap-dp/data/#media
# Create media, see https://camtrap-dp.tdwg.org/data/#media
media <-
wi_images %>%
dplyr::distinct(.data$location, .keep_all = TRUE) %>%
Expand All @@ -317,7 +317,7 @@ read_wi <- function(directory = ".") {
`_id` = NA_character_
)

# Create observations, see https://tdwg.github.io/camtrap-dp/data/#observations
# Create observations, see https://camtrap-dp.tdwg.org/data/#observations
observations <-
wi_images %>%
dplyr::transmute(
Expand Down
2 changes: 1 addition & 1 deletion R/write_dwc.R
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#' Transform Camtrap DP data to Darwin Core
#'
#' Transforms data from a [Camera Trap Data Package](
#' https://tdwg.github.io/camtrap-dp/) to [Darwin Core](https://dwc.tdwg.org/).
#' https://camtrap-dp.tdwg.org) to [Darwin Core](https://dwc.tdwg.org/).
#' The resulting CSV files can be uploaded to an [IPT](
#' https://www.gbif.org/ipt) for publication to GBIF.
#' A `meta.xml` file is included as well.
Expand Down
6 changes: 3 additions & 3 deletions R/write_eml.R
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
#' Transform Camtrap DP metadata to EML
#'
#' Transforms the metadata of a [Camera Trap Data Package](
#' https://tdwg.github.io/camtrap-dp/) to an [EML](
#' https://eml.ecoinformatics.org/) file that can be uploaded to a [GBIF IPT](
#' https://www.gbif.org/ipt) for publication.
#' https://camtrap-dp.tdwg.org) to an [EML](https://eml.ecoinformatics.org/)
#' file that can be uploaded to a [GBIF IPT](https://www.gbif.org/ipt) for
#' publication.
#'
#' @param package A Camtrap DP, as read by [read_camtrap_dp()].
#' @param directory Path to local directory to write file to.
Expand Down
Loading