Error when trying to read .json in R (Agouti)

[peterdesmet]

Reported by @katzikatz at tdwg/camtrap-dp#388 (comment)

Hi there,
I am looking for some help, I am experiencing an error when I am trying to read the file in R Studio after unzipping the Agouti Download.

I have looked at the .csv separately and they look fine, I am not sure where the problem lies. Can someone help? Thank you so much!

kind regards,
Katzi

[damianooldoni]

Hi @katzikatz! Thanks for reporting.

Based on the error message and backtrace it seems that the function cannot read the medila csv file. Can you send me the zip file datapackage? My e-mail: damiano.oldoni@inbo.be.

Meanwhile, can you post here below the output of devtools::session_info()? If you don't have the devtools package installed, utils::sessionInfo() is also good. Thanks.

[damianooldoni]

Hi @katzikatz. I could reproduce the error. No urgent need to send me a detailed session info.

After a first debugging, I see that you are trying to read a media.csv file with a field, agoutiEventID, not defined in the provided schema in datapackage.json, i.e. https://raw.githubusercontent.com/tdwg/camtrap-dp/1.0/media-table-schema.json. This issue is known now by @peterdesmet: we will work further to solve this issue.

However, if you don't need media data at the moment, you could use media = FALSE flag: read_camtrap_dp(path_to_your_json_file, media = FALSE).

It's a short term solution, I know, but I hope it helps.

[katzikatz]

Thank you! I wrote to the Agouti ppl asking about that new field. How weird, I can't find any information on their page what this is about, I last exported data a month ago and had no problem.
I think I need media data for the ctdp package I am using, I am not 100% sure, but the commands for that package also produce an error. I hope we can solve this somehow.
I am going to be cheeky and send you my R Script per mail, if that's okay. Maybe you know away around, when you see what info I need.

also this is the session_info() output:

devtools::session_info()
─ Session info ───────────────────────────────────────────────────────────────────────────────────────────────────
setting value
version R version 4.3.2 (2023-10-31 ucrt)
os Windows 10 x64 (build 19045)
system x86_64, mingw32
ui RStudio
language (EN)
collate German_Germany.utf8
ctype German_Germany.utf8
tz Europe/Berlin
date 2024-09-19
rstudio 2023.09.1+494 Desert Sunflower (desktop)
pandoc 3.1.1 @ C:/Program Files/RStudio/resources/app/bin/quarto/bin/tools/ (via rmarkdown)

─ Packages ───────────────────────────────────────────────────────────────────────────────────────────────────────
package * version date (UTC) lib source
activity * 1.3.4 2023-09-27 [1] CRAN (R 4.3.3)
assertthat 0.2.1 2019-03-21 [1] CRAN (R 4.3.3)
bit 4.0.5 2022-11-15 [1] CRAN (R 4.3.2)
bit64 4.0.5 2020-08-30 [1] CRAN (R 4.3.2)
bslib 0.6.1 2023-11-28 [1] CRAN (R 4.3.2)
cachem 1.0.8 2023-05-01 [1] CRAN (R 4.3.2)
callr 3.7.5 2024-02-19 [1] CRAN (R 4.3.2)
camtraptor * 0.25.0 2024-09-19 [1] Github (f62b937)
cli 3.6.2 2023-12-11 [1] CRAN (R 4.3.2)
colorspace 2.1-0 2023-01-23 [1] CRAN (R 4.3.2)
cowplot * 1.1.3 2024-01-22 [1] CRAN (R 4.3.3)
cpp11 0.4.7 2023-12-02 [1] CRAN (R 4.3.2)
crayon 1.5.2 2022-09-29 [1] CRAN (R 4.3.2)
crosstalk 1.2.1 2023-11-23 [1] CRAN (R 4.3.3)
ctdp * 0.4.0 2024-09-19 [1] gitlab (camtrap/ctdp@89898e1)
curl * 5.2.0 2023-12-08 [1] CRAN (R 4.3.2)
data.table * 1.15.0 2024-01-30 [1] CRAN (R 4.3.2)
desc 1.4.3 2023-12-10 [1] CRAN (R 4.3.2)
devtools 2.4.5 2022-10-11 [1] CRAN (R 4.3.3)
digest 0.6.34 2024-01-11 [1] CRAN (R 4.3.2)
dplyr * 1.1.4 2023-11-17 [1] CRAN (R 4.3.2)
DT * 0.33 2024-04-04 [1] CRAN (R 4.3.3)
dygraphs * 1.1.1.6 2018-07-11 [1] CRAN (R 4.3.3)
ellipsis 0.3.2 2021-04-29 [1] CRAN (R 4.3.2)
evaluate 0.23 2023-11-01 [1] CRAN (R 4.3.2)
fansi 1.0.6 2023-12-08 [1] CRAN (R 4.3.2)
fastmap 1.1.1 2023-02-24 [1] CRAN (R 4.3.2)
flexdashboard * 0.6.2 2023-08-11 [1] CRAN (R 4.3.3)
forcats * 1.0.0 2023-01-29 [1] CRAN (R 4.3.2)
frictionless * 1.2.0.9000 2024-09-19 [1] Github (frictionlessdata/frictionless-r@feaad88)
fs 1.6.3 2023-07-20 [1] CRAN (R 4.3.2)
generics 0.1.3 2022-07-05 [1] CRAN (R 4.3.2)
ggplot2 * 3.4.4 2023-10-12 [1] CRAN (R 4.3.2)
glue 1.7.0 2024-01-09 [1] CRAN (R 4.3.2)
gtable 0.3.4 2023-08-21 [1] CRAN (R 4.3.2)
hms 1.1.3 2023-03-21 [1] CRAN (R 4.3.2)
htmltools 0.5.7 2023-11-03 [1] CRAN (R 4.3.2)
htmlwidgets 1.6.4 2023-12-06 [1] CRAN (R 4.3.2)
httpuv 1.6.14 2024-01-26 [1] CRAN (R 4.3.2)
httr 1.4.7 2023-08-15 [1] CRAN (R 4.3.2)
jquerylib 0.1.4 2021-04-26 [1] CRAN (R 4.3.2)
jsonlite * 1.8.8 2023-12-04 [1] CRAN (R 4.3.2)
knitr 1.45 2023-10-30 [1] CRAN (R 4.3.2)
later 1.3.2 2023-12-06 [1] CRAN (R 4.3.2)
lattice 0.21-9 2023-10-01 [2] CRAN (R 4.3.2)
lazyeval 0.2.2 2019-03-15 [1] CRAN (R 4.3.3)
leaflegend * 1.2.1 2024-05-09 [1] CRAN (R 4.3.3)
leaflet * 2.2.2 2024-03-26 [1] CRAN (R 4.3.3)
lifecycle 1.0.4 2023-11-07 [1] CRAN (R 4.3.2)
lubridate * 1.9.3 2023-09-27 [1] CRAN (R 4.3.2)
magick * 2.8.4 2024-07-14 [1] CRAN (R 4.3.3)
magrittr 2.0.3 2022-03-30 [1] CRAN (R 4.3.2)
memoise 2.0.1 2021-11-26 [1] CRAN (R 4.3.2)
mime 0.12 2021-09-28 [1] CRAN (R 4.3.1)
miniUI 0.1.1.1 2018-05-18 [1] CRAN (R 4.3.2)
munsell 0.5.0 2018-06-12 [1] CRAN (R 4.3.2)
pillar 1.9.0 2023-03-22 [1] CRAN (R 4.3.2)
pkgbuild 1.4.3 2023-12-10 [1] CRAN (R 4.3.2)
pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.3.2)
pkgload 1.3.4 2024-01-16 [1] CRAN (R 4.3.2)
plotly * 4.10.4 2024-01-13 [1] CRAN (R 4.3.3)
processx 3.8.3 2023-12-10 [1] CRAN (R 4.3.2)
profvis 0.3.8 2023-05-02 [1] CRAN (R 4.3.2)
promises 1.2.1 2023-08-10 [1] CRAN (R 4.3.2)
ps 1.7.6 2024-01-18 [1] CRAN (R 4.3.2)
purrr * 1.0.2 2023-08-10 [1] CRAN (R 4.3.2)
R6 2.5.1 2021-08-19 [1] CRAN (R 4.3.2)
RColorBrewer * 1.1-3 2022-04-03 [1] CRAN (R 4.3.1)
Rcpp 1.0.12 2024-01-09 [1] CRAN (R 4.3.2)
readr * 2.1.5 2024-01-10 [1] CRAN (R 4.3.2)
remotes 2.4.2.1 2023-07-18 [1] CRAN (R 4.3.2)
rlang 1.1.3 2024-01-10 [1] CRAN (R 4.3.2)
rmarkdown 2.25 2023-09-18 [1] CRAN (R 4.3.2)
rstudioapi 0.15.0 2023-07-07 [1] CRAN (R 4.3.2)
sass 0.4.8 2023-12-06 [1] CRAN (R 4.3.2)
scales * 1.3.0 2023-11-28 [1] CRAN (R 4.3.2)
sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.3.2)
shiny 1.8.0 2023-11-17 [1] CRAN (R 4.3.2)
stringi 1.8.3 2023-12-11 [1] CRAN (R 4.3.2)
stringr * 1.5.1 2023-11-14 [1] CRAN (R 4.3.2)
tibble * 3.2.1 2023-03-20 [1] CRAN (R 4.3.2)
tidyfast * 0.4.0 2024-02-02 [1] CRAN (R 4.3.3)
tidyr * 1.3.1 2024-01-24 [1] CRAN (R 4.3.2)
tidyselect 1.2.0 2022-10-10 [1] CRAN (R 4.3.2)
tidyverse * 2.0.0 2023-02-22 [1] CRAN (R 4.3.2)
timechange 0.3.0 2024-01-18 [1] CRAN (R 4.3.2)
tzdb 0.4.0 2023-05-12 [1] CRAN (R 4.3.2)
urlchecker 1.0.1 2021-11-30 [1] CRAN (R 4.3.2)
usethis 2.2.3 2024-02-19 [1] CRAN (R 4.3.2)
utf8 1.2.4 2023-10-22 [1] CRAN (R 4.3.2)
vctrs 0.6.5 2023-12-01 [1] CRAN (R 4.3.2)
viridisLite 0.4.2 2023-05-02 [1] CRAN (R 4.3.2)
vroom 1.6.5 2023-12-05 [1] CRAN (R 4.3.2)
webshot * 0.5.5 2023-06-26 [1] CRAN (R 4.3.3)
withr 3.0.0 2024-01-16 [1] CRAN (R 4.3.2)
xfun 0.42 2024-02-08 [1] CRAN (R 4.3.2)
xtable 1.8-4 2019-04-21 [1] CRAN (R 4.3.2)
xts * 0.14.0 2024-06-05 [1] CRAN (R 4.3.3)
zoo * 1.8-12 2023-04-13 [1] CRAN (R 4.3.3)

[1] C:/Users/kajak/AppData/Local/R/win-library/4.3
[2] C:/Program Files/R/R-4.3.2/library

Thank you so much!

[katzikatz]

Good morning Domiano, I replied to you in the forum, maybe someone else is also experiencing these problems. I have attached the R Script I wrote (me and ChatGPT, we're besties). I need the ctdp package to extract the sequence length of the observation as we would like to look into the duration an animal stayed for. Maybe you know another way to do it? I hope Agouti is getting back to me soon, I already wrote to them on thursday about the json file, but there is no reply. Thank you so much!! Am Mittwoch, September 18, 2024 16:35 CEST, schrieb Damiano Oldoni ***@***.***>:     Hi @katzikatz. I could reproduce the error. No urgent need to send me a detailed session info. After a first debugging, I see that you are trying to read a media.csv file with a field, agoutiEventID, not defined in the provided schema in datapackage.json, i.e. https://raw.githubusercontent.com/tdwg/camtrap-dp/1.0/media-table-schema.json. This issue is known now by @peterdesmet: we will work further to solve this issue. However, if you don't need media data at the moment, you could use media = FALSE flag: read_camtrap_dp(path_to_your_json_file, media = FALSE). It's a short term solution, I know, but I hope it helps. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>[ { ***@***.***": "http://schema.org", ***@***.***": "EmailMessage", "potentialAction": { ***@***.***": "ViewAction", "target": "#334 (comment)", "url": "#334 (comment)", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { ***@***.***": "Organization", "name": "GitHub", "url": "https://github.com" } } ]***>

[katzikatz]

I don't seem to understand this forum, but well.

Agouti got back to me, the Agouti EventID field was only used temporarily to try and fix another issue, the new export won't include this anymore, so it should all work fine again.
Thanks for your help!!!

[peterdesmet]

This issue has been fixed in Agouti since September 18. There's also a temporary fix available for datasets before that date. See https://groups.google.com/g/agouti-users/c/OQIod-TuTLk/m/oazpV-jbAQAJ