0.11.0

🚀 Enhancement

• Update and simplify models to support automated editor generation #348 (@satra @yarikoptic @jwodder)
• Use separate session for S3 requests #330 (@jwodder)
• BF: no --develop-debug for download ATM #330 (@yarikoptic)
• RF: moved handling of dandiset identifier "deduction" into Dandiset itself #330 (@yarikoptic)
• BF(workaround): allow for "proper" identifier according to new schema #330 (@yarikoptic)
• Merge remote-tracking branch 'origin/master' into gh-320 #330 (@yarikoptic)
• Further fixes #330 (@jwodder)
• Give DandiAPIClient a dandi_authenticate() method #330 (@jwodder)
• Fixes #330 (@jwodder)
• Rename DANDI_API_KEY to DANDI_GIRDER_API_KEY #330 (@jwodder)
• Test of uploading & downloading via new API #330 (@jwodder)
• RF+ENH: support mapping for direct API urls, and use netflify insstance instead of api+ prefix #330 (@yarikoptic)
• Delint #330 (@jwodder)
• RF: account for web UI URL changes/dropped features, remove support for girder URLs #330 (@yarikoptic)
• Handle uploading already-extant assets #330 (@jwodder)
• Use new metadata schema #330 (@jwodder)
• Yield more from iter_upload() #330 (@jwodder)
• Document upload method parameters #330 (@jwodder)
• New API upload function #330 (@jwodder)
• Give dandi_instance a metadata_version field #330 (@jwodder)
• Split Docker Compose dandi_instances and fixtures in two #330 (@jwodder)
• Add dandi-api instance record #330 (@jwodder)

🐛 Bug Fix

• Add dateModified to asset metadata #452 (@jwodder)
• fix: change enum to const for single enums #449 (@satra)
• Add --new-schema option to ls command for converting output #445 (@jwodder)
• Eliminate check for session_start_time preceding date_of_birth #440 (@jwodder)
• Eliminate DANDI_SCHEMA; add get_schema_version() #442 (@jwodder)
• Discard empty "sex" and "species" fields on conversion #438 (@jwodder)
• schema: minor spotted typo fixes #435 (@yarikoptic)
• Retry requests on ConnectionErrors #437 (@jwodder)
• Include HDMF version as well into the token #434 (@yarikoptic)
• Error if sha256 digest is missing from asset being downloaded #428 (@jwodder)
• Report dandi version in User-Agent header #424 (@jwodder)
• Remove misleading log message about authenticating with new API #425 (@jwodder)
• Distinguish between pre- and post-validation when uploading #420 (@jwodder)
• Log failed HTTP connections; include PID and TID in logs; include asset path in upload log messages #418 (@jwodder)
• Revert PR #409 (Content-MD5 header) #419 (@jwodder)
• Set Content-MD5 header when uploading asset parts #409 (@jwodder)
• upload(): Only yield first "validating" status to pyout #417 (@jwodder)
• Add more logging when uploading & downloading #412 (@jwodder)
• Sleep increasing amounts while waiting for uploaded assets to validate #408 (@jwodder)
• Populate wasDerivedFrom #386 (@jwodder)
• FIX: use authorized checkout for actions #403 (@satra)
• enh: account for samples, sessions and participants #392 (@satra @jwodder @yarikoptic)
• Disabling logging to stderr while using pyout #401 (@jwodder)
• Cache file digests and check for change in digest when uploading #391 (@jwodder)
• Make existing="refresh" a synonym for "overwrite" for new upload #390 (@jwodder)
• RF/NF: Identifiable and BareAssetMeta to describe an asset anywhere #373 (@yarikoptic @jwodder)
• Give known_urls human-readable display strings #384 (@jwodder)
• Make dandi download -i <instance> run in a Dandiset download that Dandiset #383 (@jwodder)
• Give validate command a --schema VERSION option for validating assets and dandiset.yaml #379 (@jwodder)
• Support downloading folders and latest Dandiset version #377 (@jwodder @yarikoptic)
• Add include_metadata=False parameter to asset-listing DandiAPIClient methods #378 (@jwodder @yarikoptic)
• Discard invalid asset identifiers when extracting metadata #374 (@jwodder)
• Handle uploading already-present files in new API #347 (@jwodder)
• Adjust license metadata conversion #364 (@jwodder @yarikoptic)
• Give AssetMeta and DandiMeta json_dict() methods for better dictification #346 (@jwodder)
• BF: allow to handle an item with multiple files #342 (@yarikoptic)
• ENH: devel upload dandiset metadata #341 (@yarikoptic)
• Try self.listFile() again on ConnectionErrors #335 (@jwodder)
• Add functions for validating metadata against JSON Schema and use in tests #338 (@jwodder)
• Fix AttributeError: 'Resource' object has no attribute 'values' #336 (@jwodder)
• [DATALAD RUNCMD] Swap order of str and AnyUrl to be from specific to generic #334 (@yarikoptic)
• cleaning up biosample and participant #312 (@satra @jwodder)
• Tee all logs to user log directory #318 (@jwodder)
• Update for new API at https://api.dandiarchive.org/api #283 (@jwodder)

⚠️ Pushed to master

• Merge branch 'gh-320' (@yarikoptic)
• DOC: provide description for both DANDI_API_KEY and DANDI_GIRDER_API_KEY (@yarikoptic)
• ENH: log at DEBUG result.text from a failed response (@yarikoptic)
• ENH: allow for DeprecationWarning to come from requests_toolbelt, not our problem (@yarikoptic)

🏠 Internal

• Cache the individual functions called by validate_file() #446 (@jwodder)
• Simplify release workflow #444 (@jwodder)
• Check out dandi/dandi-api-datasets with direct git clone #443 (@jwodder)
• Use iter_content() instead of raw.stream() #423 (@jwodder)
• Update Black #426 (@jwodder)
• Assorted code cleanup #422 (@jwodder)
• Use fscacher #397 (@jwodder)
• Use PUT endpoint to replace pre-existing assets on upload #394 (@jwodder)
• Support passing precomputed file digest to DandiAPIClient upload methods #388 (@jwodder)
• Set asset path via metadata only #382 (@jwodder)
• Add script for migrating Dandiset metadata #366 (@jwodder)
• Add workflow for running populate_dandiset_yaml.py #363 (@jwodder)
• Configure & apply isort via pre-commit #353 (@jwodder)
• Sort install_requires #351 (@jwodder)

📝 Documentation

• List extra auto labels in DEVELOPMENT.md #381 (@jwodder)

🧪 Tests

• Set DJANGO_DANDI_SCHEMA_VERSION in docker-compose.yml #429 (@jwodder)
• Add test of upload of large file to new API #415 (@jwodder)
• Capture all dandi log messages when testing #413 (@jwodder)
• Add CI run with dev version of pynwb #399 (@jwodder @yarikoptic)
• Add sample Dandiset test fixture #380 (@jwodder @yarikoptic)
• Add DANDI_TESTS_PERSIST_DOCKER_COMPOSE envvar for reusing Docker containers across test runs #354 (@jwodder)
• Fix numpy dependency issue in tests #356 (@jwodder)
• Fetch Django test API token more robustly #323 (@jwodder)
• Require keyring backends to be initialized before running any tests #326 (@jwodder)
• Install hdf5 for Python 3.9 tests #315 (@jwodder)
• Close a file in a test case #314 (@jwodder)

Authors: 3

• John T. Wodder II (@jwodder)
• Satrajit Ghosh (@satra)
• Yaroslav Halchenko (@yarikoptic)