Skip to content

Commit 1573a8f

Browse files
sibocwstimpflistepanoveDominic-DallOstoNathanMULLER
authored
Version 1.2.0 (#229)
* change conaffinity for base terrain (by default not colliding) * Get max floor height from the arenas * functionning zstabilized camera * add config for new cameras, change the model to remove cameras, modify the simulation to be able to work with the old and new camera type * start branch for v1.2.0 * Add vscode devcontainer support * Relax torch and numpy version requirements Fixes #224 * Add official release of `flyvis` as a dependency Fixes #228 * Fix typo removing flyvision from Dockerfile * Require minimum torch version of 2.3 Otherwise code fails to run on macos 13, which pulls numpy==2.0.2 and torch==2.2.2 * Force `numpy<2` on intel macs Fixes #231 * Incorporate game (#227) * added video_game file and changes in flygym * game working with joystick AND keyboard * Game improved for keyboard * Better handling of key pressing, working version before joystick * Preparing for joystick * trying to make the joystick work * trying to make the joystick work * panic commit * fix finishline and display * final version relatively clean * Refactored code with working keyboard * version working for joystick * escaping with the keyboard only * Removing old versions * Black * Get the camera to follow the fly like a third person view * Update comments and run linter * stepping backward in single leg, better hitbox, keyboard to change mode * cleanup * Adding missed post to the hitbox. Fixed bugs with keyboard changing mode. Fixed bugs with button presses for backward * add leaderboard * add leaderboard * add adhesion and make the game more reactive (optimization of fps, actuator hain, intrinsiq freqs ect, ... * add energy consumption * different floor coor for different levels * Add instructions screen to the start of each level * Make the colours a bit less scary * Unsaved change in level 2 instructions screen * improved instruction handling and reduce adhesion forces for better single leg control * fixed the instruction with joytstick * Remove game assets * Remove arena * Remove camera following fly code * Revert changes to the mesh that made it fully capsule * Revert changes that rolled back old changes on main * Revert whitespace changes in mesh * Revert rolled back changes to config.yaml * Revert changes to turning controller * Revert change to setup.py --------- Co-authored-by: NathanMULLER <muller.nathan1867@gmail.com> Co-authored-by: Dom <dominicd7@hotmail.com> * change FlyVision to Flyvis * added new camera handling zstab, yawonly, gravityaligned and normal * remove cameras from xml * accommodate the fly and simulation with the new camera handling * fixing non decomposed forces with no perspective arrow length * remove prints * fix bugs in gravity aligned and yaw only cameras * clean up and doc * Remove old cameras and unecessary xmls * remove old camera simulaiton code + fix YawOnlyCamera when used with initial orientation * modif and test all locomotion examples * modif and test olfaction * modif and test path int * changed and test head stabilization * revert testing changes * implement and test cameras vision * update and test notebooks * black reformating * black reformating * black reformating * remove old xmls from config * removing ruff * fixing xml * fixing xml * passing the tests * removing convexhull for mujoco 3.2.5 compatibility * recent mujoco release breaking dm_control * modify unwated changes * checkout main * black and fix path exploration test * fixed typo in merge * improper camera.py merging * correct typos * rename ZStabCamera to ZStabilizedCamera and rename targeted_flies_id to targeted_fly_names and make it a list of str * make methods private * code style changes * Notebooks use fly names instead of fly ids, update tutorial docs * updating camera rst * Solving last issues * document every camera * Switch from setup.py to pyproject.toml with Poetry for package management, and update web links etc (#233) * migrate from setup.py to pyproject.toml with poetry * relax numba constraint * add ci for poetry * remove support for python 3.9 * make check for whether poetry.lock exists cross platform * make check for whether poetry.lock exists cross platform x2 * update poetry.lock * add optional dependencies in poetry install command * remove torch-related things for macos intel * fix typo * debug macos-13 * debug macos-13 x2 * debug macos-13 x3 * debug macos-13 x4 * debug macos-13 x5 * fix CI and update installation instruction on docs * remove test for macos-13 * reduce numpy version * Update links and changelog * missing ci? * fix syntax error in workflow * update lock file * remove tests for macos intel with poetry * check runner name instead of os * change syntax * separate poetry check by os * fix missing steps and typos * Reupdate config file for game mesh Somehow this got removed since #227 * improving the smooting * reformat * revert testing changes * fix mujoco and dm_control versions strictly; go down to 0.1.8 for dm-tree as it's not building on Macs * add partial tests for Macs with Intel chips * c5e0881 didn't work, reverting * Minor text/changelog updates * Final text information for changelog * found an alternative to using attachment name removing it as it can lead to errors if not set properly (camera not found) * found an alternative to using attachment name removing it as it can lead to errors if not set properly (camera not found) * removing added by mistake stl * found an alternative to using attachment name removing it as it can lead to errors if not set properly (camera not found) * found an alternative to using attachment name removing it as it can lead to errors if not set properly (camera not found) * Update NMFv2 link in notebook * removing note about birdeye camera * Clarify some wording in camera docs * fix overlook in tethered envs * fix overlook in tethered envs * Fix typo * Fix typo * Add comment about fixing numpy<2 * Fix some type annotations in the Camera class * Fix `phiml` to <=1.10.0 Fixes #236 * Updated phiflow to v3.3.0 Also fixes #236 * Add basic test for plume generation * Fix phiml version to >=1.12.0 because phiflow v3.3.0 needs it but its deps aren't correctly configured * remove obsolete comment * remove duplicated docstring * do not pass empty list of dicts * Clearing the output of the notebooks * Set hybrid controller for the fly to be on the floor * Fix missing not * Prefer `is not` to `not is` and fix typo * Clarify `target_fly_names` in new `Camera` (#238) * 1. update argument order in docstring; 2. clarify doc for targeted_fly_names argument; 3. allow a single fly to be given target * modify the typing to accept strings * modify targeted_fly_name typing's to accept strings * use strings instead of list with a single fly * use strings instead of list with a single fly * fix typo in union * reformat * Merge remote-tracking branch 'origin/dev-v1.2.0' into camera-fly-specification * Update camera that was missed in one tutorial * Removing clipping max length of force vector when normalising * Only draw contacts and gravity vectors when the camera is tracking a fly * Force `VisualTaxis` controller to take `MovingObjArena` as argument * fix #239: rerun ok with full plume dataset * upload demo data to zenodo * add build to to dev optional dependencies --------- Co-authored-by: stimpfli <victor.stimpfling@epfl.ch> Co-authored-by: stepanove <stepanove@gmail.com> Co-authored-by: Dom <dominicd7@hotmail.com> Co-authored-by: Victor Stimpfling <32440815+stimpfli@users.noreply.github.com> Co-authored-by: NathanMULLER <muller.nathan1867@gmail.com> Co-authored-by: tlam <thomas.lam@epfl.ch>
1 parent 621bb97 commit 1573a8f

File tree

87 files changed

+8820
-3399
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

87 files changed

+8820
-3399
lines changed

‎.devcontainer/Dockerfile

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
FROM python:3.12-bookworm
2+
3+
ARG USERNAME=flygym
4+
ARG USER_UID=1000
5+
ARG USER_GID=$USER_UID
6+
7+
# Create the user
8+
RUN groupadd --gid $USER_GID $USERNAME \
9+
&& useradd --uid $USER_UID --gid $USER_GID -m $USERNAME \
10+
# [Optional] Add sudo support. Omit if you don't need to install software after connecting.
11+
&& apt-get update \
12+
&& apt-get install -y sudo \
13+
&& echo $USERNAME ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/$USERNAME \
14+
&& chmod 0440 /etc/sudoers.d/$USERNAME \
15+
&& rm -rf /var/lib/apt/lists/*
16+
17+
# Install system dependencies
18+
RUN apt update && \
19+
apt-get install -y libegl1-mesa-dev ffmpeg && \
20+
rm -rf /var/lib/apt/lists/*
21+
22+
# Set renderer
23+
ENV MUJOCO_GL=egl
24+
ENV PYOPENGL_PLATFORM=egl
25+
USER $USERNAME

‎.devcontainer/devcontainer.json

+32
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
{
2+
"name": "flygym_environment",
3+
"build": {
4+
"dockerfile": "Dockerfile",
5+
"context": ".."
6+
},
7+
"customizations": {
8+
"vscode": {
9+
"extensions": [
10+
"donjayamanne.python-extension-pack"
11+
],
12+
"settings": {
13+
"terminal.integrated.defaultProfile.linux": "bash",
14+
"terminal.integrated.profiles.linux": {
15+
"bash": {
16+
"path": "bash"
17+
}
18+
}
19+
}
20+
}
21+
},
22+
"mounts": [
23+
"source=/tmp/.X11-unix,target=/tmp/.X11-unix,type=bind,consistency=cached"
24+
],
25+
"containerEnv": {
26+
"DISPLAY": "${localEnv:DISPLAY}"
27+
},
28+
"postCreateCommand": "pip install -e \".[examples,dev]\" --no-warn-script-location",
29+
"runArgs": [
30+
"--device=/dev/dri"
31+
]
32+
}

‎.github/workflows/documentation.yaml

-1
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,6 @@ jobs:
1515
pip install --upgrade pip
1616
pip cache purge
1717
pip install sphinxcontrib-googleanalytics
18-
pip install "flyvision @ https://github.com/Nely-EPFL/flyvis/archive/refs/heads/main.zip"
1918
pip cache purge
2019
pip install -e ."[dev,examples]"
2120
pip install toolz

‎.github/workflows/test_poetry_lock_macos_apple_silicon.yaml

+40
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
name: Check poetry install macOS Apple Silicon
2+
3+
on: [push]
4+
5+
env:
6+
SKIP_RENDERING: "true"
7+
8+
jobs:
9+
build:
10+
runs-on: macos-latest
11+
12+
steps:
13+
- name: Checkout code
14+
uses: actions/checkout@v4
15+
16+
- name: Set up Python
17+
uses: actions/setup-python@v5
18+
with:
19+
python-version: '3.12'
20+
21+
- name: Assert poetry.lock exists
22+
run: |
23+
if [ ! -f poetry.lock ]; then
24+
echo "Error: poetry.lock not found!"
25+
exit 1
26+
fi
27+
28+
- name: Install pipx and poetry
29+
run: |
30+
python -m pip install --upgrade pip
31+
pip install pipx
32+
pipx install poetry
33+
34+
- name: Install dependencies with poetry
35+
run: |
36+
poetry install --extras=dev --extras=examples
37+
38+
- name: Run tests
39+
run: |
40+
poetry run pytest

‎.github/workflows/test_poetry_lock_macos_intel.yaml

+40
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
name: Check poetry install macOS Intel
2+
3+
on: [push]
4+
5+
env:
6+
SKIP_RENDERING: "true"
7+
8+
jobs:
9+
build:
10+
runs-on: macos-13
11+
12+
steps:
13+
- name: Checkout code
14+
uses: actions/checkout@v4
15+
16+
- name: Set up Python
17+
uses: actions/setup-python@v5
18+
with:
19+
python-version: '3.12'
20+
21+
- name: Assert poetry.lock exists
22+
run: |
23+
if [ ! -f poetry.lock ]; then
24+
echo "Error: poetry.lock not found!"
25+
exit 1
26+
fi
27+
28+
- name: Install pipx and poetry
29+
run: |
30+
python -m pip install --upgrade pip
31+
pip install pipx
32+
pipx install poetry
33+
34+
- name: Install dependencies with poetry
35+
run: |
36+
poetry install --extras=dev
37+
38+
- name: Run tests
39+
run: |
40+
echo "Tests skipped on macOS Apple Silicon; see issue #233"

‎.github/workflows/test_poetry_lock_ubuntu.yaml

+45
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
name: Check poetry install Ubuntu
2+
3+
on: [push]
4+
5+
jobs:
6+
build:
7+
runs-on: ubuntu-latest
8+
9+
steps:
10+
- name: Checkout code
11+
uses: actions/checkout@v4
12+
13+
- name: Set up Python
14+
uses: actions/setup-python@v5
15+
with:
16+
python-version: '3.12'
17+
18+
- name: Assert poetry.lock exists
19+
run: |
20+
if [ ! -f poetry.lock ]; then
21+
echo "Error: poetry.lock not found!"
22+
exit 1
23+
fi
24+
25+
- name: Install dependencies for Ubuntu
26+
run: |
27+
sudo apt-get update
28+
sudo apt-get install -y libegl1-mesa-dev
29+
echo "MUJOCO_GL=egl" >> $GITHUB_ENV
30+
echo "PYOPENGL_PLATFORM=egl" >> $GITHUB_ENV
31+
sudo apt-get install -y ffmpeg
32+
33+
- name: Install pipx and poetry
34+
run: |
35+
python -m pip install --upgrade pip
36+
pip install pipx
37+
pipx install poetry
38+
39+
- name: Install dependencies with poetry
40+
run: |
41+
poetry install --extras=dev --extras=examples
42+
43+
- name: Run tests
44+
run: |
45+
poetry run pytest

‎.github/workflows/test_poetry_lock_windows.yaml

+41
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
name: Check poetry install Windows
2+
3+
on: [push]
4+
5+
env:
6+
SKIP_RENDERING: "true"
7+
8+
jobs:
9+
build:
10+
runs-on: windows-latest
11+
12+
steps:
13+
- name: Checkout code
14+
uses: actions/checkout@v4
15+
16+
- name: Set up Python
17+
uses: actions/setup-python@v5
18+
with:
19+
python-version: '3.12'
20+
21+
- name: Assert poetry.lock exists
22+
run: |
23+
if (!(Test-Path -Path "poetry.lock")) {
24+
Write-Host "Error: poetry.lock not found!"
25+
exit 1
26+
}
27+
shell: pwsh
28+
29+
- name: Install pipx and poetry
30+
run: |
31+
python -m pip install --upgrade pip
32+
pip install pipx
33+
pipx install poetry
34+
35+
- name: Install dependencies with poetry
36+
run: |
37+
poetry install --extras=dev --extras=examples
38+
39+
- name: Run tests
40+
run: |
41+
poetry run pytest

‎.github/workflows/tests_full.yaml

+1-3
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ jobs:
77
runs-on: ubuntu-24.04
88
strategy:
99
matrix:
10-
python-version: ["3.9", "3.10", "3.11", "3.12"]
10+
python-version: ["3.10", "3.11", "3.12"]
1111
steps:
1212
- uses: actions/checkout@v4
1313
- name: Set up Python ${{ matrix.python-version }}
@@ -25,8 +25,6 @@ jobs:
2525
sudo apt-get install ffmpeg
2626
pip install --upgrade pip
2727
pip cache purge
28-
pip install "flyvision @ https://github.com/Nely-EPFL/flyvis/archive/refs/heads/main.zip"
29-
pip cache purge
3028
pip install -e ".[dev,examples]"
3129
- name: Lint with ruff
3230
run: |

‎.github/workflows/tests_macos.yaml

-2
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,6 @@ jobs:
2222
run: |
2323
pip install --upgrade pip
2424
pip cache purge
25-
pip install "flyvision @ https://github.com/Nely-EPFL/flyvis/archive/refs/heads/main.zip"
26-
pip cache purge
2725
pip install -e ".[dev,examples]"
2826
- name: Test with pytest
2927
run: |

‎.github/workflows/tests_ubuntu22.yaml

-2
Original file line numberDiff line numberDiff line change
@@ -25,8 +25,6 @@ jobs:
2525
sudo apt-get install ffmpeg
2626
pip install --upgrade pip
2727
pip cache purge
28-
pip install "flyvision @ https://github.com/Nely-EPFL/flyvis/archive/refs/heads/main.zip"
29-
pip cache purge
3028
pip install -e ".[dev,examples]"
3129
- name: Test with pytest
3230
run: |

‎.github/workflows/tests_windows.yaml

-2
Original file line numberDiff line numberDiff line change
@@ -21,8 +21,6 @@ jobs:
2121
run: |
2222
pip install --upgrade pip
2323
pip cache purge
24-
pip install "flyvision @ https://github.com/nely-epfl/flyvis/archive/refs/heads/main.zip"
25-
pip cache purge
2624
pip install -e .[dev,examples]
2725
- name: Test with pytest
2826
run: |

‎Dockerfile

+1-2
Original file line numberDiff line numberDiff line change
@@ -19,8 +19,7 @@ ENV PYOPENGL_PLATFORM=egl
1919
ADD . $HOME/flygym/
2020

2121
# Set up virtual environment and install dependencies
22-
RUN pip install "flyvision @ https://github.com/Nely-EPFL/flyvis/archive/refs/heads/main.zip" && \
23-
pip install -e ".[examples,dev]"
22+
RUN pip install -e ".[examples,dev]"
2423

2524
# Set entrypoint
2625
ENTRYPOINT ["/bin/bash"]

‎MANIFEST.in

-2
This file was deleted.

‎README.md

+13-7
Original file line numberDiff line numberDiff line change
@@ -49,13 +49,19 @@ We are constantly working on expanding the package and improving its usability;
4949
## Citation
5050
If you use NeuroMechFly in your work, please cite the following papers:
5151
```
52-
@article{WangChen2023,
53-
author = {Sibo Wang-Chen and Victor Alfred Stimpfling and Pembe Gizem \"{O}zdil and Louise Genoud and Femke Hurtak and Pavan Ramdya},
54-
title = {NeuroMechFly 2.0, a framework for simulating embodied sensorimotor control in adult Drosophila},
55-
year = {2023},
56-
doi = {10.1101/2023.09.18.556649},
57-
URL = {https://www.biorxiv.org/content/early/2023/09/18/2023.09.18.556649},
58-
journal = {bioRxiv}
52+
@article{WangChen2024,
53+
title = {NeuroMechFly v2: simulating embodied sensorimotor control in adult Drosophila},
54+
volume = {21},
55+
ISSN = {1548-7105},
56+
url = {http://dx.doi.org/10.1038/s41592-024-02497-y},
57+
DOI = {10.1038/s41592-024-02497-y},
58+
number = {12},
59+
journal = {Nature Methods},
60+
publisher = {Springer Science and Business Media LLC},
61+
author = {Wang-Chen, Sibo and Stimpfling, Victor Alfred and Lam, Thomas Ka Chung and \"{O}zdil, Pembe Gizem and Genoud, Louise and Hurtak, Femke and Ramdya, Pavan},
62+
year = {2024},
63+
month = nov,
64+
pages = {2353–2362}
5965
}
6066
6167
@article{LobatoRios2022,

‎doc/source/api_ref/camera.rst

+18-35
Original file line numberDiff line numberDiff line change
@@ -1,41 +1,24 @@
11
Camera
22
======
33

4-
The ``Camera`` class defines how images should be rendered from a camera in the world model. Note that the ``Camera`` class does not by itself add a camera to the MuJoCo model — the camera must have already existed and can be referenced by its name. The camera can be added in two ways:
5-
6-
1. By adding a camera to the MuJoCo model file. Refer to the `section on camera <https://mujoco.readthedocs.io/en/stable/XMLreference.html#body-camera>`_ in the MuJoCo XML reference for more information. As an example, the following XML code adds a tracking camera to the MuJoCo model:
7-
8-
.. code-block:: xml
9-
10-
<worldbody>
11-
<!-- ... other things ... -->
12-
<camera name="camera_top" class="nmf" mode="track" ipd="0.068" pos="0 0 8" euler="0 0 0"/>
13-
</worldbody>
14-
15-
2. By calling ``.worldbody.add()`` on the root element of the MuJoCo model programmatically in Python. Practically, this can be done by extending an existing FlyGym Arena class and adding the camera in the ``__init__`` method. For example, the following code adds a stationary bird's eye camera to the MuJoCo model:
16-
17-
.. code-block:: python3
18-
19-
from flygym.arena import BaseArena
20-
21-
class MyArena(BaseArena):
22-
def __init__(self):
23-
super().__init__()
24-
self.birdeye_cam = self.root_element.worldbody.add(
25-
"camera",
26-
name="birdseye_cam",
27-
mode="fixed",
28-
pos=(20, 0, 20),
29-
euler=(0, 0, 0),
30-
fovy=60,
31-
)
32-
# ... other things ...
33-
34-
Once the camera is added to the MuJoCo model, it can be referenced by its name to initialize the ``Camera`` object with additional parameters assigned to it. These can include: rendering rate in frames-per-second (FPS), window size, play speed, etc. Furthermore, the ``Camera`` class has useful methods such as ``.save_video``, ``.reset``, etc. Logics such as rotating the camera in tilted terrain are also implemented in the ``Camera`` class. The ``.render`` method is the main point of entry for fetching images from the camera.
35-
36-
The full API reference of the ``Camera`` class is as follows:
37-
38-
.. autoclass:: flygym.camera.Camera
4+
The ``Camera`` class defines how images should be rendered from a camera in the world model.
5+
Cameras are added dynamically to the model by calling ``.body.add()`` on any body of the MuJoCo model programmatically in Python. In this way, the XML file does not need to contain all the preset cameras.
6+
When instantiating a ``Camera`` object, the user has to specify:
7+
- The attachment point: this can be a body or a site in the model.
8+
- The name of the camera.
9+
- The targeted fly names. Those flies will be the one(s) whose contact forces will be drawn.
10+
In order to simplify ``Camera`` instantiation, we provide a set of predefined camera parameters in config.yaml. Those parameters are used if the camera name matches the name of one of those cameras.
11+
In case the camera name does not match any of the predefined cameras, the user can specify the camera parameters manually.
12+
13+
This new logic for cameras allows a more flexible definition of the cameras update rules.
14+
We propose three different camera update rules:
15+
- ``ZStabilizedCamera``: The camera z position is fixed at a given height above the floor.
16+
- ``YawOnlyCamera``: Only the yaw and position of the camera are updated. This smoothens camera movements during locomotion in tracked cameras.
17+
- ``GravityAlignedCamera``: This camera updates its orientation based on the changes in the gravity vector. This camera is useful for tracking the fly orientation in the world frame.
18+
19+
The full API reference of the different type of camera classes is as follows:
20+
21+
.. automodule:: flygym.camera
3922
:members:
4023
:undoc-members:
4124
:show-inheritance:

‎doc/source/api_ref/mdp_specs.rst

+1-1
Original file line numberDiff line numberDiff line change
@@ -96,4 +96,4 @@ NeuroMechFly with connectome-constrained vision network (``RealisticVisionFly``)
9696

9797
**Info:** In addition to what is returned by the ``HybridTurningController``, the ``flygym.examples.vision.RealisticVisionFly`` class also provides the following in the "info" dictionary:
9898

99-
* "nn_activities": Activities of the visual system neurons as a ``flyvision.LayerActivity`` object. This is similar to ``obs["nn_activities_arr"]`` but in the form of a ``flyvision.LayerActivity`` object rather than a plain array.
99+
* "nn_activities": Activities of the visual system neurons as a ``flyvis.LayerActivity`` object. This is similar to ``obs["nn_activities_arr"]`` but in the form of a ``flyvis.LayerActivity`` object rather than a plain array.

0 commit comments

Comments
 (0)