-
+
- + + + Citation + + + + +
-
@@ -967,6 +985,17 @@
Rmagine
Rmagine allows a robot to simulate sensor data for arbitrary range sensors directly on board via raytracing. Since robots typically only have limited computational resources, Rmagine aims at being flexible and lightweight, while scaling well even to large environment maps. It runs on several platforms like Laptops or embedded computing boards like Nvidia Jetson by putting an unified API over specific proprietary libraries provided by the hardware manufacturers. This work is designed to support the future development of robotic applications depending on simulation of range data that could previously not be computed in reasonable time on mobile systems.
+Citation
+We presented this work at ICRA'23 in London. The paper gives valuable insights of the design concepts of this library. +When using the Rmagine library or any related ideas in your scientific work, please reference the following paper:
+@inproceedings{mock2023rmagine, + title={{Rmagine: 3D Range Sensor Simulation in Polygonal Maps via Ray Tracing for Embedded Hardware on Mobile Robots}}, + author={Mock, Alexander and Wiemann, Thomas and Hertzberg, Joachim}, + booktitle={IEEE International Conference on Robotics and Automation (ICRA)}, + year={2023}, + doi={10.1109/ICRA48891.2023.10161388} +} +Table of Contents
Getting Started
-
diff --git a/search/search_index.json b/search/search_index.json
index ea73e4a..b368e3f 100644
--- a/search/search_index.json
+++ b/search/search_index.json
@@ -1 +1 @@
-{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Home","text":""},{"location":"#rmagine","title":"Rmagine","text":"
- Overview
- Installation
- Integration
- Maps
- Sensors
- Simulation
- Problem Modelling
- Noise
- Concepts
- Math
- Memory
- Maps
- Tools
- Data
- News
- Embree 3
- Contributions
- Fork the repository: https://github.com/uos/rmagine_docs.
- Make changes on your forked repository.
- Check locally on your machine if mkdocs is able to compile your changes (instructions).
- Go to Github and click \"Pull Request\", select this repository's \"main\" branch as target.
- If you added new content, please provide a brief explanation of why you believe it is beneficial for the documentation.
- Send PR
- Moved the Cube 5 units along the x axis
- Moved the Cube 2 units along the y axis
- Moved the Cube 3 units along the z axis
- Rotated the Cube 45 degrees around the z axis
- Scaled the Cube with 0.2 (the resulting cube has the dimensions 0.4 x 0.4 x 0.4)
- Export DAE with default settings to \"~/utitled.dae\"
Rmagine allows a robot to simulate sensor data for arbitrary range sensors directly on board via raytracing. Since robots typically only have limited computational resources, Rmagine aims at being flexible and lightweight, while scaling well even to large environment maps. It runs on several platforms like Laptops or embedded computing boards like Nvidia Jetson by putting an unified API over specific proprietary libraries provided by the hardware manufacturers. This work is designed to support the future development of robotic applications depending on simulation of range data that could previously not be computed in reasonable time on mobile systems.
"},{"location":"#table-of-contents","title":"Table of Contents","text":"Getting Started
Library
Extra
You are welcome to contribute to the docs of Rmagine! Thorough and clear documentation is essential. You can help us by correcting mistakes, improving content, or adding examples that facilitate user navigation and usage of the project. Please submit any documentation-related issues to the repository https://github.com/uos/rmagine_docs. If you're making fixes or adding examples, don\u2019t hesitate to submit a pull request afterward!
"},{"location":"#pr-workflow","title":"PR workflow","text":"How to contribute to this documentation via pull requests:
Blender is a powerful tool to create and modify existing maps for rmagine.
Some Links to look up: - Webpage: https://www.blender.org/ - Docs: https://docs.blender.org/manual/en/latest/
"},{"location":"extra/blender/#useful-commands","title":"Useful commands","text":""},{"location":"extra/blender/#object-mode","title":"Object Mode","text":"Command Effect C Ctrl + G Move Object after: type X and \"0.5\" to move the object 0.5 along the X axis Ctrl + R Rotate Object after: type Z and \"45\" to rotate the object 45 degree around the X axis Ctrl + S Scale Object after: type X and \"2.0\" to scale the object 2.0 along the X axis"},{"location":"extra/blender/#collada-dae-exports-odyssey-of-wrong-imports","title":"Collada (DAE) exports (odyssey of wrong imports)","text":""},{"location":"extra/blender/#update","title":"UPDATE","text":"Blender plugin does everything right, Assimp writes the wrong transformation: See last sentence.
"},{"location":"extra/blender/#beginning","title":"Beginning","text":"Some strange errors happened while exporting blender's scene to collada format. I did the following in Blender (3.2.1):
After some library fixes to read the scene graph completely, any collada file generated by Blender no longer loads correctly. With PLY exports, everything works as before. So I inspected the generated outputs of the Blender Collada exports (read by Assimp):
$:~ ./bin/rmagine_map_info ~/untitled.dae\n#...\nScene Graph: \n- name: Scene\n- transform: \n M4x4[\n 1 0 0 0\n 0 0 1 0\n 0 -1 0 0\n 0 0 0 1\n ]\n- meshes: 0\n- children: 3\n#...\n Node 2\n - name: Cube\n - transform: \n M4x4[\n 0.141421 -0.141421 0 5\n 0.141421 0.141421 0 2\n 0 0 0.2 3\n 0 0 0 1\n ]\n - meshes: 1\n - mesh ref 0 -> 0\n - children: 0\nAnd exactly there is the problem. I guess the \"Cube\" transformation is correct since it come from Blender directly. The problem comes from the global matrix at node named \"Scene\". This matrix switches the y and z axes and negates the z axis (old y axis) afterwards. The complete transform of the \"Cube\" following the transformations to the root is
v[5,3,-2], E[-1.5708, 0.785398, -1.26441e-07] with scale v[0.2,0.2,0.2]as expected, the total transform holds the axis-switched version of our previously done operations. But that is not what we want. Moving something in Blender along the x axis should result in an export with something in it that was moved along the x axis and no other axis. So how to fix it? Pushing
Export -> Collodaopens a menu. Push the settings button in the top left corner. It opens a side panel, holding some values to change for the export. My first intuitive choice to set the forward axis to X and the up axis to Z were wrong, incomprehensibly. However, by trial and error I came up with setting these values as follows:Forward Axis: Z\nUp Axis: -Y\nWith these settings the scene is exported exactly as I modelled it. Nevertheless, Blender exports the weird axis flip matrix at scene root. Every other transformation is adjusted such that every total transformation are valid.
"},{"location":"extra/blender/#not-yet-fixed","title":"Not yet fixed","text":"v[5,2,3], E[0, 0, 0.785398] with scale v[0.2,0.2,0.2]Importing this exported file into Blender results in a different scene. It seems the Blender Collada Importer ignores the top level axis-switch matrix somehow? FBX export and import is correct using the axis switch by choice.
"},{"location":"extra/blender/#fixed","title":"FIXED","text":"The wrong up most transformation is written by Assimp!
- Assimp::Importer io; io.SetPropertyBool(AI_CONFIG_IMPORT_COLLADA_IGNORE_UP_DIRECTION, true)
Capsuled in
AssimpIO-Object (\"rmagine/map/AssimpIO.hpp\").The following code snipped already corrects the wrong transformation imports.
rm::AssimpIO io; \nconst aiScene* scene = io.ReadFile(\"file.dae\", 0); \nWith that, it is possible to default export Collada files with Blender, import them with rmagine and import them in Blender again, without any errors.
"},{"location":"extra/blender/#filmbox-fbx-exports","title":"Filmbox (FBX) exports","text":"Same problems as in DAE section. Set axis to:
Scale: 0.01\nForward: Y Forward\nUp: Z Up\nThen everything is exported as modelled. Imports into Blender again are working as well. Luckilly, the global scene transform is set to identity here. Maybe that is why the Import is working again: ignoring the scene transform is not important if its an identity transform.
TODO: check if Gazebo importer ignores the scene transform
"},{"location":"extra/blender/#building-a-3d-map-from-2d-building-plan","title":"Building a 3D map from 2D building plan","text":"Using Blender Version 3.3.1
"},{"location":"extra/blender/#setting-up-a-2d-map-as-reference-image","title":"Setting up a 2D Map as Reference image","text":"Make sure to be in Object Mode
- Click top right on the z-axis to make the view orthograhic top down
- In Scene press Shift+A to open the Add Panel. Then select your Image and Load it into scene. The Image should now be located on your xy-plane
- Scale the Image to a size where you think you can work good with. The exact scale can be determined later.
- Enable Opacity of Image: Select Image and go to \"Object Data Properties\" (bottom right). Checkmark the Opacity Button
To place single vertices in the scene we first have to enable an Add-On under
Edit -> PreferencescalledAdd Mesh: Extra Objects. Then with Shift+A under the entryMeshthe optionSingle Vertshould be available. SelectingSingle Vertwill place a single vertex in the origin and change toEdit Mode. A object should appear in theScene Collectionpanel top right calledVert.- Move the image so that the first vertex to set is in the origin
- Insert a single vertex. A new object should be created. The first vertex is placed in the Origin.
- Press
Eto extrude a vertex than click an endpoint to place the second vertex with an edge connecting both. - Contour: By again pressing
Eyou can create a path of vertices. I recommend to make a complete path along the contour of the building plan (without doors. Only walls). - Walls: Once you have created a contour of edges go to
Edge Select-Mode select everything. PressEand thenZto extrude the edges along the z-axis. Pull the walls to a arbitrary height (The exact scale is determined later). - Ground Faces: Go to
Vertex Select-Mode and select three ground Vertices you want to connect (Hold Shift). PressFto connect them to a Face.- If Faces are getting to long: Cut Wall-Face in two by selecting in in
Face SelectMode and then pressing Ctrl+R - Enable Statistics in Drop-Down Menu \"Overlays\" top right. If you notice during editing that there is more than one Vertex at one point: Select both of them in
Vertex Select-Mode. The Right-Click andMerge Vertices - Collapse.
- If Faces are getting to long: Cut Wall-Face in two by selecting in in
We are excited to invite you to contribute to our open source project Rmagine! Whether you're a seasoned developer, a documentation enthusiast, or someone with fresh ideas, your contributions can make a significant impact.
"},{"location":"extra/contributions/#how-you-can-help","title":"How You Can Help:","text":"-
Code Contributions: Help us improve the codebase by submitting pull requests. Whether it\u2019s fixing bugs, adding features, or optimizing existing code, every contribution counts!
-
Documentation: Clear and comprehensive documentation is crucial. Assist us by correcting errors, enhancing content, or providing examples that make it easier for users to navigate and utilize the project.
-
Feedback and Suggestions: Your insights matter! Share your thoughts on how we can improve the project. Opening issues with suggestions or feedback is a great way to get involved.
For development and testing we include some meshes inside our repository in the
"},{"location":"extra/data/#sphereply","title":"sphere.ply","text":"dat-folder:
"},{"location":"extra/data/#triangleply","title":"triangle.ply","text":"user@pc:~/rmagine/build$ ./bin/rmagine_map_info ../dat/sphere.ply\nRmagine Map Info\nInputs: \n- filename: ../dat/sphere.ply\nMeshes: 1\n Mesh 0\n - name: \n - vertices, faces: 642, 1280\n - primitives: TRIANGLE\n - normals: no\n - vertex color channels: 0\n - uv channels: 0\n - bones: 0\n - material index: 0\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\nTextures: 0\nScene Graph: \n- name: \n- transform: \n M4x4[\n 1 0 0 0\n 0 1 0 0\n 0 0 1 0\n 0 0 0 1\n ]\n- meshes: 1\n - mesh ref 0 -> 0\n- children: 0\n
"},{"location":"extra/data/#box_rot_trans_scaleddae","title":"box_rot_trans_scaled.dae","text":"user@pc:~/rmagine/build$ ./bin/rmagine_map_info ../dat/triangle.ply\nRmagine Map Info\nInputs: \n- filename: ../dat/triangle.ply\nMeshes: 1\n Mesh 0\n - name: \n - vertices, faces: 3, 1\n - primitives: TRIANGLE\n - normals: no\n - vertex color channels: 0\n - uv channels: 0\n - bones: 0\n - material index: 0\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\nTextures: 0\nScene Graph: \n- name: \n- transform: \n M4x4[\n 1 0 0 0\n 0 1 0 0\n 0 0 1 0\n 0 0 0 1\n ]\n- meshes: 1\n - mesh ref 0 -> 0\n- children: 0\n
"},{"location":"extra/data/#two_cubesdae","title":"two_cubes.dae","text":"user@pc:~/rmagine/build$ ./bin/rmagine_map_info ../dat/box_rot_trans_scaled.dae\nRmagine Map Info\nInputs: \n- filename: ../dat/box_rot_trans_scaled.dae\nMeshes: 1\n Mesh 0\n - name: Cube-mesh\n - vertices, faces: 36, 12\n - primitives: TRIANGLE\n - normals: yes\n - vertex color channels: 0\n - uv channels: 1\n - bones: 0\n - material index: 0\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\nTextures: 0\nScene Graph: \n- name: Scene\n- transform: \n M4x4[\n 1 0 0 0\n 0 1 0 0\n 0 0 1 0\n 0 0 0 1\n ]\n- meshes: 0\n- children: 3\n Node 0\n - name: Camera\n - transform: \n M4x4[\n 0.727676 0.305421 -0.61417 -6.92579\n -0.685921 0.324014 -0.651558 -7.35889\n 0 0.895396 0.445271 4.95831\n 0 0 0 1\n ]\n - meshes: 0\n - children: 0\n Node 1\n - name: Light\n - transform: \n M4x4[\n 0.955171 -0.199883 0.218391 1.00545\n 0.290865 0.771101 -0.566393 -4.07624\n -0.0551891 0.604525 0.794672 5.90386\n 0 0 0 1\n ]\n - meshes: 0\n - children: 0\n Node 2\n - name: Cube\n - transform: \n M4x4[\n 0.141421 0.141421 0 0\n -0.141421 0.141421 0 -5\n 0 0 0.2 0\n 0 0 0 1\n ]\n - meshes: 1\n - mesh ref 0 -> 0\n - children: 0\n
"},{"location":"extra/data/#many_objectsdae","title":"many_objects.dae","text":"user@pc:~/rmagine/build$ ./bin/rmagine_map_info ../dat/two_cubes.dae\nRmagine Map Info\nInputs: \n- filename: ../dat/two_cubes.dae\nMeshes: 2\n Mesh 0\n - name: Cube_001-mesh\n - vertices, faces: 36, 12\n - primitives: TRIANGLE\n - normals: yes\n - vertex color channels: 0\n - uv channels: 1\n - bones: 0\n - material index: 1\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\n Mesh 1\n - name: Cube-mesh\n - vertices, faces: 36, 12\n - primitives: TRIANGLE\n - normals: yes\n - vertex color channels: 0\n - uv channels: 1\n - bones: 0\n - material index: 0\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\nTextures: 0\nScene Graph: \n- name: Scene\n- transform: \n M4x4[\n 1 0 0 0\n 0 1 0 0\n 0 0 1 0\n 0 0 0 1\n ]\n- meshes: 0\n- children: 4\n Node 0\n - name: Cube_001\n - transform: \n M4x4[\n 1 0 0 5.01877\n 0 1 0 3.78582\n 0 0 1 1.01026\n 0 0 0 1\n ]\n - meshes: 1\n - mesh ref 0 -> 0\n - children: 0\n Node 1\n - name: Camera\n - transform: \n M4x4[\n 0.685921 -0.324014 0.651558 7.35889\n 0.727676 0.305421 -0.61417 -6.92579\n 0 0.895396 0.445271 4.95831\n 0 0 0 1\n ]\n - meshes: 0\n - children: 0\n Node 2\n - name: Light\n - transform: \n M4x4[\n -0.290865 -0.771101 0.566393 4.07624\n 0.955171 -0.199883 0.218391 1.00545\n -0.0551891 0.604525 0.794672 5.90386\n 0 0 0 1\n ]\n - meshes: 0\n - children: 0\n Node 3\n - name: Cube\n - transform: \n M4x4[\n 1.64731 1.34142 -0.299005 5.7392\n -1.28721 1.34237 -1.06938 -4.66034\n -0.481559 1.00054 1.83561 1.69496\n 0 0 0 1\n ]\n - meshes: 1\n - mesh ref 0 -> 1\n - children: 0\n
"},{"location":"extra/embree3/","title":"Embree 3","text":"user@pc:~/rmagine/build$ ./bin/rmagine_map_info ../dat/many_objects.dae\nRmagine Map Info\nInputs: \n- filename: ../dat/many_objects.dae\nMeshes: 7\n Mesh 0\n - name: Torus-mesh\n - vertices, faces: 3456, 1152\n - primitives: TRIANGLE\n - normals: yes\n - vertex color channels: 0\n - uv channels: 1\n - bones: 0\n - material index: 0\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\n Mesh 1\n - name: Suzanne-mesh\n - vertices, faces: 2901, 967\n - primitives: TRIANGLE\n - normals: yes\n - vertex color channels: 0\n - uv channels: 1\n - bones: 0\n - material index: 0\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\n Mesh 2\n - name: Cone-mesh\n - vertices, faces: 186, 62\n - primitives: TRIANGLE\n - normals: yes\n - vertex color channels: 0\n - uv channels: 1\n - bones: 0\n - material index: 0\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\n Mesh 3\n - name: Cylinder-mesh\n - vertices, faces: 372, 124\n - primitives: TRIANGLE\n - normals: yes\n - vertex color channels: 0\n - uv channels: 1\n - bones: 0\n - material index: 0\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\n Mesh 4\n - name: Plane-mesh\n - vertices, faces: 6, 2\n - primitives: TRIANGLE\n - normals: yes\n - vertex color channels: 0\n - uv channels: 1\n - bones: 0\n - material index: 0\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\n Mesh 5\n - name: Cube_001-mesh\n - vertices, faces: 36, 12\n - primitives: TRIANGLE\n - normals: yes\n - vertex color channels: 0\n - uv channels: 1\n - bones: 0\n - material index: 1\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\n Mesh 6\n - name: Cube-mesh\n - vertices, faces: 36, 12\n - primitives: TRIANGLE\n - normals: yes\n - vertex color channels: 0\n - uv channels: 1\n - bones: 0\n - material index: 0\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\nTextures: 0\nScene Graph: \n- name: Scene\n- transform: \n M4x4[\n 1 0 0 0\n 0 1 0 0\n 0 0 1 0\n 0 0 0 1\n ]\n- meshes: 0\n- children: 10\n Node 0\n - name: Torus\n - transform: \n M4x4[\n -0.0934659 -0.290695 2.78847 -9.244\n 0.942502 2.62438 0.30518 5.28275\n -2.64041 0.94707 0.0102276 3.4012\n 0 0 0 1\n ]\n - meshes: 1\n - mesh ref 0 -> 0\n - children: 0\n Node 1\n - name: Suzanne\n - transform: \n M4x4[\n -0.247416 -0.65867 0.71059 -6.41007\n 0.442643 -0.729225 -0.521823 -2.72992\n 0.861889 0.18543 0.471977 1.71339\n 0 0 0 1\n ]\n - meshes: 1\n - mesh ref 0 -> 1\n - children: 0\n Node 2\n - name: Cone\n - transform: \n M4x4[\n 1 0 0 1.73173\n 0 1 0 -7.66226\n 0 0 1 0.999599\n 0 0 0 1\n ]\n - meshes: 1\n - mesh ref 0 -> 2\n - children: 0\n Node 3\n - name: Cylinder\n - transform: \n M4x4[\n 1 0 0 1.34234\n 0 1 0 8.77874\n 0 0 1 0.959374\n 0 0 0 1\n ]\n - meshes: 1\n - mesh ref 0 -> 3\n - children: 0\n Node 4\n - name: Plane\n - transform: \n M4x4[\n 11 0 0 0\n 0 11 0 0\n 0 0 11 0\n 0 0 0 1\n ]\n - meshes: 1\n - mesh ref 0 -> 4\n - children: 0\n Node 5\n - name: Light_001\n - transform: \n M4x4[\n -0.290865 -0.771101 0.566393 4.07624\n 0.955171 -0.199883 0.218391 1.00545\n -0.0551891 0.604525 0.794672 5.90386\n 0 0 0 1\n ]\n - meshes: 0\n - children: 0\n Node 6\n - name: Cube_001\n - transform: \n M4x4[\n 0.84132 0.52049 0.145848 5.72826\n -0.452094 0.52966 0.717685 3.24672\n 0.296298 -0.669739 0.680923 3.1261\n 0 0 0 1\n ]\n - meshes: 1\n - mesh ref 0 -> 5\n - children: 0\n Node 7\n - name: Camera\n - transform: \n M4x4[\n 0.685921 -0.324014 0.651558 7.35889\n 0.727676 0.305421 -0.61417 -6.92579\n 0 0.895396 0.445271 4.95831\n 0 0 0 1\n ]\n - meshes: 0\n - children: 0\n Node 8\n - name: Light\n - transform: \n M4x4[\n -0.290865 -0.771101 0.566393 4.07624\n 0.955171 -0.199883 0.218391 1.00545\n -0.0551891 0.604525 0.794672 5.90386\n 0 0 0 1\n ]\n - meshes: 0\n - children: 0\n Node 9\n - name: Cube\n - transform: \n M4x4[\n 1 0 0 4.91178\n 0 1 0 -2.96374\n 0 0 1 1.06244\n 0 0 0 1\n ]\n - meshes: 1\n - mesh ref 0 -> 6\n - children: 0\nFollow the following steps if you want Rmagine to work with Embree 3:
user@pc:~$ git clone https://github.com/embree/embree.git --branch v3.13.5\nuser@pc:~$ mkdir embree/build && cd embree/build\nuser@pc:~/embree/build$ cmake -DCMAKE_BUILD_TYPE=Release ..\nuser@pc:~/embree/build$ make -j`nproc`\nuser@pc:~/embree/build$ sudo make install\nDuring the
cmake-process errors could occur. The following flags could fix it. You can edit them by callingccmake .in build directory.# The Intel Implicit SPMD Program Compiler is not necessarily needed \nEMBREE_ISPC_SUPPORT=OFF\n# Tasking system: TBB/Internal. You can choose INTERNAL if TBB is not installed\nEMBREE_TASKING_SYSTEM=INTERNAL \n# Tutials are not needed for library compilation\nEMBREE_TUTORIALS=OFF\nOr you can pass the arguments directly to cmake for the minimal build
"},{"location":"extra/news/","title":"News","text":""},{"location":"extra/news/#05122023","title":"05.12.2023","text":"cmake -DCMAKE_BUILD_TYPE=Release -DEMBREE_ISPC_SUPPORT=OFF -DEMBREE_TASKING_SYSTEM=INTERNAL -DEMBREE_TUTORIALS=OFF ..\nNew version 2.2.2 is available now and brings convenience updates for ROS-users. Just place Rmagine into your ROS-workspace and it will compile. Via
find_package(rmagine COMPONENTS [...])you can still find Rmagine's components as if you would install it globally on your system. We tested it with - ROS1 - noetic - ROS2 - humbleNormally you would set
OptiX_INCLUDE_DIRvia cmake flags. Now we provide an additional option: Set the environment variableOPTIX_HEADER_DIRfor example in your.bashrc-file:export OPTIX_INCLUDE_DIR=~/software/optix/NVIDIA-OptiX-SDK-7.4.0-linux64-x86_64/include\nEspecially if you place Rmagine into your ROS-workspace this option becomes very handy.
"},{"location":"extra/news/#27092023","title":"27.09.2023","text":"From version >= 2.2.0 we enabled component-wise compilation and packaging for easier installation of Rmagine. In \"Releases\" section you can find the first pre-compiled binaries. Install the core library via
$ sudo dpkg -i rmagine-core_2.2.1_amd64.deb\nThen additionally for the Embree backend:
$ sudo dpkg -i rmagine-embree_2.2.1_amd64.deb\nAnd if you have a NVIDIA GPU:
$ sudo dpkg -i rmagine-cuda_2.2.1_amd64.deb\n$ sudo dpkg -i rmagine-optix_2.2.1_amd64.deb\nUsing the pre-compiled binaries, you are not required to download the OptiX-headers anymore. However, CUDA and Embree are still required to be installed on your system.
"},{"location":"extra/styleguide/","title":"Styleguide","text":""},{"location":"extra/styleguide/#naming-conventions","title":"Naming Conventions","text":""},{"location":"extra/styleguide/#files","title":"Files","text":"- Files containing a single C++-Class are written camelcase with
.hppextension. Examples:MyClass.hpp(Header),MyClass.cpp(Code), andMyClass.tcc(Template Code). - Files holding a set of Structs, Classes, or Functions are written lowercase with
.hextension. Examples:math/types.h,conversions.h, ormath.h(Header) andconversions.cppfor code. - Files holding Functions with CUDA code are namend with a
.cuhextension:math.cuh
- Functions with underscores:
mult,my_function - Classes and Structs in camelcase beginning with a capital letter:
Sphere,SphereSimulatorEmbree - Class Functions are beginning lowercase and then camelcase:
mult,simulateRanges
TODO: revise files to meet this styleguide
"},{"location":"extra/styleguide/#special-operators-for-math-types","title":"Special Operators for Math Types","text":"~: Invert the element after
There are some helpful command line tools that are compiled alongside the main library. After installation the tools are globally available to be called.
"},{"location":"extra/tools/#rmagine_version","title":"rmagine_version","text":"Prints the rmagine version. Should match the CMakeLists version.
"},{"location":"extra/tools/#rmagine_benchmark","title":"rmagine_benchmark","text":"user@pc:~/rmagine/build$ ./bin/rmagine_version\n2.2.1\nFor every implemented computing device we compile a benchmark executable that simulates a Velodyne LiDAR sensor in a given mesh and prints out some useful run time statisitics. Thus, we can compare the run times of different implementations on several computers. The CPU / Embree version can be tested like this:
user@pc:~/rmagine/build$ ./bin/rmagine_benchmark_cpu ../dat/sphere.ply\nRmagine Benchmark CPU (Embree)\nInputs: \n- mesh: ../dat/sphere.ply\nUnit: 1 Velodyne scan (velo) = 14400 Rays\n- range of last ray: 0.998762\n-- Starting Benchmark --\n[ 129% - 2215.605926 velos/s, mean: 2387.607002 velos/s] \nResult: 2387.607002 velos/s\nAnalogously, the GPU benchmark can be started as follows:
"},{"location":"extra/tools/#rmagine_map_info","title":"rmagine_map_info","text":"user@pc:~/rmagine/build$ ./bin/rmagine_benchmark_gpu ../dat/sphere.ply\n[RMagine - CudaContext] CUDA Driver Version / Runtime Version: 12.2.0 / 12.2.0\n[RMagine - CudaContext] Construct context on device 0 - NVIDIA GeForce RTX 2060 \n[RMagine - OptixContext] Init Optix (7.3.0). Required GPU driver >= 465.84\nRmagine Benchmark GPU (OptiX)\nInputs: \n- mesh: ../dat/sphere.ply\nUnit: 1 Velodyne scan (velo) = 14400 Rays\nLast Ray:\n- range: 0.998762\n-- Starting Benchmark --\n[ 100% - 231941.938409 velos/s, mean: 231987.481164 velos/s] \nResult: 231987.481164 velos/s\nPrints useful information about the contents of a mesh file. Internally it is just printing the meta information of the assimp buffers.
"},{"location":"extra/tools/#rmagine_synthetic","title":"rmagine_synthetic","text":"user@pc:~/rmagine/build$ ./bin/rmagine_map_info ../dat/two_cubes.dae\nRmagine Map Info\nInputs: \n- filename: ../dat/two_cubes.dae\nMeshes: 2\n Mesh 0\n - name: Cube_001-mesh\n - vertices, faces: 36, 12\n - primitives: TRIANGLE\n - normals: yes\n - vertex color channels: 0\n - uv channels: 1\n - bones: 0\n - material index: 1\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\n Mesh 1\n - name: Cube-mesh\n - vertices, faces: 36, 12\n - primitives: TRIANGLE\n - normals: yes\n - vertex color channels: 0\n - uv channels: 1\n - bones: 0\n - material index: 0\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\nTextures: 0\nScene Graph: \n- name: Scene\n- transform: \n M4x4[\n 1 0 0 0\n 0 1 0 0\n 0 0 1 0\n 0 0 0 1\n ]\n- meshes: 0\n- children: 4\n Node 0\n - name: Cube_001\n - transform: \n M4x4[\n 1 0 0 5.01877\n 0 1 0 3.78582\n 0 0 1 1.01026\n 0 0 0 1\n ]\n - meshes: 1\n - mesh ref 0 -> 0\n - children: 0\n Node 1\n - name: Camera\n - transform: \n M4x4[\n 0.685921 -0.324014 0.651558 7.35889\n 0.727676 0.305421 -0.61417 -6.92579\n 0 0.895396 0.445271 4.95831\n 0 0 0 1\n ]\n - meshes: 0\n - children: 0\n Node 2\n - name: Light\n - transform: \n M4x4[\n -0.290865 -0.771101 0.566393 4.07624\n 0.955171 -0.199883 0.218391 1.00545\n -0.0551891 0.604525 0.794672 5.90386\n 0 0 0 1\n ]\n - meshes: 0\n - children: 0\n Node 3\n - name: Cube\n - transform: \n M4x4[\n 1.64731 1.34142 -0.299005 5.7392\n -1.28721 1.34237 -1.06938 -4.66034\n -0.481559 1.00054 1.83561 1.69496\n 0 0 0 1\n ]\n - meshes: 1\n - mesh ref 0 -> 1\n - children: 0\nGenerate different meshes for quick testing. Show the possible options by entering
user@pc:~/rmagine/build$ ./bin/rmagine_synthetic\nRmagine Synthetic\nUsage: ./bin/rmagine_synthetic mesh_type mesh_file\n- mesh_type: plane | cube | sphere | cylinder \nrmagine_synthetic plane plane.plyrmagine_synthetic cube cube.plyrmagine_synthetic sphere sphere.plyrmagine_synthetic cylinder cylinder.ply"},{"location":"getting_started/installation/","title":"Installation (From Source)","text":"The following instructions are tested on an Ubuntu 20.04 operating system.
"},{"location":"getting_started/installation/#dependencies","title":"Dependencies","text":""},{"location":"getting_started/installation/#assimp-open-assets-importer-library","title":"Assimp (Open Assets Importer Library)","text":"For loading commonly used mesh/scene formats.
"},{"location":"getting_started/installation/#backbones","title":"Backbones","text":"user@pc:~$ sudo apt install libassimp-dev\nRmagine provides an interface to integrate ray tracing libraries, we call backbones. All of these backbones are optional. So far we integrated Intel Embree and NVIDIA OptiX.
"},{"location":"getting_started/installation/#embree-backbone-optional","title":"Embree Backbone (optional)","text":"We support Embree in its latest version (test v4.0.1, v4.2.0):
user@pc:~$ git clone https://github.com/embree/embree.git\nuser@pc:~$ mkdir embree/build && cd embree/build\nuser@pc:~/embree/build$ cmake -DCMAKE_BUILD_TYPE=Release ..\nuser@pc:~/embree/build$ make -j`nproc`\nuser@pc:~/embree/build$ sudo make install\nFor older Embree versions we refer to this.
"},{"location":"getting_started/installation/#optix-backbone-optional","title":"OptiX Backbone (optional)","text":"Rmagine supports NVIDIA OptiX versions of 7.2 or newer. The OptiX-Library is installed via the GPU driver. The OptiX-Headers can be downloaded here. The Headers require a specific GPU driver and CUDA version to be installed on your system:
OptiX Version Minimum Driver Version 7.2 456.71 7.3 465.84 7.4 495.89 7.5 495.89 (untested) 7.6 520.00 (untested) 7.7 530.41"},{"location":"getting_started/installation/#compilation","title":"Compilation","text":"Download the Rmagine repository.
user@pc:~/rmagine$ mkdir build\nuser@pc:~/rmagine$ cd build\nuser@pc:~/rmagine/build$ cmake ..\nuser@pc:~/rmagine/build$ make\nThe path to OptiX-Headers should be specified with the CMake-Variable
"},{"location":"getting_started/installation/#bash-variable-alternative","title":"Bash Variable (Alternative)","text":"OptiX_INCLUDE_DIR. This can be done using ccmake, for example.Alternatively, cmake checks for the environment variable
OPTIX_INCLUDE_DIRto exist. After downloading the OptiX-SDK you can add the following command to your.bashrc:export OPTIX_INCLUDE_DIR=~/.../NVIDIA-OptiX-SDK-7.4.0-linux64-x86_64/include\nAfter adding this path, the project should compile without changing the cmake flags.
"},{"location":"getting_started/installation/#optional-check-compilation","title":"Optional: Check Compilation","text":"You can check if everything went wrong by running the benchmark that was build besides the library:
user@pc:~/rmagine/build$ ./bin/rmagine_benchmark_cpu ../dat/sphere.ply\n...\n[ 100% - velos/s: 6261.9, mean: 6244.84] \nResult: 6244.84 velos/s\nor if the OptiX support was successfully build:
"},{"location":"getting_started/installation/#installation","title":"Installation","text":"user@pc:~/rmagine/build$ ./bin/rmagine_benchmark_gpu ../dat/sphere.ply\n...\n[ 100% - velos/s: 383094, mean: 383457, rays/s: 5.52178e+09] \nResult: 383457 velos/s\nAfter compilation do
"},{"location":"getting_started/installation/#optional-check-installation","title":"Optional: Check Installation","text":"user@pc:~/rmagine/build$ sudo make install\nYou can check if everything went wrong by running the benchmark that was build besides the library:
user@pc:~$ rmagine_benchmark_cpu rmagine/dat/sphere.ply\n...\n[ 100% - velos/s: 6261.9, mean: 6244.84] \nResult: 6244.84 velos/s\nor if the OptiX support was successfully build:
"},{"location":"getting_started/installation/#uninstall-rmagine","title":"Uninstall Rmagine","text":"user@pc:~$ rmagine_benchmark_gpu rmagine/dat/sphere.ply\n...\n[ 100% - velos/s: 383094, mean: 383457, rays/s: 5.52178e+09] \nResult: 383457 velos/s\n
"},{"location":"getting_started/installation/#installation-debian-package-experimental","title":"Installation (Debian Package) - Experimental","text":"user@pc:~/rmagine/build$ sudo make uninstall\nWe are working on creating debian packages for easier installations.
"},{"location":"getting_started/installation/#dependencies_1","title":"Dependencies","text":"
"},{"location":"getting_started/installation/#install","title":"Install","text":"$ sudo apt install libassimp-dev libeigen3-dev\nDownload latest Rmagine debian packages from Github releases page (v2.2.2). Install the core by calling
"},{"location":"getting_started/installation/#embree-backbone","title":"Embree Backbone","text":"sudo apt install ./rmagine-core_2.2.2_amd64.deb\nWe support Embree in its latest version (tested: v4.0.1 - v4.3.0). Make sure you have Embree installed on your system.
"},{"location":"getting_started/installation/#optix-backbone","title":"OptiX Backbone","text":"sudo apt install ./rmagine-embree_2.2.2_amd64.deb\nMake sure you have a current NVIDIA driver installed, then install rmagine-cuda and rmagine-optix by:
"},{"location":"getting_started/installation/#uninstall","title":"Uninstall","text":"sudo apt install ./rmagine-cuda_2.2.2_amd64.deb\nsudo apt install ./rmagine-optix_2.2.2_amd64.deb\nTo uninstall everything related to rmagine, call:
"},{"location":"getting_started/integration/","title":"Integration","text":"sudo apt-get remove rmagine-core\nHow to use and integrate Rmagine into your own project.
"},{"location":"getting_started/integration/#cpu-embree","title":"CPU (Embree)","text":""},{"location":"getting_started/integration/#in-code","title":"In Code","text":"
"},{"location":"getting_started/integration/#cmake","title":"CMake","text":"#include <rmagine/map/EmbreeMap.hpp>\n#include <rmagine/simulation/SphereSimulatorEmbree.hpp>\n\nnamespace rm = rmagine;\n\nint main(int argc, char** argv)\n{\n std::string filename = \"path/to/mesh/file\";\n rm::EmbreeMapPtr map = rm::import_embree_map(filename);\n\n rm::SphereSimulatorEmbree sim;\n sim.setMap(map);\n\n // go on (see workflow section)\n\n return 0;\n}\nAdd to your CMakeLists.txt:
"},{"location":"getting_started/integration/#gpu-optix","title":"GPU (OptiX)","text":""},{"location":"getting_started/integration/#in-code_1","title":"In Code","text":"add_compile_options(-std=c++17)\nset(CMAKE_CXX_STANDARD 17)\n\n# find components of a specific rmagine version\n# '2.2.1...' will get the newest rmagine which \n# is greater than 2.2.1\nfind_package(rmagine 2.2.1... \n COMPONENTS\n core \n embree\n)\n\nadd_executable(my_rmagine_app \n src/my_rmagine_app.cpp)\n\n# link against rmagine targets\ntarget_link_libraries(my_rmagine_app\n rmagine::core\n rmagine::embree\n)\n
"},{"location":"getting_started/integration/#cmake_1","title":"CMake","text":"#include <rmagine/map/OptixMap.hpp>\n#include <rmagine/simulation/SphereSimulatorOptix.hpp>\n\nnamespace rm = rmagine;\n\nint main(int argc, char** argv)\n{\n std::string filename = \"path/to/mesh/file\";\n rm::OptixMapPtr map = rm::import_optix_map(filename);\n\n rm::SphereSimulatorOptix sim;\n sim.setMap(map);\n\n // go on (see workflow section)\n\n return 0;\n}\nAdd to your CMakeFile:
"},{"location":"getting_started/maps/","title":"Maps","text":"add_compile_options(-std=c++17)\nset(CMAKE_CXX_STANDARD 17)\n\n# find components of a specific rmagine version\n# '2.2.1...' will get the newest rmagine which \n# is greater than 2.2.1\nfind_package(rmagine 2.2.1... \n COMPONENTS\n core \n cuda\n optix\n)\n\nadd_executable(my_rmagine_app \n src/my_rmagine_app.cpp)\n\n# link against rmagine targets\ntarget_link_libraries(my_rmagine_app\n rmagine::core\n rmagine::cuda\n rmagine::optix\n)\nTriangle Meshes can be stored in various file formats. Rmagine utilizes the Open Asset Import Library (assimp) in order to support a wide range of well known file formats. After loading the raw scene graph buffers with Assimp it is converted into Rmagines internal scene graph structure. Dependent on the computation backend
"},{"location":"getting_started/maps/#embree-map","title":"Embree Map","text":"EmbreeorOptiXthe scene graph is prepared for fast ray traversals by building the required acceleration structures.
"},{"location":"getting_started/maps/#optix-map","title":"OptiX Map","text":"#include <rmagine/map/EmbreeMap.hpp>\n\nnamespace rm = rmagine;\n\nint main(int argc, char** argv)\n{\n std::string filename = \"path/to/mesh/file\";\n rm::EmbreeMapPtr map = rm::import_embree_map(filename);\n return 0;\n}\n
"},{"location":"getting_started/maps/#properties","title":"Properties","text":"#include <rmagine/map/OptixMap.hpp>\n\nnamespace rm = rmagine;\n\nint main(int argc, char** argv)\n{\n std::string filename = \"path/to/mesh/file\";\n rm::OptixMapPtr map = rm::import_optix_map(filename);\n return 0;\n}\nAfter loading, the map consists of a complete scene graph. It then usually passed to a simulator (see next \"Getting Started\"-sections). The advanced Map-section describes how to modify or create the internal maps from scratch.
"},{"location":"getting_started/noise/","title":"Noise","text":"Currently noise models are implemented as postprocessing steps that modify the simulated ranges. Any of the following noise models can be chained to generate complex combined noise models. The Noise models are implemented equally both for GPU and CPU. Thus the developer can apply noise to the data without downloading or uploading the data from GPU to CPU or vice versa.
- Gaussian Noise
Apply gaussian noise $N(\\mu, \\sigma)$ to simulated ranges.
Parameter DescriptionmeanMean $\\mu$ of normal distributed noisestddevstandard deviation $\\sigma$ of normal distributed noiseExample CPU:
#include <rmagine/noise/GaussianNoise.hpp>\nnamespace rm = rmagine;\n\n// what happend before:\n// - data was simulated and stored into variable 'res'\n\n// make a copy to keep the unnoised ranges\nrm::Memory<float, rm::RAM> ranges = res.ranges;\n\n// Base class: Noise\nfloat mean = 0.0;\nfloat stddev = 1.0;\nrm::NoisePtr noise = std::make_shared<rm::GaussianNoise>(\n mean, stddev);\n\n// apply noise\nnoise->apply(ranges);\n// ranges now contains noise\nExample CUDA:
#include <rmagine/noise/GaussianNoiseCuda.hpp>\nnamespace rm = rmagine;\n\n// what happend before:\n// - data was simulated and stored into variable 'res'\n\n\n// make a copy to keep unnoised ranges\nrm::Memory<float, rm::VRAM_CUDA> ranges = res.ranges;\n\n// Base class: NoiseCuda\nfloat mean = 0.0;\nfloat stddev = 1.0;\nrm::NoiseCudaPtr noise = std::make_shared<rm::GaussianNoiseCuda>(\n mean, stddev);\n\n// apply noise\nnoise->apply(ranges);\n// ranges now contains noise\n- Relative Gaussian Noise
Apply gaussian noise $N(\\mu, \\sigma_r)$ to simulated ranges. Here, the standard deviation varies depending on distance.
Parameter DescriptionmeanMean $\\mu$ of normal distributed noisestddevstandard deviation $\\sigma$ of normal distributed noiserange_exprange exponent $c$ to compute range based stddev: $ \\sigma_r = \\sigma \\cdot r^{c} $Example CPU:
#include <rmagine/noise/RelGaussianNoise.hpp>\nnamespace rm = rmagine;\n\n// what happend before:\n// - data was simulated and stored into variable 'res'\n\n// make a copy to keep the unnoised ranges\nrm::Memory<float, rm::RAM> ranges = res.ranges;\n\n// Base class: Noise\nfloat mean = 0.0;\nfloat stddev = 0.2;\nfloat range_exp = 1.1;\nrm::NoisePtr noise = std::make_shared<rm::RelGaussianNoise>(\n mean, stddev, range_exp);\n\n// apply noise\nnoise->apply(ranges);\n// ranges now contains noise\nExample CUDA:
#include <rmagine/noise/RelGaussianNoiseCuda.hpp>\nnamespace rm = rmagine;\n\n// what happend before:\n// - data was simulated and stored into variable 'res'\n\n\n// make a copy to keep unnoised ranges\nrm::Memory<float, rm::VRAM_CUDA> ranges = res.ranges;\n\n// Base class: NoiseCuda\nfloat mean = 0.0;\nfloat stddev = 0.2;\nfloat range_exp = 1.1;\nrm::NoiseCudaPtr noise = std::make_shared<rm::RelGaussianNoiseCuda>(\n mean, stddev, range_exp);\n\n// apply noise\nnoise->apply(ranges);\n// ranges now contains noise\n- Uniform Dust Noise
Apply uniform dust noise to simulated ranges. Assuming some small particles could be hit by the range sensor that are not modeled by the scene, use this noise type.
Parameters:
Parameter Descriptionhit_probProbability of a ray hitting a particle in one meter free space.return_probProbability of a ray hitting dust returns to sender depending on particle distanceExample CPU:
#include <rmagine/noise/UniformDustNoise.hpp>\nnamespace rm = rmagine;\n\n// what happend before:\n// - data was simulated and stored into variable 'res'\n\n// make a copy to keep the unnoised ranges\nrm::Memory<float, rm::RAM> ranges = res.ranges;\n\n// Base class: Noise\nfloat hit_prob = 0.0;\nfloat ret_prob = 1.0;\nrm::NoisePtr noise = std::make_shared<rm::UniformDustNoise>(\n hit_prob, ret_prob);\n\n// apply noise\nnoise->apply(ranges);\n// ranges now contains noise\nExample CUDA:
"},{"location":"getting_started/overview/","title":"Rmagine","text":""},{"location":"getting_started/overview/#3d-range-sensor-simulation-in-polygonal-maps-via-ray-tracing-for-embedded-hardware-on-mobile-robots","title":"3D Range Sensor Simulation in Polygonal Maps via Ray Tracing for Embedded Hardware on Mobile Robots","text":"#include <rmagine/noise/UniformDustNoiseCuda.hpp>\nnamespace rm = rmagine;\n\n// what happend before:\n// - data was simulated and stored into variable 'res'\n\n\n// make a copy to keep unnoised ranges\nrm::Memory<float, rm::VRAM_CUDA> ranges = res.ranges;\n\n// Base class: NoiseCuda\nfloat hit_prob = 0.0;\nfloat ret_prob = 1.0;\nrm::NoiseCudaPtr noise = std::make_shared<rm::UniformDustNoiseCuda>(\n hit_prob, ret_prob);\n\n// apply noise\nnoise->apply(ranges);\n// ranges now contains noise\nLibrary for fast sensor data simulation in large 3D environments.
"},{"location":"getting_started/overview/#design-goals","title":"Design Goals","text":"Mainly designed for robotic applications:
- Perform multiple sensor simulations simultaneously and in realtime
- Perform computations at specific computing devices (CPU, GPU..)
- Hold data at device of computation
- Minimal graphical overhead (offscreen-rendering)
- Runtime critical operations
In addition to the wiki, the following papers wrap up the library conceptually. Please reference the following papers when using the Rmagine library in your scientific work.
"},{"location":"getting_started/problem_modelling/","title":"Problem Modelling","text":"@inproceedings{mock2023rmagine,\n title={{Rmagine: 3D Range Sensor Simulation in Polygonal Maps via Ray Tracing for Embedded Hardware on Mobile Robots}}, \n author={Mock, Alexander and Wiemann, Thomas and Hertzberg, Joachim},\n booktitle={IEEE International Conference on Robotics and Automation (ICRA)}, \n year={2023}\n}\nThe general computing flow is as follows.
Tsb is the transform from sensor to base frame. Or in other words: The sensor pose relative to the robot base. The map can be either a pointer to an
EmbreeMaporOptixMap. The Prefix of theSimulatoris eitherEmbreefor CPU computation orOptixfor GPU computation. The suffix of the Simulator is dependent on which sensor model you want to simulate. A few examples:SphereSimulatorEmbree- Simulate a velodyne on CPUPinholeSimulatorOptix- Simulate a depth camera on GPUO1DnSimulatorOptix- Simulate a customO1DnModelon GPU
Now we want to construct the following pipeline.
"},{"location":"getting_started/problem_modelling/#example-2-simulate-1000-lidars-on-gpu","title":"Example 2: Simulate 1000 LiDaRs on GPU","text":"#include <rmagine/simulation/SphereSimulatorEmbree.hpp>\n\nusing namespace rmagine;\n\nSphereSimulatorEmbreePtr construct_simulator(std::string path_to_mesh)\n{\n // Default construct the SphereSimulatorEmbree as shared pointer\n SphereSimulatorEmbreePtr sim = std::make_shared<SphereSimulatorEmbree>();\n\n EmbreeMapPtr map = import_embree_map(path_to_mesh);\n sim->setMap(map);\n\n // Define sensor model\n SphericalModel model;\n // TODO: fill with model specific parameters\n sim->setModel(model);\n\n // Set static transform between sensor and base (optional)\n Transform Tsb;\n Tsb.setIdentity();\n sim->setTsb(Tsb);\n\n return sim;\n}\n\nint main(int argc, char** argv)\n{\n // Load map and set map pointer to simulator\n std::string path_to_mesh = argv[1];\n SphereSimulatorEmbreePtr sim = construct_simulator(path_to_mesh);\n\n // Define 1000 poses to simulate from\n Memory<Transform, RAM> poses(1000);\n for(int i = 0; i<poses.size(); i++)\n {\n poses[i].setIdentity();\n }\n\n // add your desired attributes at intersection here\n using ResultT = Bundle<\n Ranges<RAM> \n >;\n\n // Result: Simulate Ranges\n ResultT res = sim->simulate<ResultT>(poses_);\n // res.ranges holds a buffer to the ranges\n\n return 0;\n}\nNow we want to construct the following pipeline.
The green cells are memory objects on GPU as you see in the following code snippet.
"},{"location":"getting_started/problem_modelling/#example-3-simulate-1000-lidars-on-gpu-and-images-on-cpu","title":"Example 3: Simulate 1000 LiDaRs on GPU and Images on CPU","text":"#include <rmagine/simulation/SphereSimulatorOptix.hpp>\n\nusing namespace rmagine;\n\nSphereSimulatorOptixPtr construct_simulator(std::string path_to_mesh)\n{\n // Default construct the SphereSimulatorEmbree as shared pointer\n SphereSimulatorOptixPtr sim = std::make_shared<SphereSimulatorOptix>();\n\n OptixMapPtr map = import_optix_map(path_to_mesh);\n sim->setMap(map);\n\n // Define sensor model\n SphericalModel model;\n // TODO: fill with model specific parameters\n sim->setModel(model);\n\n // Set static transform between sensor and base (optional)\n Transform Tsb;\n Tsb.setIdentity();\n sim->setTsb(Tsb);\n\n return sim;\n}\n\nint main(int argc, char** argv)\n{\n // Load map and set map pointer to simulator\n std::string path_to_mesh = argv[1];\n SphereSimulatorOptixPtr sim = construct_simulator(path_to_mesh);\n\n // Define 1000 poses to simulate from\n Memory<Transform, RAM> poses(1000);\n for(int i = 0; i<poses.size(); i++)\n {\n poses[i].setIdentity();\n }\n\n // upload from CPU to GPU\n Memory<Transform, VRAM_CUDA> poses_ = poses;\n\n // add your desired attributes at intersection here\n using ResultT = Bundle<\n Ranges<VRAM_CUDA> \n >;\n\n // Result: Simulate Ranges\n ResultT res = sim->simulate<ResultT>(poses_);\n\n // download from GPU to CPU\n // or use CUDA buffer for other computations\n Memory<float, RAM> ranges = res.ranges;\n\n return 0;\n}\nNow we want to construct the following pipeline.
"},{"location":"getting_started/sensors/","title":"Supported Sensor Models","text":"#include <rmagine/simulation/SphereSimulatorOptix.hpp>\n#include <rmagine/simulation/PinholeSimulatorEmbree.hpp>\n\nusing namespace rmagine;\n\nint main(int argc, char** argv)\n{\n // Load map and set map pointer to simulator\n std::string path_to_mesh = argv[1];\n\n // CONSTRUCTION PART\n\n // Define Simulators\n SphereSimulatorOptix lidar_sim_gpu;\n PinholeSimulatorEmbree dcam_sim_cpu;\n\n // Load and set maps\n OptixMapPtr map_gpu = import_optix_map(path_to_mesh);\n EmbreeMapPtr map_cpu = import_embree_map(path_to_mesh);\n lidar_sim_gpu.setMap(map_gpu);\n dcam_sim_cpu.setMap(map_cpu);\n\n\n SphericalModel lidar_model;\n PinholeModel dcam_model;\n // TODO: Define models\n lidar_sim_gpu.setModel(lidar_model);\n dcam_sim_cpu.setModel(dcam_model);\n\n // Define static transforms (optional)\n Transform T_lidar_base;\n Transform T_dcam_base;\n lidar_sim_gpu.setTsb(T_lidar_base);\n dcam_sim_cpu.setTsb(T_dcam_base);\n\n // SIMULATION PART\n\n Memory<Transform, RAM> poses(1000);\n // TODO: fill poses\n\n // upload from CPU to GPU\n Memory<Transform, VRAM_CUDA> poses_ = poses;\n\n\n // Simulate Depth cameras ranges on CPU\n using ResultT_RAM = Bundle<\n Ranges<RAM> \n >;\n ResultT_RAM dcam_res\n = dcam_sim_cpu.simulate<ResultT_RAM>(poses);\n\n // Simulate LiDaRs ranges on GPU\n using ResultT_VRAM = Bundle<\n Ranges<VRAM_CUDA> \n >;\n ResultT_VRAM lidar_res\n = lidar_sim_gpu.simulate<ResultT_VRAM>(poses_);\n\n // Download lidar ranges\n Memory<float, RAM> lidar_ranges = lidar_res.ranges;\n\n // Results are in dcam_res.ranges and lidar_ranges\n\n return 0;\n}\nRmagine supports several configurations of commonly used range sensors as Spherical, Pinhole or even fully customizable O1Dn and OnDn models. The following image shows how the results could look like using these different models.
The next instructions show how to initialize each model individually.
"},{"location":"getting_started/sensors/#spherical","title":"Spherical","text":"Spherical model for LiDARs.
struct SphericalModel\n{\n DiscreteInterval phi;\n DiscreteInterval theta;\n Interval range;\n};\nExample:
"},{"location":"getting_started/sensors/#pinhole","title":"Pinhole","text":"#include <rmagine/types/sensor_models.h>\nnamespace rm = rmagine;\n\n// ...\n\nrm::SphericalModel model;\n\nmodel.theta.min = -M_PI;\nmodel.theta.inc = 0.4 * M_PI / 180.0;\nmodel.theta.size = 900;\n\nmodel.phi.min = -15.0 * M_PI / 180.0;\nmodel.phi.inc = 2.0 * M_PI / 180.0;\nmodel.phi.size = 16;\n\nmodel.range.min = 0.0;\nmodel.range.max = 100.0;\nPinhole model for depth cameras.
struct PinholeModel\n{\n uint32_t width;\n uint32_t height;\n\n Interval range;\n\n float f[2]; // focal lengths fx, fy\n float c[2]; // centroid cx, cy\n};\nExample:
"},{"location":"getting_started/sensors/#o1dn","title":"O1Dn","text":"#include <rmagine/types/sensor_models.h>\nnamespace rm = rmagine;\n\n...\n\nrm::PinholeModel model;\nmodel.width = 200;\nmodel.height = 150;\nmodel.c[0] = 100.0;\nmodel.c[1] = 75.0;\nmodel.f[0] = 100.0;\nmodel.f[1] = 100.0;\nmodel.range.min = 0.0;\nmodel.range.max = 100.0;\nFully customizable model with one origin and N directions.
struct O1DnModel\n{\n uint32_t width;\n uint32_t height;\n\n // maximum and minimum allowed range\n Interval range;\n\n // i-th ray = orig, dirs[i]\n Vector orig; // One origin\n Memory<Vector> dirs; // N directions\n};\nExample:
"},{"location":"getting_started/sensors/#ondn","title":"OnDn","text":"#include <rmagine/types/sensor_models.h>\nnamespace rm = rmagine;\n\n...\n\nrm::O1DnModel model;\n\nmodel.orig.x = 0.0;\nmodel.orig.y = 0.0;\nmodel.orig.z = 0.0;\n\nmodel.width = 200;\nmodel.height = 1;\n\nmodel.dirs.resize(model.width * model.height);\n\nfloat step_size = 0.05;\n\nfor(int i=0; i<200; i++)\n{\n float y = - static_cast<float>(i - 100) * step_size;\n float x = cos(y) * 2.0 + 2.0;\n float z = -1.0;\n\n model.dirs[i].x = x;\n model.dirs[i].y = y;\n model.dirs[i].z = z;\n\n model.dirs[i].normalize();\n}\n\nmodel.range.min = 0.0;\nmodel.range.max = 100.0;\nFully customizable model with N origins and N directions.
struct OnDnModel\n{\n uint32_t width;\n uint32_t height;\n\n Interval range;\n\n // i-th ray = origs[i], dirs[i]\n Memory<Vector> origs; // N origins\n Memory<Vector> dirs; // N directions\n};\nExample:
"},{"location":"getting_started/sensors/#predefined-models","title":"Predefined models","text":"#include <rmagine/types/sensor_models.h>\nnamespace rm = rmagine;\n\n...\n\nrm::OnDnModel model;\n\nmodel.width = 200;\nmodel.height = 1;\n\nmodel.dirs.resize(model.width * model.height);\nmodel.origs.resize(model.width * model.height);\n\nfloat step_size = 0.05;\n\nfor(int i=0; i<200; i++)\n{\n float percent = static_cast<float>(i) / static_cast<float>(200);\n float step = - static_cast<float>(i - 100) * step_size;\n float y = sin(step);\n float x = cos(step);\n\n model.origs[i].x = 0.0;\n model.origs[i].y = y * percent;\n model.origs[i].z = x * percent;\n\n model.dirs[i].x = 1.0;\n model.dirs[i].y = 0.0;\n model.dirs[i].z = 0.0;\n}\n\nmodel.range.min = 0.0;\nmodel.range.max = 100.0;\nRmagine also provides some example models for testing. These are located in the
rmagine/types/sensors.hHeader-file and will be expanded in the future to include additional range sensors.
"},{"location":"getting_started/simulation/","title":"Simulation","text":""},{"location":"getting_started/simulation/#requirements","title":"Requirements","text":"#include <rmagine/types/sensors.h>\nnamespace rm = rmagine;\n\n\n...\n\n// Velodyne VLP-16 with different horizontal resoultions\nrm::SphericalModel velo_model_1 = rm::vlp16_900();\nrm::SphericalModel velo_model_2 = rm::vlp16_360();\n\n\n// another examples for testing:\nrm::SphericalModel ex_lidar = rm::example_spherical();\nrm::PinholeModel ex_pinhole = rm::example_pinhole();\nrm::O1DnModel ex_o1dn = rm::example_o1dn();\nrm::OnDnModel ex_ondn = rm::example_ondn();\nOnce a map is loaded and a sensor is defined anything is known to simulate the first range data.
"},{"location":"getting_started/simulation/#intersection-attributes","title":"Intersection Attributes","text":"Attribute Type Stride Description Hits uint8 1 If the a face was intersected (1) or not (0) Ranges float 1 Distance from ray origin along the direction to the first intersection Points float 3 Cartesian Coordinates of the Intersection (x,y,z) Normals float 3 Normal (nx, ny, nz) of intersected face FaceIds uint32 1 The id of the face that was intersected ObjectIds uint32 1 The id of the object that was intersected GeomIds uint32 1 The id of the geometry that was intersected"},{"location":"getting_started/simulation/#handle-results","title":"Handle Results","text":"// Map\n#include <rmagine/map/EmbreeMap.hpp>\n// Sensor Models\n#include <rmagine/types/sensor_models.h>\n// Simulators\n#include <rmagine/simulation/SphereSimulatorEmbree.hpp>\n\nnamespace rm = rmagine;\n\n// ...\n\n// loading a map\nstd::string path_to_mesh = \"my_mesh.ply\";\nrm::EmbreeMapPtr map = rm::load_embree_map(path_to_mesh);\n\n// defining a model\nrm::SphericalModel velo_model = rm::vlp16_900();\n\n// construct a simulator\nrm::SphereSimulatorEmbree sim;\nsim.setMap(map);\nsim.setModel(velo_model);\n\n// simulate ranges\n// ...\n
"},{"location":"library/concepts/","title":"Key Concepts","text":"// ...\n// Defined previously\n// - namespace rm = rmagine;\n// - SphereSimulatorEmbree sim;\n\n\n// 100 Transformations between base and map. e.g. poses of the robot\nrm::Memory<rm::Transform, rm::RAM> Tbm(100);\n\nfor(size_t i=0; i < Tbm.size(); i++)\n{\n rm::Transform T = rm::Transform::Identity();\n T.t = {2.0, 0.0, 0.0}; // position (2,0,0)\n rm::EulerAngles e = {0.0, 0.0, 1.0}; // orientation (0,0,1) radian - as euler angles\n T.R.set(e); // euler internally converted to quaternion\n Tbm[i] = T; // Write Transform/Pose to Memory\n}\n\n// add your desired attributes at intersection here\n// - optimizes the code at compile time\nusing ResultT = rm::Bundle<\n rm::Hits<rm::RAM>, \n rm::Ranges<rm::RAM>\n>;\n\n// Possible Attributes (rmagine/simulation/SimulationResults.hpp):\n// - Hits\n// - Ranges\n// - Points\n// - Normals\n// - FaceIds\n// - GeomIds\n// - ObjectIds\n\n// querying every attribute with 'rm::IntAttrAny' instead of 'ResultT'\n\nResultT result = sim.simulate<ResultT>(poses);\n// result.hits, result.ranges contain the resulting attribute buffers\nstd::cout << \"printing the first ray's range: \" << result.ranges[0] << std::endl;\n\n// or slice the results for the scan of pose 5\nauto ranges5 = result.ranges(5 * model.size(), 6 * model.size());\nstd::cout << \"printing the first ray's range of the fifth scan: \" << ranges5[0] << std::endl;\n\n// slicing and other useful operations will be described at another Wiki page.\nRmagine aims to perform computations flexible on selectable computing devices (CPU, GPU, ...). It also provides mechanisms to minimize graphical overheads and unnecessary copies between devices.
"},{"location":"library/concepts/#structure","title":"Structure","text":"Rmagine has the following top-level structure of directories:
- Math -
rmagine/math - Types -
rmagine/types - Utilility -
rmagine/util - Map -
rmagine/map - Simulation -
rmagine/simulation - Noise -
rmagine/noise
Contains all math related types and functions. All math datatypes are completely CUDA compatible. See
"},{"location":"library/concepts/#types","title":"Types","text":"rmagine/math/types.hfor all types. See Math section for more details.Fundamental types required for simulations, e.g. sensor models. Depends on math types.
"},{"location":"library/concepts/#util","title":"Util","text":"Utility functions. For exaple, including
rmagine/util/prints.hlets you print every math types viastd::cout.#include <iostream>\n#include <rmagine/math/types.h>\n#include <rmagine/util/prints.h>\n\nnamespace rm = rmagine;\n\nint main(int argc, char* argv)\n{\n rm::Transform T = rm::Transform::Identity();\n std::cout << \"my transformation: \" << T << std::endl;\n return 0;\n}\nThe Stopwatch in
rmagine/util/StopWatch.hpplets you easily stop the runtime of a Code section.
"},{"location":"library/concepts/#map","title":"Map","text":"#include <iostream>\n#include <rmagine/util/StopWatch.hpp>\n\nnamespace rm = rmagine;\n\nvoid demanding_function()\n{\n // ...\n}\n\nint main(int argc, char* argv)\n{\n rm::StopWatch sw;\n double el;\n\n sw();\n demanding_function();\n el = sw();\n std::cout << \"Demanding Function took \" << el << \" s\" << std::endl;\n\n return 0;\n}\nAll classes and functions that relate to map construction and map modifications. See Getting Started - Maps for a introduction to map loading. Go to Library - Maps for a more detailed descriptions of how to build Rmagine maps.
"},{"location":"library/concepts/#simulation","title":"Simulation","text":"All classes and functions that relate to the actual simulations.
"},{"location":"library/concepts/#noise","title":"Noise","text":"All classes and functions that relate to postprocessed noising.
"},{"location":"library/maps/","title":"Maps","text":""},{"location":"library/maps/#rmagine-scene-graph","title":"Rmagine Scene Graph","text":"Rmagine provides datatypes for scene graphs. They are seperated into Embree and OptiX scene graphs to store a Scene Graph either on CPU or on GPU. Both of these Scene Graphs have a very similar interface but are slightly different. Thus are real shared interface is not implemented yet. In Rmagine, a scene graph holds Geometries, Instances and Scenes.
"},{"location":"library/maps/#geometry","title":"Geometry","text":"A geometry is an abstract description of a physical object. Implemented Geometries are:
Concept Embree OptiX Mesh EmbreeMesh OptixMesh Points EmbreePoints -We also implemented some shortcut meshes that are used in later examples. They are located in the files
ShortCut Mesh Embree OptiX Spherermagine/map/embree/embree_shapes.handrmagine/map/optix/optix_shapes.h.EmbreeSphereOptixSphereCubeEmbreeCubeOptixCubePlaneEmbreePlaneOptixPlaneCylinderEmbreeCylinderOptixCylinder"},{"location":"library/maps/#scene","title":"Scene","text":"A scene is a set of geometries or a set of instances, each of which is assigned a position, an orientation, and a scale.
"},{"location":"library/maps/#instance","title":"Instance","text":"An instance instantiates a given scene or geometry at a certain pose. Thus things can be instantiated without duplicating their memory. A classic example for that is 3D asteroids where the same asteroid geometry has to be spawned several times. Using different geometries would then bring the GPU memory to its limits. Instead, Instantiating one geometry several times leads to a more memory friendly way of solving this problem. In robotics one can think of a known geometry as a class, e.g. a chair. This chair can be placed several times in the map by instantiating it.
"},{"location":"library/maps/#scene-graph-embree","title":"Scene Graph Embree","text":""},{"location":"library/maps/#simple","title":"Simple","text":"
"},{"location":"library/maps/#instances","title":"Instances","text":"#include <rmagine/map/embree/EmbreeScene.hpp>\n// shortcut meshes\n#include <rmagine/map/embree/embree_shapes.h>\n\nnamespace rm = rmagine;\n\nint main(int argc, char** argv)\n{\n // create a scene \n rm::EmbreeScenePtr scene = std::make_shared<rm::EmbreeScene>();\n\n // create a sphere\n rm::EmbreeMeshPtr sphere = std::make_shared<rm::EmbreeSphere>(1.0);\n { // sphere settings\n Transform T = Transform::Identity();\n T.t.x = 1.0; // move sphere one unit along the x-axis \n sphere->setTransform(T);\n sphere->apply(); // apply transform related changes\n }\n sphere->commit(); // commit sphere\n\n scene->add(sphere); // add sphere to scene\n scene->commit(); // commit scene\n\n // add the scene to a map. \n rm::EmbreeMapPtr map = std::make_shared<rm::EmbreeMap>(scene);\n\n // The map can then be used for simulations ...\n return 0;\n}\n
"},{"location":"library/maps/#custom-meshes","title":"Custom Meshes","text":"#include <rmagine/map/embree/EmbreeScene.hpp>\n#include <rmagine/map/embree/EmbreeInstance.hpp>\n// shortcut meshes\n#include <rmagine/map/embree/embree_shapes.h>\n\nnamespace rm = rmagine;\n\n/**\n * Create a Scene consisting of a sphere, a cube, and a cylinder\n */\nrm::EmbreeScenePtr create_scene()\n{\n rm::EmbreeScenePtr scene = std::make_shared<rm::EmbreeScene>();\n\n rm::EmbreeMeshPtr sphere = std::make_shared<rm::EmbreeSphere>(1.0);\n { // sphere settings\n Transform T = Transform::Identity();\n T.t.y = -2.0; // move sphere two units right \n sphere->setTransform(T);\n sphere->apply(); // apply transform related changes\n }\n sphere->commit(); // commit sphere\n scene->add(sphere); // add sphere to scene\n\n rm::EmbreeMeshPtr cube = std::make_shared<rm::EmbreeCube>();\n cube->commit(); // commit cube\n scene->add(cube); // add cube to scene\n\n auto cylinder = std::make_shared<rm::EmbreeCylinder>();\n { // cylinder settings\n Transform T = Transform::Identity();\n T.t.y = 2.0; // move cylinder two units left \n cylinder->setTransform(T);\n cylinder->apply(); // apply transform related changes\n }\n cylinder->commit(); // commit cylinder\n scene->add(cylinder); // add cylinder to scene\n\n scene->commit(); // commit scene\n return scene;\n}\n\nint main(int argc, char** argv)\n{\n // create a scene \n auto scene = std::make_shared<rm::EmbreeScene>();\n\n auto subscene = create_scene();\n\n // add 100 instances of a subscene to a scene\n for(size_t i=0; i<100; i++)\n {\n rm::EmbreeInstancePtr subscene_inst = subscene->instantiate();\n { // subscene_inst settings\n rm::Transform T = rm::Transform::Identity();\n T.t.x = (float)i; // moving instance 5 units to front\n subscene_inst->setTransform(T);\n subscene_inst->apply(); // apply transform related changes\n }\n subscene_inst->commit(); // commit instances\n scene->add(subscene_inst); // add instance to scene\n }\n\n scene->commit();\n\n // add the scene to a map. \n rm::EmbreeMapPtr map = std::make_shared<rm::EmbreeMap>(scene);\n\n // The map can then be used for simulations ...\n return 0;\n}\n
"},{"location":"library/maps/#scene-graph-optix","title":"Scene Graph OptiX","text":""},{"location":"library/maps/#simple_1","title":"Simple","text":"rm::EmbreeMeshPtr create_custom_mesh()\n{\n size_t Nvertices = 3;\n size_t Nfaces = 1;\n auto mesh = std::make_shared<rm::EmbreeMesh>(Nvertices, Nfaces);\n\n // reference to data as MemoryView objects\n rm::MemoryView<rm::Vertex, rm::RAM> vertices = mesh->vertices();\n rm::MemoryView<rm::Face, rm::RAM> faces = mesh->faces();\n\n faces[0] = {0, 1, 2};\n vertices[0] = {1.0, 0.0, 0.0};\n vertices[1] = {0.0, 1.0, 0.0};\n vertices[2] = {0.0, 0.0, 0.0};\n\n return mesh;\n}\n\nint main(int argc, char** argv)\n{\n auto scene = std::make_shared<rm::EmbreeScene>();\n\n auto my_mesh = create_custom_mesh();\n my_mesh->commit();\n\n scene->add(my_mesh);\n scene->commit();\n // do something with scene ...\n return 0;\n}\n
"},{"location":"library/maps/#instances_1","title":"Instances","text":"#include <rmagine/map/optix/OptixScene.hpp>\n// shortcut meshes\n#include <rmagine/map/optix/optix_shapes.h>\n\nnamespace rm = rmagine;\n\nint main(int argc, char** argv)\n{\n // create a scene \n rm::OptixScenePtr scene = std::make_shared<rm::OptixScene>();\n\n // create a sphere\n rm::OptixMeshPtr sphere = std::make_shared<rm::OptixSphere>(1.0);\n { // sphere settings\n Transform T = Transform::Identity();\n T.t.x = 1.0; // move sphere one unit along the x-axis \n sphere->setTransform(T);\n sphere->apply(); // apply transform related changes\n }\n sphere->commit(); // commit sphere\n\n scene->add(sphere); // add sphere to scene\n scene->commit(); // commit scene\n\n // add the scene to a map. \n rm::OptixMapPtr map = std::make_shared<rm::OptixMap>(scene);\n\n // The map can then be used for simulations ...\n return 0;\n}\n
"},{"location":"library/math/","title":"Rmagine - Math-Library","text":"#include <rmagine/map/optix/OptixScene.hpp>\n#include <rmagine/map/optix/OptixInstance.hpp>\n// shortcut meshes\n#include <rmagine/map/optix/optix_shapes.h>\n\nnamespace rm = rmagine;\n\n/**\n * Create a Scene consisting of a sphere, a cube, and a cylinder\n */\nrm::OptixScenePtr create_scene()\n{\n rm::OptixScenePtr scene = std::make_shared<rm::OptixScene>();\n\n rm::OptixMeshPtr sphere = std::make_shared<rm::OptixSphere>(1.0);\n { // sphere settings\n Transform T = Transform::Identity();\n T.t.y = -2.0; // move sphere two units right \n sphere->setTransform(T);\n sphere->apply(); // apply transform related changes\n }\n sphere->commit(); // commit sphere\n scene->add(sphere); // add sphere to scene\n\n rm::OptixMeshPtr cube = std::make_shared<rm::OptixCube>();\n cube->commit(); // commit cube\n scene->add(cube); // add cube to scene\n\n auto cylinder = std::make_shared<rm::OptixCylinder>();\n { // cylinder settings\n Transform T = Transform::Identity();\n T.t.y = 2.0; // move cylinder two units left \n cylinder->setTransform(T);\n cylinder->apply(); // apply transform related changes\n }\n cylinder->commit(); // commit cylinder\n scene->add(cylinder); // add cylinder to scene\n\n scene->commit(); // commit scene\n return scene;\n}\n\nint main(int argc, char** argv)\n{\n // create a scene \n auto scene = std::make_shared<rm::OptixScene>();\n\n auto subscene = create_scene();\n\n // add 100 instances of a subscene to a scene\n for(size_t i=0; i<100; i++)\n {\n rm::OptixInstancePtr subscene_inst = subscene->instantiate();\n { // subscene_inst settings\n rm::Transform T = rm::Transform::Identity();\n T.t.x = (float)i; // moving instance 5 units to front\n subscene_inst->setTransform(T);\n subscene_inst->apply(); // apply transform related changes\n }\n subscene_inst->commit(); // commit instance\n scene->add(subscene_inst); // add instance to scene\n }\n\n scene->commit();\n\n // add the scene to a map. \n rm::OptixMapPtr map = std::make_shared<rm::OptixMap>(scene);\n\n // The map can then be used for simulations ...\n return 0;\n}\nRmagine provides it's own thin math library. It's located in
"},{"location":"library/math/#points-and-translations","title":"Points and Translations","text":"rmagine::coretarget. We decided to do so after having problems with Eigen in CUDA code (corrupt memory after using math functions). This thin math library was specifically designed to enable the sharing of functions between CPU and GPU code, and it has been tested to ensure consistent results for both CUDA and CPU implementations. The following descriptions are made reading the code located inrmagine/math/types.h. So it is recommended to open the file alongside.A floating-point coordinate can represent different things such as a point, a vector or the translational part of a transformation. For all of them, we provide the same data structure:
Vector.rm::Vector2: x,y all fp32rm::Vector3: x,y,z all fp32
Aliases: -
rm::Vector=rm::Vector3; -rm::Point=rm::Vector3; -rm::Vertex=rm::Vector3;We also implemented commonly used functions
Vector:
"},{"location":"library/math/#rotations","title":"Rotations","text":"#include <rmagine/math/types.h>\nnamespace rm = rmagine;\n\n// ...\n\n// initializations\nrm::Vector3 p1;\np1.x = 0.0;\np1.y = 1.0;\np1.z = 2.0;\nrm::Vector3 p2 = {1.0, 2.0, 3.0};\n\n// functions\nfloat p1_length = p1.l2norm();\n\n// operators\nrm::Vector3 p3;\np3 = p1 + p2;\np3 = p1 - p2;\np3 = p1 * 2.0;\np3 = p1 / 2.0;\n\n// ...\nIn Rmagine we provide three different representations of rotations: Euler angles (
rm::EulerAngles), a rotation matrix (rm::Matrix3x3) and a quaternion (rm::Quaternion). In general, we adhere to the ROS conventions, especially those that are listed in REP-103.
"},{"location":"library/math/#conversions","title":"Conversions","text":"#include <rmagine/math/types.h>\nnamespace rm = rmagine;\n\nint main(int argc, char** argv)\n{\n // EulerAngles\n rm::EulerAngles e1;\n e1.roll = 0.0;\n e1.pitch = 0.0;\n e1.yaw = M_PI / 2.0;\n rm::EulerAngles e2 = {0.0, 0.0, M_PI / 2.0};\n rm::EulerAngles eI = rm::EulerAngles::Identity();\n\n // Quaternion\n rm::Quaternion q1;\n q1.x = 0.0;\n q1.y = 0.0;\n q1.z = 0.7071068;\n q1.w = 0.7071068;\n rm::Quaternion q2 = {0.0, 0.0, 0.7071068, 0.7071068};\n rm::Quaternion qI = rm::Quaternion::Identity();\n\n // Matrix3x3\n // - Storage Order: Column-Major\n // - Access via '()'-operator: Row-Major\n // - Access via '[]'-operator: Column-Major\n rm::Matrix3x3 M1;\n M1(0,0) = 0.0; M1(0,1) = -1.0; M1(0,2) = 0.0;\n M1(1,0) = 1.0; M1(1,1) = 0.0; M1(1,2) = 0.0;\n M1(2,0) = 0.0; M1(2,1) = 0.0; M1(2,2) = 1.0;\n rm::Matrix3x3 M2 = {{\n {0.0, 1.0, 0.0},\n {-1.0, 0.0, 0.0},\n {0.0, 0.0, 1.0}\n }};\n rm::Matrix3x3 MI = rm::Matrix3x3::Identity();\n\n return 0;\n}\nWe provide conversions between different rotation representations:
#include <rmagine/math/types.h>\nnamespace rm = rmagine;\n\nint main(int argc, char** argv)\n{\n // initializations\n rm::EulerAngles e;\n rm::Matrix3x3 M;\n rm::Quaterion q; \n\n rm::Vector3 p = {1.0, 0.0, 0.0};\n\n // rotation 90 degree around z-axis counter-clockwise\n e.roll = 0.0;\n e.pitch = 0.0;\n e.yaw = M_PI / 2.0;\n\n // convert\n M.set(e); // set M values that express the same rotation as e\n q.set(e); // set q values that express the same rotation as e\n\n rm::Vector3 p1 = e * p;\n rm::Vector3 p2 = M * p;\n rm::Vector3 p3 = q * p;\n // p1, p2 and p3 should all contain the values {0.0, 1.0, 0.0}\n\n // other conversions:\n q.set(M);\n M.set(q);\n e.set(M);\n e.set(q);\n\n return 0;\n}\nand some math functions, e.g. to apply a rotation to another or to a point in space:
"},{"location":"library/math/#eigen-compatibility","title":"Eigen Compatibility","text":"#include <rmagine/math/types.h>\nnamespace rm = rmagine;\nusing namespace rmagine;\n\nint main(int argc, char** argv)\n{\n rm::EulerAngles e = {0.0, 0.0, M_PI/2.0};\n rm::Quaternion q; q.set(e);\n rm::Vector3 p = {1.0, 0.0, 0.0};\n\n // Rotate a point 90 degrees around the z axis counter-clockwise\n rm::Vector3 p1 = q * p;\n\n // Chaining Rotations\n // -> Rotate a point 90 degrees around the z axis clockwise\n rm::Quaternion q2 = q * q * q;\n rm::Vector3 p2 = q2 * p;\n\n // invert\n rm::Quaternion q2_inv = q2.inv();\n // or if 'using namespace rmagine;' was set, ~operator can be used\n q2_inv = ~q2;\n\n return 0;\n}\nWe ensure compatibility with Eigen by sharing the same memory layout (for now). This allows to do fast mappings between rmagine and Eigen types.
"},{"location":"library/math/#transformations","title":"Transformations","text":"#include <rmagine/math/types.h>\n#include <Eigen/Dense>\nnamespace rm = rmagine;\n\nint main(int argc, char** argv)\n{\n // Rmagine -> Eigen\n {\n // generate rmagine type\n rm::Matrix3x3 M_rm = rm::Matrix3x3::Identity();\n\n // rmagine's Matrix3x3 has the same Column-Major storage order as the Eigen default for Eigen::Matrix3f\n Eigen::Matrix3f& M_eig = *reinterpret_cast<Eigen::Matrix3f*>(&M_rm);\n\n // another way is to use Eigens functions for mapping raw buffers\n Eigen::Map<Eigen::Matrix3f> M_eig2(&M_rm(0,0));\n\n // changing an entry in M_rm should now change the same entry in M_eig and M_eig2 as well\n }\n\n // Eigen -> Rmagine\n {\n Eigen::Matrix3f M_eig = Eigen::Matrix3f::Identity();\n rm::Matrix3x3& M_rm = *reinterpret_cast<rm::Matrix3x3*>(&M_eig);\n }\n\n return 0;\n}\nIn Rmagine, a transformation (
rm::Transform) is an operation that maps a source Euclidean space to a target Euclidean space, implemented as isometry, since we only implemented the rotational and translational part (no scale). We decided to represent the rotational part as quaternion (rm::Quaternion) to avoid gimbal locks that occur e.g. using an Euler angles representation.In Rmagine, we use a
Transformtype for a pose as well. A sensor pose entries correspond to a transformation that maps the space with the sensor as origin to the space where the pose is located:#include <rmagine/math/types.h>\nnamespace rm = rmagine;\n\nint main(int argc, char** argv)\n{\n // pose of the sensor in the map\n rm::Vector3 s_t = {0.0, 1.0, 2.0};\n rm::Quaternion s_R = rm::Quaternion::Identity();\n\n // has the same entries as the transform form sensor -> map\n rm::Transfrom T_sensor_to_map;\n T_sensor_to_map.R = s_R;\n T_sensor_to_map.t = s_t;\n\n return 0;\n}\nTo express a transformation that is not isometric, for example because it consists of a scale part, use
rm::Matrix4x4and the linear algebra functions ofrmagine/math/linalg.hinstead:
"},{"location":"library/math/#changing-precisions","title":"Changing Precisions","text":"#include <rmagine/math/types.h>\nnamespace rm = rmagine;\n\nint main(int argc, char** argv)\n{\n // pose of the sensor in the map\n rm::Transform T = {\n rm::Quaternion::Identity(), // rotation\n {0.0, 1.0, 2.0} // translation\n };\n rm::Vector3 s = {1.2, 1.2, 1.2}; // scale each dimension with 1.2\n\n // pack to Matrix4x4\n rm::Matrix4x4 M = rm::compose(T, s);\n\n // Use M\n rm::Vector3 p_sensor = {2.0, 1.0, 0.0};\n rm::Vector3 p_map = M * p_sensor;\n\n // unpack inverse: operator~ works only because of 'using namespace rmagine;'\n // - Alternative: Matrix4x4::inv() \n rm::decompose(~M, T, s);\n\n return 0;\n}\nWhat you have used so far are all types that itself are aliases to specializations of templated math classes:
(Location:// defaults\n#define DEFAULT_FP_PRECISION 32\nusing DefaultFloatType = float;\n\n// default types\nusing Vector3 = Vector3_<DefaultFloatType>;\nusing Vector2 = Vector2_<DefaultFloatType>;\nusing Matrix2x2 = Matrix_<DefaultFloatType, 2, 2>;\nusing Matrix3x3 = Matrix_<DefaultFloatType, 3, 3>;\nusing Matrix4x4 = Matrix_<DefaultFloatType, 4, 4>;\nusing Quaternion = Quaternion_<DefaultFloatType>;\nusing EulerAngles = EulerAngles_<DefaultFloatType>;\nusing Transform = Transform_<DefaultFloatType>;\nusing AABB = AABB_<DefaultFloatType>;\nrmagine/math/types/definitions.h)However, Rmagine also supports other floating-point precisions such as double:
(Location:using Vector2f = Vector2_<float>;\nusing Vector2u = Vector2_<uint32_t>;\nusing Vector2i = Vector2_<int32_t>;\nusing Vector3f = Vector3_<float>;\nusing Matrix2x2f = Matrix_<float, 2, 2>;\nusing Matrix3x3f = Matrix_<float, 3, 3>;\nusing Matrix4x4f = Matrix_<float, 4, 4>;\nusing Quaternionf = Quaternion_<float>;\nusing EulerAnglesf = EulerAngles_<float>;\nusing Transformf = Transform_<float>;\nusing AABBf = AABB_<float>;\n\nusing Vector2d = Vector2_<double>;\nusing Vector3d = Vector3_<double>;\nusing Matrix2x2d = Matrix_<double, 2, 2>;\nusing Matrix3x3d = Matrix_<double, 3, 3>;\nusing Matrix4x4d = Matrix_<double, 4, 4>;\nusing Quaterniond = Quaternion_<double>;\nusing EulerAnglesd = EulerAngles_<double>;\nusing Transformd = Transform_<double>;\nusing AABBd = AABB_<double>;\nrmagine/math/types/definitions.h)In addition, this makes it possible to uses custom precisions, and still have the full range of functions available for both CPU and CUDA code.
WARNING: The following code is a draft and was never tested, hence it doesn't neccesarily needs to compile. It shows how one would declare and use a 16 bit float vector.
"},{"location":"library/memory/","title":"Memory","text":""},{"location":"library/memory/#memory","title":"Memory","text":"#include <cuda_fp16.hpp>\nusing MyCudaHalfVector = rm::Vector3_<__half>;\n\n// add_inplace kernel\n// compute 'A[i] += B[i]', massively parallel on the GPU\n__global__ void add_inplace_kernel(\n MyCudaHalfVector* vec_a,\n const MyCudaHalfVector* vec_b, \n size_t n)\n{\n const unsigned int tid = threadIdx.x; \n if(tid < n)\n {\n // this invokes a rmagine function using the \n // using fp16 precision\n vec_a[tid] += vec_b[tid];\n }\n}\n\nint main(int argc, char** argv)\n{\n // buffer of CUDA-half vectors located in RAM\n rm::Memory<MyCudaHalfVector, rm::RAM> buffer_cpu(100);\n // TODO fill buffer\n\n // upload all CUDA-half vectors from RAM to VRAM_CUDA, twice\n rm::Memory<MyCudaHalfVector, rm::VRAM_CUDA> buffer_a_gpu \n = buffer_cpu;\n rm::Memory<MyCudaHalfVector, rm::VRAM_CUDA> buffer_b_gpu \n = buffer_cpu;\n\n // call CUDA kernel 'add_inplace_kernel'\n add_inplace_kernel<<<buffer_gpu.size(), 1>>>(\n buffer_a_gpu.raw(),\n buffer_b_gpu.raw(),\n buffer_a_gpu.size());\n // A[i]+=B[i] done.\n // buffer_a_gpu contains changed elements now \n\n return 0;\n}\nRmagine internally uses so-called Memory objects to manage memory located on different hardware. The location where the actual memory should be allocated can be passed as a template argument using one of the following keywords:
RAM(RAM memory)RAM_CUDA(pinned CUDA host memory)VRAM_CUDA(CUDA device memory)
After allocating the memory, accessing elements is similar to using
std::vector's:- access element with
[] - resize the memory with
resize() - access raw data pointer with
raw()function
The following code samples are describing how to work with Memory objects and how to transfer Memory to other hardware.
Example CPU-only:
#include <rmagine/types/Memory.hpp>\n\nnamespace rm = rmagine;\n\nint main(int argc, char** argv)\n{\n // allocate memory with 1000 elements\n rm::Memory<float, rm::RAM> mem(1000);\n\n // shrink memory\n mem.resize(100);\n\n // set some values\n mem[0] = 10.0;\n mem[10] = 5.0;\n\n return 0;\n}\nExample GPU-only:
#include <rmagine/types/MemoryCuda.hpp>\n\nnamespace rm = rmagine;\n\n__global__\nvoid set_value(float* data, unsigned int id, float val)\n{\n data[id] = val;\n}\n\nint main(int argc, char** argv)\n{\n // allocate memory with 1000 elements on GPU\n rm::Memory<float, rm::VRAM_CUDA> mem(1000);\n\n // shrink memory on GPU\n mem.resize(100);\n\n // this would cause a segfault. since the underlying memory is not available\n // on the device this code is executed:\n // mem[0] = 10.0;\n //\n // set some values in Cuda kernels instead\n set_value<<<1,1>>>(mem.raw(), 0, 10.0);\n set_value<<<1,1>>>(mem.raw(), 10, 5.0);\n\n return 0;\n}\nExample CPU <-> CPU:
"},{"location":"library/memory/#writing-memory-dependent-functions","title":"Writing Memory dependent Functions","text":"#include <rmagine/types/Memory.hpp>\n#include <rmagine/types/MemoryCuda.hpp>\n\nnamespace rm = rmagine;\n\n__global__ my_kernel(float* data, unsigned int N)\n{\n // ...\n}\n\nint main(int argc, char** argv)\n{\n // allocate memory with 1000 float elements\n rm::Memory<float, rm::RAM> mem(1000);\n\n // set some values\n mem[0] = 10.0;\n mem[10] = 5.0;\n\n // copy the whole memory to GPU\n rm::Memory<float, rm::VRAM_CUDA> mem_ = mem;\n\n // run some kernels\n my_kernel<<<mem_.size(), 1>>>(mem_.raw(), mem_.size());\n\n // copy back\n mem = mem_;\n\n return 0;\n}\nWith the memory objects Rmagine offers at the same time the possibility to make function calls dependent on the location of the memory. The next example adds two vectors and creates a new one. The actual addition should be executed on the device where the memory is currently stored on.
rm::Memory<float, rm::RAM> vec1(1000);\nrm::Memory<float, rm::RAM> vec2(1000);\n\n// fill vec1, vec2 ...\n\n// copy to GPU\nrm::Memory<float, rm::VRAM_CUDA> vec1_ = vec1;\nrm::Memory<float, rm::VRAM_CUDA> vec2_ = vec2;\n\n// we want to achieve this:\nauto vec3 = add(vec1, vec2);\nauto vec3_ = add(vec1_, vec2_);\n\n// ...\nSo we try to create a function
"},{"location":"library/memory/#signatures-header","title":"Signatures (Header):","text":"add(a, b)whose code will be executed on the CPU onceaandbare in RAM. However, as soon asaandbare stored on the GPU the code should be executed on the GPU as well as the function returns a GPU memory object. This can be done as follows:
"},{"location":"library/memory/#code-cpu","title":"Code CPU","text":"rm::Memory<float, rm::RAM> add(\n const rm::Memory<float, rm::RAM>& a, \n const rm::Memory<float, rm::RAM>& b);\n\nrm::Memory<float, rm::VRAM_CUDA> add(\n const rm::Memory<float, rm::VRAM_CUDA>& a, \n const rm::Memory<float, rm::VRAM_CUDA>& b); \n
"},{"location":"library/memory/#code-gpu","title":"Code GPU","text":"rm::Memory<float, rm::RAM> add(\n const rm::Memory<float, rm::RAM>& a, \n const rm::Memory<float, rm::RAM>& b)\n{\n rm::Memory<float, rm::RAM> c(a.size());\n for(size_t i=0; i < a.size(); i++)\n {\n c[i] = a[i] + b[i];\n }\n return c;\n}\n__global__\nvoid add_kernel(const float* a, const float* b, float* c, unsigned int N)\n{\n const unsigned int id = blockIdx.x * blockDim.x + threadIdx.x;\n if(id < N)\n {\n c[id] = a[id] + b[id];\n }\n}\n\nrm::Memory<float, rm::VRAM_CUDA> add(\n const rm::Memory<float, rm::VRAM_CUDA>& a, \n const rm::Memory<float, rm::VRAM_CUDA>& b)\n{\n rm::Memory<float, rm::VRAM_CUDA> c(a.size());\n add_kernel<<<c.size(), 1>>>(a.raw(), b.raw(), c.raw(), c.size());\n return c;\n}\nHaving these functions defined allows us to very flexible chain operations:
"},{"location":"library/memory/#slicing-and-memoryviews","title":"Slicing and MemoryViews","text":"// Simple (as above):\nauto vec3 = add(vec1, vec2);\nauto vec3_ = add(vec1_, vec2_);\n\n// Advanced\n// - add two vectors on CPU and upload to GPU on return\nrm::Memory<float, rm::VRAM_CUDA> vec3_ = add(vec1, vec2);\n// - add two vectors on GPU and download to CPU on return\nrm::Memory<float, rm::RAM> vec3 = add(vec1_, vec2_);\nRmagine also provides mechanisms to slice these Memory objects and handling shallow copies through so-called Memory Views.
rm::Memory<float, RAM> mem(1000);\n// MemoryView to the elements [100: 200]\nrm::MemoryView<float, RAM> slice = mem(100, 200);\n// this sets slice[0] and mem[100] to 10\nslice[0] = 10.0;\nWith that it is possible to access existing memory very flexible:
"},{"location":"library/memory/#application-example-1","title":"Application Example 1","text":"rm::Memory<int, rm::RAM> mem(1000);\nrm::Memory<int, rm::VRAM_CUDA> mem_(1000);\n\n// copy [100:200] to [0:100]\nmem(0, 100) = mem(100, 200)\nmem_(0, 100) = mem_(100, 200)\n\n// or even transfer memory slice-wise\nmem_(0, 100) = mem(500, 600); // upload a slice\nmem(400, 500) = mem_(100, 200); // download a slice\nfor debuging purposes sometimes it is required to print a fetch a single element out of a GPU buffer. Here we just want to print the first element of a GPU memory object as follows:
- copy one element to CPU through
"},{"location":"library/memory/#application-example-2","title":"Application Example 2","text":"rm::Memory<int, rm::VRAM_CUDA> mem_(1000);\n\n// download [0:1] to CPU\nrm::Memory<int, rm::RAM> one_elem_mem = mem_(0,1);\nstd::cout << one_elem_mem[0] << std::endl;\nOftentimes the GPU has a very limited amount of Memory. In Code this can be overcome using Rmagine's slices as follows:
"},{"location":"library/memory/#cuda-isolated-library-creation","title":"Cuda Isolated Library Creation","text":"// max available CPU mem: 1000\nrm::Memory<int, RAM> mem(1000);\n// max available GPU mem: 10\nrm::Memory<int, VRAM_CUDA> mem_(10);\n\n// i = [0, 10, 20, 30, ..., 990]\nfor(size_t i=0; i<mem.size(); i += mem_.size())\n{\n mem_ = mem(i, i + mem_.size());\n // process algorithm on GPU\n}\nIn order to ship a library and its headers, each CUDA piece of code should be invisable after compilation. That means the shipped header should be free of code that can be only proccessed by the NVCC compiler. Exeptions for that are, if the library is clearly marked as to use with CUDA. The following example shows how to achieve that using
"},{"location":"library/memory/#signatures-header_1","title":"Signatures (Header):","text":"Memoryobjects and the functionaddfrom above. In this example theadd-function is further improved to handle slices.File:
add.hrm::Memory<float, rm::RAM> add(\n const rm::MemoryView<float, rm::RAM>& a, \n const rm::MemoryView<float, rm::RAM>& b);\n\nrm::Memory<float, rm::VRAM_CUDA> add(\n const rm::MemoryView<float, rm::VRAM_CUDA>& a, \n const rm::MemoryView<float, rm::VRAM_CUDA>& b);\nFile
add.cuh
"},{"location":"library/memory/#code","title":"Code","text":"rm::Memory<float, rm::VRAM_CUDA> add(\n const rm::MemoryView<float, rm::VRAM_CUDA>& a, \n const rm::MemoryView<float, rm::VRAM_CUDA>& b);\nFile
add.cpp#include \"add.h\"\n\nrm::Memory<float, rm::RAM> add(\n const rm::MemoryView<float, rm::RAM>& a, \n const rm::MemoryView<float, rm::RAM>& b)\n{\n rm::Memory<float, rm::RAM> c(a.size());\n for(size_t i=0; i < a.size(); i++)\n {\n c[i] = a[i] + b[i];\n }\n return c;\n}\nFile:
add.cu
"},{"location":"library/memory/#main-and-cmake","title":"Main and CMake","text":"#include \"add.cuh\"\n\n__global__\nvoid add_kernel(const float* a, const float* b, float* c, unsigned int N)\n{\n const unsigned int id = blockIdx.x * blockDim.x + threadIdx.x;\n if(id < N)\n {\n c[id] = a[id] + b[id];\n }\n}\n\nrm::Memory<float, rm::VRAM_CUDA> add(\n const rm::MemoryView<float, rm::VRAM_CUDA>& a, \n const rm::MemoryView<float, rm::VRAM_CUDA>& b)\n{\n rm::Memory<float, rm::VRAM_CUDA> c(a.size());\n add_kernel<<<c.size(), 1>>>(a.raw(), b.raw(), c.raw(), c.size());\n return c;\n}\nFile:
Main.cpp#include <rmagine/types/Memory.hpp>\n#include \"add.h\"\n#include <rmagine/types/MemoryCuda.hpp>\n#include \"add.cuh\"\n\nint main(int argc, char** argv)\n{\n // CPU\n rm::Memory<float, rm::RAM> vec1(100);\n rm::Memory<float, rm::RAM> vec2(100);\n auto vec3 = add(vec1, vec2);\n auto vec3_slice = add(vec1(0, 10), vec2(10, 20));\n\n // GPU\n rm::Memory<float, rm::RAM> vec1_(100);\n rm::Memory<float, rm::RAM> vec2_(100);\n auto vec3_ = add(vec1_, vec2_);\n auto vec3_slice_ = add(vec1_(0, 10), vec2_(10, 20));\n\n return 0;\n}\nFile:
CMakeLists.txt# ...\n\nadd_library(my_add add.cpp)\ncuda_add_library(my_add_cuda add.cu)\n\nadd_executable(Main Main.cpp)\ntarget_link_libraries(Main\n my_add\n my_add_cuda\n)\n\n# ...\nThe Main.cpp and potential other code thus can be compiled with another compiler than the NVCC host compiler even though the CUDA code is executed internally.
"}]} \ No newline at end of file +{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Home","text":""},{"location":"#rmagine","title":"Rmagine","text":"Rmagine allows a robot to simulate sensor data for arbitrary range sensors directly on board via raytracing. Since robots typically only have limited computational resources, Rmagine aims at being flexible and lightweight, while scaling well even to large environment maps. It runs on several platforms like Laptops or embedded computing boards like Nvidia Jetson by putting an unified API over specific proprietary libraries provided by the hardware manufacturers. This work is designed to support the future development of robotic applications depending on simulation of range data that could previously not be computed in reasonable time on mobile systems.
"},{"location":"#citation","title":"Citation","text":"We presented this work at ICRA'23 in London. The paper gives valuable insights of the design concepts of this library. When using the Rmagine library or any related ideas in your scientific work, please reference the following paper:
"},{"location":"#table-of-contents","title":"Table of Contents","text":"@inproceedings{mock2023rmagine,\n title={{Rmagine: 3D Range Sensor Simulation in Polygonal Maps via Ray Tracing for Embedded Hardware on Mobile Robots}}, \n author={Mock, Alexander and Wiemann, Thomas and Hertzberg, Joachim},\n booktitle={IEEE International Conference on Robotics and Automation (ICRA)}, \n year={2023},\n doi={10.1109/ICRA48891.2023.10161388}\n}\nGetting Started
- Overview
- Installation
- Integration
- Maps
- Sensors
- Simulation
- Problem Modelling
- Noise
Library
- Concepts
- Math
- Memory
- Maps
Extra
- Tools
- Data
- News
- Embree 3
- Contributions
You are welcome to contribute to the docs of Rmagine! Thorough and clear documentation is essential. You can help us by correcting mistakes, improving content, or adding examples that facilitate user navigation and usage of the project. Please submit any documentation-related issues to the repository https://github.com/uos/rmagine_docs. If you're making fixes or adding examples, don\u2019t hesitate to submit a pull request afterward!
"},{"location":"#pr-workflow","title":"PR workflow","text":"How to contribute to this documentation via pull requests:
- Fork the repository: https://github.com/uos/rmagine_docs.
- Make changes on your forked repository.
- Check locally on your machine if mkdocs is able to compile your changes (instructions).
- Go to Github and click \"Pull Request\", select this repository's \"main\" branch as target.
- If you added new content, please provide a brief explanation of why you believe it is beneficial for the documentation.
- Send PR
Blender is a powerful tool to create and modify existing maps for rmagine.
Some Links to look up: - Webpage: https://www.blender.org/ - Docs: https://docs.blender.org/manual/en/latest/
"},{"location":"extra/blender/#useful-commands","title":"Useful commands","text":""},{"location":"extra/blender/#object-mode","title":"Object Mode","text":"Command Effect C Ctrl + G Move Object after: type X and \"0.5\" to move the object 0.5 along the X axis Ctrl + R Rotate Object after: type Z and \"45\" to rotate the object 45 degree around the X axis Ctrl + S Scale Object after: type X and \"2.0\" to scale the object 2.0 along the X axis"},{"location":"extra/blender/#collada-dae-exports-odyssey-of-wrong-imports","title":"Collada (DAE) exports (odyssey of wrong imports)","text":""},{"location":"extra/blender/#update","title":"UPDATE","text":"Blender plugin does everything right, Assimp writes the wrong transformation: See last sentence.
"},{"location":"extra/blender/#beginning","title":"Beginning","text":"Some strange errors happened while exporting blender's scene to collada format. I did the following in Blender (3.2.1):
- Moved the Cube 5 units along the x axis
- Moved the Cube 2 units along the y axis
- Moved the Cube 3 units along the z axis
- Rotated the Cube 45 degrees around the z axis
- Scaled the Cube with 0.2 (the resulting cube has the dimensions 0.4 x 0.4 x 0.4)
- Export DAE with default settings to \"~/utitled.dae\"
After some library fixes to read the scene graph completely, any collada file generated by Blender no longer loads correctly. With PLY exports, everything works as before. So I inspected the generated outputs of the Blender Collada exports (read by Assimp):
$:~ ./bin/rmagine_map_info ~/untitled.dae\n#...\nScene Graph: \n- name: Scene\n- transform: \n M4x4[\n 1 0 0 0\n 0 0 1 0\n 0 -1 0 0\n 0 0 0 1\n ]\n- meshes: 0\n- children: 3\n#...\n Node 2\n - name: Cube\n - transform: \n M4x4[\n 0.141421 -0.141421 0 5\n 0.141421 0.141421 0 2\n 0 0 0.2 3\n 0 0 0 1\n ]\n - meshes: 1\n - mesh ref 0 -> 0\n - children: 0\nAnd exactly there is the problem. I guess the \"Cube\" transformation is correct since it come from Blender directly. The problem comes from the global matrix at node named \"Scene\". This matrix switches the y and z axes and negates the z axis (old y axis) afterwards. The complete transform of the \"Cube\" following the transformations to the root is
v[5,3,-2], E[-1.5708, 0.785398, -1.26441e-07] with scale v[0.2,0.2,0.2]as expected, the total transform holds the axis-switched version of our previously done operations. But that is not what we want. Moving something in Blender along the x axis should result in an export with something in it that was moved along the x axis and no other axis. So how to fix it? Pushing
Export -> Collodaopens a menu. Push the settings button in the top left corner. It opens a side panel, holding some values to change for the export. My first intuitive choice to set the forward axis to X and the up axis to Z were wrong, incomprehensibly. However, by trial and error I came up with setting these values as follows:Forward Axis: Z\nUp Axis: -Y\nWith these settings the scene is exported exactly as I modelled it. Nevertheless, Blender exports the weird axis flip matrix at scene root. Every other transformation is adjusted such that every total transformation are valid.
"},{"location":"extra/blender/#not-yet-fixed","title":"Not yet fixed","text":"v[5,2,3], E[0, 0, 0.785398] with scale v[0.2,0.2,0.2]Importing this exported file into Blender results in a different scene. It seems the Blender Collada Importer ignores the top level axis-switch matrix somehow? FBX export and import is correct using the axis switch by choice.
"},{"location":"extra/blender/#fixed","title":"FIXED","text":"The wrong up most transformation is written by Assimp!
- Assimp::Importer io; io.SetPropertyBool(AI_CONFIG_IMPORT_COLLADA_IGNORE_UP_DIRECTION, true)
Capsuled in
AssimpIO-Object (\"rmagine/map/AssimpIO.hpp\").The following code snipped already corrects the wrong transformation imports.
rm::AssimpIO io; \nconst aiScene* scene = io.ReadFile(\"file.dae\", 0); \nWith that, it is possible to default export Collada files with Blender, import them with rmagine and import them in Blender again, without any errors.
"},{"location":"extra/blender/#filmbox-fbx-exports","title":"Filmbox (FBX) exports","text":"Same problems as in DAE section. Set axis to:
Scale: 0.01\nForward: Y Forward\nUp: Z Up\nThen everything is exported as modelled. Imports into Blender again are working as well. Luckilly, the global scene transform is set to identity here. Maybe that is why the Import is working again: ignoring the scene transform is not important if its an identity transform.
TODO: check if Gazebo importer ignores the scene transform
"},{"location":"extra/blender/#building-a-3d-map-from-2d-building-plan","title":"Building a 3D map from 2D building plan","text":"Using Blender Version 3.3.1
"},{"location":"extra/blender/#setting-up-a-2d-map-as-reference-image","title":"Setting up a 2D Map as Reference image","text":"Make sure to be in Object Mode
- Click top right on the z-axis to make the view orthograhic top down
- In Scene press Shift+A to open the Add Panel. Then select your Image and Load it into scene. The Image should now be located on your xy-plane
- Scale the Image to a size where you think you can work good with. The exact scale can be determined later.
- Enable Opacity of Image: Select Image and go to \"Object Data Properties\" (bottom right). Checkmark the Opacity Button
To place single vertices in the scene we first have to enable an Add-On under
Edit -> PreferencescalledAdd Mesh: Extra Objects. Then with Shift+A under the entryMeshthe optionSingle Vertshould be available. SelectingSingle Vertwill place a single vertex in the origin and change toEdit Mode. A object should appear in theScene Collectionpanel top right calledVert.- Move the image so that the first vertex to set is in the origin
- Insert a single vertex. A new object should be created. The first vertex is placed in the Origin.
- Press
Eto extrude a vertex than click an endpoint to place the second vertex with an edge connecting both. - Contour: By again pressing
Eyou can create a path of vertices. I recommend to make a complete path along the contour of the building plan (without doors. Only walls). - Walls: Once you have created a contour of edges go to
Edge Select-Mode select everything. PressEand thenZto extrude the edges along the z-axis. Pull the walls to a arbitrary height (The exact scale is determined later). - Ground Faces: Go to
Vertex Select-Mode and select three ground Vertices you want to connect (Hold Shift). PressFto connect them to a Face.- If Faces are getting to long: Cut Wall-Face in two by selecting in in
Face SelectMode and then pressing Ctrl+R - Enable Statistics in Drop-Down Menu \"Overlays\" top right. If you notice during editing that there is more than one Vertex at one point: Select both of them in
Vertex Select-Mode. The Right-Click andMerge Vertices - Collapse.
- If Faces are getting to long: Cut Wall-Face in two by selecting in in
We are excited to invite you to contribute to our open source project Rmagine! Whether you're a seasoned developer, a documentation enthusiast, or someone with fresh ideas, your contributions can make a significant impact.
"},{"location":"extra/contributions/#how-you-can-help","title":"How You Can Help:","text":"-
Code Contributions: Help us improve the codebase by submitting pull requests. Whether it\u2019s fixing bugs, adding features, or optimizing existing code, every contribution counts!
-
Documentation: Clear and comprehensive documentation is crucial. Assist us by correcting errors, enhancing content, or providing examples that make it easier for users to navigate and utilize the project.
-
Feedback and Suggestions: Your insights matter! Share your thoughts on how we can improve the project. Opening issues with suggestions or feedback is a great way to get involved.
For development and testing we include some meshes inside our repository in the
"},{"location":"extra/data/#sphereply","title":"sphere.ply","text":"dat-folder:
"},{"location":"extra/data/#triangleply","title":"triangle.ply","text":"user@pc:~/rmagine/build$ ./bin/rmagine_map_info ../dat/sphere.ply\nRmagine Map Info\nInputs: \n- filename: ../dat/sphere.ply\nMeshes: 1\n Mesh 0\n - name: \n - vertices, faces: 642, 1280\n - primitives: TRIANGLE\n - normals: no\n - vertex color channels: 0\n - uv channels: 0\n - bones: 0\n - material index: 0\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\nTextures: 0\nScene Graph: \n- name: \n- transform: \n M4x4[\n 1 0 0 0\n 0 1 0 0\n 0 0 1 0\n 0 0 0 1\n ]\n- meshes: 1\n - mesh ref 0 -> 0\n- children: 0\n
"},{"location":"extra/data/#box_rot_trans_scaleddae","title":"box_rot_trans_scaled.dae","text":"user@pc:~/rmagine/build$ ./bin/rmagine_map_info ../dat/triangle.ply\nRmagine Map Info\nInputs: \n- filename: ../dat/triangle.ply\nMeshes: 1\n Mesh 0\n - name: \n - vertices, faces: 3, 1\n - primitives: TRIANGLE\n - normals: no\n - vertex color channels: 0\n - uv channels: 0\n - bones: 0\n - material index: 0\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\nTextures: 0\nScene Graph: \n- name: \n- transform: \n M4x4[\n 1 0 0 0\n 0 1 0 0\n 0 0 1 0\n 0 0 0 1\n ]\n- meshes: 1\n - mesh ref 0 -> 0\n- children: 0\n
"},{"location":"extra/data/#two_cubesdae","title":"two_cubes.dae","text":"user@pc:~/rmagine/build$ ./bin/rmagine_map_info ../dat/box_rot_trans_scaled.dae\nRmagine Map Info\nInputs: \n- filename: ../dat/box_rot_trans_scaled.dae\nMeshes: 1\n Mesh 0\n - name: Cube-mesh\n - vertices, faces: 36, 12\n - primitives: TRIANGLE\n - normals: yes\n - vertex color channels: 0\n - uv channels: 1\n - bones: 0\n - material index: 0\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\nTextures: 0\nScene Graph: \n- name: Scene\n- transform: \n M4x4[\n 1 0 0 0\n 0 1 0 0\n 0 0 1 0\n 0 0 0 1\n ]\n- meshes: 0\n- children: 3\n Node 0\n - name: Camera\n - transform: \n M4x4[\n 0.727676 0.305421 -0.61417 -6.92579\n -0.685921 0.324014 -0.651558 -7.35889\n 0 0.895396 0.445271 4.95831\n 0 0 0 1\n ]\n - meshes: 0\n - children: 0\n Node 1\n - name: Light\n - transform: \n M4x4[\n 0.955171 -0.199883 0.218391 1.00545\n 0.290865 0.771101 -0.566393 -4.07624\n -0.0551891 0.604525 0.794672 5.90386\n 0 0 0 1\n ]\n - meshes: 0\n - children: 0\n Node 2\n - name: Cube\n - transform: \n M4x4[\n 0.141421 0.141421 0 0\n -0.141421 0.141421 0 -5\n 0 0 0.2 0\n 0 0 0 1\n ]\n - meshes: 1\n - mesh ref 0 -> 0\n - children: 0\n
"},{"location":"extra/data/#many_objectsdae","title":"many_objects.dae","text":"user@pc:~/rmagine/build$ ./bin/rmagine_map_info ../dat/two_cubes.dae\nRmagine Map Info\nInputs: \n- filename: ../dat/two_cubes.dae\nMeshes: 2\n Mesh 0\n - name: Cube_001-mesh\n - vertices, faces: 36, 12\n - primitives: TRIANGLE\n - normals: yes\n - vertex color channels: 0\n - uv channels: 1\n - bones: 0\n - material index: 1\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\n Mesh 1\n - name: Cube-mesh\n - vertices, faces: 36, 12\n - primitives: TRIANGLE\n - normals: yes\n - vertex color channels: 0\n - uv channels: 1\n - bones: 0\n - material index: 0\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\nTextures: 0\nScene Graph: \n- name: Scene\n- transform: \n M4x4[\n 1 0 0 0\n 0 1 0 0\n 0 0 1 0\n 0 0 0 1\n ]\n- meshes: 0\n- children: 4\n Node 0\n - name: Cube_001\n - transform: \n M4x4[\n 1 0 0 5.01877\n 0 1 0 3.78582\n 0 0 1 1.01026\n 0 0 0 1\n ]\n - meshes: 1\n - mesh ref 0 -> 0\n - children: 0\n Node 1\n - name: Camera\n - transform: \n M4x4[\n 0.685921 -0.324014 0.651558 7.35889\n 0.727676 0.305421 -0.61417 -6.92579\n 0 0.895396 0.445271 4.95831\n 0 0 0 1\n ]\n - meshes: 0\n - children: 0\n Node 2\n - name: Light\n - transform: \n M4x4[\n -0.290865 -0.771101 0.566393 4.07624\n 0.955171 -0.199883 0.218391 1.00545\n -0.0551891 0.604525 0.794672 5.90386\n 0 0 0 1\n ]\n - meshes: 0\n - children: 0\n Node 3\n - name: Cube\n - transform: \n M4x4[\n 1.64731 1.34142 -0.299005 5.7392\n -1.28721 1.34237 -1.06938 -4.66034\n -0.481559 1.00054 1.83561 1.69496\n 0 0 0 1\n ]\n - meshes: 1\n - mesh ref 0 -> 1\n - children: 0\n
"},{"location":"extra/embree3/","title":"Embree 3","text":"user@pc:~/rmagine/build$ ./bin/rmagine_map_info ../dat/many_objects.dae\nRmagine Map Info\nInputs: \n- filename: ../dat/many_objects.dae\nMeshes: 7\n Mesh 0\n - name: Torus-mesh\n - vertices, faces: 3456, 1152\n - primitives: TRIANGLE\n - normals: yes\n - vertex color channels: 0\n - uv channels: 1\n - bones: 0\n - material index: 0\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\n Mesh 1\n - name: Suzanne-mesh\n - vertices, faces: 2901, 967\n - primitives: TRIANGLE\n - normals: yes\n - vertex color channels: 0\n - uv channels: 1\n - bones: 0\n - material index: 0\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\n Mesh 2\n - name: Cone-mesh\n - vertices, faces: 186, 62\n - primitives: TRIANGLE\n - normals: yes\n - vertex color channels: 0\n - uv channels: 1\n - bones: 0\n - material index: 0\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\n Mesh 3\n - name: Cylinder-mesh\n - vertices, faces: 372, 124\n - primitives: TRIANGLE\n - normals: yes\n - vertex color channels: 0\n - uv channels: 1\n - bones: 0\n - material index: 0\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\n Mesh 4\n - name: Plane-mesh\n - vertices, faces: 6, 2\n - primitives: TRIANGLE\n - normals: yes\n - vertex color channels: 0\n - uv channels: 1\n - bones: 0\n - material index: 0\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\n Mesh 5\n - name: Cube_001-mesh\n - vertices, faces: 36, 12\n - primitives: TRIANGLE\n - normals: yes\n - vertex color channels: 0\n - uv channels: 1\n - bones: 0\n - material index: 1\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\n Mesh 6\n - name: Cube-mesh\n - vertices, faces: 36, 12\n - primitives: TRIANGLE\n - normals: yes\n - vertex color channels: 0\n - uv channels: 1\n - bones: 0\n - material index: 0\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\nTextures: 0\nScene Graph: \n- name: Scene\n- transform: \n M4x4[\n 1 0 0 0\n 0 1 0 0\n 0 0 1 0\n 0 0 0 1\n ]\n- meshes: 0\n- children: 10\n Node 0\n - name: Torus\n - transform: \n M4x4[\n -0.0934659 -0.290695 2.78847 -9.244\n 0.942502 2.62438 0.30518 5.28275\n -2.64041 0.94707 0.0102276 3.4012\n 0 0 0 1\n ]\n - meshes: 1\n - mesh ref 0 -> 0\n - children: 0\n Node 1\n - name: Suzanne\n - transform: \n M4x4[\n -0.247416 -0.65867 0.71059 -6.41007\n 0.442643 -0.729225 -0.521823 -2.72992\n 0.861889 0.18543 0.471977 1.71339\n 0 0 0 1\n ]\n - meshes: 1\n - mesh ref 0 -> 1\n - children: 0\n Node 2\n - name: Cone\n - transform: \n M4x4[\n 1 0 0 1.73173\n 0 1 0 -7.66226\n 0 0 1 0.999599\n 0 0 0 1\n ]\n - meshes: 1\n - mesh ref 0 -> 2\n - children: 0\n Node 3\n - name: Cylinder\n - transform: \n M4x4[\n 1 0 0 1.34234\n 0 1 0 8.77874\n 0 0 1 0.959374\n 0 0 0 1\n ]\n - meshes: 1\n - mesh ref 0 -> 3\n - children: 0\n Node 4\n - name: Plane\n - transform: \n M4x4[\n 11 0 0 0\n 0 11 0 0\n 0 0 11 0\n 0 0 0 1\n ]\n - meshes: 1\n - mesh ref 0 -> 4\n - children: 0\n Node 5\n - name: Light_001\n - transform: \n M4x4[\n -0.290865 -0.771101 0.566393 4.07624\n 0.955171 -0.199883 0.218391 1.00545\n -0.0551891 0.604525 0.794672 5.90386\n 0 0 0 1\n ]\n - meshes: 0\n - children: 0\n Node 6\n - name: Cube_001\n - transform: \n M4x4[\n 0.84132 0.52049 0.145848 5.72826\n -0.452094 0.52966 0.717685 3.24672\n 0.296298 -0.669739 0.680923 3.1261\n 0 0 0 1\n ]\n - meshes: 1\n - mesh ref 0 -> 5\n - children: 0\n Node 7\n - name: Camera\n - transform: \n M4x4[\n 0.685921 -0.324014 0.651558 7.35889\n 0.727676 0.305421 -0.61417 -6.92579\n 0 0.895396 0.445271 4.95831\n 0 0 0 1\n ]\n - meshes: 0\n - children: 0\n Node 8\n - name: Light\n - transform: \n M4x4[\n -0.290865 -0.771101 0.566393 4.07624\n 0.955171 -0.199883 0.218391 1.00545\n -0.0551891 0.604525 0.794672 5.90386\n 0 0 0 1\n ]\n - meshes: 0\n - children: 0\n Node 9\n - name: Cube\n - transform: \n M4x4[\n 1 0 0 4.91178\n 0 1 0 -2.96374\n 0 0 1 1.06244\n 0 0 0 1\n ]\n - meshes: 1\n - mesh ref 0 -> 6\n - children: 0\nFollow the following steps if you want Rmagine to work with Embree 3:
user@pc:~$ git clone https://github.com/embree/embree.git --branch v3.13.5\nuser@pc:~$ mkdir embree/build && cd embree/build\nuser@pc:~/embree/build$ cmake -DCMAKE_BUILD_TYPE=Release ..\nuser@pc:~/embree/build$ make -j`nproc`\nuser@pc:~/embree/build$ sudo make install\nDuring the
cmake-process errors could occur. The following flags could fix it. You can edit them by callingccmake .in build directory.# The Intel Implicit SPMD Program Compiler is not necessarily needed \nEMBREE_ISPC_SUPPORT=OFF\n# Tasking system: TBB/Internal. You can choose INTERNAL if TBB is not installed\nEMBREE_TASKING_SYSTEM=INTERNAL \n# Tutials are not needed for library compilation\nEMBREE_TUTORIALS=OFF\nOr you can pass the arguments directly to cmake for the minimal build
"},{"location":"extra/news/","title":"News","text":""},{"location":"extra/news/#05122023","title":"05.12.2023","text":"cmake -DCMAKE_BUILD_TYPE=Release -DEMBREE_ISPC_SUPPORT=OFF -DEMBREE_TASKING_SYSTEM=INTERNAL -DEMBREE_TUTORIALS=OFF ..\nNew version 2.2.2 is available now and brings convenience updates for ROS-users. Just place Rmagine into your ROS-workspace and it will compile. Via
find_package(rmagine COMPONENTS [...])you can still find Rmagine's components as if you would install it globally on your system. We tested it with - ROS1 - noetic - ROS2 - humbleNormally you would set
OptiX_INCLUDE_DIRvia cmake flags. Now we provide an additional option: Set the environment variableOPTIX_HEADER_DIRfor example in your.bashrc-file:export OPTIX_INCLUDE_DIR=~/software/optix/NVIDIA-OptiX-SDK-7.4.0-linux64-x86_64/include\nEspecially if you place Rmagine into your ROS-workspace this option becomes very handy.
"},{"location":"extra/news/#27092023","title":"27.09.2023","text":"From version >= 2.2.0 we enabled component-wise compilation and packaging for easier installation of Rmagine. In \"Releases\" section you can find the first pre-compiled binaries. Install the core library via
$ sudo dpkg -i rmagine-core_2.2.1_amd64.deb\nThen additionally for the Embree backend:
$ sudo dpkg -i rmagine-embree_2.2.1_amd64.deb\nAnd if you have a NVIDIA GPU:
$ sudo dpkg -i rmagine-cuda_2.2.1_amd64.deb\n$ sudo dpkg -i rmagine-optix_2.2.1_amd64.deb\nUsing the pre-compiled binaries, you are not required to download the OptiX-headers anymore. However, CUDA and Embree are still required to be installed on your system.
"},{"location":"extra/styleguide/","title":"Styleguide","text":""},{"location":"extra/styleguide/#naming-conventions","title":"Naming Conventions","text":""},{"location":"extra/styleguide/#files","title":"Files","text":"- Files containing a single C++-Class are written camelcase with
.hppextension. Examples:MyClass.hpp(Header),MyClass.cpp(Code), andMyClass.tcc(Template Code). - Files holding a set of Structs, Classes, or Functions are written lowercase with
.hextension. Examples:math/types.h,conversions.h, ormath.h(Header) andconversions.cppfor code. - Files holding Functions with CUDA code are namend with a
.cuhextension:math.cuh
- Functions with underscores:
mult,my_function - Classes and Structs in camelcase beginning with a capital letter:
Sphere,SphereSimulatorEmbree - Class Functions are beginning lowercase and then camelcase:
mult,simulateRanges
TODO: revise files to meet this styleguide
"},{"location":"extra/styleguide/#special-operators-for-math-types","title":"Special Operators for Math Types","text":"~: Invert the element after
There are some helpful command line tools that are compiled alongside the main library. After installation the tools are globally available to be called.
"},{"location":"extra/tools/#rmagine_version","title":"rmagine_version","text":"Prints the rmagine version. Should match the CMakeLists version.
"},{"location":"extra/tools/#rmagine_benchmark","title":"rmagine_benchmark","text":"user@pc:~/rmagine/build$ ./bin/rmagine_version\n2.2.1\nFor every implemented computing device we compile a benchmark executable that simulates a Velodyne LiDAR sensor in a given mesh and prints out some useful run time statisitics. Thus, we can compare the run times of different implementations on several computers. The CPU / Embree version can be tested like this:
user@pc:~/rmagine/build$ ./bin/rmagine_benchmark_cpu ../dat/sphere.ply\nRmagine Benchmark CPU (Embree)\nInputs: \n- mesh: ../dat/sphere.ply\nUnit: 1 Velodyne scan (velo) = 14400 Rays\n- range of last ray: 0.998762\n-- Starting Benchmark --\n[ 129% - 2215.605926 velos/s, mean: 2387.607002 velos/s] \nResult: 2387.607002 velos/s\nAnalogously, the GPU benchmark can be started as follows:
"},{"location":"extra/tools/#rmagine_map_info","title":"rmagine_map_info","text":"user@pc:~/rmagine/build$ ./bin/rmagine_benchmark_gpu ../dat/sphere.ply\n[RMagine - CudaContext] CUDA Driver Version / Runtime Version: 12.2.0 / 12.2.0\n[RMagine - CudaContext] Construct context on device 0 - NVIDIA GeForce RTX 2060 \n[RMagine - OptixContext] Init Optix (7.3.0). Required GPU driver >= 465.84\nRmagine Benchmark GPU (OptiX)\nInputs: \n- mesh: ../dat/sphere.ply\nUnit: 1 Velodyne scan (velo) = 14400 Rays\nLast Ray:\n- range: 0.998762\n-- Starting Benchmark --\n[ 100% - 231941.938409 velos/s, mean: 231987.481164 velos/s] \nResult: 231987.481164 velos/s\nPrints useful information about the contents of a mesh file. Internally it is just printing the meta information of the assimp buffers.
"},{"location":"extra/tools/#rmagine_synthetic","title":"rmagine_synthetic","text":"user@pc:~/rmagine/build$ ./bin/rmagine_map_info ../dat/two_cubes.dae\nRmagine Map Info\nInputs: \n- filename: ../dat/two_cubes.dae\nMeshes: 2\n Mesh 0\n - name: Cube_001-mesh\n - vertices, faces: 36, 12\n - primitives: TRIANGLE\n - normals: yes\n - vertex color channels: 0\n - uv channels: 1\n - bones: 0\n - material index: 1\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\n Mesh 1\n - name: Cube-mesh\n - vertices, faces: 36, 12\n - primitives: TRIANGLE\n - normals: yes\n - vertex color channels: 0\n - uv channels: 1\n - bones: 0\n - material index: 0\n - tangents and bitangents: no\n - aabb: AABB [v[0,0,0] - v[0,0,0]]\nTextures: 0\nScene Graph: \n- name: Scene\n- transform: \n M4x4[\n 1 0 0 0\n 0 1 0 0\n 0 0 1 0\n 0 0 0 1\n ]\n- meshes: 0\n- children: 4\n Node 0\n - name: Cube_001\n - transform: \n M4x4[\n 1 0 0 5.01877\n 0 1 0 3.78582\n 0 0 1 1.01026\n 0 0 0 1\n ]\n - meshes: 1\n - mesh ref 0 -> 0\n - children: 0\n Node 1\n - name: Camera\n - transform: \n M4x4[\n 0.685921 -0.324014 0.651558 7.35889\n 0.727676 0.305421 -0.61417 -6.92579\n 0 0.895396 0.445271 4.95831\n 0 0 0 1\n ]\n - meshes: 0\n - children: 0\n Node 2\n - name: Light\n - transform: \n M4x4[\n -0.290865 -0.771101 0.566393 4.07624\n 0.955171 -0.199883 0.218391 1.00545\n -0.0551891 0.604525 0.794672 5.90386\n 0 0 0 1\n ]\n - meshes: 0\n - children: 0\n Node 3\n - name: Cube\n - transform: \n M4x4[\n 1.64731 1.34142 -0.299005 5.7392\n -1.28721 1.34237 -1.06938 -4.66034\n -0.481559 1.00054 1.83561 1.69496\n 0 0 0 1\n ]\n - meshes: 1\n - mesh ref 0 -> 1\n - children: 0\nGenerate different meshes for quick testing. Show the possible options by entering
user@pc:~/rmagine/build$ ./bin/rmagine_synthetic\nRmagine Synthetic\nUsage: ./bin/rmagine_synthetic mesh_type mesh_file\n- mesh_type: plane | cube | sphere | cylinder \nrmagine_synthetic plane plane.plyrmagine_synthetic cube cube.plyrmagine_synthetic sphere sphere.plyrmagine_synthetic cylinder cylinder.ply"},{"location":"getting_started/installation/","title":"Installation (From Source)","text":"The following instructions are tested on an Ubuntu 20.04 operating system.
"},{"location":"getting_started/installation/#dependencies","title":"Dependencies","text":""},{"location":"getting_started/installation/#assimp-open-assets-importer-library","title":"Assimp (Open Assets Importer Library)","text":"For loading commonly used mesh/scene formats.
"},{"location":"getting_started/installation/#backbones","title":"Backbones","text":"user@pc:~$ sudo apt install libassimp-dev\nRmagine provides an interface to integrate ray tracing libraries, we call backbones. All of these backbones are optional. So far we integrated Intel Embree and NVIDIA OptiX.
"},{"location":"getting_started/installation/#embree-backbone-optional","title":"Embree Backbone (optional)","text":"We support Embree in its latest version (test v4.0.1, v4.2.0):
user@pc:~$ git clone https://github.com/embree/embree.git\nuser@pc:~$ mkdir embree/build && cd embree/build\nuser@pc:~/embree/build$ cmake -DCMAKE_BUILD_TYPE=Release ..\nuser@pc:~/embree/build$ make -j`nproc`\nuser@pc:~/embree/build$ sudo make install\nFor older Embree versions we refer to this.
"},{"location":"getting_started/installation/#optix-backbone-optional","title":"OptiX Backbone (optional)","text":"Rmagine supports NVIDIA OptiX versions of 7.2 or newer. The OptiX-Library is installed via the GPU driver. The OptiX-Headers can be downloaded here. The Headers require a specific GPU driver and CUDA version to be installed on your system:
OptiX Version Minimum Driver Version 7.2 456.71 7.3 465.84 7.4 495.89 7.5 495.89 (untested) 7.6 520.00 (untested) 7.7 530.41"},{"location":"getting_started/installation/#compilation","title":"Compilation","text":"Download the Rmagine repository.
user@pc:~/rmagine$ mkdir build\nuser@pc:~/rmagine$ cd build\nuser@pc:~/rmagine/build$ cmake ..\nuser@pc:~/rmagine/build$ make\nThe path to OptiX-Headers should be specified with the CMake-Variable
"},{"location":"getting_started/installation/#bash-variable-alternative","title":"Bash Variable (Alternative)","text":"OptiX_INCLUDE_DIR. This can be done using ccmake, for example.Alternatively, cmake checks for the environment variable
OPTIX_INCLUDE_DIRto exist. After downloading the OptiX-SDK you can add the following command to your.bashrc:export OPTIX_INCLUDE_DIR=~/.../NVIDIA-OptiX-SDK-7.4.0-linux64-x86_64/include\nAfter adding this path, the project should compile without changing the cmake flags.
"},{"location":"getting_started/installation/#optional-check-compilation","title":"Optional: Check Compilation","text":"You can check if everything went wrong by running the benchmark that was build besides the library:
user@pc:~/rmagine/build$ ./bin/rmagine_benchmark_cpu ../dat/sphere.ply\n...\n[ 100% - velos/s: 6261.9, mean: 6244.84] \nResult: 6244.84 velos/s\nor if the OptiX support was successfully build:
"},{"location":"getting_started/installation/#installation","title":"Installation","text":"user@pc:~/rmagine/build$ ./bin/rmagine_benchmark_gpu ../dat/sphere.ply\n...\n[ 100% - velos/s: 383094, mean: 383457, rays/s: 5.52178e+09] \nResult: 383457 velos/s\nAfter compilation do
"},{"location":"getting_started/installation/#optional-check-installation","title":"Optional: Check Installation","text":"user@pc:~/rmagine/build$ sudo make install\nYou can check if everything went wrong by running the benchmark that was build besides the library:
user@pc:~$ rmagine_benchmark_cpu rmagine/dat/sphere.ply\n...\n[ 100% - velos/s: 6261.9, mean: 6244.84] \nResult: 6244.84 velos/s\nor if the OptiX support was successfully build:
"},{"location":"getting_started/installation/#uninstall-rmagine","title":"Uninstall Rmagine","text":"user@pc:~$ rmagine_benchmark_gpu rmagine/dat/sphere.ply\n...\n[ 100% - velos/s: 383094, mean: 383457, rays/s: 5.52178e+09] \nResult: 383457 velos/s\n
"},{"location":"getting_started/installation/#installation-debian-package-experimental","title":"Installation (Debian Package) - Experimental","text":"user@pc:~/rmagine/build$ sudo make uninstall\nWe are working on creating debian packages for easier installations.
"},{"location":"getting_started/installation/#dependencies_1","title":"Dependencies","text":"
"},{"location":"getting_started/installation/#install","title":"Install","text":"$ sudo apt install libassimp-dev libeigen3-dev\nDownload latest Rmagine debian packages from Github releases page (v2.2.2). Install the core by calling
"},{"location":"getting_started/installation/#embree-backbone","title":"Embree Backbone","text":"sudo apt install ./rmagine-core_2.2.2_amd64.deb\nWe support Embree in its latest version (tested: v4.0.1 - v4.3.0). Make sure you have Embree installed on your system.
"},{"location":"getting_started/installation/#optix-backbone","title":"OptiX Backbone","text":"sudo apt install ./rmagine-embree_2.2.2_amd64.deb\nMake sure you have a current NVIDIA driver installed, then install rmagine-cuda and rmagine-optix by:
"},{"location":"getting_started/installation/#uninstall","title":"Uninstall","text":"sudo apt install ./rmagine-cuda_2.2.2_amd64.deb\nsudo apt install ./rmagine-optix_2.2.2_amd64.deb\nTo uninstall everything related to rmagine, call:
"},{"location":"getting_started/integration/","title":"Integration","text":"sudo apt-get remove rmagine-core\nHow to use and integrate Rmagine into your own project.
"},{"location":"getting_started/integration/#cpu-embree","title":"CPU (Embree)","text":""},{"location":"getting_started/integration/#in-code","title":"In Code","text":"
"},{"location":"getting_started/integration/#cmake","title":"CMake","text":"#include <rmagine/map/EmbreeMap.hpp>\n#include <rmagine/simulation/SphereSimulatorEmbree.hpp>\n\nnamespace rm = rmagine;\n\nint main(int argc, char** argv)\n{\n std::string filename = \"path/to/mesh/file\";\n rm::EmbreeMapPtr map = rm::import_embree_map(filename);\n\n rm::SphereSimulatorEmbree sim;\n sim.setMap(map);\n\n // go on (see workflow section)\n\n return 0;\n}\nAdd to your CMakeLists.txt:
"},{"location":"getting_started/integration/#gpu-optix","title":"GPU (OptiX)","text":""},{"location":"getting_started/integration/#in-code_1","title":"In Code","text":"add_compile_options(-std=c++17)\nset(CMAKE_CXX_STANDARD 17)\n\n# find components of a specific rmagine version\n# '2.2.1...' will get the newest rmagine which \n# is greater than 2.2.1\nfind_package(rmagine 2.2.1... \n COMPONENTS\n core \n embree\n)\n\nadd_executable(my_rmagine_app \n src/my_rmagine_app.cpp)\n\n# link against rmagine targets\ntarget_link_libraries(my_rmagine_app\n rmagine::core\n rmagine::embree\n)\n
"},{"location":"getting_started/integration/#cmake_1","title":"CMake","text":"#include <rmagine/map/OptixMap.hpp>\n#include <rmagine/simulation/SphereSimulatorOptix.hpp>\n\nnamespace rm = rmagine;\n\nint main(int argc, char** argv)\n{\n std::string filename = \"path/to/mesh/file\";\n rm::OptixMapPtr map = rm::import_optix_map(filename);\n\n rm::SphereSimulatorOptix sim;\n sim.setMap(map);\n\n // go on (see workflow section)\n\n return 0;\n}\nAdd to your CMakeFile:
"},{"location":"getting_started/maps/","title":"Maps","text":"add_compile_options(-std=c++17)\nset(CMAKE_CXX_STANDARD 17)\n\n# find components of a specific rmagine version\n# '2.2.1...' will get the newest rmagine which \n# is greater than 2.2.1\nfind_package(rmagine 2.2.1... \n COMPONENTS\n core \n cuda\n optix\n)\n\nadd_executable(my_rmagine_app \n src/my_rmagine_app.cpp)\n\n# link against rmagine targets\ntarget_link_libraries(my_rmagine_app\n rmagine::core\n rmagine::cuda\n rmagine::optix\n)\nTriangle Meshes can be stored in various file formats. Rmagine utilizes the Open Asset Import Library (assimp) in order to support a wide range of well known file formats. After loading the raw scene graph buffers with Assimp it is converted into Rmagines internal scene graph structure. Dependent on the computation backend
"},{"location":"getting_started/maps/#embree-map","title":"Embree Map","text":"EmbreeorOptiXthe scene graph is prepared for fast ray traversals by building the required acceleration structures.
"},{"location":"getting_started/maps/#optix-map","title":"OptiX Map","text":"#include <rmagine/map/EmbreeMap.hpp>\n\nnamespace rm = rmagine;\n\nint main(int argc, char** argv)\n{\n std::string filename = \"path/to/mesh/file\";\n rm::EmbreeMapPtr map = rm::import_embree_map(filename);\n return 0;\n}\n
"},{"location":"getting_started/maps/#properties","title":"Properties","text":"#include <rmagine/map/OptixMap.hpp>\n\nnamespace rm = rmagine;\n\nint main(int argc, char** argv)\n{\n std::string filename = \"path/to/mesh/file\";\n rm::OptixMapPtr map = rm::import_optix_map(filename);\n return 0;\n}\nAfter loading, the map consists of a complete scene graph. It then usually passed to a simulator (see next \"Getting Started\"-sections). The advanced Map-section describes how to modify or create the internal maps from scratch.
"},{"location":"getting_started/noise/","title":"Noise","text":"Currently noise models are implemented as postprocessing steps that modify the simulated ranges. Any of the following noise models can be chained to generate complex combined noise models. The Noise models are implemented equally both for GPU and CPU. Thus the developer can apply noise to the data without downloading or uploading the data from GPU to CPU or vice versa.
- Gaussian Noise
Apply gaussian noise $N(\\mu, \\sigma)$ to simulated ranges.
Parameter DescriptionmeanMean $\\mu$ of normal distributed noisestddevstandard deviation $\\sigma$ of normal distributed noiseExample CPU:
#include <rmagine/noise/GaussianNoise.hpp>\nnamespace rm = rmagine;\n\n// what happend before:\n// - data was simulated and stored into variable 'res'\n\n// make a copy to keep the unnoised ranges\nrm::Memory<float, rm::RAM> ranges = res.ranges;\n\n// Base class: Noise\nfloat mean = 0.0;\nfloat stddev = 1.0;\nrm::NoisePtr noise = std::make_shared<rm::GaussianNoise>(\n mean, stddev);\n\n// apply noise\nnoise->apply(ranges);\n// ranges now contains noise\nExample CUDA:
#include <rmagine/noise/GaussianNoiseCuda.hpp>\nnamespace rm = rmagine;\n\n// what happend before:\n// - data was simulated and stored into variable 'res'\n\n\n// make a copy to keep unnoised ranges\nrm::Memory<float, rm::VRAM_CUDA> ranges = res.ranges;\n\n// Base class: NoiseCuda\nfloat mean = 0.0;\nfloat stddev = 1.0;\nrm::NoiseCudaPtr noise = std::make_shared<rm::GaussianNoiseCuda>(\n mean, stddev);\n\n// apply noise\nnoise->apply(ranges);\n// ranges now contains noise\n- Relative Gaussian Noise
Apply gaussian noise $N(\\mu, \\sigma_r)$ to simulated ranges. Here, the standard deviation varies depending on distance.
Parameter DescriptionmeanMean $\\mu$ of normal distributed noisestddevstandard deviation $\\sigma$ of normal distributed noiserange_exprange exponent $c$ to compute range based stddev: $ \\sigma_r = \\sigma \\cdot r^{c} $Example CPU:
#include <rmagine/noise/RelGaussianNoise.hpp>\nnamespace rm = rmagine;\n\n// what happend before:\n// - data was simulated and stored into variable 'res'\n\n// make a copy to keep the unnoised ranges\nrm::Memory<float, rm::RAM> ranges = res.ranges;\n\n// Base class: Noise\nfloat mean = 0.0;\nfloat stddev = 0.2;\nfloat range_exp = 1.1;\nrm::NoisePtr noise = std::make_shared<rm::RelGaussianNoise>(\n mean, stddev, range_exp);\n\n// apply noise\nnoise->apply(ranges);\n// ranges now contains noise\nExample CUDA:
#include <rmagine/noise/RelGaussianNoiseCuda.hpp>\nnamespace rm = rmagine;\n\n// what happend before:\n// - data was simulated and stored into variable 'res'\n\n\n// make a copy to keep unnoised ranges\nrm::Memory<float, rm::VRAM_CUDA> ranges = res.ranges;\n\n// Base class: NoiseCuda\nfloat mean = 0.0;\nfloat stddev = 0.2;\nfloat range_exp = 1.1;\nrm::NoiseCudaPtr noise = std::make_shared<rm::RelGaussianNoiseCuda>(\n mean, stddev, range_exp);\n\n// apply noise\nnoise->apply(ranges);\n// ranges now contains noise\n- Uniform Dust Noise
Apply uniform dust noise to simulated ranges. Assuming some small particles could be hit by the range sensor that are not modeled by the scene, use this noise type.
Parameters:
Parameter Descriptionhit_probProbability of a ray hitting a particle in one meter free space.return_probProbability of a ray hitting dust returns to sender depending on particle distanceExample CPU:
#include <rmagine/noise/UniformDustNoise.hpp>\nnamespace rm = rmagine;\n\n// what happend before:\n// - data was simulated and stored into variable 'res'\n\n// make a copy to keep the unnoised ranges\nrm::Memory<float, rm::RAM> ranges = res.ranges;\n\n// Base class: Noise\nfloat hit_prob = 0.0;\nfloat ret_prob = 1.0;\nrm::NoisePtr noise = std::make_shared<rm::UniformDustNoise>(\n hit_prob, ret_prob);\n\n// apply noise\nnoise->apply(ranges);\n// ranges now contains noise\nExample CUDA:
"},{"location":"getting_started/overview/","title":"Rmagine","text":""},{"location":"getting_started/overview/#3d-range-sensor-simulation-in-polygonal-maps-via-ray-tracing-for-embedded-hardware-on-mobile-robots","title":"3D Range Sensor Simulation in Polygonal Maps via Ray Tracing for Embedded Hardware on Mobile Robots","text":"#include <rmagine/noise/UniformDustNoiseCuda.hpp>\nnamespace rm = rmagine;\n\n// what happend before:\n// - data was simulated and stored into variable 'res'\n\n\n// make a copy to keep unnoised ranges\nrm::Memory<float, rm::VRAM_CUDA> ranges = res.ranges;\n\n// Base class: NoiseCuda\nfloat hit_prob = 0.0;\nfloat ret_prob = 1.0;\nrm::NoiseCudaPtr noise = std::make_shared<rm::UniformDustNoiseCuda>(\n hit_prob, ret_prob);\n\n// apply noise\nnoise->apply(ranges);\n// ranges now contains noise\nLibrary for fast sensor data simulation in large 3D environments.
"},{"location":"getting_started/overview/#design-goals","title":"Design Goals","text":"Mainly designed for robotic applications:
- Perform multiple sensor simulations simultaneously and in realtime
- Perform computations at specific computing devices (CPU, GPU..)
- Hold data at device of computation
- Minimal graphical overhead (offscreen-rendering)
- Runtime critical operations
In addition to the wiki, the following papers wrap up the library conceptually. Please reference the following papers when using the Rmagine library in your scientific work.
"},{"location":"getting_started/problem_modelling/","title":"Problem Modelling","text":"@inproceedings{mock2023rmagine,\n title={{Rmagine: 3D Range Sensor Simulation in Polygonal Maps via Ray Tracing for Embedded Hardware on Mobile Robots}}, \n author={Mock, Alexander and Wiemann, Thomas and Hertzberg, Joachim},\n booktitle={IEEE International Conference on Robotics and Automation (ICRA)}, \n year={2023}\n}\nThe general computing flow is as follows.
Tsb is the transform from sensor to base frame. Or in other words: The sensor pose relative to the robot base. The map can be either a pointer to an
EmbreeMaporOptixMap. The Prefix of theSimulatoris eitherEmbreefor CPU computation orOptixfor GPU computation. The suffix of the Simulator is dependent on which sensor model you want to simulate. A few examples:SphereSimulatorEmbree- Simulate a velodyne on CPUPinholeSimulatorOptix- Simulate a depth camera on GPUO1DnSimulatorOptix- Simulate a customO1DnModelon GPU
Now we want to construct the following pipeline.
"},{"location":"getting_started/problem_modelling/#example-2-simulate-1000-lidars-on-gpu","title":"Example 2: Simulate 1000 LiDaRs on GPU","text":"#include <rmagine/simulation/SphereSimulatorEmbree.hpp>\n\nusing namespace rmagine;\n\nSphereSimulatorEmbreePtr construct_simulator(std::string path_to_mesh)\n{\n // Default construct the SphereSimulatorEmbree as shared pointer\n SphereSimulatorEmbreePtr sim = std::make_shared<SphereSimulatorEmbree>();\n\n EmbreeMapPtr map = import_embree_map(path_to_mesh);\n sim->setMap(map);\n\n // Define sensor model\n SphericalModel model;\n // TODO: fill with model specific parameters\n sim->setModel(model);\n\n // Set static transform between sensor and base (optional)\n Transform Tsb;\n Tsb.setIdentity();\n sim->setTsb(Tsb);\n\n return sim;\n}\n\nint main(int argc, char** argv)\n{\n // Load map and set map pointer to simulator\n std::string path_to_mesh = argv[1];\n SphereSimulatorEmbreePtr sim = construct_simulator(path_to_mesh);\n\n // Define 1000 poses to simulate from\n Memory<Transform, RAM> poses(1000);\n for(int i = 0; i<poses.size(); i++)\n {\n poses[i].setIdentity();\n }\n\n // add your desired attributes at intersection here\n using ResultT = Bundle<\n Ranges<RAM> \n >;\n\n // Result: Simulate Ranges\n ResultT res = sim->simulate<ResultT>(poses_);\n // res.ranges holds a buffer to the ranges\n\n return 0;\n}\nNow we want to construct the following pipeline.
The green cells are memory objects on GPU as you see in the following code snippet.
"},{"location":"getting_started/problem_modelling/#example-3-simulate-1000-lidars-on-gpu-and-images-on-cpu","title":"Example 3: Simulate 1000 LiDaRs on GPU and Images on CPU","text":"#include <rmagine/simulation/SphereSimulatorOptix.hpp>\n\nusing namespace rmagine;\n\nSphereSimulatorOptixPtr construct_simulator(std::string path_to_mesh)\n{\n // Default construct the SphereSimulatorEmbree as shared pointer\n SphereSimulatorOptixPtr sim = std::make_shared<SphereSimulatorOptix>();\n\n OptixMapPtr map = import_optix_map(path_to_mesh);\n sim->setMap(map);\n\n // Define sensor model\n SphericalModel model;\n // TODO: fill with model specific parameters\n sim->setModel(model);\n\n // Set static transform between sensor and base (optional)\n Transform Tsb;\n Tsb.setIdentity();\n sim->setTsb(Tsb);\n\n return sim;\n}\n\nint main(int argc, char** argv)\n{\n // Load map and set map pointer to simulator\n std::string path_to_mesh = argv[1];\n SphereSimulatorOptixPtr sim = construct_simulator(path_to_mesh);\n\n // Define 1000 poses to simulate from\n Memory<Transform, RAM> poses(1000);\n for(int i = 0; i<poses.size(); i++)\n {\n poses[i].setIdentity();\n }\n\n // upload from CPU to GPU\n Memory<Transform, VRAM_CUDA> poses_ = poses;\n\n // add your desired attributes at intersection here\n using ResultT = Bundle<\n Ranges<VRAM_CUDA> \n >;\n\n // Result: Simulate Ranges\n ResultT res = sim->simulate<ResultT>(poses_);\n\n // download from GPU to CPU\n // or use CUDA buffer for other computations\n Memory<float, RAM> ranges = res.ranges;\n\n return 0;\n}\nNow we want to construct the following pipeline.
"},{"location":"getting_started/sensors/","title":"Supported Sensor Models","text":"#include <rmagine/simulation/SphereSimulatorOptix.hpp>\n#include <rmagine/simulation/PinholeSimulatorEmbree.hpp>\n\nusing namespace rmagine;\n\nint main(int argc, char** argv)\n{\n // Load map and set map pointer to simulator\n std::string path_to_mesh = argv[1];\n\n // CONSTRUCTION PART\n\n // Define Simulators\n SphereSimulatorOptix lidar_sim_gpu;\n PinholeSimulatorEmbree dcam_sim_cpu;\n\n // Load and set maps\n OptixMapPtr map_gpu = import_optix_map(path_to_mesh);\n EmbreeMapPtr map_cpu = import_embree_map(path_to_mesh);\n lidar_sim_gpu.setMap(map_gpu);\n dcam_sim_cpu.setMap(map_cpu);\n\n\n SphericalModel lidar_model;\n PinholeModel dcam_model;\n // TODO: Define models\n lidar_sim_gpu.setModel(lidar_model);\n dcam_sim_cpu.setModel(dcam_model);\n\n // Define static transforms (optional)\n Transform T_lidar_base;\n Transform T_dcam_base;\n lidar_sim_gpu.setTsb(T_lidar_base);\n dcam_sim_cpu.setTsb(T_dcam_base);\n\n // SIMULATION PART\n\n Memory<Transform, RAM> poses(1000);\n // TODO: fill poses\n\n // upload from CPU to GPU\n Memory<Transform, VRAM_CUDA> poses_ = poses;\n\n\n // Simulate Depth cameras ranges on CPU\n using ResultT_RAM = Bundle<\n Ranges<RAM> \n >;\n ResultT_RAM dcam_res\n = dcam_sim_cpu.simulate<ResultT_RAM>(poses);\n\n // Simulate LiDaRs ranges on GPU\n using ResultT_VRAM = Bundle<\n Ranges<VRAM_CUDA> \n >;\n ResultT_VRAM lidar_res\n = lidar_sim_gpu.simulate<ResultT_VRAM>(poses_);\n\n // Download lidar ranges\n Memory<float, RAM> lidar_ranges = lidar_res.ranges;\n\n // Results are in dcam_res.ranges and lidar_ranges\n\n return 0;\n}\nRmagine supports several configurations of commonly used range sensors as Spherical, Pinhole or even fully customizable O1Dn and OnDn models. The following image shows how the results could look like using these different models.
The next instructions show how to initialize each model individually.
"},{"location":"getting_started/sensors/#spherical","title":"Spherical","text":"Spherical model for LiDARs.
struct SphericalModel\n{\n DiscreteInterval phi;\n DiscreteInterval theta;\n Interval range;\n};\nExample:
"},{"location":"getting_started/sensors/#pinhole","title":"Pinhole","text":"#include <rmagine/types/sensor_models.h>\nnamespace rm = rmagine;\n\n// ...\n\nrm::SphericalModel model;\n\nmodel.theta.min = -M_PI;\nmodel.theta.inc = 0.4 * M_PI / 180.0;\nmodel.theta.size = 900;\n\nmodel.phi.min = -15.0 * M_PI / 180.0;\nmodel.phi.inc = 2.0 * M_PI / 180.0;\nmodel.phi.size = 16;\n\nmodel.range.min = 0.0;\nmodel.range.max = 100.0;\nPinhole model for depth cameras.
struct PinholeModel\n{\n uint32_t width;\n uint32_t height;\n\n Interval range;\n\n float f[2]; // focal lengths fx, fy\n float c[2]; // centroid cx, cy\n};\nExample:
"},{"location":"getting_started/sensors/#o1dn","title":"O1Dn","text":"#include <rmagine/types/sensor_models.h>\nnamespace rm = rmagine;\n\n...\n\nrm::PinholeModel model;\nmodel.width = 200;\nmodel.height = 150;\nmodel.c[0] = 100.0;\nmodel.c[1] = 75.0;\nmodel.f[0] = 100.0;\nmodel.f[1] = 100.0;\nmodel.range.min = 0.0;\nmodel.range.max = 100.0;\nFully customizable model with one origin and N directions.
struct O1DnModel\n{\n uint32_t width;\n uint32_t height;\n\n // maximum and minimum allowed range\n Interval range;\n\n // i-th ray = orig, dirs[i]\n Vector orig; // One origin\n Memory<Vector> dirs; // N directions\n};\nExample:
"},{"location":"getting_started/sensors/#ondn","title":"OnDn","text":"#include <rmagine/types/sensor_models.h>\nnamespace rm = rmagine;\n\n...\n\nrm::O1DnModel model;\n\nmodel.orig.x = 0.0;\nmodel.orig.y = 0.0;\nmodel.orig.z = 0.0;\n\nmodel.width = 200;\nmodel.height = 1;\n\nmodel.dirs.resize(model.width * model.height);\n\nfloat step_size = 0.05;\n\nfor(int i=0; i<200; i++)\n{\n float y = - static_cast<float>(i - 100) * step_size;\n float x = cos(y) * 2.0 + 2.0;\n float z = -1.0;\n\n model.dirs[i].x = x;\n model.dirs[i].y = y;\n model.dirs[i].z = z;\n\n model.dirs[i].normalize();\n}\n\nmodel.range.min = 0.0;\nmodel.range.max = 100.0;\nFully customizable model with N origins and N directions.
struct OnDnModel\n{\n uint32_t width;\n uint32_t height;\n\n Interval range;\n\n // i-th ray = origs[i], dirs[i]\n Memory<Vector> origs; // N origins\n Memory<Vector> dirs; // N directions\n};\nExample:
"},{"location":"getting_started/sensors/#predefined-models","title":"Predefined models","text":"#include <rmagine/types/sensor_models.h>\nnamespace rm = rmagine;\n\n...\n\nrm::OnDnModel model;\n\nmodel.width = 200;\nmodel.height = 1;\n\nmodel.dirs.resize(model.width * model.height);\nmodel.origs.resize(model.width * model.height);\n\nfloat step_size = 0.05;\n\nfor(int i=0; i<200; i++)\n{\n float percent = static_cast<float>(i) / static_cast<float>(200);\n float step = - static_cast<float>(i - 100) * step_size;\n float y = sin(step);\n float x = cos(step);\n\n model.origs[i].x = 0.0;\n model.origs[i].y = y * percent;\n model.origs[i].z = x * percent;\n\n model.dirs[i].x = 1.0;\n model.dirs[i].y = 0.0;\n model.dirs[i].z = 0.0;\n}\n\nmodel.range.min = 0.0;\nmodel.range.max = 100.0;\nRmagine also provides some example models for testing. These are located in the
rmagine/types/sensors.hHeader-file and will be expanded in the future to include additional range sensors.
"},{"location":"getting_started/simulation/","title":"Simulation","text":""},{"location":"getting_started/simulation/#requirements","title":"Requirements","text":"#include <rmagine/types/sensors.h>\nnamespace rm = rmagine;\n\n\n...\n\n// Velodyne VLP-16 with different horizontal resoultions\nrm::SphericalModel velo_model_1 = rm::vlp16_900();\nrm::SphericalModel velo_model_2 = rm::vlp16_360();\n\n\n// another examples for testing:\nrm::SphericalModel ex_lidar = rm::example_spherical();\nrm::PinholeModel ex_pinhole = rm::example_pinhole();\nrm::O1DnModel ex_o1dn = rm::example_o1dn();\nrm::OnDnModel ex_ondn = rm::example_ondn();\nOnce a map is loaded and a sensor is defined anything is known to simulate the first range data.
"},{"location":"getting_started/simulation/#intersection-attributes","title":"Intersection Attributes","text":"Attribute Type Stride Description Hits uint8 1 If the a face was intersected (1) or not (0) Ranges float 1 Distance from ray origin along the direction to the first intersection Points float 3 Cartesian Coordinates of the Intersection (x,y,z) Normals float 3 Normal (nx, ny, nz) of intersected face FaceIds uint32 1 The id of the face that was intersected ObjectIds uint32 1 The id of the object that was intersected GeomIds uint32 1 The id of the geometry that was intersected"},{"location":"getting_started/simulation/#handle-results","title":"Handle Results","text":"// Map\n#include <rmagine/map/EmbreeMap.hpp>\n// Sensor Models\n#include <rmagine/types/sensor_models.h>\n// Simulators\n#include <rmagine/simulation/SphereSimulatorEmbree.hpp>\n\nnamespace rm = rmagine;\n\n// ...\n\n// loading a map\nstd::string path_to_mesh = \"my_mesh.ply\";\nrm::EmbreeMapPtr map = rm::load_embree_map(path_to_mesh);\n\n// defining a model\nrm::SphericalModel velo_model = rm::vlp16_900();\n\n// construct a simulator\nrm::SphereSimulatorEmbree sim;\nsim.setMap(map);\nsim.setModel(velo_model);\n\n// simulate ranges\n// ...\n
"},{"location":"library/concepts/","title":"Key Concepts","text":"// ...\n// Defined previously\n// - namespace rm = rmagine;\n// - SphereSimulatorEmbree sim;\n\n\n// 100 Transformations between base and map. e.g. poses of the robot\nrm::Memory<rm::Transform, rm::RAM> Tbm(100);\n\nfor(size_t i=0; i < Tbm.size(); i++)\n{\n rm::Transform T = rm::Transform::Identity();\n T.t = {2.0, 0.0, 0.0}; // position (2,0,0)\n rm::EulerAngles e = {0.0, 0.0, 1.0}; // orientation (0,0,1) radian - as euler angles\n T.R.set(e); // euler internally converted to quaternion\n Tbm[i] = T; // Write Transform/Pose to Memory\n}\n\n// add your desired attributes at intersection here\n// - optimizes the code at compile time\nusing ResultT = rm::Bundle<\n rm::Hits<rm::RAM>, \n rm::Ranges<rm::RAM>\n>;\n\n// Possible Attributes (rmagine/simulation/SimulationResults.hpp):\n// - Hits\n// - Ranges\n// - Points\n// - Normals\n// - FaceIds\n// - GeomIds\n// - ObjectIds\n\n// querying every attribute with 'rm::IntAttrAny' instead of 'ResultT'\n\nResultT result = sim.simulate<ResultT>(poses);\n// result.hits, result.ranges contain the resulting attribute buffers\nstd::cout << \"printing the first ray's range: \" << result.ranges[0] << std::endl;\n\n// or slice the results for the scan of pose 5\nauto ranges5 = result.ranges(5 * model.size(), 6 * model.size());\nstd::cout << \"printing the first ray's range of the fifth scan: \" << ranges5[0] << std::endl;\n\n// slicing and other useful operations will be described at another Wiki page.\nRmagine aims to perform computations flexible on selectable computing devices (CPU, GPU, ...). It also provides mechanisms to minimize graphical overheads and unnecessary copies between devices.
"},{"location":"library/concepts/#structure","title":"Structure","text":"Rmagine has the following top-level structure of directories:
- Math -
rmagine/math - Types -
rmagine/types - Utilility -
rmagine/util - Map -
rmagine/map - Simulation -
rmagine/simulation - Noise -
rmagine/noise
Contains all math related types and functions. All math datatypes are completely CUDA compatible. See
"},{"location":"library/concepts/#types","title":"Types","text":"rmagine/math/types.hfor all types. See Math section for more details.Fundamental types required for simulations, e.g. sensor models. Depends on math types.
"},{"location":"library/concepts/#util","title":"Util","text":"Utility functions. For exaple, including
rmagine/util/prints.hlets you print every math types viastd::cout.#include <iostream>\n#include <rmagine/math/types.h>\n#include <rmagine/util/prints.h>\n\nnamespace rm = rmagine;\n\nint main(int argc, char* argv)\n{\n rm::Transform T = rm::Transform::Identity();\n std::cout << \"my transformation: \" << T << std::endl;\n return 0;\n}\nThe Stopwatch in
rmagine/util/StopWatch.hpplets you easily stop the runtime of a Code section.
"},{"location":"library/concepts/#map","title":"Map","text":"#include <iostream>\n#include <rmagine/util/StopWatch.hpp>\n\nnamespace rm = rmagine;\n\nvoid demanding_function()\n{\n // ...\n}\n\nint main(int argc, char* argv)\n{\n rm::StopWatch sw;\n double el;\n\n sw();\n demanding_function();\n el = sw();\n std::cout << \"Demanding Function took \" << el << \" s\" << std::endl;\n\n return 0;\n}\nAll classes and functions that relate to map construction and map modifications. See Getting Started - Maps for a introduction to map loading. Go to Library - Maps for a more detailed descriptions of how to build Rmagine maps.
"},{"location":"library/concepts/#simulation","title":"Simulation","text":"All classes and functions that relate to the actual simulations.
"},{"location":"library/concepts/#noise","title":"Noise","text":"All classes and functions that relate to postprocessed noising.
"},{"location":"library/maps/","title":"Maps","text":""},{"location":"library/maps/#rmagine-scene-graph","title":"Rmagine Scene Graph","text":"Rmagine provides datatypes for scene graphs. They are seperated into Embree and OptiX scene graphs to store a Scene Graph either on CPU or on GPU. Both of these Scene Graphs have a very similar interface but are slightly different. Thus are real shared interface is not implemented yet. In Rmagine, a scene graph holds Geometries, Instances and Scenes.
"},{"location":"library/maps/#geometry","title":"Geometry","text":"A geometry is an abstract description of a physical object. Implemented Geometries are:
Concept Embree OptiX Mesh EmbreeMesh OptixMesh Points EmbreePoints -We also implemented some shortcut meshes that are used in later examples. They are located in the files
ShortCut Mesh Embree OptiX Spherermagine/map/embree/embree_shapes.handrmagine/map/optix/optix_shapes.h.EmbreeSphereOptixSphereCubeEmbreeCubeOptixCubePlaneEmbreePlaneOptixPlaneCylinderEmbreeCylinderOptixCylinder"},{"location":"library/maps/#scene","title":"Scene","text":"A scene is a set of geometries or a set of instances, each of which is assigned a position, an orientation, and a scale.
"},{"location":"library/maps/#instance","title":"Instance","text":"An instance instantiates a given scene or geometry at a certain pose. Thus things can be instantiated without duplicating their memory. A classic example for that is 3D asteroids where the same asteroid geometry has to be spawned several times. Using different geometries would then bring the GPU memory to its limits. Instead, Instantiating one geometry several times leads to a more memory friendly way of solving this problem. In robotics one can think of a known geometry as a class, e.g. a chair. This chair can be placed several times in the map by instantiating it.
"},{"location":"library/maps/#scene-graph-embree","title":"Scene Graph Embree","text":""},{"location":"library/maps/#simple","title":"Simple","text":"
"},{"location":"library/maps/#instances","title":"Instances","text":"#include <rmagine/map/embree/EmbreeScene.hpp>\n// shortcut meshes\n#include <rmagine/map/embree/embree_shapes.h>\n\nnamespace rm = rmagine;\n\nint main(int argc, char** argv)\n{\n // create a scene \n rm::EmbreeScenePtr scene = std::make_shared<rm::EmbreeScene>();\n\n // create a sphere\n rm::EmbreeMeshPtr sphere = std::make_shared<rm::EmbreeSphere>(1.0);\n { // sphere settings\n Transform T = Transform::Identity();\n T.t.x = 1.0; // move sphere one unit along the x-axis \n sphere->setTransform(T);\n sphere->apply(); // apply transform related changes\n }\n sphere->commit(); // commit sphere\n\n scene->add(sphere); // add sphere to scene\n scene->commit(); // commit scene\n\n // add the scene to a map. \n rm::EmbreeMapPtr map = std::make_shared<rm::EmbreeMap>(scene);\n\n // The map can then be used for simulations ...\n return 0;\n}\n
"},{"location":"library/maps/#custom-meshes","title":"Custom Meshes","text":"#include <rmagine/map/embree/EmbreeScene.hpp>\n#include <rmagine/map/embree/EmbreeInstance.hpp>\n// shortcut meshes\n#include <rmagine/map/embree/embree_shapes.h>\n\nnamespace rm = rmagine;\n\n/**\n * Create a Scene consisting of a sphere, a cube, and a cylinder\n */\nrm::EmbreeScenePtr create_scene()\n{\n rm::EmbreeScenePtr scene = std::make_shared<rm::EmbreeScene>();\n\n rm::EmbreeMeshPtr sphere = std::make_shared<rm::EmbreeSphere>(1.0);\n { // sphere settings\n Transform T = Transform::Identity();\n T.t.y = -2.0; // move sphere two units right \n sphere->setTransform(T);\n sphere->apply(); // apply transform related changes\n }\n sphere->commit(); // commit sphere\n scene->add(sphere); // add sphere to scene\n\n rm::EmbreeMeshPtr cube = std::make_shared<rm::EmbreeCube>();\n cube->commit(); // commit cube\n scene->add(cube); // add cube to scene\n\n auto cylinder = std::make_shared<rm::EmbreeCylinder>();\n { // cylinder settings\n Transform T = Transform::Identity();\n T.t.y = 2.0; // move cylinder two units left \n cylinder->setTransform(T);\n cylinder->apply(); // apply transform related changes\n }\n cylinder->commit(); // commit cylinder\n scene->add(cylinder); // add cylinder to scene\n\n scene->commit(); // commit scene\n return scene;\n}\n\nint main(int argc, char** argv)\n{\n // create a scene \n auto scene = std::make_shared<rm::EmbreeScene>();\n\n auto subscene = create_scene();\n\n // add 100 instances of a subscene to a scene\n for(size_t i=0; i<100; i++)\n {\n rm::EmbreeInstancePtr subscene_inst = subscene->instantiate();\n { // subscene_inst settings\n rm::Transform T = rm::Transform::Identity();\n T.t.x = (float)i; // moving instance 5 units to front\n subscene_inst->setTransform(T);\n subscene_inst->apply(); // apply transform related changes\n }\n subscene_inst->commit(); // commit instances\n scene->add(subscene_inst); // add instance to scene\n }\n\n scene->commit();\n\n // add the scene to a map. \n rm::EmbreeMapPtr map = std::make_shared<rm::EmbreeMap>(scene);\n\n // The map can then be used for simulations ...\n return 0;\n}\n
"},{"location":"library/maps/#scene-graph-optix","title":"Scene Graph OptiX","text":""},{"location":"library/maps/#simple_1","title":"Simple","text":"rm::EmbreeMeshPtr create_custom_mesh()\n{\n size_t Nvertices = 3;\n size_t Nfaces = 1;\n auto mesh = std::make_shared<rm::EmbreeMesh>(Nvertices, Nfaces);\n\n // reference to data as MemoryView objects\n rm::MemoryView<rm::Vertex, rm::RAM> vertices = mesh->vertices();\n rm::MemoryView<rm::Face, rm::RAM> faces = mesh->faces();\n\n faces[0] = {0, 1, 2};\n vertices[0] = {1.0, 0.0, 0.0};\n vertices[1] = {0.0, 1.0, 0.0};\n vertices[2] = {0.0, 0.0, 0.0};\n\n return mesh;\n}\n\nint main(int argc, char** argv)\n{\n auto scene = std::make_shared<rm::EmbreeScene>();\n\n auto my_mesh = create_custom_mesh();\n my_mesh->commit();\n\n scene->add(my_mesh);\n scene->commit();\n // do something with scene ...\n return 0;\n}\n
"},{"location":"library/maps/#instances_1","title":"Instances","text":"#include <rmagine/map/optix/OptixScene.hpp>\n// shortcut meshes\n#include <rmagine/map/optix/optix_shapes.h>\n\nnamespace rm = rmagine;\n\nint main(int argc, char** argv)\n{\n // create a scene \n rm::OptixScenePtr scene = std::make_shared<rm::OptixScene>();\n\n // create a sphere\n rm::OptixMeshPtr sphere = std::make_shared<rm::OptixSphere>(1.0);\n { // sphere settings\n Transform T = Transform::Identity();\n T.t.x = 1.0; // move sphere one unit along the x-axis \n sphere->setTransform(T);\n sphere->apply(); // apply transform related changes\n }\n sphere->commit(); // commit sphere\n\n scene->add(sphere); // add sphere to scene\n scene->commit(); // commit scene\n\n // add the scene to a map. \n rm::OptixMapPtr map = std::make_shared<rm::OptixMap>(scene);\n\n // The map can then be used for simulations ...\n return 0;\n}\n
"},{"location":"library/math/","title":"Rmagine - Math-Library","text":"#include <rmagine/map/optix/OptixScene.hpp>\n#include <rmagine/map/optix/OptixInstance.hpp>\n// shortcut meshes\n#include <rmagine/map/optix/optix_shapes.h>\n\nnamespace rm = rmagine;\n\n/**\n * Create a Scene consisting of a sphere, a cube, and a cylinder\n */\nrm::OptixScenePtr create_scene()\n{\n rm::OptixScenePtr scene = std::make_shared<rm::OptixScene>();\n\n rm::OptixMeshPtr sphere = std::make_shared<rm::OptixSphere>(1.0);\n { // sphere settings\n Transform T = Transform::Identity();\n T.t.y = -2.0; // move sphere two units right \n sphere->setTransform(T);\n sphere->apply(); // apply transform related changes\n }\n sphere->commit(); // commit sphere\n scene->add(sphere); // add sphere to scene\n\n rm::OptixMeshPtr cube = std::make_shared<rm::OptixCube>();\n cube->commit(); // commit cube\n scene->add(cube); // add cube to scene\n\n auto cylinder = std::make_shared<rm::OptixCylinder>();\n { // cylinder settings\n Transform T = Transform::Identity();\n T.t.y = 2.0; // move cylinder two units left \n cylinder->setTransform(T);\n cylinder->apply(); // apply transform related changes\n }\n cylinder->commit(); // commit cylinder\n scene->add(cylinder); // add cylinder to scene\n\n scene->commit(); // commit scene\n return scene;\n}\n\nint main(int argc, char** argv)\n{\n // create a scene \n auto scene = std::make_shared<rm::OptixScene>();\n\n auto subscene = create_scene();\n\n // add 100 instances of a subscene to a scene\n for(size_t i=0; i<100; i++)\n {\n rm::OptixInstancePtr subscene_inst = subscene->instantiate();\n { // subscene_inst settings\n rm::Transform T = rm::Transform::Identity();\n T.t.x = (float)i; // moving instance 5 units to front\n subscene_inst->setTransform(T);\n subscene_inst->apply(); // apply transform related changes\n }\n subscene_inst->commit(); // commit instance\n scene->add(subscene_inst); // add instance to scene\n }\n\n scene->commit();\n\n // add the scene to a map. \n rm::OptixMapPtr map = std::make_shared<rm::OptixMap>(scene);\n\n // The map can then be used for simulations ...\n return 0;\n}\nRmagine provides it's own thin math library. It's located in
"},{"location":"library/math/#points-and-translations","title":"Points and Translations","text":"rmagine::coretarget. We decided to do so after having problems with Eigen in CUDA code (corrupt memory after using math functions). This thin math library was specifically designed to enable the sharing of functions between CPU and GPU code, and it has been tested to ensure consistent results for both CUDA and CPU implementations. The following descriptions are made reading the code located inrmagine/math/types.h. So it is recommended to open the file alongside.A floating-point coordinate can represent different things such as a point, a vector or the translational part of a transformation. For all of them, we provide the same data structure:
Vector.rm::Vector2: x,y all fp32rm::Vector3: x,y,z all fp32
Aliases: -
rm::Vector=rm::Vector3; -rm::Point=rm::Vector3; -rm::Vertex=rm::Vector3;We also implemented commonly used functions
Vector:
"},{"location":"library/math/#rotations","title":"Rotations","text":"#include <rmagine/math/types.h>\nnamespace rm = rmagine;\n\n// ...\n\n// initializations\nrm::Vector3 p1;\np1.x = 0.0;\np1.y = 1.0;\np1.z = 2.0;\nrm::Vector3 p2 = {1.0, 2.0, 3.0};\n\n// functions\nfloat p1_length = p1.l2norm();\n\n// operators\nrm::Vector3 p3;\np3 = p1 + p2;\np3 = p1 - p2;\np3 = p1 * 2.0;\np3 = p1 / 2.0;\n\n// ...\nIn Rmagine we provide three different representations of rotations: Euler angles (
rm::EulerAngles), a rotation matrix (rm::Matrix3x3) and a quaternion (rm::Quaternion). In general, we adhere to the ROS conventions, especially those that are listed in REP-103.
"},{"location":"library/math/#conversions","title":"Conversions","text":"#include <rmagine/math/types.h>\nnamespace rm = rmagine;\n\nint main(int argc, char** argv)\n{\n // EulerAngles\n rm::EulerAngles e1;\n e1.roll = 0.0;\n e1.pitch = 0.0;\n e1.yaw = M_PI / 2.0;\n rm::EulerAngles e2 = {0.0, 0.0, M_PI / 2.0};\n rm::EulerAngles eI = rm::EulerAngles::Identity();\n\n // Quaternion\n rm::Quaternion q1;\n q1.x = 0.0;\n q1.y = 0.0;\n q1.z = 0.7071068;\n q1.w = 0.7071068;\n rm::Quaternion q2 = {0.0, 0.0, 0.7071068, 0.7071068};\n rm::Quaternion qI = rm::Quaternion::Identity();\n\n // Matrix3x3\n // - Storage Order: Column-Major\n // - Access via '()'-operator: Row-Major\n // - Access via '[]'-operator: Column-Major\n rm::Matrix3x3 M1;\n M1(0,0) = 0.0; M1(0,1) = -1.0; M1(0,2) = 0.0;\n M1(1,0) = 1.0; M1(1,1) = 0.0; M1(1,2) = 0.0;\n M1(2,0) = 0.0; M1(2,1) = 0.0; M1(2,2) = 1.0;\n rm::Matrix3x3 M2 = {{\n {0.0, 1.0, 0.0},\n {-1.0, 0.0, 0.0},\n {0.0, 0.0, 1.0}\n }};\n rm::Matrix3x3 MI = rm::Matrix3x3::Identity();\n\n return 0;\n}\nWe provide conversions between different rotation representations:
#include <rmagine/math/types.h>\nnamespace rm = rmagine;\n\nint main(int argc, char** argv)\n{\n // initializations\n rm::EulerAngles e;\n rm::Matrix3x3 M;\n rm::Quaterion q; \n\n rm::Vector3 p = {1.0, 0.0, 0.0};\n\n // rotation 90 degree around z-axis counter-clockwise\n e.roll = 0.0;\n e.pitch = 0.0;\n e.yaw = M_PI / 2.0;\n\n // convert\n M.set(e); // set M values that express the same rotation as e\n q.set(e); // set q values that express the same rotation as e\n\n rm::Vector3 p1 = e * p;\n rm::Vector3 p2 = M * p;\n rm::Vector3 p3 = q * p;\n // p1, p2 and p3 should all contain the values {0.0, 1.0, 0.0}\n\n // other conversions:\n q.set(M);\n M.set(q);\n e.set(M);\n e.set(q);\n\n return 0;\n}\nand some math functions, e.g. to apply a rotation to another or to a point in space:
"},{"location":"library/math/#eigen-compatibility","title":"Eigen Compatibility","text":"#include <rmagine/math/types.h>\nnamespace rm = rmagine;\nusing namespace rmagine;\n\nint main(int argc, char** argv)\n{\n rm::EulerAngles e = {0.0, 0.0, M_PI/2.0};\n rm::Quaternion q; q.set(e);\n rm::Vector3 p = {1.0, 0.0, 0.0};\n\n // Rotate a point 90 degrees around the z axis counter-clockwise\n rm::Vector3 p1 = q * p;\n\n // Chaining Rotations\n // -> Rotate a point 90 degrees around the z axis clockwise\n rm::Quaternion q2 = q * q * q;\n rm::Vector3 p2 = q2 * p;\n\n // invert\n rm::Quaternion q2_inv = q2.inv();\n // or if 'using namespace rmagine;' was set, ~operator can be used\n q2_inv = ~q2;\n\n return 0;\n}\nWe ensure compatibility with Eigen by sharing the same memory layout (for now). This allows to do fast mappings between rmagine and Eigen types.
"},{"location":"library/math/#transformations","title":"Transformations","text":"#include <rmagine/math/types.h>\n#include <Eigen/Dense>\nnamespace rm = rmagine;\n\nint main(int argc, char** argv)\n{\n // Rmagine -> Eigen\n {\n // generate rmagine type\n rm::Matrix3x3 M_rm = rm::Matrix3x3::Identity();\n\n // rmagine's Matrix3x3 has the same Column-Major storage order as the Eigen default for Eigen::Matrix3f\n Eigen::Matrix3f& M_eig = *reinterpret_cast<Eigen::Matrix3f*>(&M_rm);\n\n // another way is to use Eigens functions for mapping raw buffers\n Eigen::Map<Eigen::Matrix3f> M_eig2(&M_rm(0,0));\n\n // changing an entry in M_rm should now change the same entry in M_eig and M_eig2 as well\n }\n\n // Eigen -> Rmagine\n {\n Eigen::Matrix3f M_eig = Eigen::Matrix3f::Identity();\n rm::Matrix3x3& M_rm = *reinterpret_cast<rm::Matrix3x3*>(&M_eig);\n }\n\n return 0;\n}\nIn Rmagine, a transformation (
rm::Transform) is an operation that maps a source Euclidean space to a target Euclidean space, implemented as isometry, since we only implemented the rotational and translational part (no scale). We decided to represent the rotational part as quaternion (rm::Quaternion) to avoid gimbal locks that occur e.g. using an Euler angles representation.In Rmagine, we use a
Transformtype for a pose as well. A sensor pose entries correspond to a transformation that maps the space with the sensor as origin to the space where the pose is located:#include <rmagine/math/types.h>\nnamespace rm = rmagine;\n\nint main(int argc, char** argv)\n{\n // pose of the sensor in the map\n rm::Vector3 s_t = {0.0, 1.0, 2.0};\n rm::Quaternion s_R = rm::Quaternion::Identity();\n\n // has the same entries as the transform form sensor -> map\n rm::Transfrom T_sensor_to_map;\n T_sensor_to_map.R = s_R;\n T_sensor_to_map.t = s_t;\n\n return 0;\n}\nTo express a transformation that is not isometric, for example because it consists of a scale part, use
rm::Matrix4x4and the linear algebra functions ofrmagine/math/linalg.hinstead:
"},{"location":"library/math/#changing-precisions","title":"Changing Precisions","text":"#include <rmagine/math/types.h>\nnamespace rm = rmagine;\n\nint main(int argc, char** argv)\n{\n // pose of the sensor in the map\n rm::Transform T = {\n rm::Quaternion::Identity(), // rotation\n {0.0, 1.0, 2.0} // translation\n };\n rm::Vector3 s = {1.2, 1.2, 1.2}; // scale each dimension with 1.2\n\n // pack to Matrix4x4\n rm::Matrix4x4 M = rm::compose(T, s);\n\n // Use M\n rm::Vector3 p_sensor = {2.0, 1.0, 0.0};\n rm::Vector3 p_map = M * p_sensor;\n\n // unpack inverse: operator~ works only because of 'using namespace rmagine;'\n // - Alternative: Matrix4x4::inv() \n rm::decompose(~M, T, s);\n\n return 0;\n}\nWhat you have used so far are all types that itself are aliases to specializations of templated math classes:
(Location:// defaults\n#define DEFAULT_FP_PRECISION 32\nusing DefaultFloatType = float;\n\n// default types\nusing Vector3 = Vector3_<DefaultFloatType>;\nusing Vector2 = Vector2_<DefaultFloatType>;\nusing Matrix2x2 = Matrix_<DefaultFloatType, 2, 2>;\nusing Matrix3x3 = Matrix_<DefaultFloatType, 3, 3>;\nusing Matrix4x4 = Matrix_<DefaultFloatType, 4, 4>;\nusing Quaternion = Quaternion_<DefaultFloatType>;\nusing EulerAngles = EulerAngles_<DefaultFloatType>;\nusing Transform = Transform_<DefaultFloatType>;\nusing AABB = AABB_<DefaultFloatType>;\nrmagine/math/types/definitions.h)However, Rmagine also supports other floating-point precisions such as double:
(Location:using Vector2f = Vector2_<float>;\nusing Vector2u = Vector2_<uint32_t>;\nusing Vector2i = Vector2_<int32_t>;\nusing Vector3f = Vector3_<float>;\nusing Matrix2x2f = Matrix_<float, 2, 2>;\nusing Matrix3x3f = Matrix_<float, 3, 3>;\nusing Matrix4x4f = Matrix_<float, 4, 4>;\nusing Quaternionf = Quaternion_<float>;\nusing EulerAnglesf = EulerAngles_<float>;\nusing Transformf = Transform_<float>;\nusing AABBf = AABB_<float>;\n\nusing Vector2d = Vector2_<double>;\nusing Vector3d = Vector3_<double>;\nusing Matrix2x2d = Matrix_<double, 2, 2>;\nusing Matrix3x3d = Matrix_<double, 3, 3>;\nusing Matrix4x4d = Matrix_<double, 4, 4>;\nusing Quaterniond = Quaternion_<double>;\nusing EulerAnglesd = EulerAngles_<double>;\nusing Transformd = Transform_<double>;\nusing AABBd = AABB_<double>;\nrmagine/math/types/definitions.h)In addition, this makes it possible to uses custom precisions, and still have the full range of functions available for both CPU and CUDA code.
WARNING: The following code is a draft and was never tested, hence it doesn't neccesarily needs to compile. It shows how one would declare and use a 16 bit float vector.
"},{"location":"library/memory/","title":"Memory","text":""},{"location":"library/memory/#memory","title":"Memory","text":"#include <cuda_fp16.hpp>\nusing MyCudaHalfVector = rm::Vector3_<__half>;\n\n// add_inplace kernel\n// compute 'A[i] += B[i]', massively parallel on the GPU\n__global__ void add_inplace_kernel(\n MyCudaHalfVector* vec_a,\n const MyCudaHalfVector* vec_b, \n size_t n)\n{\n const unsigned int tid = threadIdx.x; \n if(tid < n)\n {\n // this invokes a rmagine function using the \n // using fp16 precision\n vec_a[tid] += vec_b[tid];\n }\n}\n\nint main(int argc, char** argv)\n{\n // buffer of CUDA-half vectors located in RAM\n rm::Memory<MyCudaHalfVector, rm::RAM> buffer_cpu(100);\n // TODO fill buffer\n\n // upload all CUDA-half vectors from RAM to VRAM_CUDA, twice\n rm::Memory<MyCudaHalfVector, rm::VRAM_CUDA> buffer_a_gpu \n = buffer_cpu;\n rm::Memory<MyCudaHalfVector, rm::VRAM_CUDA> buffer_b_gpu \n = buffer_cpu;\n\n // call CUDA kernel 'add_inplace_kernel'\n add_inplace_kernel<<<buffer_gpu.size(), 1>>>(\n buffer_a_gpu.raw(),\n buffer_b_gpu.raw(),\n buffer_a_gpu.size());\n // A[i]+=B[i] done.\n // buffer_a_gpu contains changed elements now \n\n return 0;\n}\nRmagine internally uses so-called Memory objects to manage memory located on different hardware. The location where the actual memory should be allocated can be passed as a template argument using one of the following keywords:
RAM(RAM memory)RAM_CUDA(pinned CUDA host memory)VRAM_CUDA(CUDA device memory)
After allocating the memory, accessing elements is similar to using
std::vector's:- access element with
[] - resize the memory with
resize() - access raw data pointer with
raw()function
The following code samples are describing how to work with Memory objects and how to transfer Memory to other hardware.
Example CPU-only:
#include <rmagine/types/Memory.hpp>\n\nnamespace rm = rmagine;\n\nint main(int argc, char** argv)\n{\n // allocate memory with 1000 elements\n rm::Memory<float, rm::RAM> mem(1000);\n\n // shrink memory\n mem.resize(100);\n\n // set some values\n mem[0] = 10.0;\n mem[10] = 5.0;\n\n return 0;\n}\nExample GPU-only:
#include <rmagine/types/MemoryCuda.hpp>\n\nnamespace rm = rmagine;\n\n__global__\nvoid set_value(float* data, unsigned int id, float val)\n{\n data[id] = val;\n}\n\nint main(int argc, char** argv)\n{\n // allocate memory with 1000 elements on GPU\n rm::Memory<float, rm::VRAM_CUDA> mem(1000);\n\n // shrink memory on GPU\n mem.resize(100);\n\n // this would cause a segfault. since the underlying memory is not available\n // on the device this code is executed:\n // mem[0] = 10.0;\n //\n // set some values in Cuda kernels instead\n set_value<<<1,1>>>(mem.raw(), 0, 10.0);\n set_value<<<1,1>>>(mem.raw(), 10, 5.0);\n\n return 0;\n}\nExample CPU <-> CPU:
"},{"location":"library/memory/#writing-memory-dependent-functions","title":"Writing Memory dependent Functions","text":"#include <rmagine/types/Memory.hpp>\n#include <rmagine/types/MemoryCuda.hpp>\n\nnamespace rm = rmagine;\n\n__global__ my_kernel(float* data, unsigned int N)\n{\n // ...\n}\n\nint main(int argc, char** argv)\n{\n // allocate memory with 1000 float elements\n rm::Memory<float, rm::RAM> mem(1000);\n\n // set some values\n mem[0] = 10.0;\n mem[10] = 5.0;\n\n // copy the whole memory to GPU\n rm::Memory<float, rm::VRAM_CUDA> mem_ = mem;\n\n // run some kernels\n my_kernel<<<mem_.size(), 1>>>(mem_.raw(), mem_.size());\n\n // copy back\n mem = mem_;\n\n return 0;\n}\nWith the memory objects Rmagine offers at the same time the possibility to make function calls dependent on the location of the memory. The next example adds two vectors and creates a new one. The actual addition should be executed on the device where the memory is currently stored on.
rm::Memory<float, rm::RAM> vec1(1000);\nrm::Memory<float, rm::RAM> vec2(1000);\n\n// fill vec1, vec2 ...\n\n// copy to GPU\nrm::Memory<float, rm::VRAM_CUDA> vec1_ = vec1;\nrm::Memory<float, rm::VRAM_CUDA> vec2_ = vec2;\n\n// we want to achieve this:\nauto vec3 = add(vec1, vec2);\nauto vec3_ = add(vec1_, vec2_);\n\n// ...\nSo we try to create a function
"},{"location":"library/memory/#signatures-header","title":"Signatures (Header):","text":"add(a, b)whose code will be executed on the CPU onceaandbare in RAM. However, as soon asaandbare stored on the GPU the code should be executed on the GPU as well as the function returns a GPU memory object. This can be done as follows:
"},{"location":"library/memory/#code-cpu","title":"Code CPU","text":"rm::Memory<float, rm::RAM> add(\n const rm::Memory<float, rm::RAM>& a, \n const rm::Memory<float, rm::RAM>& b);\n\nrm::Memory<float, rm::VRAM_CUDA> add(\n const rm::Memory<float, rm::VRAM_CUDA>& a, \n const rm::Memory<float, rm::VRAM_CUDA>& b); \n
"},{"location":"library/memory/#code-gpu","title":"Code GPU","text":"rm::Memory<float, rm::RAM> add(\n const rm::Memory<float, rm::RAM>& a, \n const rm::Memory<float, rm::RAM>& b)\n{\n rm::Memory<float, rm::RAM> c(a.size());\n for(size_t i=0; i < a.size(); i++)\n {\n c[i] = a[i] + b[i];\n }\n return c;\n}\n__global__\nvoid add_kernel(const float* a, const float* b, float* c, unsigned int N)\n{\n const unsigned int id = blockIdx.x * blockDim.x + threadIdx.x;\n if(id < N)\n {\n c[id] = a[id] + b[id];\n }\n}\n\nrm::Memory<float, rm::VRAM_CUDA> add(\n const rm::Memory<float, rm::VRAM_CUDA>& a, \n const rm::Memory<float, rm::VRAM_CUDA>& b)\n{\n rm::Memory<float, rm::VRAM_CUDA> c(a.size());\n add_kernel<<<c.size(), 1>>>(a.raw(), b.raw(), c.raw(), c.size());\n return c;\n}\nHaving these functions defined allows us to very flexible chain operations:
"},{"location":"library/memory/#slicing-and-memoryviews","title":"Slicing and MemoryViews","text":"// Simple (as above):\nauto vec3 = add(vec1, vec2);\nauto vec3_ = add(vec1_, vec2_);\n\n// Advanced\n// - add two vectors on CPU and upload to GPU on return\nrm::Memory<float, rm::VRAM_CUDA> vec3_ = add(vec1, vec2);\n// - add two vectors on GPU and download to CPU on return\nrm::Memory<float, rm::RAM> vec3 = add(vec1_, vec2_);\nRmagine also provides mechanisms to slice these Memory objects and handling shallow copies through so-called Memory Views.
rm::Memory<float, RAM> mem(1000);\n// MemoryView to the elements [100: 200]\nrm::MemoryView<float, RAM> slice = mem(100, 200);\n// this sets slice[0] and mem[100] to 10\nslice[0] = 10.0;\nWith that it is possible to access existing memory very flexible:
"},{"location":"library/memory/#application-example-1","title":"Application Example 1","text":"rm::Memory<int, rm::RAM> mem(1000);\nrm::Memory<int, rm::VRAM_CUDA> mem_(1000);\n\n// copy [100:200] to [0:100]\nmem(0, 100) = mem(100, 200)\nmem_(0, 100) = mem_(100, 200)\n\n// or even transfer memory slice-wise\nmem_(0, 100) = mem(500, 600); // upload a slice\nmem(400, 500) = mem_(100, 200); // download a slice\nfor debuging purposes sometimes it is required to print a fetch a single element out of a GPU buffer. Here we just want to print the first element of a GPU memory object as follows:
- copy one element to CPU through
"},{"location":"library/memory/#application-example-2","title":"Application Example 2","text":"rm::Memory<int, rm::VRAM_CUDA> mem_(1000);\n\n// download [0:1] to CPU\nrm::Memory<int, rm::RAM> one_elem_mem = mem_(0,1);\nstd::cout << one_elem_mem[0] << std::endl;\nOftentimes the GPU has a very limited amount of Memory. In Code this can be overcome using Rmagine's slices as follows:
"},{"location":"library/memory/#cuda-isolated-library-creation","title":"Cuda Isolated Library Creation","text":"// max available CPU mem: 1000\nrm::Memory<int, RAM> mem(1000);\n// max available GPU mem: 10\nrm::Memory<int, VRAM_CUDA> mem_(10);\n\n// i = [0, 10, 20, 30, ..., 990]\nfor(size_t i=0; i<mem.size(); i += mem_.size())\n{\n mem_ = mem(i, i + mem_.size());\n // process algorithm on GPU\n}\nIn order to ship a library and its headers, each CUDA piece of code should be invisable after compilation. That means the shipped header should be free of code that can be only proccessed by the NVCC compiler. Exeptions for that are, if the library is clearly marked as to use with CUDA. The following example shows how to achieve that using
"},{"location":"library/memory/#signatures-header_1","title":"Signatures (Header):","text":"Memoryobjects and the functionaddfrom above. In this example theadd-function is further improved to handle slices.File:
add.hrm::Memory<float, rm::RAM> add(\n const rm::MemoryView<float, rm::RAM>& a, \n const rm::MemoryView<float, rm::RAM>& b);\n\nrm::Memory<float, rm::VRAM_CUDA> add(\n const rm::MemoryView<float, rm::VRAM_CUDA>& a, \n const rm::MemoryView<float, rm::VRAM_CUDA>& b);\nFile
add.cuh
"},{"location":"library/memory/#code","title":"Code","text":"rm::Memory<float, rm::VRAM_CUDA> add(\n const rm::MemoryView<float, rm::VRAM_CUDA>& a, \n const rm::MemoryView<float, rm::VRAM_CUDA>& b);\nFile
add.cpp#include \"add.h\"\n\nrm::Memory<float, rm::RAM> add(\n const rm::MemoryView<float, rm::RAM>& a, \n const rm::MemoryView<float, rm::RAM>& b)\n{\n rm::Memory<float, rm::RAM> c(a.size());\n for(size_t i=0; i < a.size(); i++)\n {\n c[i] = a[i] + b[i];\n }\n return c;\n}\nFile:
add.cu
"},{"location":"library/memory/#main-and-cmake","title":"Main and CMake","text":"#include \"add.cuh\"\n\n__global__\nvoid add_kernel(const float* a, const float* b, float* c, unsigned int N)\n{\n const unsigned int id = blockIdx.x * blockDim.x + threadIdx.x;\n if(id < N)\n {\n c[id] = a[id] + b[id];\n }\n}\n\nrm::Memory<float, rm::VRAM_CUDA> add(\n const rm::MemoryView<float, rm::VRAM_CUDA>& a, \n const rm::MemoryView<float, rm::VRAM_CUDA>& b)\n{\n rm::Memory<float, rm::VRAM_CUDA> c(a.size());\n add_kernel<<<c.size(), 1>>>(a.raw(), b.raw(), c.raw(), c.size());\n return c;\n}\nFile:
Main.cpp#include <rmagine/types/Memory.hpp>\n#include \"add.h\"\n#include <rmagine/types/MemoryCuda.hpp>\n#include \"add.cuh\"\n\nint main(int argc, char** argv)\n{\n // CPU\n rm::Memory<float, rm::RAM> vec1(100);\n rm::Memory<float, rm::RAM> vec2(100);\n auto vec3 = add(vec1, vec2);\n auto vec3_slice = add(vec1(0, 10), vec2(10, 20));\n\n // GPU\n rm::Memory<float, rm::RAM> vec1_(100);\n rm::Memory<float, rm::RAM> vec2_(100);\n auto vec3_ = add(vec1_, vec2_);\n auto vec3_slice_ = add(vec1_(0, 10), vec2_(10, 20));\n\n return 0;\n}\nFile:
CMakeLists.txt# ...\n\nadd_library(my_add add.cpp)\ncuda_add_library(my_add_cuda add.cu)\n\nadd_executable(Main Main.cpp)\ntarget_link_libraries(Main\n my_add\n my_add_cuda\n)\n\n# ...\nThe Main.cpp and potential other code thus can be compiled with another compiler than the NVCC host compiler even though the CUDA code is executed internally.
"}]} \ No newline at end of file