Skip to content

Commit 783841e

Browse files
feat(readme): - added thesis url
1 parent cb2c782 commit 783841e

File tree

1 file changed

+64
-72
lines changed

1 file changed

+64
-72
lines changed

README.md

+64-72
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,43 @@
11
# KinectPython
2+
23
Human - Robot Collaboration for fabric folding using RGB/D and Kalman Filters.
34

5+
<div align="center">
6+
<h3>
7+
<a href="https://nemertes.library.upatras.gr/jspui/bitstream/10889/13753/1/DT_M_%ce%91%ce%93%ce%93%ce%95%ce%9b%ce%9f%ce%a0%ce%9f%ce%a5%ce%9b%ce%9f%ce%a3_%ce%9a%ce%a9%ce%9d%ce%a3%ce%a4%ce%91%ce%9d%ce%a4%ce%99%ce%9d%ce%9f%ce%a3_246690.pdf">
8+
→ Full Master Thesis ←
9+
</a>
10+
</h3>
11+
</div>
12+
413
## Table of Contents
14+
515
1. [Description](#Description)
616
2. [Installation](#Installation)
7-
- [Kinect Setup](#Kinect)
8-
- [Python Setup](#Python)
9-
- [RoboDK Setup](#RoboDK)
10-
- [Reflex One Setup](#ReflexOne)
11-
- [ATI Gamma FT Setup](#AtiFT)
17+
- [Kinect Setup](#Kinect)
18+
- [Python Setup](#Python)
19+
- [RoboDK Setup](#RoboDK)
20+
- [Reflex One Setup](#ReflexOne)
21+
- [ATI Gamma FT Setup](#AtiFT)
1222
3. [Usage](#Usage)
1323
4. [Examples](#Examples)
1424

15-
***
25+
---
26+
1627
## Description
1728

18-
This algorithm is designed to utilize the [Kinect for windows 2 RGB/D sensor](https://developer.microsoft.com/el-gr/windows/kinect/),
19-
the [KUKA LWR IV+](https://www.kuka.com/en-de/products/robot-systems/industrial-robots) industrial robot,
20-
the [ATI Force Torque Gamma Sensor](https://www.ati-ia.com/products/ft/ft_models.aspx?id=Gamma),
21-
the [Reflex One](https://www.labs.righthandrobotics.com/reflexhand) gripper
22-
and non Linear [Kalman Filter](http://web.mit.edu/kirtley/kirtley/binlustuff/literature/control/Kalman%20filter.pdf) estimation to enable the
29+
This algorithm is designed to utilize the [Kinect for windows 2 RGB/D sensor](https://developer.microsoft.com/el-gr/windows/kinect/),
30+
the [KUKA LWR IV+](https://www.kuka.com/en-de/products/robot-systems/industrial-robots) industrial robot,
31+
the [ATI Force Torque Gamma Sensor](https://www.ati-ia.com/products/ft/ft_models.aspx?id=Gamma),
32+
the [Reflex One](https://www.labs.righthandrobotics.com/reflexhand) gripper
33+
and non Linear [Kalman Filter](http://web.mit.edu/kirtley/kirtley/binlustuff/literature/control/Kalman%20filter.pdf) estimation to enable the
2334
Human - Robot Collaboration for fabric folding. This algorithm is an extension of this [publication](https://www.researchgate.net/publication/278028040_Human_Robot_Collaboration_for_Folding_Fabrics_Based_on_ForceRGB-D_Feedback).
2435
First the algorithm identifies the laid fabric using the background subtraction technique and map the fabric corner points to the corresponding
25-
wolrd space coordinates. After locating the fabric, the operator can enter the collaborative space and grab a fabric corner. Then the decision model will compute the robot's starting point and command the robot to approach the fabric and grab the
36+
wolrd space coordinates. After locating the fabric, the operator can enter the collaborative space and grab a fabric corner. Then the decision model will compute the robot's starting point and command the robot to approach the fabric and grab the
2637
fabric using the appropriate grasping model. After the robot has grabbed the fabric, it starts to follow the operator's movement
2738
to properly fold the fabric.
2839
The constructed framework and the hardware used can be seen in the figure below.
40+
2941
<p align="center">
3042
<img src="images/framework.png"/>
3143
</p>
@@ -58,59 +70,59 @@ To install the drivers for the Kinect 2 for windows download and install the [Ki
5870

5971
### Python Environment Installation <a name="Python"></a>
6072

61-
The constructed architecture works only on Windows and is tested with Python 3.6.
73+
The constructed architecture works only on Windows and is tested with Python 3.6.
6274
First create a fresh conda virtual environment with anaconda that uses Python 3.6 with
6375
the following steps:
6476

65-
1. Download and Install Anaconda for Windows [using this link](https://www.anaconda.com/products/individual#windows).
77+
1. Download and Install Anaconda for Windows [using this link](https://www.anaconda.com/products/individual#windows).
6678

67-
2. Create a new virtual env with Python 3.6. Open the Anaconda Prompt and type the following command.
79+
2. Create a new virtual env with Python 3.6. Open the Anaconda Prompt and type the following command.
6880

6981
```
7082
conda create -n name_of_your_environment python=3.6
7183
```
7284
73-
3. Activate the constructed environment.
85+
3. Activate the constructed environment.
7486
```
7587
conda activate name_of_your_environment
7688
```
77-
4. Install all **requirements** from requirements.txt using the following command.
89+
4. Install all **requirements** from requirements.txt using the following command.
7890
```
7991
pip install -r requirements.txt
8092
```
81-
5. Download all files using git clone or the .zip option and place them all in a
82-
folder wherever you want.
93+
5. Download all files using git clone or the .zip option and place them all in a
94+
folder wherever you want.
8395
84-
6. Open the Anaconda Prompt and type the following commands to find the directory
85-
of the installed python in the conda environment.
86-
```
96+
6. Open the Anaconda Prompt and type the following commands to find the directory
97+
of the installed python in the conda environment.
98+
````
8799
conda activate name_of_your_environment
88-
89-
where python
100+
where python
101+
```
102+
````
103+
7. Navigate to the Python's displayed directory, for example
90104
```
91-
7. Navigate to the Python's displayed directory, for example
92-
```
93-
C:\Users\UserName\.conda\envs\name_of_your_environment
94-
```
95-
8. Navigate inside the pykinect2 installed Library of the Python.
105+
C:\Users\UserName\.conda\envs\name_of_your_environment
106+
```
107+
8. Navigate inside the pykinect2 installed Library of the Python.
96108
```
97109
C:\Users\UserName\.conda\envs\name_of_your_environment\Lib\site-packages\pykinect2
98110
```
99-
9. Replace all the files inside the pykinect2 installed Library with the files located in the
100-
pykinect2_original folder inside the repository's downloaded files.
111+
9. Replace all the files inside the pykinect2 installed Library with the files located in the
112+
pykinect2_original folder inside the repository's downloaded files.
101113
102114
10. Add Python Directory to the systems Environment Variables PATH.
103115
- Search for _Edit the system environment variables_.
104116
- From the Advanced Tab click on the environment variables.
105117
- From the System variables scroll down, select Path and click on Edit.
106118
- By clicking on New add the following paths.
107-
```
108-
C:\Users\UserName\.conda\envs\name_of_your_environment
109-
C:\Users\UserName\.conda\envs\name_of_your_environment\python.exe
110-
C:\Users\UserName\.conda\envs\name_of_your_environment\Library\bin
111-
C:\Users\UserName\.conda\envs\name_of_your_environment\Scripts
112-
```
113-
119+
```
120+
C:\Users\UserName\.conda\envs\name_of_your_environment
121+
C:\Users\UserName\.conda\envs\name_of_your_environment\python.exe
122+
C:\Users\UserName\.conda\envs\name_of_your_environment\Library\bin
123+
C:\Users\UserName\.conda\envs\name_of_your_environment\Scripts
124+
```
125+
114126
### RoboDK Installation <a name="RoboDK"></a>
115127
116128
1. To configure RoboDK download and install the latest version of RoboDK [using this link.](https://robodk.com/download)
@@ -125,7 +137,7 @@ pykinect2_original folder inside the repository's downloaded files.
125137
126138
### Reflex One Gripper Installation <a name="ReflexOne"></a>
127139
128-
The Refex One Gripper works only with ROS Jade on Ubuntu 14.04 LTS. In order to install and configure the Reflex One software follow the instructions on the [_Gripper/instructions.txt_](Gripper/instructions.txt) file.
140+
The Refex One Gripper works only with ROS Jade on Ubuntu 14.04 LTS. In order to install and configure the Reflex One software follow the instructions on the [_Gripper/instructions.txt_](Gripper/instructions.txt) file.
129141
130142
### ATI Gamma FT Sensor Installation <a name="AtiFT"></a>
131143
@@ -152,7 +164,8 @@ If the installation is completed and everything works, then follow the next step
152164
8. Connect the Kinect via USB to the computer.
153165
154166
9. Open the [_track_v3.py_](track_v3.py) file and change the following flags according to what you want to use:
155-
```
167+
168+
```
156169
dim = True # Flag for finding the fabric's dimensions
157170
cal = False # Flag for calibrating the camera
158171
Sim = True # Flag for starting RoboDK
@@ -166,20 +179,26 @@ If the installation is completed and everything works, then follow the next step
166179
cloudPointInit = False # Flag to import the workspace as a pointCloud in RoboDK
167180
full_screen = False # flag to open pygame in fullscreen
168181
```
182+
169183
Then change the following parameters to your own configurations. Specifically, the Robot's controller IP and port:
184+
170185
```
171186
"""======================== ROBOT CONFIGS ==========================="""
172187
ROBOT_IP = '169.254.98.120' # KRC2 LAN IP
173188
ROBOT_PORT = 7000 # KRC2 LAN port
174189
```
190+
175191
The Gripper ROS API IP, port and encryption flag same as the server's value:
192+
176193
```
177194
"""============================= Gripper Configs ==========================="""
178195
VM_IP = '192.168.56.2' # Vm with Ubuntu Host only Static IP
179196
VM_PORT = 20000 # Port to communicate with Ubuntu running ROS
180197
VM_SERVER_ENCRYPTION = True
181198
```
199+
182200
The ATI Controller Server IP, port, encryption flag (same as the server's value) and the COM port that the ATI FT controller is connected to.
201+
183202
```
184203
"""========================== ATI FT Sensor Configs ====================="""
185204
ATI_FT_IP = 'localhost'
@@ -192,45 +211,18 @@ If the installation is completed and everything works, then follow the next step
192211
10. Save and run the [_track_v3.py_](track_v3.py) file.
193212
194213
11. If everything is correct you will see the following lines on the screen:
195-
```
196-
+-----------------------------+
197-
[MAIN] Elapsed Time: seconds
198-
[MAIN] Loaded: 100%
199-
[MAIN] Starting...
200-
+-----------------------------+
201-
[ATI FT CLIENT]: Message from Server: Hello UDP Client
202-
[ATI FT CLIENT]: Message from Server: Hello UDP Client
203-
[ATI FT CLIENT]: Message from Server: Started ATI FT... You can grab...
204-
+-------------------+
205-
Connecting to Gripper Server
206-
[GRIPPER CLIENT]: Message from Server: Hello UDP Client
207-
[GRIPPER CLIENT]: Message from Server: Hello UDP Client
208-
[GRIPPER CLIENT]: Message from Server: Started ROS... You can publish...
209-
[GRIPPER CLIENT]: Message from Server: Hello UDP Client
210-
[GRIPPER CLIENT]: Message from Server: Opened Gripper
211-
[ROBODK]: Connection Successful
212-
[ROBODK]: Robot Status: Connected
213-
+-------------------+
214-
Fabric Detected
215-
Width (mm):
216-
Height (mm):
217-
World Center Point (XYZ in mm):
218-
Color2World Fabric Points (mm):
219-
ISO Speed Calculated
220-
+-------------------+
221-
+-------------------+
222-
Starting Tracking
223-
+-------------------+
224-
```
225-
<br>
214+
`+-----------------------------+ [MAIN] Elapsed Time: seconds [MAIN] Loaded: 100% [MAIN] Starting... +-----------------------------+ [ATI FT CLIENT]: Message from Server: Hello UDP Client [ATI FT CLIENT]: Message from Server: Hello UDP Client [ATI FT CLIENT]: Message from Server: Started ATI FT... You can grab... +-------------------+ Connecting to Gripper Server [GRIPPER CLIENT]: Message from Server: Hello UDP Client [GRIPPER CLIENT]: Message from Server: Hello UDP Client [GRIPPER CLIENT]: Message from Server: Started ROS... You can publish... [GRIPPER CLIENT]: Message from Server: Hello UDP Client [GRIPPER CLIENT]: Message from Server: Opened Gripper [ROBODK]: Connection Successful [ROBODK]: Robot Status: Connected +-------------------+ Fabric Detected Width (mm): Height (mm): World Center Point (XYZ in mm): Color2World Fabric Points (mm): ISO Speed Calculated +-------------------+ +-------------------+ Starting Tracking +-------------------+`
215+
<br>
226216
227217
When the Starting tracking message show up then the operator can enter the collaborative space and grab a fabric corner.
228-
Then the decision model will compute the robot's starting point and command the robot to approach the fabric and grab the
218+
Then the decision model will compute the robot's starting point and command the robot to approach the fabric and grab the
229219
fabric using the appropriate grasping model. After the robot has grabbed the fabric, it starts to follow the operator's movement
230220
to properly fold the fabric.
231221
232222
## Examples
223+
233224
The constructed collaborative space can be seen inside the RoboDK simulation space:
225+
234226
<table>
235227
<tr>
236228
<td><img src="images/workspace.png" width=920 height=480></td>

0 commit comments

Comments
 (0)