Skip to content

Commit

Permalink
Merge pull request #182 from fabric-testbed/fablib1.3.3_working
Browse files Browse the repository at this point in the history 
added demos and tutorials
  • Loading branch information
@kthare10
kthare10 authored Dec 2, 2022
2 parents d1c1aaa + 057c98d commit 63ed3c6
Show file tree
Hide file tree
Showing 92 changed files with 7,330 additions and 44 deletions.
0 CHANGELOG.md → Changelog.md
View file
File renamed without changes.
279 changes: 279 additions & 0 deletions configure.ipynb
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,279 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"tags": []
},
"source": [
"# Configure your Jupyter Environment\n",
"\n",
"The following notebook can be used to configure your Jupyter environment. The product of running this notebook are the following:\n",
"\n",
"- `requirements.txt`: Used to upgrade/downgrade your environment with the FABLIB version used by these example notebooks.\n",
"- `fabric_rc`: File used to configure a FABlib application. \n",
"- `ssh_config`: File used to ssh from from a terminal to FABRIC VM by jumping through the FABRIC bastion host. "
]
},
{
"cell_type": "markdown",
"metadata": {
"tags": []
},
"source": [
"## Copy your Bastion Key to your JupyterLab Environment\n",
"\n",
"In order to minimize security incidents on FABRIC, access to VMs and other resources administered by users is controlled using a bastion host. You will need to set up an ssh keypair that will be used to jump through the bastion host to your VMs and other resources. This keypair is unique to you and is only used to set up ssh proxy connections through the bastion host to your FABRIC resources. \n",
"\n",
"The following notebook walks through creating and submitting a bastion keypair necessary to use FABRIC. More information about how to access your experiment through the bastion host can be found [here](https://learn.fabric-testbed.net/knowledge-base/logging-into-fabric-vms/).\n",
"\n",
"If you have not created a bastion host keypair, create a new one in the [Experiments](https://portal.fabric-testbed.net/experiments#sshKeys) section of the FABRIC Portal.\n",
"\n",
"Copy your bastion private key to your Jupyter container. Drag/drop your key from your local machine to the file browser in your Juypyter browser window. Note the name of your key and location where you put it."
]
},
{
"cell_type": "markdown",
"metadata": {
"tags": []
},
"source": [
"## Set Bastion Username and Project ID\n",
"\n",
"Edit the following cell by entering your FABRIC bastion username and the correct path to the copy of your bastion private key in your Jupyter container. \n",
"\n",
"- Your bastion user name can be found on your [user profile page](https://portal.fabric-testbed.net/user) in the FABRIC portal (click \"My SSH Keys\").\n",
"- The path to your bastion key was determined when you copied it to the Jupyter container. \n",
"\n",
"The Project ID can be from any of your projects. The ID can be found on the [projects](https://portal.fabric-testbed.net/projects) tab in the FABRIC portal."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"export FABRIC_BASTION_USERNAME=<BASTION_USERNAME>\n",
"export FABRIC_PROJECT_ID=<PROJECT_ID>\n",
"\n",
"export FABRIC_BASTION_PRIVATE_KEY_LOCATION=${HOME}/work/fabric_config/fabric_bastion_key\n",
"\n",
"export FABRIC_BASTION_SSH_CONFIG_FILE=${HOME}'/work/fabric_config/ssh_config'\n",
"export FABRIC_RC_FILE=${HOME}'/work/fabric_config/fabric_rc'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Set other Tokens and Keys\n",
"\n",
"The the path to where you put your bastion key.The defaults below should work within the Jupyter environment hosted by FABRIC. Custom installations will require modifications to this configuration.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"export FABRIC_TOKEN_FILE=${HOME}'/.tokens.json'\n",
"export FABRIC_SLICE_PRIVATE_KEY_FILE=${HOME}/work/fabric_config/slice_key\n",
"export FABRIC_SLICE_PUBLIC_KEY_FILE=${FABRIC_SLICE_PRIVATE_KEY_FILE}.pub"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Optionally, create a new slice keypair. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#ssh-keygen -t rsa -b 3072 -f $FABRIC_SLICE_PRIVATE_KEY_FILE -q -N \"\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"\n",
"Set the Permissions on your bastion key and private slice key. Your private bastion key must not have it permissions too open."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"chmod 600 ${FABRIC_BASTION_PRIVATE_KEY_LOCATION}\n",
"chmod 600 ${FABRIC_SLICE_PRIVATE_KEY_FILE}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## FABLIB Version\n",
"\n",
"This package of example notebooks was created using the version of fabrictestbed-extensions (i.e. the FABLIB package) specified in the local `requirements.txt` file. "
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"fabrictestbed-extensions==1.3.3b3\n"
]
}
],
"source": [
"cat requirements.txt"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"These examples may work with your currently installed default FABLIB version. However, you may want to upgrade/downgrade the FABLIB version to the one used to develop these examples. The following cell will synchronize your current FABLIB version with these examples."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"pip install -r requirements.txt --user --quiet"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Your Jupyter container will often be updated with the current default FABLIB version. You can permanently override these updates by setting your desired version in\n",
"`~/work/fabric_config/requirements.txt`. If you would like to permanently configure your Jupyter environment to use the FABLIB version used for this set of examples, copy the local `requirnments.txt` file to `~/work/fabric_config/requirements.txt` using the following command."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"cp requirements.txt ~/work/fabric_config/requirements.txt"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Create the FABRIC Config File \n",
"\n",
"\n",
"The following cell creates the `fabric_rc` file using the values configured above. \n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"cat <<EOF > ${FABRIC_RC_FILE}\n",
"export FABRIC_CREDMGR_HOST=cm.fabric-testbed.net\n",
"export FABRIC_ORCHESTRATOR_HOST=orchestrator.fabric-testbed.net\n",
"\n",
"export FABRIC_PROJECT_ID=${FABRIC_PROJECT_ID}\n",
"export FABRIC_TOKEN_LOCATION=${FABRIC_TOKEN_FILE}\n",
"\n",
"export FABRIC_BASTION_HOST=bastion-1.fabric-testbed.net\n",
"export FABRIC_BASTION_USERNAME=${FABRIC_BASTION_USERNAME}\n",
"\n",
"export FABRIC_BASTION_KEY_LOCATION=${FABRIC_BASTION_PRIVATE_KEY_LOCATION}\n",
"#export FABRIC_BASTION_KEY_PASSPHRASE=\n",
"\n",
"export FABRIC_SLICE_PRIVATE_KEY_FILE=${FABRIC_SLICE_PRIVATE_KEY_FILE}\n",
"export FABRIC_SLICE_PUBLIC_KEY_FILE=${FABRIC_SLICE_PUBLIC_KEY_FILE} \n",
"#export FABRIC_SLICE_PRIVATE_KEY_PASSPHRASE=\n",
"\n",
"export FABRIC_LOG_FILE=${HOME}/fablib.log\n",
"export FABRIC_LOG_LEVEL=INFO \n",
"\n",
"export FABRIC_AVOID=''\n",
"\n",
"export FABRIC_SSH_COMMAND_LINE=\"ssh -i {{ node.private_ssh_key_file }} -F ${FABRIC_BASTION_SSH_CONFIG_FILE} {{ node.username }}@{{ node.management_ip }}\"\n",
"EOF"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Create the SSH Config File\n",
"\n",
"The following cell creates an SSH config file using the values configure above. The config file allows you to use ssh to jump through the bastion host from a terminal command line. After creating this file, you can ssh and pass the file with the `-F` option.\n",
"\n",
"Alternatively, you can directly edit your [ssh config](~/fabric_config/ssh_config) file.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"cat <<EOF > ${FABRIC_BASTION_SSH_CONFIG_FILE}\n",
"UserKnownHostsFile /dev/null\n",
"StrictHostKeyChecking no\n",
"ServerAliveInterval 120 \n",
"\n",
"Host bastion-?.fabric-testbed.net\n",
" User ${FABRIC_BASTION_USERNAME}\n",
" ForwardAgent yes\n",
" Hostname %h\n",
" IdentityFile ${FABRIC_BASTION_PRIVATE_KEY_LOCATION}\n",
" IdentitiesOnly yes\n",
"\n",
"Host * !bastion-?.fabric-testbed.net\n",
" ProxyJump ${FABRIC_BASTION_USERNAME}@bastion-1.fabric-testbed.net:22\n",
"EOF"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Bash",
"language": "bash",
"name": "bash"
},
"language_info": {
"codemirror_mode": "shell",
"file_extension": ".sh",
"mimetype": "text/x-sh",
"name": "bash"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
54 changes: 54 additions & 0 deletions fabric_examples/public_demos/FABRIC-Workshop-Spring2021/index.ipynb
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "0c2da6f1",
"metadata": {},
"source": [
"# FABRIC Demos and Tutorials\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "19651fde-76e0-4e82-ada1-29e3b8d0541a",
"metadata": {},
"source": [
"## Internet2 TechEx (Dec 2022)\n",
"\n",
"- [Configure Environment](./configure.ipynb): Configure you Environment including creating the `fabric_rc` and `ssh_config` files.\n",
"\n",
"## SC22 (Nov 2022)\n",
"\n",
"## KNIT5 (Sept 2022)\n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "76146b01",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Bash",
"language": "bash",
"name": "bash"
},
"language_info": {
"codemirror_mode": "shell",
"file_extension": ".sh",
"mimetype": "text/x-sh",
"name": "bash"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
38 changes: 38 additions & 0 deletions fabric_examples/public_demos/KNIT5/tutorials_index.ipynb
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "0c2da6f1",
"metadata": {},
"source": [
"# KNIT5 \n",
"\n",
"- [Tutorial](./KNIT5_Creating_FABnet_Networks/KNIT5_Tutorial_Creating_FABnet_Networks.ipynb)\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "521c3386-45df-4355-986f-24b38d0bc756",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Bash",
"language": "bash",
"name": "bash"
},
"language_info": {
"codemirror_mode": "shell",
"file_extension": ".sh",
"mimetype": "text/x-sh",
"name": "bash"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
1 change: 1 addition & 0 deletions fabric_examples/public_demos/SC22/FRRouting/config/FRRouting_OSPF_SC22_2_data.json
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"slice": {}, "networks": [{"name": "dall", "site": "DALL", "subnet": "192.168.1.0/24", "router": {"name": "dall_router", "facility": "FABRIC", "network": "dall", "ip": "192.168.1.1", "cores": "32", "ram": "128", "disk": "10", "static_routes": []}}, {"name": "salt", "site": "SALT", "subnet": "192.168.2.0/24", "router": {"name": "salt_router", "facility": "FABRIC", "network": "salt", "ip": "192.168.2.1", "cores": "32", "ram": "128", "disk": "10", "static_routes": []}}, {"name": "utah", "site": "UTAH", "subnet": "192.168.3.0/24", "router": {"name": "utah_router", "facility": "FABRIC", "network": "utah", "ip": "192.168.3.1", "cores": "32", "ram": "128", "disk": "10", "static_routes": []}}, {"name": "wash", "site": "WASH", "subnet": "192.168.4.0/24", "router": {"name": "wash_router", "facility": "FABRIC", "network": "wash", "ip": "192.168.4.1", "cores": "32", "ram": "128", "disk": "10", "static_routes": []}}], "links": [{"name": "link0", "subnet": "192.168.100.0/24", "endpoints": [{"router_name": "dall_router", "ip": "192.168.100.1"}, {"router_name": "wash_router", "ip": "192.168.100.2"}]}, {"name": "link1", "subnet": "192.168.101.0/24", "endpoints": [{"router_name": "dall_router", "ip": "192.168.101.1"}, {"router_name": "salt_router", "ip": "192.168.101.2"}]}, {"name": "link2", "subnet": "192.168.102.0/24", "endpoints": [{"router_name": "salt_router", "ip": "192.168.102.1"}, {"router_name": "wash_router", "ip": "192.168.102.2"}]}, {"name": "link3", "subnet": "192.168.103.0/24", "endpoints": [{"router_name": "salt_router", "ip": "192.168.103.1"}, {"router_name": "utah_router", "ip": "192.168.103.2"}]}], "nodes": [{"name": "dall_node1", "facility": "FABRIC", "network": "dall", "ip": "192.168.1.100", "cores": "16", "ram": "64", "disk": "10", "static_routes": [["192.168.0.0/16", "192.168.1.1"]]}, {"name": "dall_node2", "facility": "FABRIC", "network": "dall", "ip": "192.168.1.101", "cores": "16", "ram": "64", "disk": "10", "static_routes": [["192.168.0.0/16", "192.168.1.1"]]}, {"name": "salt_node1", "facility": "FABRIC", "network": "salt", "ip": "192.168.2.100", "cores": "16", "ram": "64", "disk": "10", "static_routes": [["192.168.0.0/16", "192.168.2.1"]]}, {"name": "salt_node2", "facility": "FABRIC", "network": "salt", "ip": "192.168.2.101", "cores": "16", "ram": "64", "disk": "10", "static_routes": [["192.168.0.0/16", "192.168.2.1"]]}, {"name": "utah_node1", "facility": "FABRIC", "network": "utah", "ip": "192.168.3.100", "cores": "16", "ram": "64", "disk": "10", "static_routes": [["192.168.0.0/16", "192.168.3.1"]]}, {"name": "utah_node2", "facility": "FABRIC", "network": "utah", "ip": "192.168.3.101", "cores": "16", "ram": "64", "disk": "10", "static_routes": [["192.168.0.0/16", "192.168.3.1"]]}, {"name": "wash_node1", "facility": "FABRIC", "network": "wash", "ip": "192.168.4.100", "cores": "16", "ram": "64", "disk": "10", "static_routes": [["192.168.0.0/16", "192.168.4.1"]]}, {"name": "wash_node2", "facility": "FABRIC", "network": "wash", "ip": "192.168.4.101", "cores": "16", "ram": "64", "disk": "10", "static_routes": [["192.168.0.0/16", "192.168.4.1"]]}]}
1 change: 1 addition & 0 deletions fabric_examples/public_demos/SC22/FRRouting/config/FRRouting_OSPF_SC22_data.json
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"slice": {}, "networks": [{"name": "chameleon", "site": "STAR", "facility": "CHI@UC", "subnet": "192.168.1.0/24", "allocation_pool_start": "192.168.1.10", "allocation_pool_end": "192.168.1.250", "chameleon_gateway_ip": "192.168.1.2", "router": {"name": "chameleon_router", "facility": "FABRIC", "site": "MICH", "network": "cham", "ip": "192.168.1.1", "cores": "32", "ram": "128", "disk": "10", "static_routes": []}}, {"name": "dall", "site": "DALL", "facility": "FABRIC", "subnet": "192.168.2.0/24", "router": {"name": "dall_router", "facility": "FABRIC", "network": "dall", "site": "DALL", "ip": "192.168.2.1", "cores": "32", "ram": "128", "disk": "10", "static_routes": []}}, {"name": "utah", "site": "UTAH", "facility": "FABRIC", "subnet": "192.168.3.0/24", "router": {"name": "utah_router", "facility": "FABRIC", "network": "utah", "site": "UTAH", "ip": "192.168.3.1", "cores": "8", "ram": "32", "disk": "10", "static_routes": []}}, {"name": "salt", "site": "SALT", "facility": "FABRIC", "subnet": "192.168.4.0/24", "router": {"name": "salt_router", "facility": "FABRIC", "network": "salt", "site": "SALT", "ip": "192.168.4.1", "cores": "32", "ram": "128", "disk": "10", "static_routes": []}}], "links": [{"name": "link0", "subnet": "192.168.100.0/24", "endpoints": [{"router_name": "dall_router", "ip": "192.168.100.1"}, {"router_name": "chameleon_router", "ip": "192.168.100.2"}]}, {"name": "link1", "subnet": "192.168.101.0/24", "endpoints": [{"router_name": "dall_router", "ip": "192.168.101.1"}, {"router_name": "salt_router", "ip": "192.168.101.2"}]}, {"name": "link2", "subnet": "192.168.102.0/24", "endpoints": [{"router_name": "salt_router", "ip": "192.168.102.1"}, {"router_name": "chameleon_router", "ip": "192.168.102.2"}]}, {"name": "link3", "subnet": "192.168.103.0/24", "endpoints": [{"router_name": "utah_router", "ip": "192.168.103.1"}, {"router_name": "salt_router", "ip": "192.168.103.2"}]}], "nodes": [{"name": "chameleon_node0", "facility": "CHI@UC", "compute_reservation_id": "51a07b0c-d667-4908-9400-049ce3bb26fe", "network": "chameleon", "ip": "192.168.1.197", "cores": "16", "ram": "64", "disk": "10", "static_routes": [["192.168.0.0/16", "192.168.1.1"]], "management_ip": "192.5.86.251"}, {"name": "chameleon_node1", "facility": "CHI@UC", "compute_reservation_id": "51a07b0c-d667-4908-9400-049ce3bb26fe", "network": "chameleon", "ip": "192.168.1.11", "cores": "16", "ram": "64", "disk": "10", "static_routes": [["192.168.0.0/16", "192.168.1.1"]], "management_ip": "192.5.86.218"}, {"name": "dall_node1", "facility": "FABRIC", "network": "dall", "ip": "192.168.2.100", "cores": "16", "ram": "64", "disk": "10", "static_routes": [["192.168.0.0/16", "192.168.2.1"]]}, {"name": "dall_node2", "facility": "FABRIC", "network": "dall", "ip": "192.168.2.101", "cores": "16", "ram": "64", "disk": "10", "static_routes": [["192.168.0.0/16", "192.168.2.1"]]}, {"name": "utah_node1", "facility": "FABRIC", "network": "utah", "ip": "192.168.3.100", "cores": "8", "ram": "32", "disk": "10", "static_routes": [["192.168.0.0/16", "192.168.3.1"]]}, {"name": "utah_node2", "facility": "FABRIC", "network": "utah", "ip": "192.168.3.101", "cores": "8", "ram": "32", "disk": "10", "static_routes": [["192.168.0.0/16", "192.168.3.1"]]}, {"name": "salt_node1", "facility": "FABRIC", "network": "salt", "ip": "192.168.4.100", "cores": "16", "ram": "64", "disk": "10", "static_routes": [["192.168.0.0/16", "192.168.4.1"]]}, {"name": "salt_node2", "facility": "FABRIC", "network": "salt", "ip": "192.168.4.101", "cores": "16", "ram": "64", "disk": "10", "static_routes": [["192.168.0.0/16", "192.168.4.1"]]}]}
Loading

0 comments on commit 63ed3c6

Please sign in to comment.