PyTorch_1/1_tensors.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Tensors\n",
    "## Tensor creation\n",
    "\n",
    "Tensors are Pytorch's data structure for representing vectors, matrices and higher dimensional tensors.\n",
    "Tensors contain homogeneous data of fixed size.\n",
    "\n",
    "Tensors can be initialized from Python objects, e.g., a list of lists of integers, making up a $n\\times m$ dimensional array.\n",
    "You just call `torch.tensor(data)` where `data` is the Python object.\n",
    "The data type is automatically inferred, but you can override this behavior by setting the `dtype` argument when calling `torch.tensor`, where `dtype` is one of the PyTorch dtypes as listed at https://pytorch.org/docs/stable/tensors.html#data-types.\n",
    "\n",
    "### Tensors from functions with random or constant values\n",
    "PyTorch provides several functions to create tensors with random or constant values:\n",
    "* `torch.ones(size)`: Creates a tensor filled with ones.\n",
    "* `torch.zeros(size)`: Creates a tensor filled with zeros.\n",
    "* `torch.eye(size)`: Creates a 2D tensor with ones on the diagonal and zeros elsewhere (identity matrix).\n",
    "* `torch.rand(size)`: Creates a tensor with random values uniformly distributed between 0 and 1.\n",
    "* `torch.randn(size)`: Creates a tensor with random values from a normal distribution with mean 0 and variance 1.\n",
    "* `torch.full(size, fill_value)`: Creates a tensor filled with the specified value.\n",
    "\n",
    "### Tensors from other tensors\n",
    "You can create new tensors from existing tensors. This can be done using various functions:\n",
    "* `torch.clone(tensor)`: Creates a copy of the tensor.\n",
    "* `tensor.new_tensor(data)`: Creates a new tensor with the same properties (dtype, device) as the original tensor.\n",
    "* `tensor.expand(size)`: Expands the tensor to a larger size without copying data.\n",
    "* `tensor.view(size)`: Returns a new tensor with the same data but different shape.\n",
    "* `tensor.to(dtype)`: Converts the tensor to a different data type.\n",
    "\n",
    "Note that `size` is a sequence of integers defining the shape of the output tensor, and `dtype` is of type `torch.dtype`, e.g., `torch.float32`.\n",
    "\n",
    "## Device allocation\n",
    "\n",
    "You can also specify the device where the tensor lives, e.g., on CPU (default) or GPU.\n",
    "For this, you can pass the `device` argument to the tensor creation functions, which takes an object of type `torch.device` ([documentation](https://pytorch.org/docs/stable/tensor_attributes.html#torch-device)).\n",
    "These can be created by calling `torch.device` with a device type, e.g., `'cpu'` or `'cuda'` (for GPU), and an ordinal specifying the the device id.\n",
    "\n",
    "## Tensor attributes\n",
    "\n",
    "Attributes allow inspecting some properties of the tensors, e.g., `shape`, `dtype` and `device`.\n",
    "They can be valuable when creating neural networks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Task: Create tensors\n",
    "\n",
    "1. Create a tensor `A` $\\in \\mathbb{R}^{3 \\times 3}$ using the `torch.tensor()` function. Initialize it with the following values:\n",
    "$\n",
    "A = \\begin{bmatrix}\n",
    "1 & 2 & 3 \\\\\n",
    "4 & 5 & 6 \\\\\n",
    "7 & 8 & 9 \\\\\n",
    "\\end{bmatrix}\n",
    "$\n",
    "1. Create a tensor `B` $\\in \\mathbb{R}^{3 \\times 3}$ using the `torch.ones()` function.\n",
    "2. Create a tensor `C` with shape (3, 3) using the `torch.zeros()` function.\n",
    "3. Create a randomly initialized float tensor `D` with the same properties as `C`.\n",
    "4. Inspect the tensors' shape, datatype and device.\n",
    "\n",
    "> Hint: PyTorch tensors have a nice `str` representation and can be printed on the console."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tensor operations\n",
    "### Basic operations\n",
    "\n",
    "There are several operations that can be applied to tensors, including arithmetic, linear algebra, sampling, matrix manipulation, logical, conversion, moving device and more.\n",
    "For example, two tensors can be added using `+` or `torch.add`, concatenated using `torch.cat` or moved to a new device using `torch.to`.\n",
    "Conversion to float is done using `Tensor.float`, e.g., `a.float()`.\n",
    "\n",
    "Basic operations:\n",
    "* Add two tensors using `torch.add`.\n",
    "* Multiply two tensors element-wise using `torch.mul`.\n",
    "* Divide two tensors  element-wise using `torch.div`.\n",
    "\n",
    "> Note that most tensor operations from `torch` can directly be called on a tensor: `torch.add(a, b)` is equivalent to `a.add(b)`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task: Basic tensor operations\n",
    "\n",
    "1. Perform the following operations and print the results:\n",
    "\n",
    "    a) Add tensor `A` and tensor `B` element-wise.\n",
    "\n",
    "    b) Multiply tensor `A` and tensor `C` element-wise.\n",
    "\n",
    "    c) Divide tensor `A` by tensor `B` element-wise.\n",
    "\n",
    "    d) Divide tensor `B` by tensor `C` element-wise."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Advanced operations\n",
    "\n",
    "Advanced tensor operations include matrix multiplication, dot product, reshaping, and comparison operations.\n",
    "\n",
    "* `torch.matmul(tensor1, tensor2)`: Performs matrix multiplication between two tensors.\n",
    "* `torch.dot(tensor1, tensor2)`: Computes the dot product of two 1D tensors.\n",
    "* `torch.squeeze(tensor)`: Removes dimensions of size 1 from the shape of a tensor.\n",
    "* `torch.eq(tensor1, tensor2)`: Compares two tensors element-wise and returns a tensor of the same shape with boolean values.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task: Advanced tensor operations\n",
    "\n",
    "1. Create a tensor `D` with shape (3, 3) using the `torch.eye()` function.\n",
    "2. Perform the following operations and print the results:\n",
    "    1. Multiply tensor `A` and tensor `D` using the `torch.matmul()` function.\n",
    "    2. Calculate the matrix multiplication of tensor `A` and tensor `D` using the `torch.dot()` function.\n",
    "3. Create a tensor `E` with shape (2, 1) using the `torch.rand()` function.\n",
    "4. Print the size of tensor `E` and then squeeze its dimensions using the `torch.squeeze()` function. Print the size of the squeezed tensor.\n",
    "5. Compare tensor `A` and tensor `B` element-wise using the `torch.eq()` function and print the result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "intro-dl-ws-2024-25",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}