Overview

Contents

• Codec
  • Color Conversion
  • Sampling
  • Transform Coding
  • Quantization
• References
  • Class lectures
  • OpenCV cheat sheet
  • OpenCV matrix types
  • Example code
    • JPEG compression for a smooth image block (Fig 9.2)
    • JPEG encoding algorithm description
• Extra
  • Qt
  • OpenCV and Qt
  • Source control

 

JPEG Codec

Input

• Description: Original input RGB image
• Type: Mat3b matrix

Output

• Description: Compressed RGB image
• Type: Mat3b matrix

Encode

• Preprocess original image
  1. Color conversion RGB to YUV
  2. Chroma subsampling 4:2:0
• Iterate over each 8×8 block
  1. Apply 2D DCT transformation
  2. Quantize DCT coefficients
  4. ...

Decode

1. 2D IDCT on image blocks
2. Reverse 4:2:0 subsampling
3. Color conversion YUV to RGB

Note: The decoding process reverses Encoding steps, except the quantization because it is irreversible

 

Color Conversion

• RGB YUV
• YUV RGB

Input

• Type: Mat3b matrix

Output

• Type: Mat3b matrix

RGB to YUV

1. Matrix (4) RGB_YUV:

0.299	0.587	0.114
−0.14713	−0.28886	0.436
0.615	−0.51499	−0.10001

2. Compute matrix product, YUVColor, of RGB_YUV and RGBColor, for all.

for (row : rows) {
    for (col : columns) {
        RGBColor = RGBImage[row, col]
        YUVColor = RGB_YUV * RGBColor
        YUVImage[row, col] = YUVColor 
    }
}

YUV to RGB

• Matrix (5) YUV_RGB:

1.00	0	1.1398
1.00	−0.3946	−0.58060
1.00	2.03211	0

Resources

• Color converter
• YUV data range
• YUV adjustment
• Scale Step 4

 

Sampling

• 4:2:0 ratio
• 4:2:0 reverse

Input

• Description: YUV image
• Type: Mat3b matrix

Output

• Description: ?
• Type: Mat3b matrix

4:2:0 Ratio

4:2:0 Reverse

Resources

• Conversion with different sampling
• YUV subsample
• Chroma subsampling Wikipedia

 

Transform Coding

• DCT
• IDCT

Discrete Cosine Transform

DCT

• T is a DCT-matrix defined as:

    T[i, j]     = 1 / √(N)                              if i = 0
                = √(2/N) * cos((2j+1) * iπ) / 2N)       if i > 0

• For block size N=8, DCT-matrix T_8:

    T_8[i, j]     = 1 / (2*√(2))                        if i = 0
                  = 1/2 * cos((2j+1) * iπ) / 16)        if i > 0 

2D DCT

• 2D DCT is implemented by two consecutive 1D DCT matrix multiplications

    F(u, v) = T * f(i, j) * Transpose(T)

Code

• Transform.h defines 2D DCT

    template<typename _Tp>
    Mat1d Transform::dct2(const Mat_<_Tp> &matrix) {
        return mul(mul(Transform::DCT, matrix), Transform::DCT_T);
    }

• Codec.h applies transformation to each block

    using BlockDataType = short;
    using BlockTransform = std::function<Mat_<double>(Mat_<BlockDataType>)>;
     
    BlockTransform dct2 = Transform::dct2<BlockDataType>;
    block.transform(dct2);

• ImageBlock.h applies transformation to each channel within a block

template<typename _Tp, int cn>
void ImageBlock<_Tp, cn>::transform(BlockTransform transformFunc) {
    for (int c = 0; c < cn; c++) {
        this->at(c) = transformFunc(this->at(c));
    }
}

Inverse Discrete Cosine Transform

2D IDCT

• The equation of 2D inverse DCT based on:

    f(i, j) = Transpose(T) * F(u, v) * T

• Every 8×8 block is decoded with 2D IDCT

Note: DCT matrix is orthogonal: Transpose(T) = Inverse(T)

Resources

• Chapter 8 p. 30
• Concepts and applications

 

Quantization

Input

• Description: DCT coefficients F(u, v)
• Type: ImageBlock

Output

• Description: Quantized DCT coefficients F^(u, v)
• Type: ImageBlock

Implementation

• Quantization reduces high frequency DCT coefficients. Quantization formula:

    F^(u, v) = round( F(u, v) / Q(u, v) )

Resources

• Standard JPEG quantization tables p. 9; tables 9.1, 9.2

 

Extra

Qt

• The code uses the Qt framework. It is a cross-platform GUI framework for C++. You can open and run the code using an IDE called Qt creator. The code basically creates a window, add two buttons and labels. When the open button is clicked, it loads an image and displays it using the first label. When the convert button is clicked, it computes the Y channel of the loaded image and displays it using second label.

• Qt for beginners

• Download Qt

Note: remember to download Qt creator and a version of Qt, for example, Qt 5.12.

OpenCV with Qt

• macOS
  • Note: you may need to type in the following command after you successfully installed OpenCV. Basically, you need to create a link for opencv4.pc

    ln /usr/local/Cellar/opencv/4.0.1/lib/pkgconfig/opencv4.pc /usr/local/Cellar/opencv/4.0.1/lib/pkgconfig/opencv.pc

• Windows

• Ubuntu

Git Commands

• Clone this repository

    git clone https://csil-git1.cs.surrey.sfu.ca/A2-365/JPEG-Image-Compression.git

• Commit all changes and push

    git add .
    git commit -m "Message"
    git push

• Create a new branch

    git checkout -b "Branch_Name"
    git push --set-upstream origin "Branch_Name"