PyPCAPKit - Stream PCAP File Extractor

For any technical and/or maintenance information, please kindly refer to the Official Documentation.

The PyPCAPKit project is an open source Python program focus on PCAP parsing and analysis, which works as a stream PCAP file extractor. With support of DictDumper, it shall support multiple output report formats.

The whole project supports Python 3.6 or later.

About

PyPCAPKit is an independent open source library, using only DictDumper as its formatted output dumper.

There is a project called jspcapy works on pcapkit, which is a command line tool for PCAP extraction.

Note

The jspcapy project is deprecated and has been merged into the PyPCAPKit project as its CLI support since PyPCAPKit v0.8.0.

Unlike popular PCAP file extractors, such as Scapy, DPKT, PyShark, and etc, pcapkit uses streaming strategy to read input files. That is to read frame by frame, decrease occupation on memory, as well as enhance efficiency in some way.

Module Structure

In pcapkit, all files can be described as following eight parts.

• Interface (pcapkit.interface)

  User interface for the pcapkit library, which standardises and simplifies the usage of this library.

• Foundation (pcapkit.foundation)

  Synthesises file I/O and protocol analysis, coordinates information exchange in all network layers, as well as provides the foundamental functions for pcapkit.

• Protocols (pcapkit.protocols)

  Collection of all protocol family, with detailed implementation and methods.

• Utilities (pcapkit.utilities)

  Auxiliary functions and tools for pcapkit.

• CoreKit (pcapkit.corekit)

  Core utilities for pcapkit implementation, mainly for internal data structure and processing.

• ToolKit (pcapkit.toolkit)

  Auxiliary tools for pcapkit to support the multiple extraction engines with a unified interface.

• DumpKit (pcapkit.dumpkit)

  File output formatters for pcapkit.

• Constants (pcapkit.const)

  Constant enumerations used in pcapkit for protocol family extraction and representation.

Engine Comparison

Due to the general overhead of pcapkit, its extraction procedure takes around 0.0008 seconds per packet, which is already impressive but not enough comparing to other popular extration engines availbale on the market. Thus pcapkit introduced alternative extractionengines to accelerate this procedure. By now pcapkit supports Scapy, DPKT, and PyShark.

Test Environment

Operating System	macOS Monterey
Processor Name	Intel Core i7
Processor Speed	2.6 GHz
Total Number of Cores	6
Memory	16 GB

Test Results

Engine	Performance (seconds per packet)
dpkt	0.00006832083066304525
scapy	0.0002489296595255534
pcapkit	0.0008274253209431966
pyshark	0.039607704480489093

Installation

Note

pcapkit supports Python versions since 3.6.

Simply run the following to install the current version from PyPI:

pip install pypcapkit

Or install the latest version from the gi repository:

git clone https://github.com/JarryShaw/PyPCAPKit.git
cd pypcapkit
pip install -e .
# and to update at any time
git pull

And since pcapkit supports various extraction engines, and extensive plug-in functions, you may want to install the optional ones:

# for DPKT only
pip install pypcapkit[DPKT]
# for Scapy only
pip install pypcapkit[Scapy]
# for PyShark only
pip install pypcapkit[PyShark]
# and to install all the optional packages
pip install pypcapkit[all]
# or to do this explicitly
pip install pypcapkit dpkt scapy pyshark

For CLI usage, you will need to install the optional packages:

pip install pypcapkit[cli]
# or explicitly...
pip install pypcapkit emoji