Apparent performance regression

[emeneker38a]

Commit 44e9085 appears to have introduced a performance regression when compared with v25.05.
The performance regression is present in the current master branch.

Attached is a simple benchmark I used to measure the performance difference between libpcap and the current pcapplusplus.

Platform details

((v25.05))> lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.5 LTS
Release:        22.04
Codename:       jammy

• GCC version: gcc version 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04.2)
• libpcap version: 1.10.1

Build and comparison

Performance between the master branch and the v25.05 tag were compared by building

cmake -DCMAKE_INSTALL_PREFIX=(realpath ~/local_install/pcapplusplus/) -DPCAPPP_BUILD_EXAMPLES=ON -B build .
cd build
make
make install

g++ -O2 benchmark.cpp -o benchmark -I ~/local_install/pcapplusplus/include/pcapplusplus -L ~/local_install/pcapplusplus/lib/ -lPcap++ -lPacket++ -lCommon++ -lpcap -lpthread

Performance comparison

Regressed performance beginning in commit 44e9085

Running libpcap benchmark:
Benchmarking PCAP file with libpcap: pcap/file.pcap
Reading all packets...

=== libpcap Benchmark Results ===
Packets read: 485760
Total bytes: 232226544 bytes (221.47 MB)
Time taken: 0.077176 seconds
Packets/second: 6294184.72
Throughput: 2869.65 MB/s

--------------------------------------------------

Running PcapPlusPlus benchmark:

=== PcapPlusPlus Benchmark Results ===
Packets read: 485760
Total bytes: 232226544 bytes (221.47 MB)
Time taken: 1.857491 seconds
Packets/second: **261514.05**
Throughput: **119.23 MB/s**

High performance in v25.05

Running libpcap benchmark:
Benchmarking PCAP file with libpcap: pcap/file.pcap
Reading all packets...

=== libpcap Benchmark Results ===
Packets read: 485760
Total bytes: 232226544 bytes (221.47 MB)
Time taken: 0.065889 seconds
Packets/second: 7372399.03
Throughput: 3361.24 MB/s

--------------------------------------------------

Running PcapPlusPlus benchmark:
=== PcapPlusPlus Benchmark Results === 
Packets read: 485760
Total bytes: 232226544 bytes (221.47 MB)
Time taken: 0.084271 seconds
Packets/second: 5764260.54
Throughput: 2628.05 MB/s

The attached benchmark.cpp is 80% Claude Sonnet written.

[emeneker38a]

It seems that uploading my benchmark.cpp fails, so the code is pasted below:

/*
This benchmark performs a basic comparison of speed between PcapPlusPlus and libpcap reading data from a pcap file without interpretation of the data read.

This benchmark can be built manually with a command like:
g++ -O2 benchmark.cpp -o benchmark -I </path/to>/pcapplusplus/include/pcapplusplus -L </path/to>/pcapplusplus/lib/ -lPcap++ -lPacket++ -lCommon++ -lpcap -lpthread

To run the benchmark, build the project and then execute the resulting binary with the path to a pcap file as an argument:

    ./benchmark <path_to_pcap_file>

This will run both PcapPlusPlus and libpcap benchmarks on the same file for comparison.

*/


#include <iostream>
#include <chrono>
#include <string>
#include <iomanip>
#include <PcapFileDevice.h>
#include <Packet.h>
#include <pcap.h>

void printUsage(const char* programName) {
    std::cout << "Usage: " << programName << " <pcap_file>" << std::endl;
    std::cout << "  pcap_file: Path to the PCAP file to benchmark" << std::endl;
}

void printBenchmarkResults(std::string name, size_t packetCount, size_t totalBytes, double seconds) {
    std::cout << std::endl;
    std::cout << "=== " << name << " Benchmark Results ===" << std::endl;
    std::cout << "Packets read: " << packetCount << std::endl;
    std::cout << "Total bytes: " << totalBytes << " bytes (" << std::fixed << std::setprecision(2) 
              << totalBytes / (1024.0 * 1024.0) << " MB)" << std::endl;
    std::cout << "Time taken: " << std::fixed << std::setprecision(6) << seconds << " seconds" << std::endl;
    std::cout << "Packets/second: " << std::fixed << std::setprecision(2) 
              << packetCount / seconds << std::endl;
    std::cout << "Throughput: " << std::fixed << std::setprecision(2) 
              << (totalBytes / (1024.0 * 1024.0)) / seconds << " MB/s" << std::endl;
}

int pcapplusplus_benchmark(std::string const& pcapFilePath) {
    pcpp::PcapFileReaderDevice reader(pcapFilePath);
    
    if (!reader.open()) {
        std::cerr << "Error: Cannot open PCAP file: " << pcapFilePath << std::endl;
        return 1;
    }

    auto startTime = std::chrono::high_resolution_clock::now();
    
    // Read all packets
    pcpp::RawPacket rawPacket;
    size_t packetCount = 0;
    size_t totalBytes = 0;
    
    while (reader.getNextPacket(rawPacket)) {
        packetCount++;
        totalBytes += rawPacket.getRawDataLen();
    }
    
    auto endTime = std::chrono::high_resolution_clock::now();
    auto duration = std::chrono::duration_cast<std::chrono::microseconds>(endTime - startTime);
    double seconds = duration.count() / 1000000.0;
    
    reader.close();
    
    printBenchmarkResults("PcapPlusPlus", packetCount, totalBytes, seconds);
    
    return 0;
}

int libpcap_benchmark(std::string const& pcapFilePath) {
    
    // Open pcap file using libpcap
    char errbuf[PCAP_ERRBUF_SIZE];
    pcap_t* handle = pcap_open_offline(pcapFilePath.c_str(), errbuf);
    
    if (handle == nullptr) {
        std::cerr << "Error: Cannot open PCAP file: " << pcapFilePath << std::endl;
        std::cerr << "libpcap error: " << errbuf << std::endl;
        return 1;
    }

    std::cout << "Benchmarking PCAP file with libpcap: " << pcapFilePath << std::endl;
    std::cout << "Reading all packets..." << std::endl;

    // Start timing
    auto startTime = std::chrono::high_resolution_clock::now();
    
    // Read all packets
    struct pcap_pkthdr* header;
    const u_char* pktData;
    size_t packetCount = 0;
    size_t totalBytes = 0;
    
    int result;
    while ((result = pcap_next_ex(handle, &header, &pktData)) >= 0) {
        packetCount++;
        totalBytes += header->caplen;
    }
    
    auto endTime = std::chrono::high_resolution_clock::now();
    auto duration = std::chrono::duration_cast<std::chrono::microseconds>(endTime - startTime);
    double seconds = duration.count() / 1000000.0;
    
    // Close the pcap handle
    pcap_close(handle);
    
    // Output results
    printBenchmarkResults("libpcap", packetCount, totalBytes, seconds);

    return 0;
}

int main(int argc, char* argv[]) {
    // Check command line arguments
    if (argc != 2) {
        printUsage(argv[0]);
        return 1;
    }
    std::string pcapFilePath = argv[1];

    // Run libpcap benchmark
    std::cout << "Running libpcap benchmark:" << std::endl;
    int result2 = libpcap_benchmark(pcapFilePath);

    std::cout << std::endl << std::string(50, '-') << std::endl << std::endl;

    // Run PcapPlusPlus benchmark
    std::cout << "Running PcapPlusPlus benchmark:" << std::endl;
    int result1 = pcapplusplus_benchmark(pcapFilePath);
    
    return 0;
}

[Dimi1010]

Thanks for bringing this to our attention.

I will look it over. I suspect something strange might be happeing with the buffering during reads?

[Dimi1010]

The issue was excessive allocator pressure introduced with the commit you specified. :/
I opened #2072 which should fix the issue.

[seladb]

@emeneker38a the commit you specified changed PcapPlusPlus to not rely on libpcap when reading pcap files. I guess it degraded the performance.

@Dimi1010 thanks for looking into it! 🙏