Skip to content

Testing

Jump to bottom Edit New page
Daniel Markstedt edited this page Aug 15, 2025 · 21 revisions

Netatalk Testing

At present, Netatalk has two test automation suites, both written in pure C:

  • afpd component integration tests (executed during build)
  • afptest (testsuite) system integration test suite

The former can be run stateless through the Meson test runner.

The latter requires a test environment with a correctly configured and running netatalk instance, but provides more advanced testing options.

The component integration test code is located in test/afpd, and the system integration tests in test/testsuite.

Testing policy

Any changeset to the Netatalk AFP server components should include updates to the test suites, when needed. Any novel test failures should be fixed, and new AFP functionality should be covered by new tests.

It is strongly recommended to run the tests locally before submitting a PR.

Run the afpd stateless tests during build

In order to run the tests, build Netatalk with tests enabled, then run the meson test target from within the build directory:

meson setup build -Dwith-tests=true
meson compile -C build
cd build
meson test

Passing results should look something like this:

ninja: Entering directory `/home/atalk/devel/netatalk/build'
ninja: no work to do.
1/2 afpd integration tests - setup        OK              0.42s
2/2 afpd integration tests - run          OK              1.98s

Ok:                 2   
Expected Fail:      0   
Fail:               0   
Unexpected Pass:    0   
Skipped:            0   
Timeout:            0   

Full log written to /home/atalk/devel/netatalk/build/meson-logs/testlog.txt

Note that the suite contains multiple tests. See test.c for the full list of assertions.

Docker - Run testsuite in Container for software changes

The prefered method to test Netatalk software changes is using Docker. The Netatalk codebase includes two Dockerfiles for testing; testsuite_alp.Dockerfile and testsuite_deb.Dockerfile

The two Dockerfiles use different base images: Alpine Linux and Debian Linux, respectively. There are otherwise no notable differences bewteen the two.

After installing Docker, or Docker Desktop;

Build Container Image: docker build --no-cache -f testsuite_alp.Dockerfile -t netatalk-test:latest .

Initialize Container and run Netatalk:

docker run -d --network host --cap-add=NET_ADMIN \
--volume "<path to local folder>:/mnt/afpshare" \
--volume "<path to local folder>:/mnt/afpbackup" \
--env AFP_USER=test --env AFP_PASS=test --env AFP_GROUP=test --env INSECURE_AUTH=true \
--env SHARE_NAME='File Sharing' \
--name netatalk-test netatalk-test:latest

The container should now be up with Netatalk running inside.

Read Container Logs: docker logs netatalk-test

Attach to Container and run afp_lantest Test:

docker exec -it netatalk-test /usr/local/bin/afp_lantest -h localhost -p 548 -u test -w test -s "File Sharing" -n 2

Stop Container: docker stop netatalk-test

Start Container: docker start netatalk-test netatalk-test

List Containers: docker ps -a

Delete Container Instance: docker rm netatalk-test

Purge Container Image: docker image rm netatalk-test

Purge ALL Images (including any non-netatalk images): docker rmi $(docker images -q)

Alternatively, rather than attaching to the Container - Initilise, start Netatalk, run test interactively and shutdown:

docker run --rm -it --network host --cap-add=NET_ADMIN \
--volume "<path to local folder>:/mnt/afpshare" \
--volume "<path to local folder>:/mnt/afpbackup" \
--env AFP_USER=test --env AFP_PASS=test --env AFP_GROUP=test --env INSECURE_AUTH=true \
--env SHARE_NAME='File Sharing' \
--env TESTSUITE=lan \
--env TEST_FLAGS="-n 2" \
--name netatalk-test netatalk-test:latest

The above example runs the afp_lantest (TESTSUITE=lan), and passes additional arguments (-n 2).

Other tests can be run with; TESTSUITE=login for afp_logintest, TESTSUITE=spectest for afp_spectest, or TESTSUITE=speed for afp_speedtest.

Run the Container in developer mode

Running afp_lantest with IO performance monitoring enabled (Linux only), requires elevated privileges (--privileged) and permissive proc mount inside the container (IO_MONITORING=1);

docker run --rm -it --privileged --network host --cap-add=NET_ADMIN \
--volume "<path to local folder>:/mnt/afpshare" \
--volume "<path to local folder>:/mnt/afpbackup" \
--env AFP_USER=test --env AFP_PASS=test --env AFP_GROUP=test --env INSECURE_AUTH=true \
--env SHARE_NAME='File Sharing' \
--env IO_MONITORING=1 \
--env TESTSUITE=lan \
--env TEST_FLAGS="-n 2" \
--name netatalk-test netatalk-test:latest

Local - Run testsuite natively

This can be used to test distribution specific issues, or for testing network performance between different client and server hosts.

Build netatalk with the testsuite enabled

meson setup build -Dwith-testsuite=true
meson compile -C build
meson install -C build

Note: Installation may need root privileges.

Stop netatalk if running, and then configure the environment.

Create users and groups

We only set up the requirements for the tier 1 spectests, namely those that can run remotely, without test suite access to the host file system.

Root privileges are required to execute these commands.

Debian Linux

The following steps were run on Debian Linux, but may work on other Linux flavors.

groupadd -f afpusers
useradd -G afpusers atalk1
useradd -G afpusers atalk2
echo "atalk1:afpafp" | chpasswd
echo "atalk2:afpafp" | chpasswd

FreeBSD

pw groupadd afpusers
pw useradd atalk1 -m -G afpusers
pw useradd atalk2 -m -G afpusers
passwd atalk1
passwd atalk2

OpenBSD

groupadd afpusers
useradd -m -G afpusers atalk1
useradd -m -G afpusers atalk2
passwd atalk1
passwd atalk2

Create test volume dirs

The paths are arbitrary, but need to match up with afp.conf below.

mkdir -p /tmp/afptest1
mkdir -p /tmp/afptest2
chown atalk1:afpusers /tmp/afptest1
chown atalk1:afpusers /tmp/afptest2
chmod 2775 /tmp/afptest1
chmod 2775 /tmp/afptest2

Configure Netatalk

Modify the netatalk config files with test users, volumes, and UAMs.

cat <<AFP > /usr/local/etc/afp.conf
[Global]
uam list = uams_clrtxt.so uams_guest.so

[test1]
appledouble = ea
path = /tmp/afptest1
valid users = @afpusers

[test2]
appledouble = ea
path = /tmp/afptest2
valid users = @afpusers
AFP
cat <<EXT > /usr/local/etc/extmap.conf
.         "????"  "????"      Unix Binary                    Unix                      application/octet-stream
.doc      "WDBN"  "MSWD"      Word Document                  Microsoft Word            application/msword
.pdf      "PDF "  "CARO"      Portable Document Format       Acrobat Reader            application/pdf
EXT

Restart netatalk

Once all configurations are done, start the netatalk daemon and get ready to run the tests.

spectest (AFP Specification Conformance)

The AFP spectest test suite ensures there are no breakages in AFP specification conformance. The test runner binary is called afp_spectest.

The IP address or domain name you pass to the -h argument is the netatalk host to test against. It can be a remote machine, or the localhost.

When run against a remote host, only the so-called tier 1 tests are executed.

afp_spectest -h 192.168.0.2 -u atalk1 -d atalk2 -w afpafp -s test1 -S test2

When run on localhost, and you pass the local file system path to the primary shared volume with the -c parameter, the so-called tier 2 tests are executed as well. These are tests are involve making local file system modifications to set up certain test preconditions.

afp_spectest -h localhost -u atalk1 -d atalk2 -w afpafp -s test1 -S test2 -c /tmp/afptest1

For additional instructions, see the afp_spectest man page.

The majority of the spec tests are also running in the GitHub CI workflow, the only exceptions being a handful that require special setup that is cumbersome to replicate or take too long time to run.

Inspect implemented test functions

To get an authoritative list of test methods that are actually implemented, e.g. not commented out, you can inspect the compiled test binaries.

Build the netatalk source code with testsuite enabled, then run the following commands.

cd build/test/testsuite
nm afp_logintest afp_spectest | cut -d " " -f 3 | egrep "^test[[:digit:]]+" | sort -n -k 1.5

This should give you an exhaustive list of tests.

afp_lantest (Netatalk Performance Testing)

As above, you will need to build Netatalk with the testsuite (-Dwith-testsuite=true).

afp_lantest is a comprehensive AFP (Apple Filing Protocol) protocol performance testing tool designed to benchmark various aspects of AFP servers. The tool runs a series of tests that measure file operations, directory traversal, and caching efficiency.

It includes both traditional file system benchmarks and specialized cache-focused tests that highlight directory cache validation and probabilistic validation features.

$ afp_lantest -n 2 -7 -h 127.0.0.1 -p 548 -u test -w test -s 'File Sharing'
Connecting to host 127.0.0.1:548
IO monitoring: /proc_io is available
Looking for cnid_dbd processes with -u test in command line
Found cnid_dbd process: PID 40
Looking for afpd processes owned by user 'test' (UID: 1000)
Found privilege-dropped afpd process: PID 36
IO monitoring enabled (afpd: 36, cnid_dbd: 40)

Run 1 => Open, stat and read 512 bytes from 1000 files [8,000 AFP ops]        1923 ms
        IO Operations; afpd: 6000 READs, 7002 WRITEs | cnid_dbd: 0 READs, 2 WRITEs
Run 1 => Writing one large file [103 AFP ops]                                  136 ms for 100 MB (avg. 771 MB/s)
        IO Operations; afpd: 0 READs, 299 WRITEs | cnid_dbd: 0 READs, 0 WRITEs
Run 1 => Reading one large file [102 AFP ops]                                   39 ms for 100 MB (avg. 2688 MB/s)
        IO Operations; afpd: 100 READs, 100 WRITEs | cnid_dbd: 0 READs, 0 WRITEs
Run 1 => Locking/Unlocking 10000 times each [20,000 AFP ops]                   799 ms
        IO Operations; afpd: 0 READs, 20000 WRITEs | cnid_dbd: 0 READs, 0 WRITEs
Run 1 => Creating dir with 2000 files [4,000 AFP ops]                         4061 ms
        IO Operations; afpd: 2000 READs, 10005 WRITEs | cnid_dbd: 4 READs, 6150 WRITEs
Run 1 => Enumerate dir with 2000 files [~51 AFP ops]                           637 ms
        IO Operations; afpd: 1960 READs, 49 WRITEs | cnid_dbd: 0 READs, 0 WRITEs
Run 1 => Deleting dir with 2000 files [2,000 AFP ops]                         3176 ms
        IO Operations; afpd: 4000 READs, 4004 WRITEs | cnid_dbd: 2 READs, 6104 WRITEs
Run 1 => Create directory tree with 1000 dirs [1,110 AFP ops]                 1885 ms
        IO Operations; afpd: 0 READs, 4445 WRITEs | cnid_dbd: 4 READs, 2351 WRITEs
Run 1 => Directory cache hits (100 dirs + 1000 files) [11,000 AFP ops]        3625 ms
        IO Operations; afpd: 10000 READs, 11100 WRITEs | cnid_dbd: 0 READs, 100 WRITEs
Run 1 => Mixed cache operations (create/stat/enum/delete) [820 AFP ops]       1134 ms
        IO Operations; afpd: 820 READs, 1621 WRITEs | cnid_dbd: 0 READs, 1201 WRITEs
Run 1 => Deep path traversal (nested directory navigation) [3,500 AFP ops]     965 ms
        IO Operations; afpd: 2500 READs, 3550 WRITEs | cnid_dbd: 0 READs, 50 WRITEs
Run 1 => Cache validation efficiency (metadata changes) [30,000 AFP ops]      8529 ms
        IO Operations; afpd: 30000 READs, 30100 WRITEs | cnid_dbd: 0 READs, 100 WRITEs
Run 2 => Open, stat and read 512 bytes from 1000 files [8,000 AFP ops]        2453 ms
        IO Operations; afpd: 6000 READs, 7002 WRITEs | cnid_dbd: 0 READs, 2 WRITEs
Run 2 => Writing one large file [103 AFP ops]                                   87 ms for 100 MB (avg. 1205 MB/s)
        IO Operations; afpd: 0 READs, 299 WRITEs | cnid_dbd: 0 READs, 0 WRITEs
Run 2 => Reading one large file [102 AFP ops]                                   36 ms for 100 MB (avg. 2912 MB/s)
        IO Operations; afpd: 100 READs, 100 WRITEs | cnid_dbd: 0 READs, 0 WRITEs
Run 2 => Locking/Unlocking 10000 times each [20,000 AFP ops]                   769 ms
        IO Operations; afpd: 0 READs, 20000 WRITEs | cnid_dbd: 0 READs, 0 WRITEs
Run 2 => Creating dir with 2000 files [4,000 AFP ops]                         3442 ms
        IO Operations; afpd: 2000 READs, 10005 WRITEs | cnid_dbd: 7 READs, 6140 WRITEs
Run 2 => Enumerate dir with 2000 files [~51 AFP ops]                           805 ms
        IO Operations; afpd: 1960 READs, 49 WRITEs | cnid_dbd: 0 READs, 0 WRITEs
Run 2 => Deleting dir with 2000 files [2,000 AFP ops]                         2475 ms
        IO Operations; afpd: 4000 READs, 4003 WRITEs | cnid_dbd: 4 READs, 6180 WRITEs
Run 2 => Create directory tree with 1000 dirs [1,110 AFP ops]                 1701 ms
        IO Operations; afpd: 0 READs, 4442 WRITEs | cnid_dbd: 2 READs, 2267 WRITEs
Run 2 => Directory cache hits (100 dirs + 1000 files) [11,000 AFP ops]        2962 ms
        IO Operations; afpd: 10000 READs, 11100 WRITEs | cnid_dbd: 0 READs, 100 WRITEs
Run 2 => Mixed cache operations (create/stat/enum/delete) [820 AFP ops]        598 ms
        IO Operations; afpd: 820 READs, 1621 WRITEs | cnid_dbd: 2 READs, 1242 WRITEs
Run 2 => Deep path traversal (nested directory navigation) [3,500 AFP ops]     796 ms
        IO Operations; afpd: 2500 READs, 3550 WRITEs | cnid_dbd: 0 READs, 50 WRITEs
Run 2 => Cache validation efficiency (metadata changes) [30,000 AFP ops]      8431 ms
        IO Operations; afpd: 30000 READs, 30100 WRITEs | cnid_dbd: 0 READs, 100 WRITEs

Netatalk Lantest Results (Averages and standard deviations (±) for all tests, across 2 iterations (default))
============================================================================================================

Test                                                                Time_ms  Time± AFPD_R AFPD_R± AFPD_W AFPD_W± CNID_R CNID_R± CNID_W CNID_W±   MB/s
------------------------------------------------------------------ -------- ------ ------ ------- ------ ------- ------ ------- ------ ------- ------
Open, stat and read 512 bytes from 1000 files [8,000 AFP ops]          2188  374.8   6000     0.0   7002     0.0      0     0.0      2     0.0      0
Writing one large file [103 AFP ops]                                    111   34.7      0     0.0    299     0.0      0     0.0      0     0.0    900
Reading one large file [102 AFP ops]                                     37    2.2    100     0.0    100     0.0      0     0.0      0     0.0   2702
Locking/Unlocking 10000 times each [20,000 AFP ops]                     784   21.2      0     0.0  20000     0.0      0     0.0      0     0.0      0
Creating dir with 2000 files [4,000 AFP ops]                           3751  437.7   2000     0.0  10005     0.0      5     2.2   6145     7.1      0
Enumerate dir with 2000 files [~51 AFP ops]                             721  118.8   1960     0.0     49     0.0      0     0.0      0     0.0      0
Deleting dir with 2000 files [2,000 AFP ops]                           2825  495.7   4000     0.0   4003     1.0      3     1.4   6142    53.7      0
Create directory tree with 1000 dirs [1,110 AFP ops]                   1793  130.1      0     0.0   4443     2.2      3     1.4   2309    59.4      0
Directory cache hits (100 dirs + 1000 files) [11,000 AFP ops]          3293  468.8  10000     0.0  11100     0.0      0     0.0    100     0.0      0
Mixed cache operations (create/stat/enum/delete) [820 AFP ops]          866  379.0    820     0.0   1621     0.0      2     0.0   1221    29.0      0
Deep path traversal (nested directory navigation) [3,500 AFP ops]       880  119.5   2500     0.0   3550     0.0      0     0.0     50     0.0      0
Cache validation efficiency (metadata changes) [30,000 AFP ops]        8480   69.3  30000     0.0  30100     0.0      0     0.0    100     0.0      0
------------------------------------------------------------------ -------- ------ ------ ------- ------ ------- ------ ------- ------ ------- ------
Sum of all AFP OPs = 80686                                            25729         57380          92272             13          16069               

Aggregates Summary:
-------------------
Average Time per AFP OP: 0.319 ms
Average AFPD Reads per AFP OP: 0.711
Average AFPD Writes per AFP OP: 1.144

Result Columns

Time(ms) = Test runtime in milliseconds
Time±    = Test runtime standard deviation
AFPD_R   = afpd process IO Read operations
AFPD_R±  = afpd process IO Read operation standard deviation
AFPD_W   = afpd process IO Write operations
AFPD_W±  = afpd process IO Write operation standard deviation
CNID_*   = IO measurements for the cnid_dbd process (optional)

IO Monitoring

The afp_lantest IO Monitoring capability is an analysis mechanism to enable Netatalk developers to validate end-to-end performance across the Netatalk codebase by quantifying AFP Client Operations -to- Netatalk Server Storage IO Operations.

IO Monitoring requires the afp_lantest client to be run on the same host as the Netatalk server, so it can watch the afpd server processes disk IO via the Linux proc virtual filesystem.

NB; Distributions running systemd do not allow existing /proc filesystem to be remounted with the required permissions, therefore a secondary proc mount is required at /proc_io by default.

Example - Run as root to allow 'root' GID access to /proc_io for other UIDs (when running afp_lantest as user other than root, update gid accordingly): mkdir -p /proc_io && mount -t proc -o hidepid=0,gid=0 proc /proc_io

IO Monitoring also works with the Netatalk test Docker container (see testsuite_alp.Dockerfile), to enable simple performance and regression testing.

DSI Quantum Size Impact

The DSI (Data Stream Interface) protocol default quantum size is 1MB (1048576 bytes), which affects how many read/write operations are performed for large data transfers. Ie, for a 1MB file, with the default 1MB quantum, only 1 AFP operation and 1 disk IO operation is required to read or write 1MB.

AFP Operation Counts Analysis

The number of AFP (Apple Filing Protocol) operations performed by each test in lantest.c (afp_lantest).

The AFP operation counts are measured between starttimer() and stoptimer() calls to understand the actual workload each test generates.

1. TEST_OPENSTATREAD - Open, stat and read 512 bytes from 1000 files

Total AFP Operations: 8,000

For each of the 1000 files, the test performs:

  • 1 is_there() call (FPGetFileDirParams)
  • 2 additional FPGetFileDirParams calls
  • 1 FPOpenFork operation
  • 2 FPGetForkParam operations
  • 1 FPRead operation (512 bytes)
  • 1 FPCloseFork operation

Operations per file: 8

  • Open operations: 1
  • Stat operations (GetFileDirParams + GetForkParam): 5
  • Read operations: 1
  • Close operations: 1

Total: 8 × 1000 files = 8,000 AFP operations

2. TEST_WRITE100MB - Writing one large file

Total AFP Operations: 103

  • 1 FPCreateExt (create file)
  • 1 FPOpenFork
  • 100 FPWrite operations (100MB @1MB quantum)
  • 1 FPCloseFork

Total: 103 AFP operations (100MB @1MB quantum)

3. TEST_READ100MB - Reading one large file

Total AFP Operations: 102

  • 1 FPOpenFork
  • 100 FPRead operations (100MB @1MB quantum)
  • 1 FPCloseFork

Total: 102 AFP operations (100MB @1MB quantum)

4. TEST_LOCKUNLOCK - Locking/Unlocking 10000 times each

Total AFP Operations: 20,000

  • 10,000 FPByteRangeLock operations (lock)
  • 10,000 FPByteRangeLock operations (unlock)

Total: 20,000 AFP operations

5. TEST_CREATE2000FILES - Creating dir with 2000 files

Total AFP Operations: 4,000

  • 1 FPCreateDir (Directory creation outside timed section)
  • 2,000 FPCreateExt (one per file)
  • 2,000 FPGetFileDirParams (one per file after creation)

Total: 4,000 AFP operations

6. TEST_ENUM2000FILES - Enumerate dir with 2000 files

Total AFP Operations: ~51

  • ~51 FPEnumerate operations (based on response packet size)
    • Each FPEnumerate can return ~40 entries
    • 2000 files ÷ 40 per call ≈ 50 calls
    • Plus 1 final call to confirm end

This test demonstrates the efficiency of enumeration through batching.

7. TEST_DELETE2000FILES - Deleting dir with 2000 files

Total AFP Operations: 2,000

  • 2,000 FPDelete operations (one per file)
  • 1 FPDelete (Directory deletion outside timed section)

Total: 2,000 AFP operations

8. TEST_CREATEDIR - Create directory tree with 1000 dirs

Total AFP Operations: 1,110

Creates nested structure: 10 × 10 × 10 directories + 10 top-level

  • 10 FPCreateDir (level 1)
  • 100 FPCreateDir (level 2: 10 × 10)
  • 1,000 FPCreateDir (level 3: 10 × 10 × 10)

Total: 10 + 100 + 1,000 dirs = 1,110 AFP operations

9. TEST_DIRCACHE_HITS - Directory cache hits

Total AFP Operations: 11,100

  • 100 FPCreateDir operations (10 × 10 directories)
  • 1,000 FPCreateExt operations (100 dirs × 10 files per dir)
  • 10,000 FPGetFileDirParams operations (100 iterations × 100 dirs)

Total: 100 + 1,000 + 10,000 dirs = 1,110 AFP operations

10. TEST_DIRCACHE_MIXED - Mixed cache operations

Total AFP Operations: 820

For each of 10 iterations (with 20 files each):

  • 1 FPCreateDir
  • 10 FPCreateExt (files)
  • 20 FPGetFileDirParams (2 × 10 stats)
  • 41 FPEnumerate (~40 entries per call + 1)
  • 10 FPDelete (files)

Operations per iteration: 82 Total for 10 iterations: 820 AFP operations

11. TEST_DIRCACHE_TRAVERSE - Deep path traversal

Total AFP Operations: 3,500

Creates 20-level deep directory structure with 50 files in the deepest directory, then performs 50 traversals:

  • Initial Directory creation: 20 FPCreateDir operations (outside timed section)
  • Initial File creation: 50 FPCreateFile operations (outside timed section)

For each of 50 traversals (within timed section):

  • 20 FPGetFileDirParams (navigating down 20 directory levels using is_there() call)
  • 50 FPGetFileDirParams (accessing all 50 files in the deepest directory)

Operations per traversal: 70 Total: 50 traversals × 70 operations = 3,500 AFP operations

12. TEST_CACHE_VALIDATION - Cache validation efficiency

Total AFP Operations: 30,000

  • 100 FPCreateDir
  • 1,000 FPCreateExt (10 files per dir)
  • 10,000 FPGetFileDirParams (validation before modifications)
  • 1,000 FPSetFileParms (modify timestamps)
  • 1,000 FPWrite operations (trigger metadata changes)
  • 10,000 FPGetFileDirParams (revalidation after changes)
  • 1,000 FPRename operations (test cache coherency)
  • 6,000 FPGetFileDirParams (verify renames - partial dirs)

Total: ~30,000 AFP operations

Test Key Insights

  1. Enumeration Efficiency: The enumeration test (TEST_ENUM2000FILES) performs only ~51 operations for 2000 files due to batching, making it the most efficient operation per item.

  2. Read/Write Quantum Impact: With 1MB quantum size, the 100MB file tests perform only ~100 operations each, making them very efficient for bulk data transfer.

  3. Cache Validation Overhead: The cache validation test performs the most operations (30,000), making it ideal for stress testing and cache coherency validation.

  4. Lock Operations: The lock/unlock test provides a pure protocol overhead measurement with 20,000 rapid operations that don't involve actual data transfer.

  5. Directory Operations: Directory creation and traversal tests (1,110-1,500 operations) provide good measurements for metadata operation performance.

These operation counts help identify which tests are best suited for:

  • Protocol efficiency testing: Enumeration, large file I/O
  • Stress testing: Cache validation, lock/unlock, open/stat/read
  • Metadata performance: Directory operations, cache hits
  • Real-world simulation: Mixed operations, open/stat/read patterns
Add a custom footer

Resources

OS Specific Guides

Tech Notes

Retro AFP

Development

Clone this wiki locally