Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Performance #977

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Performance #977

wants to merge 4 commits into from

Commits on May 2, 2024

  1. trace cache operation 

    Can be enabled with LFS_CACHE_TRACE
it will output information with LFS_DEBUG
    @matthieu-c-tagheuer
    matthieu-c-tagheuer committed May 2, 2024
    Configuration menu
    Copy the full SHA
    fd31c12 View commit details
    Browse the repository at this point in the history

  2. lfs_dir_traverse: better try to use cache 

    cache trace show lots of possible caching in lfs_dir_traverse

cache recache for block 835 number 4938 line=1683
cache recache for block 882 number 4938 line=1683
cache recache for block 930 number 4938 line=1683
cache recache for block 978 number 4938 line=1683

test

config
    .read_size = 4,
    .prog_size = 16,
    .block_size = 4096,
    .block_count = (8*1024),
    .cache_size = 1024,
    .metadata_max = 1024,

        for (int i = 0; i <= 1000; i++) {
                char test_path[32];
                if (i > 100) {
                        sprintf(test_path, "testdir/test%d", i-100);
                        err = lfs_remove(&lfs, test_path);
                }

                sprintf(test_path, "testdir/test%d", i);
                uint32_t prng = i;
                lfs_size_t size = 4096 + (test_prng(&prng) & 3);

                lfs_file_t file;
                err = lfs_file_open(&lfs, &file, test_path,
                                LFS_O_WRONLY | LFS_O_CREAT | LFS_O_TRUNC);
                lfs_ssize_t res = lfs_file_write(&lfs, &file, chunk, size);
                err = lfs_file_close(&lfs, &file);
        }

before patch

top read operation (number of flash read size, number of flash op)
106060, 16976,
106060, 16976,
106560, 19645,
107084, 16977,
108820, 17624,

total read/write
total read=26174188 B write=4287328 B
total num read=1445383 op write=7732 op

after patch

15044, 41
15044, 41
15044, 41
15044, 41
16068, 42,

total read=18809040 B write=4287328 B
total num read=49477 op write=7732 op
    @matthieu-c-tagheuer
    matthieu-c-tagheuer committed May 2, 2024
    Configuration menu
    Copy the full SHA
    cb1f921 View commit details
    Browse the repository at this point in the history

  3. lfs_dir_getslice: reverse caching 

    cache trace show possible caching in lfs_dir_getslice

cache recache for block 83 number 23 line=790
cache recache for block 881 number 23 line=790
cache recache for block 929 number 23 line=790
cache recache for block 977 number 23 line=790
cache recache for block 128 number 24 line=790

For the same test than previous patch

before patch

top read operation (number of flash read size, number of flash op)
15044, 41
15044, 41
15044, 41
15044, 41
16068, 42

total read/write
total read=18809040 B write=4287328 B
total num read=49477 op write=7732 op

after patch
top read operation (number of flash read size, number of flash op)
15044, 41
15044, 41
15044, 41
15044, 41
16068, 42

total read/write
total read=18852728 B write=4287328 B
total num read=45678 op write=7732 op

cache recache for block 882 number 2 line=998
cache recache for block 911 number 2 line=998
cache recache for block 930 number 2 line=998
cache recache for block 959 number 2 line=998
cache recache for block 978 number 2 line=998
    @matthieu-c-tagheuer
    matthieu-c-tagheuer committed May 2, 2024
    Configuration menu
    Copy the full SHA
    30da07f View commit details
    Browse the repository at this point in the history

  4. do direct write if file data doesn't fit in cache

    @matthieu-c-tagheuer
    matthieu-c-tagheuer committed May 2, 2024
    Configuration menu
    Copy the full SHA
    9270114 View commit details
    Browse the repository at this point in the history