Skip to content

Optimize RefcountWithFlags::decRef() implementation #31

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 62 commits into
base: develop
Choose a base branch
from

Conversation

vinser52
Copy link

@vinser52 vinser52 commented Oct 20, 2022

This change is Reviewable

igchor and others added 30 commits September 7, 2022 07:58
It's implementation is mostly based on PosixShmSegment.

Also, extend ShmManager and ShmSegmentOpts to support this new
segment type.
After introducing file segment type, nameToKey_ does not provide
enough information to recover/remove segments on restart.

This commit fixes that by replacing nameToKey_ with nameToOpts_.

Previously, the Key from nameToKey_ map was only used in a single
DCHECK().
* New class MemoryTierCacheConfig allows to configure a memory tier.
  Setting tier size and location of a file for file-backed memory are
  supported in this initial implementation;
* New member, vector of memory tiers, is added to class CacheAllocatorConfig.
* New test suite, chelib/allocator/tests/MemoryTiersTest.cpp,
  demonstrates the usage of and tests extended config API.
to allow using new configureMemoryTiers() API with legacy behavior.

Move validation code for memory tiers to validate() method and convert
ratios to sizes lazily (on get)..
It wrongly assumed that the only possible segment type is
PosixSysV segment.
Now it's size is 8 bytes intead of 4.

Original CompressedPtr stored only some offset with a memory Allocator.
For multi-tier implementation, this is not enough. We must also store
tierId and when uncompressing, select a proper allocator.

An alternative could be to just resign from CompressedPtr but they
are leveraged to allow the cache to be mapped to different addresses on shared memory.

Changing CompressedPtr impacted CacheItem size - it increased from 32 to 44 bytes.
Without this fix removeCb called even in case when Item is moved between
tiers.
It fails because CentOS is EOL. We might want to consider
using CentOS Streams but for now, just remove it.

Right now, we rely on build-cachelib-centos workflow anyway.
Disabled test suite allocator-test-AllocatorTypeTest to skip sporadically failing tests.
…m#43)

Compensation results in ratios being different than originially specified.
Return a sum of sizes of each tier instead of just 1st tier's size.
byrnedj and others added 17 commits September 9, 2022 11:51
Hot queue iterator for 2Q. Will start at Hot queue and move to Warm queue if hot queue is exhausted. Useful for promotion semantics if using 2Q replacement. rebased on to develop and added some tests.
Initial support of NUMA bindings
The issue was caused by incorrect behaviour of the
CacheAllocator<CacheTrait>::tryEvictToNextMemoryTier method in case the
evicted item is expired. We cannot simply return a handle to it, but we need
to remove it from the access container and MM container.
Fix ReaperSkippingSlabTraversalWhileSlabReleasing test
Fix moveRegularItemWithSync and add tests
byrnedj pushed a commit that referenced this pull request Jul 23, 2023
Extend cachbench with value validation
@byrnedj byrnedj force-pushed the develop branch 2 times, most recently from 09d7bab to ebfca17 Compare May 21, 2024 13:24
@byrnedj byrnedj force-pushed the develop branch 4 times, most recently from 9ba4e79 to c1ff397 Compare June 26, 2024 18:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants