mirror of
http://github.com/valkey-io/valkey
synced 2024-11-23 03:33:28 +00:00
d4e7ffb38c
Background: Following the upgrade to jemalloc 5.2, there was a test that used to be flaky and started failing consistently (on 32bit), so we disabled it (see #9645). This is a test that i introduced in #7289 when i attempted to solve a rare stagnation problem, and it later turned out i failed to solve it, ans what's more i added a test that caused it to be not so rare, and as i mentioned, now in jemalloc 5.2 it became consistent on 32bit. Stagnation can happen when all the slabs of the bin are equally utilized, so the decision to move an allocation from a relatively empty slab to a relatively full one, will never happen, and in that test all the slabs are at 50% utilization, so the defragger could just keep scanning the keyspace and not move anything. What this PR changes: * First, finally in jemalloc 5.2 we have the count of non-full slabs, so when we compare the utilization of the current slab, we can compare it to the average utilization of the non-full slabs in our bin, instead of the total average of our bin. this takes the full slabs out of the game, since they're not candidates for migration (neither source nor target). * Secondly, We add some 12% (100/8) to the decision to defrag an allocation, this is the part that aims to avoid stagnation, and it's especially important since the above mentioned change can get us closer to stagnation. * Thirdly, since jemalloc 5.2 adds sharded bins, we take into account all shards (something that's missing from the original PR that merged it), this isn't expected to make any difference since anyway there should be just one shard. How this was benchmarked. What i did was run the memefficiency test unit with `--verbose` and compare the defragger hits and misses the tests reported. At first, when i took into consideration only the non-full slabs, it got a lot worse (i got into stagnation, or just got a lot of misses and a lot of hits), but when i added the 10% i got back to results that were slightly better than the ones of the jemalloc 5.1 branch. i.e. full defragmentation was achieved with fewer hits (relocations), and fewer misses (keyspace scans). |
||
|---|---|---|
| .. | ||
| bin | ||
| build-aux | ||
| doc | ||
| include | ||
| m4 | ||
| msvc | ||
| scripts | ||
| src | ||
| test | ||
| .appveyor.yml | ||
| .autom4te.cfg | ||
| .cirrus.yml | ||
| .gitattributes | ||
| .gitignore | ||
| .travis.yml | ||
| autogen.sh | ||
| ChangeLog | ||
| config.stamp.in | ||
| configure | ||
| configure.ac | ||
| COPYING | ||
| INSTALL.md | ||
| jemalloc.pc.in | ||
| Makefile.in | ||
| README | ||
| run_tests.sh | ||
| TUNING.md | ||
| VERSION | ||
jemalloc is a general purpose malloc(3) implementation that emphasizes fragmentation avoidance and scalable concurrency support. jemalloc first came into use as the FreeBSD libc allocator in 2005, and since then it has found its way into numerous applications that rely on its predictable behavior. In 2010 jemalloc development efforts broadened to include developer support features such as heap profiling and extensive monitoring/tuning hooks. Modern jemalloc releases continue to be integrated back into FreeBSD, and therefore versatility remains critical. Ongoing development efforts trend toward making jemalloc among the best allocators for a broad range of demanding applications, and eliminating/mitigating weaknesses that have practical repercussions for real world applications. The COPYING file contains copyright and licensing information. The INSTALL file contains information on how to configure, build, and install jemalloc. The ChangeLog file contains a brief summary of changes for each release. URL: http://jemalloc.net/