* feat: Support non-root paths for json.merge
Pass path argument and rewrite the JSON.MERGE code similar to OpToggle
or other mutating functions. Currently works only with --experimental_flat_json=false.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* chore: comments
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
1. Add background offloading stats
2. remove direct_fd override - helio is already updated with default=false, so it's not needed anymore.
3. remove redundant tiered_storage_memory_margin flag
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
json.merge would throw an exception when the json object did not contain the element to replace because RecursiveMerge functions used &dest->at(k_v.key()) which threw the exception. Remove RecursiveMerge completely and use the one implemented in jsoncons lib.
* add test
* replace RecursiveMerge with mergepatch::apply_merge_patch
* add exception handling for that flow
---------
Signed-off-by: kostas <kostas@dragonflydb.io>
The recent changes of the serialization_max_chunk_size set to 1 for extreme testing increased the running time of the tests causing them sometimes to timeout.
* increase timeout on reg tests from 40 to 50
---------
Signed-off-by: kostas <kostas@dragonflydb.io>
DastTable::Traverse is error prone when the callback passed preempts because the segment might change. This is problematic and we need atomicity while traversing segments with preemption. The fix is to add Traverse in DbSlice and protect the traversal via ThreadLocalMutex.
* add ConditionFlag to DbSlice
* add Traverse in DbSlice and protect it with the ConditionFlag
* remove condition flag from snapshot
* remove condition flag from streamer
---------
Signed-off-by: kostas <kostas@dragonflydb.io>
* feat: stabilize non-coordinated omission mode
1. Our latency/RPS computations were off because we started measuring before drivers
started running. Now, Run/Start phases are separated, so the start time is measured more precisely
(after the start phase)
2. Introduced progress per connection - one of my discoveries is that driver
connections progress with differrent pace when running in coordinated omission mode.
This can reach x5 speed differrences. Now we measure and output fastest/slowest progress.
3. Coordinated omission is great when the Server Under Test is able to sustain the required RPS.
But if the actual RPS is lower than the one is sent than the final latencies will be infinitely high.
We fix it by introducing self-adjusting sleep interval, so if the actual RPS is lower
we will increase the interval to be closer to the actual RPS.
Show p99 latency and maximum pending requests per connection.
Co-authored-by: Shahar Mike <chakaz@users.noreply.github.com>
Signed-off-by: Roman Gershman <romange@gmail.com>
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Signed-off-by: Roman Gershman <romange@gmail.com>
Co-authored-by: Shahar Mike <chakaz@users.noreply.github.com>
* chore: Support setting the value of `replica-priority`
This PR adds a small refactor to the way we set and get config names
which have dashes (`-`) and underscores (`_`).
Until now, words were separated by underscores because this is how our
flags library (absl) works. However, this is incompatible with Valkey,
which uses dashes as a word separator.
Once merged, we will support both underscores and dashes in config
names, but will only return the name with dashes. **This is a behavior
change**.
We're doing this in order to be compatible with `replica-priority` and
possibly other config names that Valkey uses.
* Flag restore
* normalize to '_'
* fix: reenable macos builds
Also, add debug function that prints local state if deadlocks occure.
* fix: free cold memory for non-cache mode as well
* chore: disable FreeMemWithEvictionStep again
Because it heavily affects the performance when performing evictions.
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
1. Fully support tiered_experimental_cooling for all operations
2. Offset cool storage usage when computing memory pressure situations in Hearbeat.
3. Introduce realtime entry counting per db_slice and provide DCHECK to verify it vs the old approach.
Later we will switch to realtime entry and free memory computations when computing bytes per object,
and remove the old approach in CacheStats().
4. Show hit rate during the run of dfly_bench loadtest.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
None does not exist in Valkey and its entry was missing from the indexes we use to map categories to commands leading to an out of bounds access and causing a segfault.
* remove none from acl categories
---------
Signed-off-by: kostas <kostas@dragonflydb.io>
1. Use introsive::list for CoolQueue.
2. Make sure that we ignore cool memory usage when computing average object size to
prevent evictions during dashtable growth attempts.
3. Remove items from the cool storage before evicting them from the dash table.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Update the flag for extreme testing. We should remove this before the release.
* set serialization_max_chunk_size to 1 byte
---------
Signed-off-by: kostas <kostas@dragonflydb.io>
The problem is that the test test_big_value_serialization_memory_limit will try to shutdown dragonfly at the end with a timeout of 15 seconds. Dragonfly during shutdown takes a snapshot which might take more than 15 seconds and the test fails.
* call flushall before we exit the test
---------
Signed-off-by: kostas <kostas@dragonflydb.io>
Inline transactions do not acquire any locks and therefore they should not preempt. This is no longer true when db_slice has registered callbacks.
* disable inline transactions when db_slice has registered callbacks
---------
Signed-off-by: kostas <kostas@dragonflydb.io>
For big value serialization it is required to support preemption when db_slice::RegisterOnChange is called to avoid UB when a code path is iterating over the change_cb_ and preempts because it serializes a big value. As this is problematic and can lead to data inconsistencies I replace the std::vector with std::list and bound the iteration of change_cb_ on paths that preempt.
* replace std::vector with std::list for change_cb_
* bound iteration of change_cb_ on paths that preempt
---------
Signed-off-by: kostas <kostas@dragonflydb.io>
We might preempt when we serialize a big value and the code in journal was protected by an atomic guard triggering a check failed.
* remove fiber guard from non atomic section
* move LocalBlockingCounter to common
---------
Signed-off-by: kostas <kostas@dragonflydb.io>
* chore: reenable evictions upon insertion to avoid OOM rejections
Before: when running dragonfly with --cache_mode we could get OOM rejections
even though the eviction policy allowed to evict items to free memory.
Ideally, dragonfly in cache mode should not respond with the OOM error.
This PR reuses the same Eviction step we have in the Heartbeat and conditionally applies it
during the insertion. In my test the OOM errors went from 500K to 0 and the server
still respected memory limit.
Also, remove the old heuristics that has never been used.
Test:
./dfly_bench --key_prefix=bar: -d 1024 --ratio=1:0 --qps=200 -n 3000
./dragonfly --dbfilename= --proactor_threads=2 --maxmemory=600M --cache_mode
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* fix: Fix `test_take_over_seeder`
There are a few issues with the test:
1. Not using the admin port, which could cause pause to deadlock
2. Not waiting for some of the `task`s (although that won't cause a
failure)
But also in the product code:
1. We used to `std::move()` the same pointer multiple times
2. We assigned to the same status object from multiple threads
Hopefully this fixes the test. It used to fail every ~100 attempts on my
machine, now it's been >1,000 and they all passed.
* add comments
* remove shard_ptr param
* chore: introduce a cool queue that gradually retires cool items
This PR introduces a new state in which the offloaded value is not freed from memory but instead stays
in the cool queue.
Upon Read we convert the cool value back to hot table and delete it from storage.
When we low on memory we retire oldest cool values until we are above the threshold.
The PR does not fully finish the feature but it is workable enough to start (load)testing.
Missing:
a) Handle Modify operations
b) Retire cool items in more cases where we are low on memory. Specifically, refrain from evictions as long as cool items exist.
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* default serialization_max_chunk_size to 10 mb
* add test for big values
* small rename of enum to conform style guide
---------
Signed-off-by: kostas <kostas@dragonflydb.io>
* chore: simplify computation of used_mem_current
Before - each thread updated its own variable and then,
the global "used_mem_current" was updated by summing used memory from each thread.
Now, each thread updates used_mem_current directly. The code is simpler and also provides more precise
results more frequently.
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Also, add the according API to compact object.
Now external objects can be in two states: Cool and Offloaded.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Our area of attack during concurrent transaction access is the call to DisarmInShard and DisarmInShardWhen, which only access is_armed - an atomic varible. It is not safe to arbitrarily call GetNamespace() if we write to it in InitBase
Solution: Don't write to it post first initialization
* chore: fix test_parser_memory_stats flakiness
1. Added a robust assert_eventually decorator for pytests
2. Improved the assertion condition in TieredStorageTest.BackgroundOffloading
3. Added total_uploaded stats for tiering that tells how many times offloaded values
were promoted back to RAM.
* chore: skip test_cluster_fuzzymigration
1. Moved CommandGenerator to thread scope - there is no need to maintain separate command generator per connection.
2. Added "done" metric - to know how much was done so far.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
