* test(server): test transaction locked keys
1. add test utility class that will add suspension to transaction
execution.
2. add test for locked keys in transaction
Signed-off-by: adi_holden <adi@dragonflydb.io>
* introduce `--replicaof` flag
Closes#1381.
The behvaiour of `--replicaof` is similar to `REPLICAOF`. On startup, the instance continuously attempts to connect to master. Stop using the normal `REPLICAOF NO ONE` command.
The flag expects format `<IPv4/host>:<port>` or `[<IPv6>]:<port>`.
---------
Signed-off-by: talbii <ido@dragonflydb.io>
Signed-off-by: talbii <41526934+talbii@users.noreply.github.com>
* opt(server): Execute lua on target shard, if it's 1
This will save hops by short-circuiting execution of commands.
* Reuse unique shard id from tx
Only switch threads for LOCK_AHEAD
* Signedness
1. add tcp_keepalive flag
2. support config set for the following flags: tcp_keepalive,dir,requirepass,masterauth
3. more print details when failing on dbfilename flag validation
Signed-off-by: adi_holden <adi@dragonflydb.io>
* chore: Refactor SortedMap
1. intoduce variant of Redis and Dragonfly implementation and
wrap each operation with a dedicated dispatcher operator.
The dragonfly implementation path is a noop and not implemented.
2. Introduce a very basic sorted_map_test around Add and move the unrelated old test
from sorted_map_test to bptree_set_test.
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
update denyoom flag for all commands
DENYOOM command flag marks which commands will be denied when memory usage crosses the "red zone".
Signed-off-by: adi_holden <adi@dragonflydb.io>
1. add flag maxmemory_ratio
2. When current used memory * maxmemory_ratio > maxmemory_limit denyoom
commands will return oom error.
Signed-off-by: adi_holden <adi@dragonflydb.io>
* chore: Save snapshots as `.tmp` first to avoid file corruption
Snapshotting begins by saving to `<file-format>.ext.tmp` (for RDB this is a single file, for DFS multiple). *Only after* saving is done we rename the files to remove the `.tmp` extension.
---------
Signed-off-by: talbii <ido@dragonflydb.io>
Implemented `DeleteRange(KeyT first, KeyT last, std::function<void(KeyT)> cb,
bool inclusive_first, bool inclusive_last)`
method. We need inclusive parameters because with doubles it harder to reduce
the query to a standard open-closed range.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Also, fix Deletion bug inside DigRight function.
DeleteRangeByRank is implemented naively but it should be still
faster than zsl given that zsl needs to deallocate metadata per each item deleted.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* chore: Iterate over bptree_set.
Add iterate by rank functions.
For that we introduce BPTreePath::Next() and BPTreePath::Prev() functions
that allow iterating over immutable tree.
---------
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
This map should serve as a replacement for dict that currently is used internally by the SortedMap.
Specifically, it maps sds strings to scores to provide a reverse mapping.
It should be more efficient in memory and CPU - requires a single allocation to store both the
key and the value, in contrast to dict that needs two allocations.
In addition, it also reduces the object size needed for the zset score set:
zsl stores both the sds pointer and its double, i.e. requires 16 bytes not including the metadata.
By passing to bptree_set the internal ScoreMap objects that are allocated by ScoreMap::AddOrUpdate
we can just store 8-byte pointers, not including the bptree metadata.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Snapshot caused endless save loop on exit due to fiber not finishing.
The fix is to add a `break` to the `while` loop after `schedule_done_`
has notified.
Signed-off-by: Dor Avrahami <da19965@gmail.com>
feat: Introduce DEBUG OBJHIST command
Fixes#1618.
Also added to CompactObject::Size() OBJ_HASH case that was not handled before.
To get the statistics run `redis-cli --raw debug objhist`
It requires `--raw` because we send muilti-line output which
redis-cli escapes by default.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Also add a benchmark test that shows 2-3 times better performance for Find
vs redis zslGetRank when number of items grows.
For Insert/Delete I expect the differrence be even bigger.
Benchmark Time CPU Iterations
----------------------------------------------------------------------
BM_FindRandomBPTree/1024 56000 ns 55996 ns 75079
BM_FindRandomBPTree/65536 9406738 ns 9406552 ns 450
BM_FindRandomBPTree/1048576 390559398 ns 390539398 ns 11
BM_FindRandomZSL/1024 87991 ns 87989 ns 49196
BM_FindRandomZSL/65536 17917190 ns 17916618 ns 196
BM_FindRandomZSL/1048576 1343231616 ns 1343130709 ns 3
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
It's a partial implementation for only the insertion API.
We add the "tree count" variable, present in each inner node. This variable is updated upon each insertion
along the insertion path.
Deletion api will follow up shortly.
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
* feat(server): Add `--lock_on_hashtags` mode.
This new mode effectively locks hashtags (i.e. strings within {curly
braces}) instead of the full keys being used.
This can allow scripts to access undeclared keys if they all use a
common hashtag, like for the case of BullMQ.
To make sure this mode is tested, I added a way to specify flags via env
variables, and modified `ci.yml` to run all tests using this mode as well.
While at it, I also added `--cluster_mode=emulated` mode to CI.
Requested by #1590.
Introducing a new flag --snapshot_cron, enabling users to use cronjob expressions to time snapshot saves.
Cronjob expressions are parsed using a third party library croncpp.
This PR continues #1599, updating cron expressions to crontab style,
up to minutes resolution instead of seconds.
Signed-off-by: Dor Avrahami <da19965@gmail.com>
Introducing a new flag `--snapshot_cron`, which enables users
to use cron expressions to time snapshot saves.
Signed-off-by: Dor Avrahami <da19965@gmail.com>
* bug(server): multi atomicity fix
The bug: when multi transaction run OOO we removed it from trasaction
queue, causing non atomic execution.
The fix: When we run multi transaction unless it is the head in txq we
remove it inside unlock multi from txq.
Signed-off-by: adi_holden <adi@dragonflydb.io>
This PR would have been 1 line change instead of 11 files, but it
required some plumbing and refactoring:
* Now ClusterConfig is aware of emulated cluster mode
* As a result, this API was moved from ClusterFamily
* And so was the flag & its parsing
* ClusterFamily doesn't need is_emulated_cluster_ member
* ServerFamily no longer needs ClusterFamily* member (because the API is
static)
* I also changed `ClusterConfig::IsClusterEnabled()` to
`ClusterConfig::IsEnabled()` to be shorter
