| .github/workflows | ||
| .vscode | ||
| helio@408d201f36 | ||
| patches | ||
| src | ||
| tests | ||
| tools/docker | ||
| .clang-format | ||
| .dockerignore | ||
| .gitignore | ||
| .gitmodules | ||
| .gitorderfile | ||
| CMakeLists.txt | ||
| LICENSE | ||
| README.md | ||
| TODO.md | ||
Dragonfly
A novel memory store that supports Redis and Memcached commands. For more detailed status of what's implemented - see below.
Features include:
- High throughput reaching millions of QPS on a single node.
- TLS support.
- Pipelining mode.
- A novel cache design, which does not require specifying eviction policies.
- Memory efficiency that can save 20-40% for regular workloads and even more for cache like workloads
Building from source
I've tested the build on Ubuntu 21.04+.
git clone --recursive https://github.com/romange/dragonfly
cd dragonfly && ./helio/blaze.sh -release
cd build-opt && ninja dragonfly
Running
./dragonfly --logtostderr
for more options, run ./dragonfly --help
Milestone Egg 🥚
API 1.0
- String family
- SET
- SETNX
- GET
- DECR
- INCR
- DECRBY
- GETSET
- INCRBY
- MGET
- MSET
- MSETNX
- SUBSTR
- Generic family
- DEL
- ECHO
- EXISTS
- EXPIRE
- EXPIREAT
- PING
- RENAME
- RENAMENX
- SELECT
- TTL
- TYPE
- SORT
- Server Family
- QUIT
- DBSIZE
- BGSAVE
- SAVE
- DBSIZE
- DEBUG
- EXEC
- FLUSHALL
- FLUSHDB
- INFO
- MULTI
- SHUTDOWN
- LASTSAVE
- SLAVEOF/REPLICAOF
- SYNC
- Set Family
- SADD
- SCARD
- SDIFF
- SDIFFSTORE
- SINTER
- SINTERSTORE
- SISMEMBER
- SMOVE
- SPOP
- SRANDMEMBER
- SREM
- SMEMBERS
- SUNION
- SUNIONSTORE
- List Family
- LINDEX
- LLEN
- LPOP
- LPUSH
- LRANGE
- LREM
- LSET
- LTRIM
- RPOP
- RPOPLPUSH
- RPUSH
- SortedSet Family
- ZADD
- ZCARD
- ZINCRBY
- ZRANGE
- ZRANGEBYSCORE
- ZREM
- ZREMRANGEBYSCORE
- ZREVRANGE
- ZSCORE
- Not sure whether these are required for the initial release.
- AUTH
- BGREWRITEAOF
- KEYS
- MONITOR
- RANDOMKEY
- MOVE
In addition, we want to support efficient expiry (TTL) and cache eviction algorithms. We should implement basic memory management support. For Master/Slave replication we should design a distributed log format.
Memchache API
- set
- get
- replace
- add
- stats (partial)
- append
- prepend
- delete
- flush_all
- incr
- decr
- version
- quit
API 2.0
- List Family
- BLPOP
- BRPOP
- BRPOPLPUSH
- String Family
- SETEX
- HashSet Family
- HSET
- HMSET
- HDEL
- HEXISTS
- HGET
- HMGET
- HLEN
- HINCRBY
- HINCRBYFLOAT
- HGETALL
- HKEYS
- PubSub family
- PUBLISH
- PUBSUB
- PUBSUB CHANNELS
- SUBSCRIBE
- UNSUBSCRIBE
- Server Family
- WATCH
- UNWATCH
- DISCARD
- Generic Family
- SCAN
- PEXPIREAT
- String Family
- APPEND
- PREPEND (dragonfly specific)
Commands that I prefer not implement before launch:
- PUNSUBSCRIBE
- PSUBSCRIBE
Also, I would omit keyspace notifications. For that I would like to deep dive and learn exact use-cases for this API.
Random commands we implemented along the way
- ROLE (2.8) decorator for for master withour replicas
- UNLINK (4.0) decorator for DEL command
- BGSAVE
Milestone Nymph
API 2,3,4 without cluster support, without modules, without memory inspection commands. Without support for keyspace notifications.
Design config support. ~140 commands overall...
Milestone Molt
API 5,6 - without cluster and modules. Streams support. ~80 commands overall.
Milestone Adult
TBD.
Design decisions along the way
Expiration deadlines with relative accuracy
I decided to limit the expiration range to 365 days. Moreover, expiration deadlines
with millisecond precision (PEXPIRE/PSETEX etc) will be rounded to closest second
**for deadlines greater than 33554431ms (approximately 560 minutes). In other words,
expiries of PEXPIRE key 10010 will expire exactly after 10 seconds and 10ms. However,
PEXPIRE key 34000300 will expire after 34000 seconds (i.e. 300ms earlier). Similarly,
PEXPIRE key 34000800 will expire after 34001 seconds, i.e. 200ms later.
Such rounding has at most 0.002% error which I hope is acceptable for large ranges. If it breaks your use-cases - talk to me or open an issue and explain your case.