Commit Graph

1331 Commits

Author SHA1 Message Date
Roman Gershman
8ebfd15166
fix: extend CI running time (#1749)
Also, add sccache debug log in hope to understand
why we get 0 hits sometimes.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-08-27 10:57:29 +03:00
adiholden
6aaa0ee072
feat(server): support multi eval in lock ahead mode (#1662)
* feat(server): support multi eval in lock ahead mode

1. remove validation to allow multi eval only in global script mode
2. send error if there is a mode conflict when running eval inside multi
3. reset uniqe_keys_ when transaction finishes
2023-08-27 09:13:22 +03:00
Roman Gershman
62a9313a38
fix: the bug where iobuf should be enlarged during the replication (#1744)
If an empty buffer is passed to the socket Recv function - it returns error 103.
Even if we returned success, this would lead to the endless loop since the parser
requires more data to parse the load.
Fixes #1680

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-08-25 22:38:04 +03:00
Vladislav
f1ac4b0f35
fix: fix index loading (#1742)
* fix: fix index loading

Signed-off-by: Vladislav <vladislav.oleshko@gmail.com>
Co-authored-by: Kostas Kyrimis <kostaskyrim@gmail.com>
2023-08-25 21:24:57 +03:00
Kostas Kyrimis
50545fc176
feat(AclFamily): add AUTH for acl members (#1732)
* Extends AUTH command to authenticate ACL users
* Add tests
2023-08-25 18:56:15 +03:00
Roman Gershman
eaaf2c5287
chore: cover zset over bptree implementation in the CI. (#1736)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-08-25 11:45:17 +03:00
Roman Gershman
7d619d2d4b
chore: integrate sccache (#1738)
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-08-25 10:58:11 +03:00
Vladislav
3520697b22
fix: fix search test (#1739)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-08-25 10:01:36 +03:00
Roman Gershman
5aa63a68dd
chore: add clear metric for the fragmentation waste (#1735) 2023-08-24 20:17:12 +03:00
Vladislav
731f36edd2
feat: Defragmentation for hash values (#1727)
* feat: defrag hash

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>

* fix: change tests to real scenarios

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>

* fix: handle link nodes

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-08-24 18:17:28 +03:00
Roman Gershman
e4ea868175
fix: Fix inconsistency when deleting dash entries (#1734)
Partially addresses #1730.

Beforehand we removed entries from mcflag only inside DbSlice::Del function
but we delete entries in other cases too (like expiration) and in this case we left
dangling entries in mcflag table. The fix adds the deletion of mcflag entries into PerformDeletion function.

It by itself does not explain the crash but
dangling entries cause UB (because the keys are references to non-existent PrimeKeys)
so maybe this somehow causes inconsistency where a key is not present in mcflags.

I am not sure I fixed the original issue but I surely fixed a corruption bug.
In addition, I removed the redundant check fail and added error logs to follow up on this later.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-08-24 17:02:02 +03:00
Roy Jacobson
b0718abdd1
chore: Split the build & test into 2 stages (#1733) 2023-08-24 16:52:24 +03:00
Roy Jacobson
5042cb2f6d
fix(ACL): Fix the iteration over categories loop in AclToString (#1731)
* fix(ACL): Fix the iteration over categories loop in AclToString

* Use another approach.
2023-08-24 15:51:34 +02:00
Kostas Kyrimis
69e24e7a43
chore(gh-workflows): add regression tests to release job (#1665)
* Run regression tests on Release build (only for x86)
* Make regression tests a composite action 
* Refactor to use regression tests action
2023-08-24 15:24:21 +02:00
Kostas Kyrimis
631c8e945a
fix: add proper printing of sha256 passwords (#1728)
* Properly print sha256 passwords by converting hex values to characters
2023-08-24 10:51:15 +00:00
Vladislav
84871b8dce
feat: search index persistence (#1721)
* feat: WIP search index persistence

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>

* Update src/server/search/doc_index.cc

Co-authored-by: Kostas Kyrimis  <kostaskyrim@gmail.com>
Signed-off-by: Vladislav <vladislav.oleshko@gmail.com>

* fix: foxes

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>

---------

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
Signed-off-by: Vladislav <vladislav.oleshko@gmail.com>
Co-authored-by: Kostas Kyrimis <kostaskyrim@gmail.com>
2023-08-24 13:41:10 +03:00
Kostas Kyrimis
bd87fb75fa
feat(AclFamily): add acl setuser command (#1725)
* Add `ACL SETUSER` command
* Add tests
2023-08-24 13:24:18 +03:00
Kostas Kyrimis
7f89bf37d8
fix(AclFamily): commands parsing and error handling (#1726)
* Fixed broken ACL command (prev df crashed)
* Fixed broken ACL LIST STR (now prints error)
* Added tests
2023-08-23 14:23:41 +03:00
talbii
74fcd3ed06
fix: Correctly read container limits when running under k8s (#1692)
Signed-off-by: talbii <ido@dragonflydb.io>
2023-08-23 14:21:49 +03:00
Roy Jacobson
331e6a4d47
chore: Mark slow pytests as 'slow' so they can be excluded. (#1720) 2023-08-23 13:04:49 +03:00
Roy Jacobson
d12c6e3639
chore: Add a daily coverage run (#1437)
* Add a coverage workflow to dragonfly.

* Try and make the testing permissive

* Apply suggestions from code review

Signed-off-by: Roy Jacobson <roi.jacobson1@gmail.com>

* Remove redundant parts

---------

Signed-off-by: Roy Jacobson <roi.jacobson1@gmail.com>
2023-08-22 20:00:21 +02:00
Kostas Kyrimis
898061d738
feat(AclFamilly): add acl list command (#1722)
* Add acl-family source and header
* Add `ACL LIST` command
* Add a simple test to check the default user
2023-08-22 15:33:14 +00:00
Vladislav
eae02a16da
fix: Make restore accept ttl in ms (#1724)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-08-22 12:17:12 +03:00
Vladislav
8b6de914fc
tools: Hash defrag script (#1723)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-08-22 09:11:34 +03:00
Roman Gershman
eb40ff8ed9
feat: implement DeleteRange functions in DfImpl (#1716)
Remove DeleteRange function from bptree_set API and
implement them externally inside sorted_map.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-08-21 13:53:55 +03:00
Vladislav
afb39287ce
chore: clean up save (#1709)
* chore: clean up save

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-08-21 13:32:41 +03:00
Roy Jacobson
969133b95e
replication: fix misleading log message about full sync (#1717) 2023-08-21 12:30:27 +02:00
Vladislav
ac79167530
fix: Add small timeout to monitor (#1718)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-08-21 10:20:43 +00:00
Kostas Kyrimis
cfc04cf952
chore: add acl categories to all commands (#1711)
1. Add acl categories to each command
2. Extend `facade::CommandId` to include acl category
3. Add dragonfly extensions to acl categories for modules like search, json etc (since modules in redis do not have acl categories by default but in dragonfly these are not implemented as modules)
2023-08-20 13:27:14 +03:00
Roman Gershman
98c6aac4e7
feat: implement GetRange/GetLexRange for bptree. (#1715)
Also, implement GetLexCount.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-08-19 17:22:19 +03:00
Roman Gershman
f4e1e1e1b3
feat: introduce partial DfImpl implementation for sorted_map (#1714)
Specifically add comparison operator that supports lex mode, score mode
and infinity abstractions for both modes.

Also, introduce LEQ/GEQ range queries for bptree_set.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-08-18 22:02:49 +03:00
Vladislav
e0f3684e5d
feat: json paths in search (#1695)
* feat: json paths in search

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
Signed-off-by: Vladislav <vladislav.oleshko@gmail.com>
Co-authored-by: Roy Jacobson <roi.jacobson1@gmail.com>
2023-08-18 15:40:59 +03:00
Vladislav
5198622a15
feat: Support unicode strings in search (#1698)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
Signed-off-by: Vladislav <vlad@dragonflydb.io>
2023-08-18 15:40:37 +03:00
Vladislav
003d2031b5
fix: fix batching during dispatch mode changes (#1713)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-08-18 15:39:34 +03:00
Vladislav
c65b9cf63d
fix: Fix squashing, pytest arg formatting (#1712)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-08-18 09:28:19 +03:00
Vladislav
4fbd0e38dd
feat: Pipeline squashing (#1619)
* feat: Pipeline squashing

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
Signed-off-by: Vladislav <vladislav.oleshko@gmail.com>
Co-authored-by: Kostas Kyrimis <kostaskyrim@gmail.com>
2023-08-17 16:06:48 +03:00
Vladislav
71fa2f275e
fix: MONITOR now works for multi transactions (#1675)
* fix: fix monitoring for multi transactions

Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-08-17 12:50:16 +03:00
Vladislav
d8ab016792
feat: Update search index on expiry (#1697)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-08-16 16:25:59 +03:00
Vladislav
66e11de274
fix: allow integer tokens in search queries (#1701)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-08-16 16:25:49 +03:00
Roy Jacobson
773cee12f4
security: Fix the password hashing in UserRegistry. (#1702) 2023-08-15 17:28:50 +02:00
Roman Gershman
ec22e73a28
chore: several improvements around sorted map (#1699)
* chore: several improvements around sorted map

1. Pass memory_resource to sorted_map.
2. Get rid of GetDict leaky accessor in SortedMap and introduce a proper
   Scan method.
3. Introduce correct BPTree type inside SortedMap::DFImpl.
4. Added a test for bptree_test that covers sds comparison
   (apparently, sdscmp can return values outside of [-1, 1] range).
   Fixed bptree code to support a proper spec for three-way comparison.
5. Expose pointers to internal objects allocated by score_map so we could insert them
   into bptree.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

* chore: fix comments

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

---------

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-08-15 18:08:59 +03:00
Kostas Kyrimis
e22c131b7c
chore: implement User and UserRegistry classes (#1693) 2023-08-15 11:57:21 +02:00
David Mutia
82c3690e75
feat(helm) - Enable configuring service type LoadBalancer with a Static IP (#1689)
* Enable configuring service type LoadBalancer IP

Signed-off-by: David Mutia <davidmutia47@gmail.com>

* Add tests for loadBalancerIP

Signed-off-by: David Mutia <davidmutia47@gmail.com>

* chore: get rid of cmdstats_map (#1687)

cmdstats_map were on the hotpath and are needed only to update the command stats.
Instead, I introduced the stats withing the CommandId itself that we lookup anyways.
Also, it removes fragile dependency on naked command name char* pointers.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>

---------

Signed-off-by: David Mutia <davidmutia47@gmail.com>
Signed-off-by: Roman Gershman <roman@dragonflydb.io>
Co-authored-by: Roman Gershman <roman@dragonflydb.io>
2023-08-14 14:04:35 +05:30
Tarun Pothulapati
df2745df64
fix(ci): use auth mechanism with tls helm values (#1696)
fix(ci): Fix Chart install Tests

One of the install test seems to fail, This PR tries to
fix that by also including password with tls test
2023-08-14 11:24:12 +05:30
Roman Gershman
e234833423
chore: move RdImpl functions together (#1694)
No functional changes are made.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-08-12 11:15:49 +03:00
Roman Gershman
a743d7577a
chore: get rid of cmdstats_map (#1687)
cmdstats_map were on the hotpath and are needed only to update the command stats.
Instead, I introduced the stats withing the CommandId itself that we lookup anyways.
Also, it removes fragile dependency on naked command name char* pointers.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-08-10 20:13:02 +03:00
Shahar Mike
3a832b5620
opt(server): Run Lua on remote thread only if it's a different thread (#1684)
* opt(server): Run Lua on remote thread only if it's a different thread

* Fixes
2023-08-10 10:05:36 +03:00
Abhradeep Chakraborty
0922cdb2b1
fix(daily-builds): temporarily comment out bullmq tests (#1679)
Signed-off-by: Abhradeep Chakraborty <abhradeep@dragonflydb.io>
2023-08-10 10:04:59 +03:00
Roman Gershman
f2ddbe7307
chore: refactor a common code into a function (#1685)
Also, fix unreliable time measurement for http latency metric.

Signed-off-by: Roman Gershman <roman@dragonflydb.io>
2023-08-10 05:50:43 +03:00
Vladislav
663c1f9e1b
fix: fix dchecks (#1681)
Signed-off-by: Vladislav Oleshko <vlad@dragonflydb.io>
2023-08-09 22:44:44 +03:00