diff --git a/doc/BlpopCommand.html b/doc/BlpopCommand.html index 9c64390af..7627ed717 100644 --- a/doc/BlpopCommand.html +++ b/doc/BlpopCommand.html @@ -16,7 +16,7 @@
-BlpopCommand: Contents
  BLPOP _key1_ _key2_ ... _keyN_ _timeout_ (Redis >
  BRPOP _key1_ _key2_ ... _keyN_ _timeout_ (Redis >
    Non blocking behavior
    Blocking behavior
    Multiple clients blocking for the same keys
    Return value +BlpopCommand: Contents
  BLPOP _key1_ _key2_ ... _keyN_ _timeout_ (Redis >
  BRPOP _key1_ _key2_ ... _keyN_ _timeout_ (Redis >
    Non blocking behavior
    Blocking behavior
    Multiple clients blocking for the same keys
    blocking POP inside a MULTI/EXEC transaction
    Return value

BlpopCommand

@@ -35,8 +35,10 @@

Blocking behavior

If none of the specified keys exist or contain non empty lists, BLPOPblocks until some other client performs a LPUSH oran RPUSH operation against one of the lists.
Once new data is present on one of the lists, the client finally returnswith the name of the key unblocking it and the popped value.
When blocking, if a non-zero timeout is specified, the client will unblockreturning a nil special value if the specified amount of seconds passedwithout a push operation against at least one of the specified keys.
-
A timeout of zero means instead to block forever.
+
The timeout argument is interpreted as an integer value. A timeout of zero means instead to block forever.

Multiple clients blocking for the same keys

Multiple clients can block for the same key. They are put intoa queue, so the first to be served will be the one that started to waitearlier, in a first-blpopping first-served fashion.
+

blocking POP inside a MULTI/EXEC transaction

BLPOP and BRPOP can be used with pipelining (sending multiple commands and reading the replies in batch), but it does not make sense to use BLPOP or BRPOP inside a MULTI/EXEC block (a Redis transaction).
+
The behavior of BLPOP inside MULTI/EXEC when the list is empty is to return a multi-bulk nil reply, exactly what happens when the timeout is reached. If you like science fiction, think at it like if inside MULTI/EXEC the time will flow at infinite speed :)

Return value

BLPOP returns a two-elements array via a multi bulk reply in order to returnboth the unblocking key and the popped value.
When a non-zero timeout is specified, and the BLPOP operation timed out,the return value is a nil multi bulk reply. Most client values will returnfalse or nil accordingly to the programming language used.
Multi bulk reply diff --git a/doc/CommandReference.html b/doc/CommandReference.html index 647c1b0c4..7021bd7e4 100644 --- a/doc/CommandReference.html +++ b/doc/CommandReference.html @@ -16,7 +16,7 @@

CommandReference

@@ -26,18 +26,18 @@
- = Redis Command Reference =

Every command name links to a specific wiki page describing the behavior of the command.

Connection handling

-

Commands operating on all the kind of values

-

Commands operating on string values

-

Commands operating on lists

-

Commands operating on sets

-

Commands operating on sorted sets (zsets, Redis version >

1.1) ==

-

Commands operating on hashes

-

Sorting

-

Transactions

-

Publish/Subscribe

-

Persistence control commands

-

Remote server control commands

+ = Redis Command Reference =

Every command name links to a specific wiki page describing the behavior of the command.

Categorized Command List

Connection handling

Command Parameters Description
QUIT - close the connection
AUTH password simple password authentication if enabled
+

Commands operating on all value types

Command Parameters Description
EXISTS key test if a key exists
DEL key delete a key
TYPE key return the type of the value stored at key
KEYS pattern return all the keys matching a given pattern
RANDOMKEY - return a random key from the key space
RENAME oldname newname rename the old key in the new one, destroying the newname key if it already exists
RENAMENX oldname newname rename the oldname key to newname, if the newname key does not already exist
DBSIZE - return the number of keys in the current db
EXPIRE - set a time to live in seconds on a key
PERSIST - remove the expire from a key
TTL - get the time to live in seconds of a key
SELECT index Select the DB with the specified index
MOVE key dbindex Move the key from the currently selected DB to the dbindex DB
FLUSHDB - Remove all the keys from the currently selected DB
FLUSHALL - Remove all the keys from all the databases
+

Commands operating on string values

Command Parameters Description
SET key value Set a key to a string value
GET key Return the string value of the key
GETSET key value Set a key to a string returning the old value of the key
MGET key1 key2 ... keyN Multi-get, return the strings values of the keys
SETNX key value Set a key to a string value if the key does not exist
SETEX key time value Set+Expire combo command
MSET key1 value1 key2 value2 ... keyN valueN Set multiple keys to multiple values in a single atomic operation
MSETNX key1 value1 key2 value2 ... keyN valueN Set multiple keys to multiple values in a single atomic operation if none of the keys already exist
INCR key Increment the integer value of key
INCRBY key integer Increment the integer value of key by integer
DECR key Decrement the integer value of key
DECRBY key integer Decrement the integer value of key by integer
APPEND key value Append the specified string to the string stored at key
SUBSTR key start end Return a substring of a larger string
+

Commands operating on lists

Command Parameters Description
RPUSH key value Append an element to the tail of the List value at key
LPUSH key value Append an element to the head of the List value at key
LLEN key Return the length of the List value at key
LRANGE key start end Return a range of elements from the List at key
LTRIM key start end Trim the list at key to the specified range of elements
LINDEX key index Return the element at index position from the List at key
LSET key index value Set a new value as the element at index position of the List at key
LREM key count value Remove the first-N, last-N, or all the elements matching value from the List at key
LPOP key Return and remove (atomically) the first element of the List at key
RPOP key Return and remove (atomically) the last element of the List at key
BLPOP key1 key2 ... keyN timeout Blocking LPOP
BRPOP key1 key2 ... keyN timeout Blocking RPOP
RPOPLPUSH srckey dstkey Return and remove (atomically) the last element of the source List stored at srckey and push the same element to the destination List stored at dstkey
+

Commands operating on sets

Command Parameters Description
SADD key member Add the specified member to the Set value at key
SREM key member Remove the specified member from the Set value at key
SPOP key Remove and return (pop) a random element from the Set value at key
SMOVE srckey dstkey member Move the specified member from one Set to another atomically
SCARD key Return the number of elements (the cardinality) of the Set at key
SISMEMBER key member Test if the specified value is a member of the Set at key
SINTER key1 key2 ... keyN Return the intersection between the Sets stored at key1, key2, ..., keyN
SINTERSTORE dstkey key1 key2 ... keyN Compute the intersection between the Sets stored at key1, key2, ..., keyN, and store the resulting Set at dstkey
SUNION key1 key2 ... keyN Return the union between the Sets stored at key1, key2, ..., keyN
SUNIONSTORE dstkey key1 key2 ... keyN Compute the union between the Sets stored at key1, key2, ..., keyN, and store the resulting Set at dstkey
SDIFF key1 key2 ... keyN Return the difference between the Set stored at key1 and all the Sets key2, ..., keyN
SDIFFSTORE dstkey key1 key2 ... keyN Compute the difference between the Set key1 and all the Sets key2, ..., keyN, and store the resulting Set at dstkey
SMEMBERS key Return all the members of the Set value at key
SRANDMEMBER key Return a random member of the Set value at key
+

Commands operating on sorted zsets (sorted sets)

Command Parameters Description
ZADD key score member Add the specified member to the Sorted Set value at key or update the score if it already exist
ZREM key member Remove the specified member from the Sorted Set value at key
ZINCRBY key increment member If the member already exists increment its score by increment, otherwise add the member setting increment as score
ZRANK key member Return the rank (or index) or member in the sorted set at key, with scores being ordered from low to high
ZREVRANK key member Return the rank (or index) or member in the sorted set at key, with scores being ordered from high to low
ZRANGE key start end Return a range of elements from the sorted set at key
ZREVRANGE key start end Return a range of elements from the sorted set at key, exactly like ZRANGE, but the sorted set is ordered in traversed in reverse order, from the greatest to the smallest score
ZRANGEBYSCORE key min max Return all the elements with score >= min and score <= max (a range query) from the sorted set
ZCOUNT key min max Return the number of elements with score >= min and score <= max in the sorted set
ZCARD key Return the cardinality (number of elements) of the sorted set at key
ZSCORE key element Return the score associated with the specified element of the sorted set at key
ZREMRANGEBYRANK key min max Remove all the elements with rank >= min and rank <= max from the sorted set
ZREMRANGEBYSCORE key min max Remove all the elements with score >= min and score <= max from the sorted set
ZUNIONSTORE / ZINTERSTORE dstkey N key1 ... keyN WEIGHTS w1 ... wN AGGREGATE SUM|MIN|MAX Perform a union or intersection over a number of sorted sets with optional weight and aggregate
+

Commands operating on hashes

Command Parameters Description
HSET key field value Set the hash field to the specified value. Creates the hash if needed.
HGET key field Retrieve the value of the specified hash field.
HMGET key field1 ... fieldN Get the hash values associated to the specified fields.
HMSET key field1 value1 ... fieldN valueN Set the hash fields to their respective values.
HINCRBY key field integer Increment the integer value of the hash at key on field with integer.
HEXISTS key field Test for existence of a specified field in a hash
HDEL key field Remove the specified field from a hash
HLEN key Return the number of items in a hash.
HKEYS key Return all the fields in a hash.
HVALS key Return all the values in a hash.
HGETALL key Return all the fields and associated values in a hash.
+

Sorting

Command Parameters Description
SORT key BY pattern LIMIT start end GET pattern ASC|DESC ALPHA Sort a Set or a List accordingly to the specified parameters
+

Transactions

Command Parameters Description
MULTI/EXEC/DISCARD/WATCH/UNWATCH - Redis atomic transactions
+

Publish/Subscribe

Command Parameters Description
SUBSCRIBE/UNSUBSCRIBE/PUBLISH - Redis Public/Subscribe messaging paradigm implementation
+

Persistence control commands

Command Parameters Description
SAVE - Synchronously save the DB on disk
BGSAVE - Asynchronously save the DB on disk
LASTSAVE - Return the UNIX time stamp of the last successfully saving of the dataset on disk
SHUTDOWN - Synchronously save the DB on disk, then shutdown the server
BGREWRITEAOF - Rewrite the append only file in background when it gets too big
+

Remote server control commands

Command Parameters Description
INFO - Provide information and statistics about the server
MONITOR - Dump all the received requests in real time
SLAVEOF - Change the replication settings
CONFIG - Configure a Redis server at runtime
diff --git a/doc/DelCommand.html b/doc/DelCommand.html index 8d063ce7d..3a7ac69cd 100644 --- a/doc/DelCommand.html +++ b/doc/DelCommand.html @@ -27,12 +27,11 @@
#sidebar GenericCommandsSidebar

DEL _key1_ _key2_ ... _keyN_

-Time complexity: O(1)
Remove the specified keys. If a given key does not existno operation is performed for this key. The commnad returns the number ofkeys removed.
+Time complexity: O(1)
Remove the specified keys. If a given key does not existno operation is performed for this key. The command returns the number ofkeys removed.

Return value

Integer reply, specifically:

 an integer greater than 0 if one or more keys were removed
 0 if none of the specified key existed
 
-
diff --git a/doc/ExpireCommand.html b/doc/ExpireCommand.html index a3dbbe5be..cebac8b8e 100644 --- a/doc/ExpireCommand.html +++ b/doc/ExpireCommand.html @@ -16,7 +16,7 @@
-ExpireCommand: Contents
  EXPIRE _key_ _seconds_
  EXPIREAT _key_ _unixtime_ (Redis >
    How the expire is removed from a key
    Restrictions with write operations against volatile keys
    Setting the timeout again on already volatile keys
    Enhanced Lazy Expiration algorithm
      Version 1.0
      Version 1.1
    Return value
    FAQ: Can you explain better why Redis deletes keys with an EXPIRE on write operations? +ExpireCommand: Contents
  EXPIRE _key_ _seconds_
  EXPIREAT _key_ _unixtime_ (Redis >
  PERSIST _key_
    How the expire is removed from a key
    Restrictions with write operations against volatile keys
    Restrictions for write operations with volatile keys as sources
    Setting the timeout again on already volatile keys
    Enhanced Lazy Expiration algorithm
      Version 1.0
      Version 1.1
    Return value
    FAQ: Can you explain better why Redis < 2.1.3 deletes keys with an EXPIRE on write operations?
    FAQ: How this limitations were solved in Redis versions > 2.1.3?

ExpireCommand

@@ -28,12 +28,15 @@
#sidebar GenericCommandsSidebar

EXPIRE _key_ _seconds_

EXPIREAT _key_ _unixtime_ (Redis >

1.1)= +

PERSIST _key_

Time complexity: O(1)
Set a timeout on the specified key. After the timeout the key will beautomatically delete by the server. A key with an associated timeout issaid to be volatile in Redis terminology.
-
Voltile keys are stored on disk like the other keys, the timeout is persistenttoo like all the other aspects of the dataset. Saving a dataset containingthe dataset and stopping the server does not stop the flow of time as Redisregisters on disk when the key will no longer be available as Unix time, andnot the remaining seconds.
+
Voltile keys are stored on disk like the other keys, the timeout is persistenttoo like all the other aspects of the dataset. Saving a dataset containingexpires and stopping the server does not stop the flow of time as Redisstores on disk the time when the key will no longer be available as Unixtime, and not the remaining seconds.
EXPIREAT works exctly like EXPIRE but instead to get the number of secondsrepresenting the Time To Live of the key as a second argument (that is arelative way of specifing the TTL), it takes an absolute one in the form ofa UNIX timestamp (Number of seconds elapsed since 1 Gen 1970).
-
EXPIREAT was introduced in order to implement [Persistence append only saving mode] so that EXPIRE commands are automatically translated into EXPIREAT commands for the append only file. Of course EXPIREAT can alsoused by programmers that need a way to simply specify that a given key should expire at a given time in the future.
-

How the expire is removed from a key

When the key is set to a new value using the SET command, the INCR commandor any other command that modify the value stored at key the timeout isremoved from the key and the key becomes non volatile.
-

Restrictions with write operations against volatile keys

Write operations like LPUSH, LSET and every other command that has theeffect of modifying the value stored at a volatile key have a special semantic:basically a volatile key is destroyed when it is target of a write operation.See for example the following usage pattern:
+
EXPIREAT was introduced in order to implement the Append Only File persistence modeso that EXPIRE commands are automatically translated into EXPIREAT commands for the append only file. Of course EXPIREAT can alsoused by programmers that need a way to simply specify that a given key should expire at a given time in the future.
+
Since Redis 2.1.3 you can update the value of the timeout of a key alreadyhaving an expire set. It is also possible to undo the expire at allturning the key into a normal key using the PERSIST command.
+

How the expire is removed from a key

When the key is set to a new value using the SET command, or when a keyis destroied via DEL, the timeout is removed from the key.
+

Restrictions with write operations against volatile keys

IMPORTANT: Since Redis 2.1.3 or greater, there are no restrictions aboutthe operations you can perform against volatile keys, however older versionsof Redis, including the current stable version 2.0.0, has the followinglimitations:
+
Write operations like LPUSH, LSET and every other command that has theeffect of modifying the value stored at a volatile key have a special semantic:basically a volatile key is destroyed when it is target of a write operation.See for example the following usage pattern:
 % ./redis-cli lpush mylist foobar /Users/antirez/hack/redis
 OK
@@ -45,8 +48,13 @@ OK
 OK
 % ./redis-cli lrange mylist 0 -1  /Users/antirez/hack/redis
 1. newelement
-
What happened here is that lpush against the key with a timeout set deletedthe key before to perform the operation. There is so a simple rule, writeoperations against volatile keys will destroy the key before to perform theoperation. Why Redis uses this behavior? In order to retain an importantproperty: a server that receives a given number of commands in the samesequence will end with the same dataset in memory. Without the delete-on-writesemantic what happens is that the state of the server depends on the timeof the commands to. This is not a desirable property in a distributed databasethat supports replication.
-

Setting the timeout again on already volatile keys

Trying to call EXPIRE against a key that already has an associated timeoutwill not change the timeout of the key, but will just return 0. If insteadthe key does not have a timeout associated the timeout will be set and EXPIREwill return 1.
+
What happened here is that LPUSH against the key with a timeout set deletedthe key before to perform the operation. There is so a simple rule, writeoperations against volatile keys will destroy the key before to perform theoperation. Why Redis uses this behavior? In order to retain an importantproperty: a server that receives a given number of commands in the samesequence will end with the same dataset in memory. Without the delete-on-writesemantic what happens is that the state of the server depends on the timethe commands were issued. This is not a desirable property in a distributed databasethat supports replication.
+

Restrictions for write operations with volatile keys as sources

Even when the volatile key is not modified as part of a write operation, if it is +read in a composite write operation (such as SINTERSTORE) it will be cleared at the +start of the operation. This is done to avoid concurrency issues in replication. +Imagine a key that is about to expire and the composite operation is run against it. +On a slave node, this key might already be expired, which leaves you with a +desync in your dataset.

Setting the timeout again on already volatile keys

Trying to call EXPIRE against a key that already has an associated timeoutwill not change the timeout of the key, but will just return 0. If insteadthe key does not have a timeout associated the timeout will be set and EXPIREwill return 1.

Enhanced Lazy Expiration algorithm

Redis does not constantly monitor keys that are going to be expired.Keys are expired simply when some client tries to access a key, andthe key is found to be timed out.
Of course this is not enough as there are expired keys that will neverbe accessed again. This keys should be expired anyway, so once everysecond Redis test a few keys at random among keys with an expire set.All the keys that are already expired are deleted from the keyspace.

Version 1.0

Each time a fixed number of keys where tested (100 by default). So ifyou had a client setting keys with a very short expire faster than 100for second the memory continued to grow. When you stopped to insertnew keys the memory started to be freed, 100 keys every second in thebest conditions. Under a peak Redis continues to use more and more RAMeven if most keys are expired in each sweep.
@@ -56,8 +64,10 @@ OK
This means that at any given moment the maximum amount of keys alreadyexpired that are using memory is at max equal to max setting operations per second divided by 4.

Return value

Integer reply, specifically:

 1: the timeout was set.
-0: the timeout was not set since the key already has an associated timeout, or the key does not exist.
-

FAQ: Can you explain better why Redis deletes keys with an EXPIRE on write operations?

+0: the timeout was not set since the key already has an associated timeout + (this may happen only in Redis versions < 2.1.3, Redis >= 2.1.3 will + happily update the timeout), or the key does not exist. +

FAQ: Can you explain better why Redis < 2.1.3 deletes keys with an EXPIRE on write operations?

Ok let's start with the problem:
 redis> set a 100
@@ -76,7 +86,8 @@ EXPIRE a 5
 INCR a
 
Imagine a Redis version that does not implement the "Delete keys with an expire set on write operation" semantic. -Running the above example with the 10 seconds pause will lead to 'a' being set to the value of 1, as it no longer exists when INCR is called 10 seconds later.

Instead if we drop the 10 seconds pause, the result is that 'a' is set to 101.

And in the practice timing changes! For instance the client may wait 10 seconds before INCR, but the sequence written in the Append Only File (and later replayed-back as fast as possible when Redis is restarted) will not have the pause. Even if we add a timestamp in the AOF, when the time difference is smaller than our timer resolution, we have a race condition.

The same happens with master-slave replication. Again, consider the example above: the client will use the same sequence of commands without the 10 seconds pause, but the replication link will slow down for a few seconds due to a network problem. Result? The master will contain 'a' set to 101, the slave 'a' set to 1.

The only way to avoid this but at the same time have reliable non time dependent timeouts on keys is to destroy volatile keys when a write operation is attempted against it.

After all Redis is one of the rare fully persistent databases that will give you EXPIRE. This comes to a cost :) +Running the above example with the 10 seconds pause will lead to 'a' being set to the value of 1, as it no longer exists when INCR is called 10 seconds later.

Instead if we drop the 10 seconds pause, the result is that 'a' is set to 101.

And in the practice timing changes! For instance the client may wait 10 seconds before INCR, but the sequence written in the Append Only File (and later replayed-back as fast as possible when Redis is restarted) will not have the pause. Even if we add a timestamp in the AOF, when the time difference is smaller than our timer resolution, we have a race condition.

The same happens with master-slave replication. Again, consider the example above: the client will use the same sequence of commands without the 10 seconds pause, but the replication link will slow down for a few seconds due to a network problem. Result? The master will contain 'a' set to 101, the slave 'a' set to 1.

The only way to avoid this but at the same time have reliable non time dependent timeouts on keys is to destroy volatile keys when a write operation is attempted against it.

After all Redis is one of the rare fully persistent databases that will give you EXPIRE. This comes to a cost :)

FAQ: How this limitations were solved in Redis versions > 2.1.3?

Since Redis 2.1.3 there are no longer restrictions in the use you can do of write commands against volatile keys, still the replication and AOF file are guaranteed to be fully consistent.

In order to obtain a correct behavior without sacrificing consistency now when a key expires, a DEL operation is synthesized in both the AOF file and against all the attached slaves. This way the expiration process is centralized in the master instance, and there is no longer a chance of consistency errors.

However while the slaves while connected to a master will not expire keys independently, they'll still take the full state of the expires existing in the dataset, so when a slave is elected to a master it will be able to expire the keys independently, fully acting as a master. +
diff --git a/doc/FAQ.html b/doc/FAQ.html index 7c012b2cf..531fb708e 100644 --- a/doc/FAQ.html +++ b/doc/FAQ.html @@ -58,7 +58,7 @@ Redis for the same objects. This happens because when data is in memory is full of pointers, reference counters and other metadata. Add to this malloc fragmentation and need to return word-aligned chunks of memory and you have a clear picture of what happens. So this means to -have 10 times the I/O between memory and disk than otherwise needed.

Is there something I can do to lower the Redis memory usage?

Yes, try to compile it with 32 bit target if you are using a 64 bit box.

If you are using Redis >= 1.3, try using the Hash data type, it can save a lot of memory.

If you are using hashes or any other type with values bigger than 128 bytes try also this to lower the RSS usage (Resident Set Size): EXPORT MMAP_THRESHOLD=4096

I have an empty Redis server but INFO and logs are reporting megabytes of memory in use!

This may happen and it's prefectly ok. Redis objects are small C structures allocated and freed a lot of times. This costs a lot of CPU so instead of being freed, released objects are taken into a free list and reused when needed. This memory is taken exactly by this free objects ready to be reused.

What happens if Redis runs out of memory?

With modern operating systems malloc() returning NULL is not common, usually the server will start swapping and Redis performances will be disastrous so you'll know it's time to use more Redis servers or get more RAM.

The INFO command (work in progress in this days) will report the amount of memory Redis is using so you can write scripts that monitor your Redis servers checking for critical conditions.

You can also use the "maxmemory" option in the config file to put a limit to the memory Redis can use. If this limit is reached Redis will start to reply with an error to write commands (but will continue to accept read-only commands).

Does Redis use more memory running in 64 bit boxes? Can I use 32 bit Redis in 64 bit systems?

Redis uses a lot more memory when compiled for 64 bit target, especially if the dataset is composed of many small keys and values. Such a database will, for instance, consume 50 MB of RAM when compiled for the 32 bit target, and 80 MB for 64 bit! That's a big difference.

You can run 32 bit Redis binaries in a 64 bit Linux and Mac OS X system without problems. For OS X just use make 32bit. For Linux instead, make sure you have libc6-dev-i386 installed, then use make 32bit if you are using the latest Git version. Instead for Redis <= 1.2.2 you have to edit the Makefile and replace "-arch i386" with "-m32".

If your application is already able to perform application-level sharding, it is very advisable to run N instances of Redis 32bit against a big 64 bit Redis box (with more than 4GB of RAM) instead than a single 64 bit instance, as this is much more memory efficient.

How much time it takes to load a big database at server startup?

Just an example on normal hardware: It takes about 45 seconds to restore a 2 GB database on a fairly standard system, no RAID. This can give you some kind of feeling about the order of magnitude of the time needed to load data when you restart the server.

Background saving is failing with a fork() error under Linux even if I've a lot of free RAM!

Short answer: echo 1 > /proc/sys/vm/overcommit_memory :)

And now the long one:

Redis background saving schema relies on the copy-on-write semantic of fork in modern operating systems: Redis forks (creates a child process) that is an exact copy of the parent. The child process dumps the DB on disk and finally exits. In theory the child should use as much memory as the parent being a copy, but actually thanks to the copy-on-write semantic implemented by most modern operating systems the parent and child process will share the common memory pages. A page will be duplicated only when it changes in the child or in the parent. Since in theory all the pages may change while the child process is saving, Linux can't tell in advance how much memory the child will take, so if the overcommit_memory setting is set to zero fork will fail unless there is as much free RAM as required to really duplicate all the parent memory pages, with the result that if you have a Redis dataset of 3 GB and just 2 GB of free memory it will fail.

Setting overcommit_memory to 1 says Linux to relax and perform the fork in a more optimistic allocation fashion, and this is indeed what you want for Redis.

Are Redis on disk snapshots atomic?

Yes, redis background saving process is always fork(2)ed when the server is outside of the execution of a command, so every command reported to be atomic in RAM is also atomic from the point of view of the disk snapshot.

Redis is single threaded, how can I exploit multiple CPU / cores?

Simply start multiple instances of Redis in different ports in the same box and threat them as different servers! Given that Redis is a distributed database anyway in order to scale you need to think in terms of multiple computational units. At some point a single box may not be enough anyway.

In general key-value databases are very scalable because of the property that different keys can stay on different servers independently.

In Redis there are client libraries such Redis-rb (the Ruby client) that are able to handle multiple servers automatically using consistent hashing. We are going to implement consistent hashing in all the other major client libraries. If you use a different language you can implement it yourself otherwise just hash the key before to SET / GET it from a given server. For example imagine to have N Redis servers, server-0, server-1, ..., server-N. You want to store the key "foo", what's the right server where to put "foo" in order to distribute keys evenly among different servers? Just perform the crc = CRC32("foo"), then servernum = crc % N (the rest of the division for N). This will give a number between 0 and N-1 for every key. Connect to this server and store the key. The same for gets.

This is a basic way of performing key partitioning, consistent hashing is much better and this is why after Redis 1.0 will be released we'll try to implement this in every widely used client library starting from Python and PHP (Ruby already implements this support).

I'm using some form of key hashing for partitioning, but what about SORT BY?

With SORT BY you need that all the weight keys are in the same Redis instance of the list/set you are trying to sort. In order to make this possible we developed a concept called key tags. A key tag is a special pattern inside a key that, if preset, is the only part of the key hashed in order to select the server for this key. For example in order to hash the key "foo" I simply perform the CRC32 checksum of the whole string, but if this key has a pattern in the form of the characters {...} I only hash this substring. So for example for the key "foo{bared}" the key hashing code will simply perform the CRC32 of "bared". This way using key tags you can ensure that related keys will be stored on the same Redis instance just using the same key tag for all this keys. Redis-rb already implements key tags.

What is the maximum number of keys a single Redis instance can hold? and what the max number of elements in a List, Set, Ordered Set?

In theory Redis can handle up to 232 keys, and was tested in practice to handle at least 150 million of keys per instance. We are working in order to experiment with larger values.

Every list, set, and ordered set, can hold 2
32 elements.

Actually Redis internals are ready to allow up to 264 elements but the current disk dump format don't support this, and there is a lot time to fix this issues in the future as currently even with 128 GB of RAM it's impossible to reach 232 elements.

What Redis means actually?

Redis means two things: +have 10 times the I/O between memory and disk than otherwise needed.

Is there something I can do to lower the Redis memory usage?

Yes, try to compile it with 32 bit target if you are using a 64 bit box.

If you are using Redis >= 1.3, try using the Hash data type, it can save a lot of memory.

If you are using hashes or any other type with values bigger than 128 bytes try also this to lower the RSS usage (Resident Set Size): EXPORT MMAP_THRESHOLD=4096

I have an empty Redis server but INFO and logs are reporting megabytes of memory in use!

This may happen and it's prefectly ok. Redis objects are small C structures allocated and freed a lot of times. This costs a lot of CPU so instead of being freed, released objects are taken into a free list and reused when needed. This memory is taken exactly by this free objects ready to be reused.

What happens if Redis runs out of memory?

With modern operating systems malloc() returning NULL is not common, usually the server will start swapping and Redis performances will be disastrous so you'll know it's time to use more Redis servers or get more RAM.

The INFO command (work in progress in this days) will report the amount of memory Redis is using so you can write scripts that monitor your Redis servers checking for critical conditions.

You can also use the "maxmemory" option in the config file to put a limit to the memory Redis can use. If this limit is reached Redis will start to reply with an error to write commands (but will continue to accept read-only commands).

Does Redis use more memory running in 64 bit boxes? Can I use 32 bit Redis in 64 bit systems?

Redis uses a lot more memory when compiled for 64 bit target, especially if the dataset is composed of many small keys and values. Such a database will, for instance, consume 50 MB of RAM when compiled for the 32 bit target, and 80 MB for 64 bit! That's a big difference.

You can run 32 bit Redis binaries in a 64 bit Linux and Mac OS X system without problems. For OS X just use make 32bit. For Linux instead, make sure you have libc6-dev-i386 installed, then use make 32bit if you are using the latest Git version. Instead for Redis <= 1.2.2 you have to edit the Makefile and replace "-arch i386" with "-m32".

If your application is already able to perform application-level sharding, it is very advisable to run N instances of Redis 32bit against a big 64 bit Redis box (with more than 4GB of RAM) instead than a single 64 bit instance, as this is much more memory efficient.

How much time it takes to load a big database at server startup?

Just an example on normal hardware: It takes about 45 seconds to restore a 2 GB database on a fairly standard system, no RAID. This can give you some kind of feeling about the order of magnitude of the time needed to load data when you restart the server.

Background saving is failing with a fork() error under Linux even if I've a lot of free RAM!

Short answer: echo 1 > /proc/sys/vm/overcommit_memory :)

And now the long one:

Redis background saving schema relies on the copy-on-write semantic of fork in modern operating systems: Redis forks (creates a child process) that is an exact copy of the parent. The child process dumps the DB on disk and finally exits. In theory the child should use as much memory as the parent being a copy, but actually thanks to the copy-on-write semantic implemented by most modern operating systems the parent and child process will share the common memory pages. A page will be duplicated only when it changes in the child or in the parent. Since in theory all the pages may change while the child process is saving, Linux can't tell in advance how much memory the child will take, so if the overcommit_memory setting is set to zero fork will fail unless there is as much free RAM as required to really duplicate all the parent memory pages, with the result that if you have a Redis dataset of 3 GB and just 2 GB of free memory it will fail.

Setting overcommit_memory to 1 says Linux to relax and perform the fork in a more optimistic allocation fashion, and this is indeed what you want for Redis.

A good source to understand how Linux Virtual Memory work and other alternatives for overcommit_memory and overcommit_ratio is this classic from Red Hat Magaize, "Understanding Virtual Memory": http://www.redhat.com/magazine/001nov04/features/vm/

Are Redis on disk snapshots atomic?

Yes, redis background saving process is always fork(2)ed when the server is outside of the execution of a command, so every command reported to be atomic in RAM is also atomic from the point of view of the disk snapshot.

Redis is single threaded, how can I exploit multiple CPU / cores?

Simply start multiple instances of Redis in different ports in the same box and threat them as different servers! Given that Redis is a distributed database anyway in order to scale you need to think in terms of multiple computational units. At some point a single box may not be enough anyway.

In general key-value databases are very scalable because of the property that different keys can stay on different servers independently.

In Redis there are client libraries such Redis-rb (the Ruby client) that are able to handle multiple servers automatically using consistent hashing. We are going to implement consistent hashing in all the other major client libraries. If you use a different language you can implement it yourself otherwise just hash the key before to SET / GET it from a given server. For example imagine to have N Redis servers, server-0, server-1, ..., server-N. You want to store the key "foo", what's the right server where to put "foo" in order to distribute keys evenly among different servers? Just perform the crc = CRC32("foo"), then servernum = crc % N (the rest of the division for N). This will give a number between 0 and N-1 for every key. Connect to this server and store the key. The same for gets.

This is a basic way of performing key partitioning, consistent hashing is much better and this is why after Redis 1.0 will be released we'll try to implement this in every widely used client library starting from Python and PHP (Ruby already implements this support).

I'm using some form of key hashing for partitioning, but what about SORT BY?

With SORT BY you need that all the weight keys are in the same Redis instance of the list/set you are trying to sort. In order to make this possible we developed a concept called key tags. A key tag is a special pattern inside a key that, if preset, is the only part of the key hashed in order to select the server for this key. For example in order to hash the key "foo" I simply perform the CRC32 checksum of the whole string, but if this key has a pattern in the form of the characters {...} I only hash this substring. So for example for the key "foo{bared}" the key hashing code will simply perform the CRC32 of "bared". This way using key tags you can ensure that related keys will be stored on the same Redis instance just using the same key tag for all this keys. Redis-rb already implements key tags.

What is the maximum number of keys a single Redis instance can hold? and what the max number of elements in a List, Set, Ordered Set?

In theory Redis can handle up to 232 keys, and was tested in practice to handle at least 150 million of keys per instance. We are working in order to experiment with larger values.

Every list, set, and ordered set, can hold 2
32 elements.

Actually Redis internals are ready to allow up to 264 elements but the current disk dump format don't support this, and there is a lot time to fix this issues in the future as currently even with 128 GB of RAM it's impossible to reach 232 elements.

What Redis means actually?

Redis means two things:

Why did you started the Redis project?

In order to scale LLOOGG. But after I got the basic server working I liked the idea to share the work with other guys, and Redis was turned into an open source project. diff --git a/doc/GenericCommandsSidebar.html b/doc/GenericCommandsSidebar.html index d2dd6aa70..0d25cb222 100644 --- a/doc/GenericCommandsSidebar.html +++ b/doc/GenericCommandsSidebar.html @@ -26,7 +26,7 @@
- == Generic Commands ==

+ == Generic Commands ==

diff --git a/doc/HashCommandsSidebar.html b/doc/HashCommandsSidebar.html index f4808af20..c0b84670f 100644 --- a/doc/HashCommandsSidebar.html +++ b/doc/HashCommandsSidebar.html @@ -26,7 +26,7 @@
- == Hash Commands ==

+ == Hash Commands ==

diff --git a/doc/KeysCommand.html b/doc/KeysCommand.html index f1a6e070c..6d79428a8 100644 --- a/doc/KeysCommand.html +++ b/doc/KeysCommand.html @@ -32,7 +32,8 @@
the slow commands that may ruin the DB performance if not usedwith care*.
In other words this command is intended only for debugging and *special* operations like creating a script to change the DB schema. Don't use it in your normal code. Use Redis Sets in order to group together a subset of objects.
Glob style patterns examples: -
* h?llo will match hello hallo hhllo* h*llo will match hllo heeeello* h[ae]llo will match hello and hallo, but not hillo
Use \ to escape special chars if you want to match them verbatim.

Return value

Bulk reply, specifically a string in the form of space separated list of keys. Note that most client libraries will return an Array of keys and not a single string with space separated keys (that is, split by " " is performed in the client library usually). +
* h?llo will match hello hallo hhllo* h*llo will match hllo heeeello* h[ae]llo will match hello and hallo, but not hillo
Use \ to escape special chars if you want to match them verbatim.

Return value

+Multi bulk reply diff --git a/doc/LindexCommand.html b/doc/LindexCommand.html index 4af80530f..0e36634b9 100644 --- a/doc/LindexCommand.html +++ b/doc/LindexCommand.html @@ -28,10 +28,9 @@
#sidebar ListCommandsSidebar

LINDEX _key_ _index_

Time complexity: O(n) (with n being the length of the list)
Return the specified element of the list stored at the specifiedkey. 0 is the first element, 1 the second and so on. Negative indexesare supported, for example -1 is the last element, -2 the penultimateand so on.
-
If the value stored at key is not of list type an error is returned.If the index is out of range an empty string is returned.
+
If the value stored at key is not of list type an error is returned.If the index is out of range a 'nil' reply is returned.
Note that even if the average time complexity is O(n) asking forthe first or the last element of the list is O(1).

Return value

Bulk reply, specifically the requested element. -
diff --git a/doc/MultiExecCommand.html b/doc/MultiExecCommand.html index e0a41983a..65ec67a00 100644 --- a/doc/MultiExecCommand.html +++ b/doc/MultiExecCommand.html @@ -16,7 +16,7 @@
-MultiExecCommand: Contents
  MULTI
  COMMAND_1 ...
  COMMAND_2 ...
  COMMAND_N ...
  EXEC or DISCARD
    Usage
    The DISCARD command
    Return value +MultiExecCommand: Contents
  WATCH key1 key2 ... keyN (Redis >
  UNWATCH
  MULTI
  COMMAND_1 ...
  COMMAND_2 ...
  COMMAND_N ...
  EXEC or DISCARD
    Usage
    The DISCARD command
    Check and Set (CAS) transactions using WATCH
    WATCH explained
    WATCH used to implement ZPOP
    Return value

MultiExecCommand

@@ -26,15 +26,21 @@
- #sidebar GenericCommandsSidebar

MULTI

+ #sidebar GenericCommandsSidebar

WATCH key1 key2 ... keyN (Redis >

2.1.0)= +

UNWATCH

+

MULTI

COMMAND_1 ...

COMMAND_2 ...

COMMAND_N ...

-

EXEC or DISCARD

MULTI, EXEC and DISCARD commands are the fundation of Redis Transactions.A Redis Transaction allows to execute a group of Redis commands in a singlestep, with two important guarantees:
- -

Usage

A Redis transaction is entered using the MULTI command. The command alwaysreplies with OK. At this point the user can issue multiple commands. Insteadto execute this commands Redis will "queue" them. All the commands areexecuted once EXEC is called.
-
Calling DISCARD instead will flush the transaction queue and will exitthe transaction.
-
The following is an example using the Ruby client:
+

EXEC or DISCARD

MULTI, EXEC, DISCARD and WATCH commands are the fundation of Redis Transactions. +A Redis Transaction allows the execution of a group of Redis commands in a single +step, with two important guarantees:

+Since Redis 2.1.0, it's also possible to add a further guarantee to the above two, in the form of optimistic locking of a set of keys in a way very similar to a CAS (check and set) operation. This is documented later in this manual page.

Usage

A Redis transaction is entered using the MULTI command. The command always +replies with OK. At this point the user can issue multiple commands. Instead +to execute this commands Redis will "queue" them. All the commands are +executed once EXEC is called.

Calling DISCARD instead will flush the transaction queue and will exit +the transaction.

The following is an example using the Ruby client: +
 ?> r.multi
 => "OK"
 >> r.incr "foo"
@@ -46,9 +52,14 @@
 >> r.exec
 => [1, 1, 2]
 
-
As it is possible to see from the session above, MULTI returns an "array" ofreplies, where every element is the reply of a single command in thetransaction, in the same order the commands were queued.
-
When a Redis connection is in the context of a MULTI request, all the commandswill reply with a simple string "QUEUED" if they are correct from thepoint of view of the syntax and arity (number of arguments) of the commaand.Some command is still allowed to fail during execution time.
-
This is more clear if at protocol level: in the following example one commandwill fail when executed even if the syntax is right:
+As it is possible to see from the session above, MULTI returns an "array" of
+replies, where every element is the reply of a single command in the
+transaction, in the same order the commands were queued.

When a Redis connection is in the context of a MULTI request, all the commands +will reply with a simple string "QUEUED" if they are correct from the +point of view of the syntax and arity (number of arguments) of the commaand. +Some command is still allowed to fail during execution time.

This is more clear if at protocol level: in the following example one command +will fail when executed even if the syntax is right: +
 Trying 127.0.0.1...
 Connected to localhost.
 Escape character is '^]'.
@@ -64,16 +75,21 @@ EXEC
 +OK
 -ERR Operation against a key holding the wrong kind of value
 
-
MULTI returned a two elements bulk reply in witch one of this is a +OKcode and one is a -ERR reply. It's up to the client lib to find a sensibleway to provide the error to the user.
-
IMPORTANT: even when a command will raise an error, all the other commandsin the queue will be processed. Redis will NOT stop the processing ofcommands once an error is found.
-
Another example, again using the write protocol with telnet, shows howsyntax errors are reported ASAP instead:
+MULTI returned a two elements bulk reply in witch one of this is a +OK
+code and one is a -ERR reply. It's up to the client lib to find a sensible
+way to provide the error to the user.

IMPORTANT: even when a command will raise an error, all the other commandsin the queue will be processed. Redis will NOT stop the processing ofcommands once an error is found.
+Another example, again using the write protocol with telnet, shows how +syntax errors are reported ASAP instead: +
 MULTI
 +OK
 INCR a b c
 -ERR wrong number of arguments for 'incr' command
 
-
This time due to the syntax error the "bad" INCR command is not queuedat all.
-

The DISCARD command

DISCARD can be used in order to abort a transaction. No command will beexecuted, and the state of the client is again the normal one, outsideof a transaction. Example using the Ruby client:
+This time due to the syntax error the "bad" INCR command is not queued
+at all.

The DISCARD command

DISCARD can be used in order to abort a transaction. No command will be +executed, and the state of the client is again the normal one, outside +
of a transaction. Example using the Ruby client:
 ?> r.set("foo",1)
 => true
 >> r.multi
@@ -84,9 +100,64 @@ INCR a b c
 => "OK"
 >> r.get("foo")
 => "1"
-

Return value

Multi bulk reply, specifically:

-The result of a MULTI/EXEC command is a multi bulk reply where every element is the return value of every command in the atomic transaction.
+

Check and Set (CAS) transactions using WATCH

WATCH is used in order to provide a CAS (Check and Set) behavior to +Redis Transactions.

WATCHed keys are monitored in order to detect changes against this keys. +If at least a watched key will be modified before the EXEC call, the +whole transaction will abort, and EXEC will return a nil object +(A Null Multi Bulk reply) to notify that the transaction failed.

For example imagine we have the need to atomically increment the value +of a key by 1 (I know we have INCR, let's suppose we don't have it).

The first try may be the following: +
+val = GET mykey
+val = val + 1
+SET mykey $val
 
+This will work reliably only if we have a single client performing the operation in a given time. +If multiple clients will try to increment the key about at the same time +there will be a race condition. For instance client A and B will read the +old value, for instance, 10. The value will be incremented to 11 by both +the clients, and finally SET as the value of the key. So the final value +will be "11" instead of "12".

Thanks to WATCH we are able to model the problem very well: +
+WATCH mykey
+val = GET mykey
+val = val + 1
+MULTI
+SET mykey $val
+EXEC
+
+Using the above code, if there are race conditions and another client +modified the result of val in the time between our call to WATCH and +our call to EXEC, the transaction will fail.

We'll have just to re-iterate the operation hoping this time we'll not get +a new race. This form of locking is called optimistic locking and is +a very powerful form of locking as in many problems there are multiple +clients accessing a much bigger number of keys, so it's very unlikely that +there are collisions: usually operations don't need to be performed +multiple times.

WATCH explained

So what is WATCH really about? It is a command that will make the EXEC +conditional: we are asking Redis to perform the transaction only if no +other client modified any of the WATCHed keys. Otherwise the transaction is not +entered at all. (Note that if you WATCH a volatile key and Redis expires the key after you WATCHed it, EXEC will still work. More.)

WATCH can be called multiple times. Simply all the WATCH calls will +have the effects to watch for changes starting from the call, up to the +moment EXEC is called.

When EXEC is called, either if it will fail or succeed, all keys are +UNWATCHed. Also when a client connection is closed, everything gets +UNWATCHed.

It is also possible to use the UNWATCH command (without arguments) in order +to flush all the watched keys. Sometimes this is useful as we +optimistically lock a few keys, since possibly we need to perform a transaction +to alter those keys, but after reading the current content of the keys +we don't want to proceed. When this happens we just call UNWATCH so that +the connection can already be used freely for new transactions.

WATCH used to implement ZPOP

A good example to illustrate how WATCH can be used to create new atomic +operations otherwise not supported by Redis is to implement ZPOP, that is +a command that pops the element with the lower score from a sorted set +in an atomic way. This is the simplest implementation: +
+WATCH zset
+ele = ZRANGE zset 0 0
+MULTI
+ZREM zset ele
+EXEC
+
+If EXEC fails (returns a nil value) we just re-iterate the operation.

Return value

Multi bulk reply, specifically:

+The result of a MULTI/EXEC command is a multi bulk reply where every element is the return value of every command in the atomic transaction.
+
If a MULTI/EXEC transaction is aborted because of WATCH detected modified keys, a Null Multi Bulk reply is returned.
diff --git a/doc/README.html b/doc/README.html index f70fe83f0..c71d63861 100644 --- a/doc/README.html +++ b/doc/README.html @@ -26,11 +26,42 @@
- = Introduction =

Redis is a database. To be specific, Redis is a database implementing a dictionary, where every key is associated with a value. For example I can set the key "surname_1992" to the string "Smith". -What makes Redis different from many other key-value stores, is that every single value has a type. The following types are supported:

-The type of a value determines what operations (called commands) are available for the value itself. -For example you can append elements to a list stored at the key "mylist" using the LPUSH or RPUSH command in O(1). Later you'll be able to get a range of elements with LRANGE or trim the list with LTRIM. Sets are very flexible too, it is possible to add and remove elements from Sets (unsorted collections of strings), and then ask for server-side intersection, union, difference of Sets. Each command is performed through server-side atomic operations. -Please refer to the Command Reference to see the full list of operations associated to these data types.

In other words, you can look at Redis as a data structures server. A Redis user is virtually provided with an interface to Abstract Data Types, saving her from the responsibility to implement concrete data structures and algorithms. Indeed both algorithms and data structures in Redis are properly choosed in order to obtain the best performance.

All data in memory, but saved on disk

Redis loads and mantains the whole dataset into memory, but the dataset is persistent, since at the same time it is saved on disk, so that when the server is restarted data can be loaded back in memory.

There are two kind of persistence supported: the first one is called snapshotting. In this mode Redis, from time to time, writes a dump on disk asynchronously. The dataset is loaded from the dump every time the server is (re)started.

Redis can be configured to save the dataset when a certain number of changes is reached and after a given number of seconds elapses. For example, you can configure Redis to save after 1000 changes and at most 60 seconds since the last save. You can specify any combination for these numbers.

Because data is written asynchronously, when a system crash occurs, the last few queries can get lost (that is acceptable in many applications but not in all). In order to make this a non issue Redis supports another, safer persistence mode, called Append Only File, where every command received altering the dataset (so not a read-only command, but a write command) is written on an append only file ASAP. This commands are replayed when the server is restarted in order to rebuild the dataset in memory.

Redis Append Only File supports a very handy feature: the server is able to safely rebuild the append only file in background in a non-blocking fashion when it gets too long. You can find more details in the Append Only File HOWTO.

Master-Slave replication made trivial

Whatever will be the persistence mode you'll use Redis supports master-slave replications if you want to stay really safe or if you need to scale to huge amounts of reads.

Redis Replication is trivial to setup. So trivial that all you need to do in order to configure a Redis server to be a slave of another one, with automatic synchronization if the link will go down and so forth, is the following config line: slaveof 192.168.1.100 6379. We provide a Replication Howto if you want to know more about this feature.

It's persistent but supports expires

Redis can be used as a memcached on steroids because is as fast as memcached but with a number of features more. Like memcached, Redis also supports setting timeouts to keys so that this key will be automatically removed when a given amount of time passes.

Beyond key-value databases

All these features allow to use Redis as the sole DB for your scalable application without the need of any relational database. We wrote a simple Twitter clone in PHP + Redis to show a real world example, the link points to an article explaining the design and internals in very simple words.

Multiple databases support

Redis supports multiple databases with commands to atomically move keys from one database to the other. By default DB 0 is selected for every new connection, but using the SELECT command it is possible to select a different database. The MOVE operation can move an item from one DB to another atomically. This can be used as a base for locking free algorithms together with the 'RANDOMKEY' commands.

Know more about Redis!

To really get a feeling about what Redis is and how it works please try reading A fifteen minutes introduction to Redis data types.

To know a bit more about how Redis works internally continue reading.

Redis Tutorial

(note, you can skip this section if you are only interested in "formal" doc.)

Later in this document you can find detailed information about Redis commands, + = Introduction =

Redis is an extremely fast and powerful key-value store database and server implemented in ANSI C. Redis offers many different ways to do one straightforward thing: store a value ("antirez") to a key ("redis"). While the format of keys must always be simple strings, the power is with the values, which support the following data types:

+Each value type has an associated list of commands which can operate on them, and the The Redis Command Reference contains an up to date list of these commands, organized primarily by data type. The Redis source also includes a Redis command line interface which allows you to interact directly with the server, and is the means by which this introduction will provide examples. Once you walk through the Redis Quick Start Guide to get your instance of Redis running, you can follow along.

One of the most powerful aspects of Redis is the wide range of commands which are optimized to work with specific data value types and executed as atomic server-side operations. The List type is a great example - Redis implements O(1) operations such as LPUSH or RPUSH, which have accompanying LPOP and RPOP methods:

+redis> lpush programming_languages C
+OK
+redis> lpush programming_languages Ruby
+OK
+redis> rpush programming_languages Python
+OK
+redis> rpop programming_languages
+Python
+redis> lpop programming_languages
+Ruby
+
More complex operations are available for each data type as well. Continuing with lists, you can get a range of elements with LRANGE (O(start+n)) or trim the list with LTRIM (O(n)):

+redis> lpush cities NYC
+OK
+redis> lpush cities SF
+OK
+redis> lpush cities Tokyo
+OK
+redis> lpush cities London
+OK
+redis> lpush cities Paris
+OK
+redis> lrange cities 0 2
+1. Paris
+2. London
+3. Tokyo
+redis> ltrim cities 0 1
+OK
+redis> lpop cities
+Paris
+redis> lpop cities
+London
+redis> lpop cities
+(nil)
+
You can also add and remove elements from a set, and perform intersections, unions, and differences.

Redis can also be looked at as a data structures server. A Redis user is virtually provided with an interface to Abstract Data Types, saving them from the responsibility of implementing concrete data structures and algorithms -- indeed both algorithms and data structures in Redis are properly chosen in order to obtain the best performance.

All data in memory, but saved on disk

Redis loads and mantains the whole dataset into memory, but the dataset is persistent, since at the same time it is saved on disk, so that when the server is restarted data can be loaded back in memory.

There are two kinds of persistence supported: the first one is called snapshotting. In this mode Redis periodically writes to disk asynchronously. The dataset is loaded from the dump every time the server is (re)started.

Redis can be configured to save the dataset when a certain number of changes is reached and after a given number of seconds elapses. For example, you can configure Redis to save after 1000 changes and at most 60 seconds since the last save. You can specify any combination for these numbers.

Because data is written asynchronously, when a system crash occurs, the last few queries can get lost (that is acceptable in many applications but not in all). In order to make this a non issue Redis supports another, safer persistence mode, called Append Only File, where every command received altering the dataset (so not a read-only command, but a write command) is written on an append only file ASAP. This commands are replayed when the server is restarted in order to rebuild the dataset in memory.

Redis Append Only File supports a very handy feature: the server is able to safely rebuild the append only file in background in a non-blocking fashion when it gets too long. You can find more details in the Append Only File HOWTO.

Master-Slave replication made trivial

Whatever will be the persistence mode you'll use Redis supports master-slave replications if you want to stay really safe or if you need to scale to huge amounts of reads.

Redis Replication is trivial to setup. So trivial that all you need to do in order to configure a Redis server to be a slave of another one, with automatic synchronization if the link will go down and so forth, is the following config line: slaveof 192.168.1.100 6379. We provide a Replication Howto if you want to know more about this feature.

It's persistent but supports expires

Redis can be used as a memcached on steroids because is as fast as memcached but with a number of features more. Like memcached, Redis also supports setting timeouts to keys so that this key will be automatically removed when a given amount of time passes.

Beyond key-value databases

All these features allow to use Redis as the sole DB for your scalable application without the need of any relational database. We wrote a simple Twitter clone in PHP + Redis to show a real world example, the link points to an article explaining the design and internals in very simple words.

Multiple databases support

Redis supports multiple databases with commands to atomically move keys from one database to the other. By default DB 0 is selected for every new connection, but using the SELECT command it is possible to select a different database. The MOVE operation can move an item from one DB to another atomically. This can be used as a base for locking free algorithms together with the 'RANDOMKEY' commands.

Know more about Redis!

To really get a feeling about what Redis is and how it works please try reading A fifteen minutes introduction to Redis data types.

To know a bit more about how Redis works internally continue reading.

Redis Tutorial

(note, you can skip this section if you are only interested in "formal" doc.)

Later in this document you can find detailed information about Redis commands, the protocol specification, and so on. This kind of documentation is useful but... if you are new to Redis it is also BORING! The Redis protocol is designed so that is both pretty efficient to be parsed by computers, but simple enough @@ -40,7 +71,7 @@ feeling about it, and how it works.

To start just compile redis with 'm The server will start and log stuff on the standard output, if you want it to log more edit redis.conf, set the loglevel to debug, and restart it.

You can specify a configuration file as unique parameter:

./redis-server /etc/redis.conf
This is NOT required. The server will start even without a configuration file -using a default built-in configuration.

Now let's try to set a key to a given value:

+using a default built-in configuration.

Now let's try to set a key to a given value:

 $ telnet localhost 6379
 Trying 127.0.0.1...
 Connected to localhost.
@@ -59,17 +90,17 @@ the point of view of both the server and client but allows us to play with
 Redis with the telnet command easily.

The last line of the chat between server and client is "+OK". This means our key was added without problems. Actually SET can never fail but the "+OK" sent lets us know that the server received everything and -the command was actually executed.

Let's try to get the key content now:

+the command was actually executed.

Let's try to get the key content now:

 GET foo
 $3
 bar
 
Ok that's very similar to 'set', just the other way around. We sent "get foo", the server replied with a first line that is just the $ character follwed by the number of bytes the value stored at key contained, followed by the actual -bytes. Again "\r\n" are appended both to the bytes count and the actual data. In Redis slang this is called a bulk reply.

What about requesting a non existing key?

+bytes. Again "\r\n" are appended both to the bytes count and the actual data. In Redis slang this is called a bulk reply.

What about requesting a non existing key?

 GET blabla
 $-1
-
When the key does not exist instead of the length, just the "$-1" string is sent. Since a -1 length of a bulk reply has no meaning it is used in order to specifiy a 'nil' value and distinguish it from a zero length value. Another way to check if a given key exists or not is indeed the EXISTS command:

+
When the key does not exist instead of the length, just the "$-1" string is sent. Since a -1 length of a bulk reply has no meaning it is used in order to specifiy a 'nil' value and distinguish it from a zero length value. Another way to check if a given key exists or not is indeed the EXISTS command:

 EXISTS nokey
 :0
 EXISTS foo
diff --git a/doc/SortedSetCommandsSidebar.html b/doc/SortedSetCommandsSidebar.html
index 2534beb20..c2e8fae09 100644
--- a/doc/SortedSetCommandsSidebar.html
+++ b/doc/SortedSetCommandsSidebar.html
@@ -26,7 +26,7 @@
                 
- == Sorted Set Commands ==

+ == Sorted Set Commands ==

diff --git a/doc/SupportedLanguages.html b/doc/SupportedLanguages.html index 3b8156a2d..d0d06aa2b 100644 --- a/doc/SupportedLanguages.html +++ b/doc/SupportedLanguages.html @@ -28,7 +28,7 @@

Supported Languages (DRAFT)

Wondering if you can use Redis from your favorite language? Well here is the definitive guide to the available client libraries.

This libraries are intended to expose Redis commands, but you also have the option to use some higher level libraries that provide a Object Hash Mappings pretty much the same idea implemented by a classic ORM.

TODO

Features Support Matrix



The following matrix should give you a quick overviwe of the state of the different client libraries existing for each supported language.

The core command set is the one of Version 1.0, while Sharding and Pipelining are convenient client side features not tied to any Redis server version.

Version 1.1

Compatible client libraries are expected to implement the command sets specified in Version 1.0 plus:

-

Version 1.0



Compatible client libraries are expected to implement the following command sets:

Language Name Sharding Pipelining 1.1 1.0
ActionScript 3 as3redis No Yes Yes Yes
Clojure redis-clojure No No Partial Yes
Common Lisp CL-Redis No No No Yes
Erlang erldis No Looks like No Looks like
Go Go-Redis No Yes Yes Yes
Haskell haskell-redis No No No Yes
Java JDBC-Redis No No No Yes
Java JRedis No Yes Yes Yes
LUA redis-lua No No Yes Yes
Perl Redis Client No No No Yes
Perl AnyEvent::Redis No No No Yes
PHP Redis PHP Bindings No No No Yes
PHP phpredis (C) No No No Yes
PHP Predis Yes Yes Yes Yes
PHP Redisent Yes No No Yes
Python Python Client No No No Yes
Python py-redis No No Partial Yes
Python txredis No No No Yes
Ruby redis-rb Yes Yes Yes Yes
Scala scala-redis Yes No No Yes
TCL TCL No No Yes Yes
+

Version 1.0



Compatible client libraries are expected to implement the following command sets:

Language Name Sharding Pipelining 1.1 1.0
ActionScript 3 as3redis No Yes Yes Yes
Clojure redis-clojure No No Partial Yes
Common Lisp CL-Redis No No No Yes
Erlang erldis No Looks like No Looks like
Go Go-Redis No Yes Yes Yes
Haskell haskell-redis No No No Yes
Java JDBC-Redis No No No Yes
Java JRedis No Yes Yes Yes
Java Jedis No Yes Yes Yes
LUA redis-lua No No Yes Yes
Perl Redis Client No No No Yes
Perl AnyEvent::Redis No No No Yes
PHP Redis PHP Bindings No No No Yes
PHP phpredis (C) No No No Yes
PHP Predis Yes Yes Yes Yes
PHP Redisent Yes No No Yes
Python Python Client No No No Yes
Python py-redis No No Partial Yes
Python txredis No No No Yes
Ruby redis-rb Yes Yes Yes Yes
Scala scala-redis Yes No No Yes
TCL TCL No No Yes Yes

Client Libraries Reference

as3 (ActionScript 3)

redis-clojure (Clojure)

CL-Redis (Common Lisp)

@@ -36,7 +36,8 @@

Go-Redis (Go)

haskell-redis (Haskell)

Java

JDBC-Redis

-

JRedis

+

JRedis

+

Jedis

redis-lua (Lua)

Perl

Perl Client

AnyEvent::Redis

diff --git a/doc/ZincrbyCommand.html b/doc/ZincrbyCommand.html index 7e6a8458a..1f8fe2947 100644 --- a/doc/ZincrbyCommand.html +++ b/doc/ZincrbyCommand.html @@ -30,8 +30,8 @@ Time complexity O(log(N)) with N being the number of elements in the sorted set
If member already exists in the sorted set adds the increment to its scoreand updates the position of the element in the sorted set accordingly.If member does not already exist in the sorted set it is added with_increment_ as score (that is, like if the previous score was virtually zero).If key does not exist a new sorted set with the specified_member_ as sole member is crated. If the key exists but does not hold asorted set value an error is returned.
The score value can be the string representation of a double precision floatingpoint number. It's possible to provide a negative value to perform a decrement.
For an introduction to sorted sets check the Introduction to Redis data types page.
-

Return value

Integer reply, specifically:

-The score of the member after the increment is performed.
+

Return value

Bulk reply
+The new score (a double precision floating point number) represented as string.
 
diff --git a/doc/ZrangebyscoreCommand.html b/doc/ZrangebyscoreCommand.html index 583e9303f..d9b310d4e 100644 --- a/doc/ZrangebyscoreCommand.html +++ b/doc/ZrangebyscoreCommand.html @@ -16,7 +16,7 @@
-ZrangebyscoreCommand: Contents
  ZRANGEBYSCORE _key_ _min_ _max_ `[`LIMIT _offset_ _count_`]` (Redis >
  ZRANGEBYSCORE _key_ _min_ _max_ `[`LIMIT _offset_ _count_`]` `[`WITHSCORES`]` (Redis >
    Exclusive intervals and infinity
    Return value
    Examples +ZrangebyscoreCommand: Contents
  ZRANGEBYSCORE _key_ _min_ _max_ `[`LIMIT _offset_ _count_`]` (Redis >
  ZRANGEBYSCORE _key_ _min_ _max_ `[`LIMIT _offset_ _count_`]` `[`WITHSCORES`]` (Redis >
  ZCOUNT _key_ _min_ _max_
    Exclusive intervals and infinity
    Return value
    Examples

ZrangebyscoreCommand

@@ -28,9 +28,11 @@
#sidebar SortedSetCommandsSidebar

ZRANGEBYSCORE _key_ _min_ _max_ `[`LIMIT _offset_ _count_`]` (Redis >

1.1) =

ZRANGEBYSCORE _key_ _min_ _max_ `[`LIMIT _offset_ _count_`]` `[`WITHSCORES`]` (Redis >

1.3.4) = +

ZCOUNT _key_ _min_ _max_

Time complexity: O(log(N))+O(M) with N being the number of elements in the sorted set and M the number of elements returned by the command, so if M is constant (for instance you always ask for the first ten elements with LIMIT) you can consider it O(log(N))
Return the all the elements in the sorted set at key with a score between_min_ and max (including elements with score equal to min or max).
The elements having the same score are returned sorted lexicographically asASCII strings (this follows from a property of Redis sorted sets and does notinvolve further computation).
-
Using the optional LIMIT it's possible to get only a range of the matchingelements in an SQL-alike way. Note that if offset is large the commandsneeds to traverse the list for offset elements and this adds up to theO(M) figure.

Exclusive intervals and infinity

+
Using the optional LIMIT it's possible to get only a range of the matchingelements in an SQL-alike way. Note that if offset is large the commandsneeds to traverse the list for offset elements and this adds up to theO(M) figure.
+
The ZCOUNT command is similar to ZRANGEBYSCORE but instead of returningthe actual elements in the specified interval, it just returns the numberof matching elements.

Exclusive intervals and infinity

min and max can be -inf and +inf, so that you are not required to know what's the greatest or smallest element in order to take, for instance, elements "up to a given value".

Also while the interval is for default closed (inclusive) it's possible to specify open intervals prefixing the score with a "(" character, so for instance:
 ZRANGEBYSCORE zset (1.3 5
@@ -40,7 +42,7 @@ Will return all the values with score > 1.3 and <= 5, while for ins
 ZRANGEBYSCORE zset (5 (10
 
Will return all the values with score > 5 and < 10 (5 and 10 excluded). -

Return value

Multi bulk reply, specifically a list of elements in the specified score range. +

Return value

ZRANGEBYSCORE returns a Multi bulk reply specifically a list of elements in the specified score range.

ZCOUNT returns a Integer reply specifically the number of elements matching the specified score range.

Examples

 redis> zadd zset 1 foo
@@ -56,6 +58,8 @@ redis> zrangebyscore zset -inf +inf
 2. "bar"
 3. "biz"
 4. "foz"
+redis> zcount zset 1 2
+(integer) 2
 redis> zrangebyscore zset 1 2
 1. "foo"
 2. "bar"
diff --git a/doc/ZunionCommand.html b/doc/ZunionCommand.html
index edb52a9c9..cb5b844d7 100644
--- a/doc/ZunionCommand.html
+++ b/doc/ZunionCommand.html
@@ -16,7 +16,7 @@
             

ZunionCommand

@@ -27,8 +27,9 @@
-

ZUNION / ZINTER _dstkey_ _N_ _k1_ ... _kN_ `[`WEIGHTS _w1_ ... _wN_`]` `[`AGGREGATE SUM|MIN|MAX`]` (Redis >

1.3.5) =

Time complexity: O(N) + O(M log(M)) with N being the sum of the sizes of the input sorted sets, and M being the number of elements in the resulting sorted set
Creates a union or intersection of N sorted sets given by keys k1 through kN, and stores it at dstkey. It is mandatory to provide the number of input keys N, before passing the input keys and the other (optional) arguments.
-
As the terms imply, the ZINTER command requires an element to be present in each of the given inputs to be inserted in the result. The ZUNION command inserts all elements across all inputs.
+

ZUNIONSTORE _dstkey_ _N_ _k1_ ... _kN_ `[`WEIGHTS _w1_ ... _wN_`]` `[`AGGREGATE SUM|MIN|MAX`]` (Redis >

1.3.12) = +

ZINTERSTORE _dstkey_ _N_ _k1_ ... _kN_ `[`WEIGHTS _w1_ ... _wN_`]` `[`AGGREGATE SUM|MIN|MAX`]` (Redis >

1.3.12) =

Time complexity: O(N) + O(M log(M)) with N being the sum of the sizes of the input sorted sets, and M being the number of elements in the resulting sorted set
Creates a union or intersection of N sorted sets given by keys k1 through kN, and stores it at dstkey. It is mandatory to provide the number of input keys N, before passing the input keys and the other (optional) arguments.
+
As the terms imply, the ZINTERSTORE command requires an element to be present in each of the given inputs to be inserted in the result. The ZUNIONSTORE command inserts all elements across all inputs.
Using the WEIGHTS option, it is possible to add weight to each input sorted set. This means that the score of each element in the sorted set is first multiplied by this weight before being passed to the aggregation. When this option is not given, all weights default to 1.
With the AGGREGATE option, it's possible to specify how the results of the union or intersection are aggregated. This option defaults to SUM, where the score of an element is summed across the inputs where it exists. When this option is set to be either MIN or MAX, the resulting set will contain the minimum or maximum score of an element across the inputs where it exists.

Return value

Integer reply, specifically the number of elements in the sorted set at dstkey. diff --git a/doc/ZunionstoreCommand.html b/doc/ZunionstoreCommand.html index a9f743260..862c38bba 100644 --- a/doc/ZunionstoreCommand.html +++ b/doc/ZunionstoreCommand.html @@ -16,7 +16,7 @@

ZunionstoreCommand

@@ -27,8 +27,10 @@
-

ZUNION / ZINTER _dstkey_ _N_ _k1_ ... _kN_ `[`WEIGHTS _w1_ ... _wN_`]` `[`AGGREGATE SUM|MIN|MAX`]` (Redis >

1.3.5) =

Time complexity: O(N) + O(M log(M)) with N being the sum of the sizes of the input sorted sets, and M being the number of elements in the resulting sorted set
Creates a union or intersection of N sorted sets given by keys k1 through kN, and stores it at dstkey. It is mandatory to provide the number of input keys N, before passing the input keys and the other (optional) arguments.
-
As the terms imply, the ZINTER command requires an element to be present in each of the given inputs to be inserted in the result. The ZUNION command inserts all elements across all inputs.
+

ZUNIONSTORE _dstkey_ _N_ _k1_ ... _kN_ `[`WEIGHTS _w1_ ... _wN_`]` `[`AGGREGATE SUM|MIN|MAX`]` (Redis >

1.3.12) = +

ZINTERSTORE _dstkey_ _N_ _k1_ ... _kN_ `[`WEIGHTS _w1_ ... _wN_`]` `[`AGGREGATE SUM|MIN|MAX`]` (Redis >

1.3.12) = +Time complexity: O(N) + O(M log(M)) with N being the sum of the sizes of the input sorted sets, and M being the number of elements in the resulting sorted set
Creates a union or intersection of N sorted sets given by keys k1 through kN, and stores it at dstkey. It is mandatory to provide the number of input keys N, before passing the input keys and the other (optional) arguments.
+
As the terms imply, the ZINTERSTORE command requires an element to be present in each of the given inputs to be inserted in the result. The ZUNIONSTORE command inserts all elements across all inputs.
Using the WEIGHTS option, it is possible to add weight to each input sorted set. This means that the score of each element in the sorted set is first multiplied by this weight before being passed to the aggregation. When this option is not given, all weights default to 1.
With the AGGREGATE option, it's possible to specify how the results of the union or intersection are aggregated. This option defaults to SUM, where the score of an element is summed across the inputs where it exists. When this option is set to be either MIN or MAX, the resulting set will contain the minimum or maximum score of an element across the inputs where it exists.

Return value

Integer reply, specifically the number of elements in the sorted set at dstkey. diff --git a/doc/index.html b/doc/index.html index 2cf5d9a8b..1c72b230b 100644 --- a/doc/index.html +++ b/doc/index.html @@ -26,12 +26,12 @@
- = Redis Documentation =

Russian TranslationHello! The followings are pointers to different parts of the Redis Documentation.

-

HOWTOs about selected features

  • The Redis Replication HOWTO is what you need to read in order to understand how Redis master <-> slave replication works.
  • The Append Only File HOWTO explains how the alternative Redis durability mode works. AOF is an alternative to snapshotting on disk from time to time (the default).
  • Virutal Memory User Guide. A simple to understand guide about using and configuring the Redis Virtual Memory.
+ = Redis Documentation =

Russian TranslationHello! The followings are pointers to different parts of the Redis Documentation.

+

HOWTOs about selected features

  • The Redis Replication HOWTO is what you need to read in order to understand how Redis master <-> slave replication works.
  • The Append Only File HOWTO explains how the alternative Redis durability mode works. AOF is an alternative to snapshotting on disk from time to time (the default).
  • Virtual Memory User Guide. A simple to understand guide about using and configuring the Redis Virtual Memory.

Hacking

  • The Protocol Specification is all you need in order to implement a Redis client library for a missing language. PHP, Python, Ruby and Erlang are already supported.
  • Look at Redis Internals if you are interested in the implementation details of the Redis server.
-

Videos

+

Videos

diff --git a/src/solarisfixes.h b/src/solarisfixes.h index ce8e7b6fd..3cb091d47 100644 --- a/src/solarisfixes.h +++ b/src/solarisfixes.h @@ -1,6 +1,7 @@ /* Solaris specific fixes */ #if defined(__GNUC__) +#include #undef isnan #define isnan(x) \ __extension__({ __typeof (x) __x_a = (x); \