Commit Graph

42 Commits

Author SHA1 Message Date
debing.sun
4a265554ae
Expose lua os.clock() api (#12971)
Implement #12699

This PR exposing Lua os.clock() api for getting the elapsed time of Lua
code execution.

Using:
```lua
local start = os.clock()
...
do something
...
local elpased = os.clock() - start
```

---------

Co-authored-by: Meir Shpilraien (Spielrein) <meir@redis.com>
Co-authored-by: Madelyn Olson <34459052+madolson@users.noreply.github.com>
2024-02-22 11:29:52 +02:00
sundb
da9c2804a5
Avoid mostly harmless integer overflow in cjson (#12456)
This PR mainly fixes a possible integer overflow in `json_append_string()`.
When we use `cjson.encoding()` to encode a string larger than 2GB, at specific
compilation flags, an integer overflow may occur leading to truncation, resulting
in the part of the string larger than 2GB not being encoded.
On the other hand, this overflow doesn't cause any read or write out-of-range or segment fault.

1) using -O0 for lua_cjson (`make LUA_DEBUG=yes`)
    In this case, `i` will overflow and leads to truncation.
    When `i` reaches `INT_MAX+1` and overflows to INT_MIN, when compared to
    len, `i` (1000000..00) is expanded to 64 bits signed integer (1111111.....000000) .
    At this point i will be greater than len and jump out of the loop, so `for (i = 0; i < len; i++)`
    will loop up to 2^31 times, and the part of larger than 2GB will be truncated.

```asm
`i` => -0x24(%rbp)
<+253>:   addl   $0x1,-0x24(%rbp)       ; overflow if i large than 2^31
<+257>:   mov    -0x24(%rbp),%eax
<+260>:   movslq %eax,%rdx	            ; move a 32-bit value with sign extension into a 64-bit signed
<+263>:   mov    -0x20(%rbp),%rax
<+267>:   cmp    %rax,%rdx              ; check `i < len`
<+270>:   jb     0x212600 <json_append_string+148>
```
   
2) using -O2/-O3 for lua_cjson (`make LUA_DEBUG=no`, **the default**)
    In this case, because singed integer overflow is an undefined behavior, `i` will not overflow.
   `i` will be optimized by the compiler and use 64-bit registers for all subsequent instructions.

```asm
<+180>:   add    $0x1,%rbx           ; Using 64-bit register `rbx` for i++
<+184>:   lea    0x1(%rdx),%rsi
<+188>:   mov    %rsi,0x10(%rbp)
<+192>:   mov    %al,(%rcx,%rdx,1)
<+195>:   cmp    %rbx,(%rsp)         ; check `i < len`
<+199>:   ja     0x20b63a <json_append_string+154>
```

3) using 32bit
    Because `strbuf_ensure_empty_length()` preallocates memory of length (len * 6 + 2),
    in 32-bit `cjson.encode()` can only handle strings smaller than ((2 ^ 32) - 3 ) / 6.
    So 32bit is not affected.

Also change `i` in `strbuf_append_string()` to `size_t`.
Since its second argument `str` is taken from the `char2escape` string array which is never
larger than 6, so `strbuf_append_string()` is not at risk of overflow (the bug was unreachable).
2023-08-05 07:57:06 +03:00
Oran Agra
936cfa464f
Lua cjson and cmsgpack integer overflow issues (CVE-2022-24834) (#12398)
* Fix integer overflows due to using wrong integer size.
* Add assertions / panic when overflow still happens.
* Deletion of dead code to avoid need to maintain it
* Some changes are not because of bugs, but rather paranoia.
* Improve cmsgpack and cjson test coverage.

Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
2023-07-10 10:26:09 +03:00
Ozan Tezcan
c5ff163d53
Fix Lua compile warning on GCC 12.1 (#11115)
Fix Lua compile warning on GCC 12.1

GCC 12.1 prints a warning on compile: 
```
ldump.c: In function ‘DumpString’:
ldump.c:63:26: warning: the comparison will always evaluate as ‘false’ for the pointer operand in ‘s + 24’ must not be NULL [-Waddress]
   63 |  if (s==NULL || getstr(s)==NULL)

```

It seems correct, `getstr(s)` can't be `NULL`.  
Also, I see Lua v5.2 does not have that check: https://github.com/lua/lua/blob/v5-2/ldump.c#L63
2022-08-14 14:29:05 +03:00
Ozan Tezcan
c81b5e5594
Fix Lua compile warning (#10805)
Apparently, GCC 11.2.0 has a new fancy warning for misleading indentations.
It prints a warning when BRET(b) is on the same line as the loop.
2022-06-01 16:21:45 +03:00
meir
efa162bcd7 Protect any table which is reachable from globals and added globals white list.
The white list is done by setting a metatable on the global table before initializing
any library. The metatable set the `__newindex` field to a function that check
the white list before adding the field to the table. Fields which is not on the
white list are simply ignored.

After initialization phase is done we protect the global table and each table
that might be reachable from the global table. For each table we also protect
the table metatable if exists.
2022-04-27 00:37:40 +03:00
meir
8b33d813a3 Added support for Lua readonly tables.
The new feature can be turned off and on using the new `lua_enablereadonlytable` Lua API.
2022-04-27 00:20:54 +03:00
Meir Shpilraien (Spielrein)
a8c1253b6f
Fix Lua C API violation on lua msgpack lib. (#9832)
msgpack lib missed using lua_checkstack and so on rare
cases overflow the stack by at most 2 elements. This is a
violation of the Lua C API. Notice that Lua allocates
additional 5 more elements on top of lua->stack_last
so Redis does not access an invalid memory. But it is an
API violation and we should avoid it.

This PR also added a new Lua compilation option. The new
option can be enable using environment variable called
LUA_DEBUG. If set to `yes` (by default `no`), Lua will be
compiled without optimizations and with debug symbols (`-O0 -g`).
In addition, in this new mode, Lua will be compiled with the
`-DLUA_USE_APICHECK` flag that enables extended Lua C API
validations.

In addition, set LUA_DEBUG=yes on daily valgrind flow so we
will be able to catch Lua C API violations in the future.
2021-11-28 11:33:09 +02:00
Kamil Cudnik
7f88923b40
Lua: Use all characters to calculate string hash (#9449)
For a lot of long strings which have same prefix which extends beyond
hashing limit, there will be many hash collisions which result in
performance degradation using commands like KEYS
2021-09-09 12:47:26 +03:00
YoongHM
448c435b1b
Fix compilation warnings in Lua and jemalloc dependencies (#7785)
- The argument `u` in for `ar` is ignored (and generates warnings since `D` became the default.
  All it does is avoid updating unchanged objects (shouldn't have any impact on our build)
- Enable `LUA_USE_MKSTEMP` to force the use of `mkstemp()` instead of `tmpname()` (which is dead
  code in redis anyway).
- Remove unused variable `c` in `f_parser()`
- Removed misleadingly indented space in `luaL_loadfile()` and ``addfield()`

Co-authored-by: Oran Agra <oran@redislabs.com>
2020-09-29 17:10:54 +03:00
Yossi Gottlieb
d75ad774a9
Backport Lua 5.2.2 stack overflow fix. (#7733)
This fixes the issue described in CVE-2014-5461. At this time we cannot
confirm that the original issue has a real impact on Redis, but it is
included as an extra safety measure.
2020-08-31 20:42:46 +03:00
Seunghoon Woo
ef764dde1c [FIX] revisit CVE-2015-8080 vulnerability 2020-02-10 16:32:46 +09:00
antirez
e89086e09a Security: fix Lua struct package offset handling.
After the first fix to the struct package I found another similar
problem, which is fixed by this patch. It could be reproduced easily by
running the following script:

    return struct.unpack('f', "xxxxxxxxxxxxx",-3)

The above will access bytes before the 'data' pointer.
2018-06-13 12:40:33 +02:00
antirez
5ccb6f7a79 Security: more cmsgpack fixes by @soloestoy.
@soloestoy sent me this additional fixes, after searching for similar
problems to the one reported in mp_pack(). I'm committing the changes
because it was not possible during to make a public PR to protect Redis
users and give Redis providers some time to patch their systems.
2018-06-13 12:40:33 +02:00
antirez
1eb08bcd46 Security: update Lua struct package for security.
During an auditing Apple found that the "struct" Lua package
we ship with Redis (http://www.inf.puc-rio.br/~roberto/struct/) contains
a security problem. A bound-checking statement fails because of integer
overflow. The bug exists since we initially integrated this package with
Lua, when scripting was introduced, so every version of Redis with
EVAL/EVALSHA capabilities exposed is affected.

Instead of just fixing the bug, the library was updated to the latest
version shipped by the author.
2018-06-13 12:40:33 +02:00
antirez
52a00201fc Security: fix Lua cmsgpack library stack overflow.
During an auditing effort, the Apple Vulnerability Research team discovered
a critical Redis security issue affecting the Lua scripting part of Redis.

-- Description of the problem

Several years ago I merged a pull request including many small changes at
the Lua MsgPack library (that originally I authored myself). The Pull
Request entered Redis in commit 90b6337c1, in 2014.
Unfortunately one of the changes included a variadic Lua function that
lacked the check for the available Lua C stack. As a result, calling the
"pack" MsgPack library function with a large number of arguments, results
into pushing into the Lua C stack a number of new values proportional to
the number of arguments the function was called with. The pushed values,
moreover, are controlled by untrusted user input.

This in turn causes stack smashing which we believe to be exploitable,
while not very deterministic, but it is likely that an exploit could be
created targeting specific versions of Redis executables. However at its
minimum the issue results in a DoS, crashing the Redis server.

-- Versions affected

Versions greater or equal to Redis 2.8.18 are affected.

-- Reproducing

Reproduce with this (based on the original reproduction script by
Apple security team):

https://gist.github.com/antirez/82445fcbea6d9b19f97014cc6cc79f8a

-- Verification of the fix

The fix was tested in the following way:

1) I checked that the problem is no longer observable running the trigger.
2) The Lua code was analyzed to understand the stack semantics, and that
actually enough stack is allocated in all the cases of mp_pack() calls.
3) The mp_pack() function was modified in order to show exactly what items
in the stack were being set, to make sure that there is no silent overflow
even after the fix.

-- Credits

Thank you to the Apple team and to the other persons that helped me
checking the patch and coordinating this communication.
2018-06-13 12:40:33 +02:00
antirez
fcd7df5eab Remove Lua state reference from buffers in lua_cmsgpack. 2016-02-10 09:12:17 +01:00
yoav@monfort.co.il
fdbefc9d83 cmsgpack: pass correct osize values to lua allocator, update correct buf free space in cmsgpack 2016-02-07 20:24:28 +02:00
Sun He
3a47c8cfb8 lua_struct.c/getnum: throw error if overflow happen
Fix issue #2855
2015-12-13 13:47:22 +08:00
Ben Murphy
fdf9d45509 disable loading lua bytecode 2015-06-03 13:33:28 +02:00
antirez
357a40c4fc Scripting: Lua cmsgpack lib updated to include str8 support 2015-06-03 08:44:43 +02:00
antirez
66e2bdf210 Lua cmsgpack lib updated to latest version.
It fixes a bad bug that crashes the server in certain conditions
as shown in issue #2210.
2014-12-12 22:56:33 +01:00
antirez
ff25d1896d Simplify lua_cmsgpack macro and fix build on old Linux distros.
Thanks to @badboy for the help in checking the build after the fix.
2014-12-05 10:51:25 +01:00
Matt Stancliff
6064371085 Fix lua-cmsgpack 64 bit integer on 32 bit platform
This syncs lua-cmsgpack with the mattsta/lua-cmsgpack upstream.

Fixes #2161
2014-11-24 12:45:15 -05:00
antirez
0ed2c60118 lua_cjson.c Lua includes: angled -> quoted. 2014-11-14 17:16:23 +01:00
antirez
8a75ec0fca Merge remote-tracking branch 'origin/unstable' into unstable 2014-11-14 17:10:48 +01:00
Matt Stancliff
90b6337c15 Lua: upgrade cmsgpack to 0.4.0
Main reasons for upgrade:
  - Remove a warning when building Redis
  - Add multi pack/unpack
  - Improve memory usage and use Lua allocator properly
  - Fix some edge case encoding/decoding bugs
2014-11-14 17:08:51 +01:00
Matt Stancliff
da18dd34a2 Lua: remove new warning added by cjson header
clang doesn't like "extern inline" when no definition
is given right away.
2014-11-14 17:08:44 +01:00
Matt Stancliff
a9900ad38e Lua: Use Redis solaris compatability for cjson too
cjson calls isinf, but some Solaris versions don't have isinf
even with the attempted fix we have in deps/Makefile.

We can harmlessly include the Redis solarisfixes.h header to
give cjson proper isinf.

Note: cjson has a compile-time setting for using their own defined
isinf, but the Redis definition in solarisfixes.h is more complete.

Fixes antirez#1620
2014-11-14 17:08:39 +01:00
Matt Stancliff
4fdcd213f0 Lua: Upgrade cjson to 2.1.0 (2012-03-01)
The new cjson has some improvements over our current version including
increased platform compatability, a new resource limit to restrict
decode depth, and better invalid number handling.

One minor change was required to deps/Makefile because this version
of cjson doesn't export itself globally, so we added a quick little
define of -DENABLE_CJSON_GLOBAL.

cjson now has an optional higher performing float parsing interface,
but we are not including it (g_fmt.c, dtoa.c) because it requires
endianness declaration during compile time.

This commit is exactly lua_cjson.c from 2.1.0 with one minor
change of altering the two Lua includes for local search
instead of system-wide importing.
2014-11-14 17:08:33 +01:00
Matt Stancliff
3fecb96122 Lua: Add bitop
A few people have written custom C commands because bit
manipulation isn't exposed through Lua.  Let's give
them Mike Pall's bitop.

This adds bitop 1.0.2 (2012-05-08) from http://bitop.luajit.org/

bitop is imported as "bit" into the global namespace.

New Lua commands: bit.tobit, bit.tohex, bit.bnot, bit.band, bit.bor, bit.bxor,
bit.lshift, bit.rshift, bit.arshift, bit.rol, bit.ror, bit.bswap

Verification of working (the asserts would abort on error, so (nil) is correct):
127.0.0.1:6379> eval "assert(bit.tobit(1) == 1); assert(bit.band(1) == 1); assert(bit.bxor(1,2) == 3); assert(bit.bor(1,2,4,8,16,32,64,128) == 255)" 0
(nil)
127.0.0.1:6379> eval 'assert(0x7fffffff == 2147483647, "broken hex literals"); assert(0xffffffff == -1 or 0xffffffff == 2^32-1, "broken hex literals"); assert(tostring(-1) == "-1", "broken tostring()"); assert(tostring(0xffffffff) == "-1" or tostring(0xffffffff) == "4294967295", "broken tostring()")' 0
(nil)

Tests also integrated into the scripting tests and can be run with:
./runtest --single unit/scripting

Tests are excerpted from `bittest.lua` included in the bitop distribution.
2014-10-09 11:51:30 -04:00
yihuang
9bb91d198c fix lua_cmsgpack pack map as array 2013-08-27 15:19:25 +02:00
antirez
214adc5051 Lua updated to version 5.1.5. 2013-04-29 12:00:54 +02:00
Johan Bergström
a66195629b Inherit CC for Lua 2013-03-16 18:38:37 +11:00
antirez
61853a9c75 Lua struct library updated to version 0.2.
There was a bug in the previous version of this library that caused a
crash under the circumstances described in issue #901.

The newer version of the library appears to be fixed (I tested it
manually with valgrind and everything seems fine now).

For more information about this library please visit this web site:

    http://www.inf.puc-rio.br/~roberto/struct/
2013-01-23 10:50:48 +01:00
antirez
36e6637dfd lua_cmsgpack.c added 2012-02-24 15:46:25 +01:00
antirez
63505e0b59 Lua_cmsgpack added to Redis scripting. 2012-02-24 15:45:16 +01:00
lsbardel
2f75bbab02 added lua struct c extension 2012-02-13 21:05:21 +00:00
antirez
b28d0461b4 32bit build fixed, broken by a previous commit fixing build on Solaris 2011-11-08 21:00:52 +01:00
antirez
34296ae890 Lua cjson include paths now use the local includes. 2011-10-19 16:57:54 +02:00
antirez
15108778d5 JSON support for Lua scripting, based on work from @lp, thanks!. We are using the good and fast cjson by Mark Pulford. 2011-10-19 16:42:10 +02:00
antirez
21d3294c70 makefile adapted to link against lua lib and to pass the 32bit flag to Lua building system 2011-05-25 12:32:43 +02:00