mirrors/valkey
1
0
Fork 0
mirror of http://github.com/valkey-io/valkey synced 2024-11-22 00:52:38 +00:00
Code Issues Packages Projects Releases Wiki Activity
8.0
valkey/deps/Makefile

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

119 lines
3.3 KiB
Makefile
Raw Permalink Normal View History

Remove trademarked language in code comments (#223) This includes comments used for module API documentation. * Strategy for replacement: Regex search: `(//|/\*| \*|#).* ("|\()?(r|R)edis( |\. |'|\n|,|-|\)|")(?!nor the names of its contributors)(?!Ltd.)(?!Labs)(?!Contributors.)` * Don't edit copyright comments * Replace "Redis version X.X" -> "Redis OSS version X.X" to distinguish from newly licensed repository * Replace "Redis Object" -> "Object" * Exclude markdown for now * Don't edit Lua scripting comments referring to redis.X API * Replace "Redis Protocol" -> "RESP" * Replace redis-benchmark, -cli, -server, -check-aof/rdb with "valkey-" prefix * Most other places, I use best judgement to either remove "Redis", or replace with "the server" or "server" Fixes #148 --------- Signed-off-by: Jacob Murphy <jkmurphy@google.com> Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2024-04-09 08:24:03 +00:00
# Dependency Makefile
Rebuild deps/ and src/ when ARCH changes This change moves the build instructions for dependencies to a separate Makefile in deps/. The ARCH environment variable is stored in a .make-arch file in the same directory as the Makefile. The contents of this file is read and compared to the current ARCH, and, on a mismatch triggers rebuilding the entire source tree. When file .make-arch exists and matches with ARCH from the environment, the dependencies are assumed to already be built. The new "clean" target only cleans the Redis source tree, not its dependencies. To clear the dependencies as well, the "distclean" target can be used.
2011-11-15 20:40:49 +00:00
Clean up Makefiles Remove unused variables. Instead of overriding non-standard variables such as ARCH and PROF, use standard variables CFLAGS and LDFLAGS to override Makefile settings. Move dependencies generated by `make dep` to a separate file.
2012-03-25 02:25:03 +00:00
uname_S:= $(shell sh -c 'uname -s 2>/dev/null || echo not')
Rebuild deps/ and src/ when ARCH changes This change moves the build instructions for dependencies to a separate Makefile in deps/. The ARCH environment variable is stored in a .make-arch file in the same directory as the Makefile. The contents of this file is read and compared to the current ARCH, and, on a mismatch triggers rebuilding the entire source tree. When file .make-arch exists and matches with ARCH from the environment, the dependencies are assumed to already be built. The new "clean" target only cleans the Redis source tree, not its dependencies. To clear the dependencies as well, the "distclean" target can be used.
2011-11-15 20:40:49 +00:00
Fix Lua C API violation on lua msgpack lib. (#9832) msgpack lib missed using lua_checkstack and so on rare cases overflow the stack by at most 2 elements. This is a violation of the Lua C API. Notice that Lua allocates additional 5 more elements on top of lua->stack_last so Redis does not access an invalid memory. But it is an API violation and we should avoid it. This PR also added a new Lua compilation option. The new option can be enable using environment variable called LUA_DEBUG. If set to `yes` (by default `no`), Lua will be compiled without optimizations and with debug symbols (`-O0 -g`). In addition, in this new mode, Lua will be compiled with the `-DLUA_USE_APICHECK` flag that enables extended Lua C API validations. In addition, set LUA_DEBUG=yes on daily valgrind flow so we will be able to catch Lua C API violations in the future.
2021-11-28 09:33:09 +00:00
LUA_DEBUG?=no
Lua cjson and cmsgpack integer overflow issues (CVE-2022-24834) (#12398) * Fix integer overflows due to using wrong integer size. * Add assertions / panic when overflow still happens. * Deletion of dead code to avoid need to maintain it * Some changes are not because of bugs, but rather paranoia. * Improve cmsgpack and cjson test coverage. Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
2023-07-10 07:26:09 +00:00
LUA_COVERAGE?=no
Fix Lua C API violation on lua msgpack lib. (#9832) msgpack lib missed using lua_checkstack and so on rare cases overflow the stack by at most 2 elements. This is a violation of the Lua C API. Notice that Lua allocates additional 5 more elements on top of lua->stack_last so Redis does not access an invalid memory. But it is an API violation and we should avoid it. This PR also added a new Lua compilation option. The new option can be enable using environment variable called LUA_DEBUG. If set to `yes` (by default `no`), Lua will be compiled without optimizations and with debug symbols (`-O0 -g`). In addition, in this new mode, Lua will be compiled with the `-DLUA_USE_APICHECK` flag that enables extended Lua C API validations. In addition, set LUA_DEBUG=yes on daily valgrind flow so we will be able to catch Lua C API violations in the future.
2021-11-28 09:33:09 +00:00
Rebuild deps/ and src/ when ARCH changes This change moves the build instructions for dependencies to a separate Makefile in deps/. The ARCH environment variable is stored in a .make-arch file in the same directory as the Makefile. The contents of this file is read and compared to the current ARCH, and, on a mismatch triggers rebuilding the entire source tree. When file .make-arch exists and matches with ARCH from the environment, the dependencies are assumed to already be built. The new "clean" target only cleans the Redis source tree, not its dependencies. To clear the dependencies as well, the "distclean" target can be used.
2011-11-15 20:40:49 +00:00
CCCOLOR="\033[34m"
LINKCOLOR="\033[34;1m"
SRCCOLOR="\033[33m"
BINCOLOR="\033[37;1m"
MAKECOLOR="\033[32;1m"
ENDCOLOR="\033[0m"
default:
@echo "Explicit target required"
Clean up Makefiles Remove unused variables. Instead of overriding non-standard variables such as ARCH and PROF, use standard variables CFLAGS and LDFLAGS to override Makefile settings. Move dependencies generated by `make dep` to a separate file.
2012-03-25 02:25:03 +00:00
.PHONY: default
# Prerequisites target
.make-prerequisites:
@touch $@
# Clean everything when CFLAGS is different
ifneq ($(shell sh -c '[ -f .make-cflags ] && cat .make-cflags || echo none'), $(CFLAGS))
.make-cflags: distclean
-(echo "$(CFLAGS)" > .make-cflags)
.make-prerequisites: .make-cflags
Rebuild deps/ and src/ when ARCH changes This change moves the build instructions for dependencies to a separate Makefile in deps/. The ARCH environment variable is stored in a .make-arch file in the same directory as the Makefile. The contents of this file is read and compared to the current ARCH, and, on a mismatch triggers rebuilding the entire source tree. When file .make-arch exists and matches with ARCH from the environment, the dependencies are assumed to already be built. The new "clean" target only cleans the Redis source tree, not its dependencies. To clear the dependencies as well, the "distclean" target can be used.
2011-11-15 20:40:49 +00:00
endif
Clean up Makefiles Remove unused variables. Instead of overriding non-standard variables such as ARCH and PROF, use standard variables CFLAGS and LDFLAGS to override Makefile settings. Move dependencies generated by `make dep` to a separate file.
2012-03-25 02:25:03 +00:00
# Clean everything when LDFLAGS is different
ifneq ($(shell sh -c '[ -f .make-ldflags ] && cat .make-ldflags || echo none'), $(LDFLAGS))
.make-ldflags: distclean
-(echo "$(LDFLAGS)" > .make-ldflags)
.make-prerequisites: .make-ldflags
endif
Rebuild deps/ and src/ when ARCH changes This change moves the build instructions for dependencies to a separate Makefile in deps/. The ARCH environment variable is stored in a .make-arch file in the same directory as the Makefile. The contents of this file is read and compared to the current ARCH, and, on a mismatch triggers rebuilding the entire source tree. When file .make-arch exists and matches with ARCH from the environment, the dependencies are assumed to already be built. The new "clean" target only cleans the Redis source tree, not its dependencies. To clear the dependencies as well, the "distclean" target can be used.
2011-11-15 20:40:49 +00:00
distclean:
-(cd hiredis && $(MAKE) clean) > /dev/null || true
-(cd linenoise && $(MAKE) clean) > /dev/null || true
-(cd lua && $(MAKE) clean) > /dev/null || true
-(cd jemalloc && [ -f Makefile ] && $(MAKE) distclean) > /dev/null || true
Extended redis-benchmark instant metrics and overall latency report (#7600) A first step to enable a consistent full percentile analysis on query latency so that we can fully understand the performance and stability characteristics of the redis-server system we are measuring. It also improves the instantaneous reported metrics, and the csv output format.
2020-08-25 18:21:29 +00:00
-(cd hdr_histogram && $(MAKE) clean) > /dev/null || true
optimizing d2string() and addReplyDouble() with grisu2: double to string conversion based on Florian Loitsch's Grisu-algorithm (#10587) All commands / use cases that heavily rely on double to a string representation conversion, (e.g. meaning take a double-precision floating-point number like 1.5 and return a string like "1.5" ), could benefit from a performance boost by swapping snprintf(buf,len,"%.17g",value) by the equivalent [fpconv_dtoa](https://github.com/night-shift/fpconv) or any other algorithm that ensures 100% coverage of conversion. This is a well-studied topic and Projects like MongoDB. RedPanda, PyTorch leverage libraries ( fmtlib ) that use the optimized double to string conversion underneath. The positive impact can be substantial. This PR uses the grisu2 approach ( grisu explained on https://www.cs.tufts.edu/~nr/cs257/archive/florian-loitsch/printf.pdf section 5 ). test suite changes: Despite being compatible, in some cases it produces a different result from printf, and some tests had to be adjusted. one case is that `%.17g` (which means %e or %f which ever is shorter), chose to use `5000000000` instead of 5e+9, which sounds like a bug? In other cases, we changed TCL to compare numbers instead of strings to ignore minor rounding issues (`expr 0.8 == 0.79999999999999999`)
2022-10-15 09:17:41 +00:00
-(cd fpconv && $(MAKE) clean) > /dev/null || true
Clean up Makefiles Remove unused variables. Instead of overriding non-standard variables such as ARCH and PROF, use standard variables CFLAGS and LDFLAGS to override Makefile settings. Move dependencies generated by `make dep` to a separate file.
2012-03-25 02:25:03 +00:00
-(rm -f .make-*)
.PHONY: distclean
Build TLS as a loadable module * Support BUILD_TLS=module to be loaded as a module via config file or command line. e.g. redis-server --loadmodule redis-tls.so * Updates to redismodule.h to allow it to be used side by side with server.h by defining REDISMODULE_CORE_MODULE * Changes to server.h, redismodule.h and module.c to avoid repeated type declarations (gcc 4.8 doesn't like these) * Add a mechanism for non-ABI neutral modules (ones who include server.h) to refuse loading if they detect not being built together with redis (release.c) * Fix wrong signature of RedisModuleDefragFunc, this could break compilation of a module, but not the ABI * Move initialization of listeners in server.c to be after loading the modules * Config TLS after initialization of listeners * Init cluster after initialization of listeners * Add TLS module to CI * Fix a test suite race conditions: Now that the listeners are initialized later, it's not sufficient to wait for the PID message in the log, we need to wait for the "Server Initialized" message. * Fix issues with moduleconfigs test as a result from start_server waiting for "Server Initialized" * Fix issues with modules/infra test as a result of an additional module present Notes about Sentinel: Sentinel can't really rely on the tls module, since it uses hiredis to initiate connections and depends on OpenSSL (won't be able to use any other connection modules for that), so it was decided that when TLS is built as a module, sentinel does not support TLS at all. This means that it keeps using redis_tls_ctx and redis_tls_client_ctx directly. Example code of config in redis-tls.so(may be use in the future): RedisModuleString *tls_cfg = NULL; void tlsInfo(RedisModuleInfoCtx *ctx, int for_crash_report) { UNUSED(for_crash_report); RedisModule_InfoAddSection(ctx, ""); RedisModule_InfoAddFieldLongLong(ctx, "var", 42); } int tlsCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { if (argc != 2) return RedisModule_WrongArity(ctx); return RedisModule_ReplyWithString(ctx, argv[1]); } RedisModuleString *getStringConfigCommand(const char *name, void *privdata) { REDISMODULE_NOT_USED(name); REDISMODULE_NOT_USED(privdata); return tls_cfg; } int setStringConfigCommand(const char *name, RedisModuleString *new, void *privdata, RedisModuleString **err) { REDISMODULE_NOT_USED(name); REDISMODULE_NOT_USED(err); REDISMODULE_NOT_USED(privdata); if (tls_cfg) RedisModule_FreeString(NULL, tls_cfg); RedisModule_RetainString(NULL, new); tls_cfg = new; return REDISMODULE_OK; } int RedisModule_OnLoad(void *ctx, RedisModuleString **argv, int argc) { .... if (RedisModule_CreateCommand(ctx,"tls",tlsCommand,"",0,0,0) == REDISMODULE_ERR) return REDISMODULE_ERR; if (RedisModule_RegisterStringConfig(ctx, "cfg", "", REDISMODULE_CONFIG_DEFAULT, getStringConfigCommand, setStringConfigCommand, NULL, NULL) == REDISMODULE_ERR) return REDISMODULE_ERR; if (RedisModule_LoadConfigs(ctx) == REDISMODULE_ERR) { if (tls_cfg) { RedisModule_FreeString(ctx, tls_cfg); tls_cfg = NULL; } return REDISMODULE_ERR; } ... } Co-authored-by: zhenwei pi <pizhenwei@bytedance.com> Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
2022-08-22 07:53:56 +00:00
ifneq (,$(filter $(BUILD_TLS),yes module))
TLS: Connections refactoring and TLS support. * Introduce a connection abstraction layer for all socket operations and integrate it across the code base. * Provide an optional TLS connections implementation based on OpenSSL. * Pull a newer version of hiredis with TLS support. * Tests, redis-cli updates for TLS support.
2019-09-12 07:56:54 +00:00
HIREDIS_MAKE_FLAGS = USE_SSL=1
endif
Clean up Makefiles Remove unused variables. Instead of overriding non-standard variables such as ARCH and PROF, use standard variables CFLAGS and LDFLAGS to override Makefile settings. Move dependencies generated by `make dep` to a separate file.
2012-03-25 02:25:03 +00:00
hiredis: .make-prerequisites
@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
TLS: Connections refactoring and TLS support. * Introduce a connection abstraction layer for all socket operations and integrate it across the code base. * Provide an optional TLS connections implementation based on OpenSSL. * Pull a newer version of hiredis with TLS support. * Tests, redis-cli updates for TLS support.
2019-09-12 07:56:54 +00:00
cd hiredis && $(MAKE) static $(HIREDIS_MAKE_FLAGS)
Clean up Makefiles Remove unused variables. Instead of overriding non-standard variables such as ARCH and PROF, use standard variables CFLAGS and LDFLAGS to override Makefile settings. Move dependencies generated by `make dep` to a separate file.
2012-03-25 02:25:03 +00:00
.PHONY: hiredis
linenoise: .make-prerequisites
@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
cd linenoise && $(MAKE)
.PHONY: linenoise
Extended redis-benchmark instant metrics and overall latency report (#7600) A first step to enable a consistent full percentile analysis on query latency so that we can fully understand the performance and stability characteristics of the redis-server system we are measuring. It also improves the instantaneous reported metrics, and the csv output format.
2020-08-25 18:21:29 +00:00
hdr_histogram: .make-prerequisites
@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
cd hdr_histogram && $(MAKE)
.PHONY: hdr_histogram
optimizing d2string() and addReplyDouble() with grisu2: double to string conversion based on Florian Loitsch's Grisu-algorithm (#10587) All commands / use cases that heavily rely on double to a string representation conversion, (e.g. meaning take a double-precision floating-point number like 1.5 and return a string like "1.5" ), could benefit from a performance boost by swapping snprintf(buf,len,"%.17g",value) by the equivalent [fpconv_dtoa](https://github.com/night-shift/fpconv) or any other algorithm that ensures 100% coverage of conversion. This is a well-studied topic and Projects like MongoDB. RedPanda, PyTorch leverage libraries ( fmtlib ) that use the optimized double to string conversion underneath. The positive impact can be substantial. This PR uses the grisu2 approach ( grisu explained on https://www.cs.tufts.edu/~nr/cs257/archive/florian-loitsch/printf.pdf section 5 ). test suite changes: Despite being compatible, in some cases it produces a different result from printf, and some tests had to be adjusted. one case is that `%.17g` (which means %e or %f which ever is shorter), chose to use `5000000000` instead of 5e+9, which sounds like a bug? In other cases, we changed TCL to compare numbers instead of strings to ignore minor rounding issues (`expr 0.8 == 0.79999999999999999`)
2022-10-15 09:17:41 +00:00
fpconv: .make-prerequisites
@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
cd fpconv && $(MAKE)
.PHONY: fpconv
Clean up Makefiles Remove unused variables. Instead of overriding non-standard variables such as ARCH and PROF, use standard variables CFLAGS and LDFLAGS to override Makefile settings. Move dependencies generated by `make dep` to a separate file.
2012-03-25 02:25:03 +00:00
ifeq ($(uname_S),SunOS)
Spaces to tabs
2013-03-16 07:35:20 +00:00
# Make isinf() available
LUA_CFLAGS= -D__C99FEATURES__=1
Clean up Makefiles Remove unused variables. Instead of overriding non-standard variables such as ARCH and PROF, use standard variables CFLAGS and LDFLAGS to override Makefile settings. Move dependencies generated by `make dep` to a separate file.
2012-03-25 02:25:03 +00:00
endif
Replacing REDIS_STATIC with static (#691) As discussed, we want to remove the old `REDIS_STATIC` flag which is no longer relevant. When moving from Redis to Valkey we renamed all REDIS flags in Makefile. The REDIS_STATIC flag was renamed to SERVER_STATIC, but this change was not updated in some of the files. After discussing it with @madolson and @ranshid, we decided that since this was introduced 10 years ago, and in many places in the code base we simply use `static`, we should simplify and remove the flag entirely. --------- Signed-off-by: Ouri Half <ourih@amazon.com>
2024-06-26 16:47:59 +00:00
LUA_CFLAGS+= -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -DLUA_USE_MKSTEMP $(CFLAGS)
Clean up Makefiles Remove unused variables. Instead of overriding non-standard variables such as ARCH and PROF, use standard variables CFLAGS and LDFLAGS to override Makefile settings. Move dependencies generated by `make dep` to a separate file.
2012-03-25 02:25:03 +00:00
LUA_LDFLAGS+= $(LDFLAGS)
Fix Lua C API violation on lua msgpack lib. (#9832) msgpack lib missed using lua_checkstack and so on rare cases overflow the stack by at most 2 elements. This is a violation of the Lua C API. Notice that Lua allocates additional 5 more elements on top of lua->stack_last so Redis does not access an invalid memory. But it is an API violation and we should avoid it. This PR also added a new Lua compilation option. The new option can be enable using environment variable called LUA_DEBUG. If set to `yes` (by default `no`), Lua will be compiled without optimizations and with debug symbols (`-O0 -g`). In addition, in this new mode, Lua will be compiled with the `-DLUA_USE_APICHECK` flag that enables extended Lua C API validations. In addition, set LUA_DEBUG=yes on daily valgrind flow so we will be able to catch Lua C API violations in the future.
2021-11-28 09:33:09 +00:00
ifeq ($(LUA_DEBUG),yes)
LUA_CFLAGS+= -O0 -g -DLUA_USE_APICHECK
else
LUA_CFLAGS+= -O2
endif
Lua cjson and cmsgpack integer overflow issues (CVE-2022-24834) (#12398) * Fix integer overflows due to using wrong integer size. * Add assertions / panic when overflow still happens. * Deletion of dead code to avoid need to maintain it * Some changes are not because of bugs, but rather paranoia. * Improve cmsgpack and cjson test coverage. Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
2023-07-10 07:26:09 +00:00
ifeq ($(LUA_COVERAGE),yes)
LUA_CFLAGS += -fprofile-arcs -ftest-coverage
LUA_LDFLAGS += -fprofile-arcs -ftest-coverage
endif
Define AR to help with lua cross-compilation Closes #997
2013-03-12 01:04:40 +00:00
# lua's Makefile defines AR="ar rcu", which is unusual, and makes it more
# challenging to cross-compile lua (and redis). These defines make it easier
# to fit redis into cross-compilation environments, which typically set AR.
AR=ar
Fix compilation warnings in Lua and jemalloc dependencies (#7785) - The argument `u` in for `ar` is ignored (and generates warnings since `D` became the default. All it does is avoid updating unchanged objects (shouldn't have any impact on our build) - Enable `LUA_USE_MKSTEMP` to force the use of `mkstemp()` instead of `tmpname()` (which is dead code in redis anyway). - Remove unused variable `c` in `f_parser()` - Removed misleadingly indented space in `luaL_loadfile()` and ``addfield()` Co-authored-by: Oran Agra <oran@redislabs.com>
2020-09-29 14:10:54 +00:00
ARFLAGS=rc
Rebuild deps/ and src/ when ARCH changes This change moves the build instructions for dependencies to a separate Makefile in deps/. The ARCH environment variable is stored in a .make-arch file in the same directory as the Makefile. The contents of this file is read and compared to the current ARCH, and, on a mismatch triggers rebuilding the entire source tree. When file .make-arch exists and matches with ARCH from the environment, the dependencies are assumed to already be built. The new "clean" target only cleans the Redis source tree, not its dependencies. To clear the dependencies as well, the "distclean" target can be used.
2011-11-15 20:40:49 +00:00
Clean up Makefiles Remove unused variables. Instead of overriding non-standard variables such as ARCH and PROF, use standard variables CFLAGS and LDFLAGS to override Makefile settings. Move dependencies generated by `make dep` to a separate file.
2012-03-25 02:25:03 +00:00
lua: .make-prerequisites
@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
Define AR to help with lua cross-compilation Closes #997
2013-03-12 01:04:40 +00:00
cd lua/src && $(MAKE) all CFLAGS="$(LUA_CFLAGS)" MYLDFLAGS="$(LUA_LDFLAGS)" AR="$(AR) $(ARFLAGS)"
Rebuild deps/ and src/ when ARCH changes This change moves the build instructions for dependencies to a separate Makefile in deps/. The ARCH environment variable is stored in a .make-arch file in the same directory as the Makefile. The contents of this file is read and compared to the current ARCH, and, on a mismatch triggers rebuilding the entire source tree. When file .make-arch exists and matches with ARCH from the environment, the dependencies are assumed to already be built. The new "clean" target only cleans the Redis source tree, not its dependencies. To clear the dependencies as well, the "distclean" target can be used.
2011-11-15 20:40:49 +00:00
Clean up Makefiles Remove unused variables. Instead of overriding non-standard variables such as ARCH and PROF, use standard variables CFLAGS and LDFLAGS to override Makefile settings. Move dependencies generated by `make dep` to a separate file.
2012-03-25 02:25:03 +00:00
.PHONY: lua
Rebuild deps/ and src/ when ARCH changes This change moves the build instructions for dependencies to a separate Makefile in deps/. The ARCH environment variable is stored in a .make-arch file in the same directory as the Makefile. The contents of this file is read and compared to the current ARCH, and, on a mismatch triggers rebuilding the entire source tree. When file .make-arch exists and matches with ARCH from the environment, the dependencies are assumed to already be built. The new "clean" target only cleans the Redis source tree, not its dependencies. To clear the dependencies as well, the "distclean" target can be used.
2011-11-15 20:40:49 +00:00
deps/jemalloc: Do not force building in gnu99 mode (#11583) Previously, jemalloc was explicitly configured to build in `gnu99` mode. As a result, `<stdatomic.h>` was presumed to be unavailable and never used. This commit removes explicit build flags configuration and lets `autoconf` determine the supported build flags. In addition, we also no longer build C++ jemalloc code. Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
2023-02-02 14:58:16 +00:00
JEMALLOC_CFLAGS=$(CFLAGS)
JEMALLOC_LDFLAGS=$(LDFLAGS)
Rebuild deps/ and src/ when ARCH changes This change moves the build instructions for dependencies to a separate Makefile in deps/. The ARCH environment variable is stored in a .make-arch file in the same directory as the Makefile. The contents of this file is read and compared to the current ARCH, and, on a mismatch triggers rebuilding the entire source tree. When file .make-arch exists and matches with ARCH from the environment, the dependencies are assumed to already be built. The new "clean" target only cleans the Redis source tree, not its dependencies. To clear the dependencies as well, the "distclean" target can be used.
2011-11-15 20:40:49 +00:00
Handle cross-compiling when configuring jemalloc. (#9659)
2021-10-24 07:15:32 +00:00
ifneq ($(DEB_HOST_GNU_TYPE),)
JEMALLOC_CONFIGURE_OPTS += --host=$(DEB_HOST_GNU_TYPE)
endif
Clean up Makefiles Remove unused variables. Instead of overriding non-standard variables such as ARCH and PROF, use standard variables CFLAGS and LDFLAGS to override Makefile settings. Move dependencies generated by `make dep` to a separate file.
2012-03-25 02:25:03 +00:00
jemalloc: .make-prerequisites
@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
Set Jemalloc --disable-cache-oblivious to reduce memory overhead (#12315) Apparently for large size classes Jemalloc allocate some extra memory (can be up to 25% overhead for allocations of 16kb). see https://github.com/jemalloc/jemalloc/issues/1098#issuecomment-1589870476 p.s. from Redis's perspective that looks like external fragmentation, (i.e. allocated bytes will be low, and active pages bytes will be large) which can cause active-defrag to eat CPU cycles in vain. Some details about this mechanism we disable: --------------------------------------------------------------- Disabling this mechanism only affects large allocations (above 16kb) Not only that it isn't expected to cause any performance regressions, it's actually recommended, unless you have a specific workload pattern and hardware that benefit from this feature -- by default it's enabled and adds address randomization to all large buffers, by over allocating 1 page per large size class, and offsetting into that page to make the starting address of the user buffer randomized. Workloads such as scientific computation often handle multiple big matrixes at the same time, and the randomization makes sure that the cacheline level accesses don't suffer bad conflicts (when they all start from page-aligned addresses). However the downsize is also quite noticeable, like you observed that extra page per large size can cause memory overhead, plus the extra TLB entry. The other factor is, hardware in the last few years started doing the randomization at the hardware level, i.e. the address to cacheline mapping isn't a direct mapping anymore. So there's debate to disable the randomization by default, but we are still hesitant because when it matters, it could matter a lot, and having it enabled by default limits that worst case behavior, even though it means the majority of workloads suffers a regression. So in short, it's safe and offers better performance in most cases.
2023-06-18 07:30:23 +00:00
cd jemalloc && ./configure --disable-cxx --with-version=5.3.0-0-g0 --with-lg-quantum=3 --disable-cache-oblivious --with-jemalloc-prefix=je_ CFLAGS="$(JEMALLOC_CFLAGS)" LDFLAGS="$(JEMALLOC_LDFLAGS)" $(JEMALLOC_CONFIGURE_OPTS)
deps/jemalloc: Do not force building in gnu99 mode (#11583) Previously, jemalloc was explicitly configured to build in `gnu99` mode. As a result, `<stdatomic.h>` was presumed to be unavailable and never used. This commit removes explicit build flags configuration and lets `autoconf` determine the supported build flags. In addition, we also no longer build C++ jemalloc code. Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
2023-02-02 14:58:16 +00:00
cd jemalloc && $(MAKE) lib/libjemalloc.a
Rebuild deps/ and src/ when ARCH changes This change moves the build instructions for dependencies to a separate Makefile in deps/. The ARCH environment variable is stored in a .make-arch file in the same directory as the Makefile. The contents of this file is read and compared to the current ARCH, and, on a mismatch triggers rebuilding the entire source tree. When file .make-arch exists and matches with ARCH from the environment, the dependencies are assumed to already be built. The new "clean" target only cleans the Redis source tree, not its dependencies. To clear the dependencies as well, the "distclean" target can be used.
2011-11-15 20:40:49 +00:00
Clean up Makefiles Remove unused variables. Instead of overriding non-standard variables such as ARCH and PROF, use standard variables CFLAGS and LDFLAGS to override Makefile settings. Move dependencies generated by `make dep` to a separate file.
2012-03-25 02:25:03 +00:00
.PHONY: jemalloc
Reference in New Issue Copy Permalink