zitadel/cmd/setup
Tim Möhlmann a84b259e8c
perf(oidc): nest position clause for session terminated query (#8738)
# Which Problems Are Solved

Optimize the query that checks for terminated sessions in the access
token verifier. The verifier is used in auth middleware, userinfo and
introspection.


# How the Problems Are Solved

The previous implementation built a query for certain events and then
appended a single `PositionAfter` clause. This caused the postgreSQL
planner to use indexes only for the instance ID, aggregate IDs,
aggregate types and event types. Followed by an expensive sequential
scan for the position. This resulting in internal over-fetching of rows
before the final filter was applied.


![Screenshot_20241007_105803](https://github.com/user-attachments/assets/f2d91976-be87-428b-b604-a211399b821c)

Furthermore, the query was searching for events which are not always
applicable. For example, there was always a session ID search and if
there was a user ID, we would also search for a browser fingerprint in
event payload (expensive). Even if those argument string would be empty.

This PR changes:

1. Nest the position query, so that a full `instance_id, aggregate_id,
aggregate_type, event_type, "position"` index can be matched.
2. Redefine the `es_wm` index to include the `position` column.
3. Only search for events for the IDs that actually have a value. Do not
search (noop) if none of session ID, user ID or fingerpint ID are set.

New query plan:


![Screenshot_20241007_110648](https://github.com/user-attachments/assets/c3234c33-1b76-4b33-a4a9-796f69f3d775)


# Additional Changes

- cleanup how we load multi-statement migrations and make that a bit
more reusable.

# Additional Context

- Related to https://github.com/zitadel/zitadel/issues/7639
2024-10-07 12:49:55 +00:00
..
01_sql feat(eventstore): increase parallel write capabilities (#5940) 2023-10-19 12:19:10 +02:00
06 fix(setup): create indexes only if not exists (#4801) 2022-12-01 14:31:01 +01:00
07 fix(init): correct quoting of database and user (#6928) 2023-12-03 08:30:08 +00:00
08 fix(setup): optimise events indexes (#5316) 2023-03-01 00:11:23 +00:00
10 feat(eventstore): increase parallel write capabilities (#5940) 2023-10-19 12:19:10 +02:00
12 feat(login): add OTP (email and sms) (#6353) 2023-08-15 12:47:05 +00:00
13 fix: set quotas (#6597) 2023-09-22 13:06:59 +02:00
14 feat(eventstore): increase parallel write capabilities (#5940) 2023-10-19 12:19:10 +02:00
15 feat(eventstore): increase parallel write capabilities (#5940) 2023-10-19 12:19:10 +02:00
34 perf(cache): pgx pool connector (#8703) 2024-10-04 13:15:41 +00:00
35 perf(oidc): nest position clause for session terminated query (#8738) 2024-10-07 12:49:55 +00:00
01.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
02.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
03.go feat(cmd): mirror (#7004) 2024-05-30 09:35:30 +00:00
05.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
05.sql fix(database): create column only if not exists (#4812) 2022-12-02 10:48:39 +00:00
06.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
07.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
08.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
10.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
12.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
13.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
14.go perf(oidc): nest position clause for session terminated query (#8738) 2024-10-07 12:49:55 +00:00
15.go perf(oidc): nest position clause for session terminated query (#8738) 2024-10-07 12:49:55 +00:00
16.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
16.sql fix: ensure uniqueness (#6956) 2023-11-22 12:05:14 +00:00
17.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
17.sql fix(eventstore): improve pagination of handler filter (#6968) 2023-12-01 12:25:41 +00:00
18.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
18.sql fix(query): user performance (#6537) 2023-12-08 13:14:22 +01:00
19.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
19.sql fix(eventstore): increase performance on push (#7125) 2023-12-31 15:30:25 +01:00
20.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
20.sql fix(auth): efficient user session projection (#7187) 2024-01-09 18:36:46 +00:00
21.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
21.sql feat: block instances (#7129) 2024-01-17 10:16:48 +00:00
22.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
22.sql fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
23.go fix(eventstore): consider IsGlobal-flag of constraints (#7518) 2024-03-08 13:33:53 +00:00
23.sql fix(eventstore): consider IsGlobal-flag of constraints (#7518) 2024-03-08 13:33:53 +00:00
24.go feat(oidc): token exchange impersonation (#7516) 2024-03-20 10:18:46 +00:00
24.sql feat(oidc): token exchange impersonation (#7516) 2024-03-20 10:18:46 +00:00
25.go feat: password age policy (#8132) 2024-06-18 11:27:44 +00:00
25.sql feat: password age policy (#8132) 2024-06-18 11:27:44 +00:00
26.go fix(login): improve auth handlers (#7969) 2024-05-22 15:26:02 +00:00
26.sql fix(login): improve auth handlers (#7969) 2024-05-22 15:26:02 +00:00
27.go feat: Add Twilio Verification Service (#8678) 2024-09-26 09:14:33 +02:00
27.sql feat(saml): allow setting nameid-format and alternative mapping for transient format (#7979) 2024-05-23 05:04:07 +00:00
28.go feat(eventstore): add search table (#8191) 2024-07-03 15:00:56 +00:00
28.sql feat(eventstore): add search table (#8191) 2024-07-03 15:00:56 +00:00
29.go feat(eventstore): add search table (#8191) 2024-07-03 15:00:56 +00:00
30.go perf(import): optimize search for domains claimed by other organizations (#8200) 2024-07-05 09:36:00 +02:00
31.go fix(fields): add index to improve search by aggregate (#8267) 2024-07-08 15:54:19 +00:00
31.sql fix(fields): add index to improve search by aggregate (#8267) 2024-07-08 15:54:19 +00:00
32.go feat(oidc): sid claim for id_tokens issued through login V1 (#8525) 2024-09-03 13:19:00 +00:00
32.sql feat(oidc): sid claim for id_tokens issued through login V1 (#8525) 2024-09-03 13:19:00 +00:00
33.go feat: Add Twilio Verification Service (#8678) 2024-09-26 09:14:33 +02:00
33.sql feat: Add Twilio Verification Service (#8678) 2024-09-26 09:14:33 +02:00
34.go perf(cache): pgx pool connector (#8703) 2024-10-04 13:15:41 +00:00
35.go perf(oidc): nest position clause for session terminated query (#8738) 2024-10-07 12:49:55 +00:00
cleanup.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
config_change.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
config_test.go fix(setup): decode complex config strings (#7854) 2024-05-01 12:17:27 +02:00
config.go perf(oidc): nest position clause for session terminated query (#8738) 2024-10-07 12:49:55 +00:00
projections.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
setup.go perf(oidc): nest position clause for session terminated query (#8738) 2024-10-07 12:49:55 +00:00
steps.yaml docs(cmd): document that human and service user can be used in the default instance (#8229) 2024-07-04 11:39:28 +00:00