Commit Graph

3413 Commits

Author SHA1 Message Date
Tim Möhlmann
c165ed07f4
feat(cache): organization (#8903)
# Which Problems Are Solved

Organizations are ofter searched for by ID or primary domain. This
results in many redundant queries, resulting in a performance impact.

# How the Problems Are Solved

Cache Organizaion objects by ID and primary domain.

# Additional Changes

- Adjust integration test config to use all types of cache.
- Adjust integration test lifetimes so the pruner has something to do
while the tests run.

# Additional Context

- Closes #8865
- After #8902
2024-11-21 08:05:03 +02:00
zitadelraccine
041c3d9b9e
docs: add next office hours (#8922)
# Which Problems Are Solved

Updating the meeting schedule with the latest community event.

# How the Problems Are Solved

A new event invite with associated details are added to direct community
members on Github to register for our Discord event.

# Additional Changes

N/A

# Additional Context

N/A

Co-authored-by: Silvan <silvan.reusser@gmail.com>
2024-11-19 12:00:58 +00:00
Tim Möhlmann
c31b5df73b
docs: add caches documentation (#8902)
# Which Problems Are Solved

Explain the usage of the new cache mechanisms.

# How the Problems Are Solved

Provide a dedicated page on caches with reference to `defaults.yaml`.

# Additional Changes

- Fix a broken link tag in token exchange docs.

# Additional Context

- Closes #8855
2024-11-19 10:56:10 +01:00
Tim Möhlmann
65e24b67da
chore(load-test): disable userinfo after JWT profile (#8927)
# Which Problems Are Solved

Load-test requires single endpoint to be used for each test type.

# How the Problems Are Solved

Remove userinfo call from machine tests.

# Additional Changes

- Add load-test/.env to gitignore.

# Additional Context

- Related to #4424
2024-11-19 09:53:07 +01:00
Silvan
522c82876f
fix(eventstore): set application name during push to instance id (#8918)
# Which Problems Are Solved

Noisy neighbours can introduce projection latencies because the
projections only query events older than the start timestamp of the
oldest push transaction.

# How the Problems Are Solved

During push we set the application name to
`zitadel_es_pusher_<instance_id>` instead of `zitadel_es_pusher` which
is used to query events by projections.
2024-11-18 15:30:12 +00:00
dependabot[bot]
5a85c3eda8
chore(deps): bump http-proxy-middleware from 2.0.6 to 2.0.7 in /console (#8823)
Bumps
[http-proxy-middleware](https://github.com/chimurai/http-proxy-middleware)
from 2.0.6 to 2.0.7.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/chimurai/http-proxy-middleware/releases">http-proxy-middleware's
releases</a>.</em></p>
<blockquote>
<h2>v2.0.7</h2>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/chimurai/http-proxy-middleware/compare/v2.0.6...v2.0.7">https://github.com/chimurai/http-proxy-middleware/compare/v2.0.6...v2.0.7</a></p>
<h2>v2.0.7-beta.1</h2>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/chimurai/http-proxy-middleware/compare/v2.0.7-beta.0...v2.0.7-beta.1">https://github.com/chimurai/http-proxy-middleware/compare/v2.0.7-beta.0...v2.0.7-beta.1</a></p>
<h2>v2.0.7-beta.0</h2>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/chimurai/http-proxy-middleware/compare/v2.0.6...v2.0.7-beta.0">https://github.com/chimurai/http-proxy-middleware/compare/v2.0.6...v2.0.7-beta.0</a></p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/chimurai/http-proxy-middleware/blob/v2.0.7/CHANGELOG.md">http-proxy-middleware's
changelog</a>.</em></p>
<blockquote>
<h2><a
href="https://github.com/chimurai/http-proxy-middleware/releases/tag/v2.0.7">v2.0.7</a></h2>
<ul>
<li>ci(github actions): add publish.yml</li>
<li>fix(filter): handle errors</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="1e92339098"><code>1e92339</code></a>
ci(github-actions): fix npm tag</li>
<li><a
href="90afb7c9a6"><code>90afb7c</code></a>
chore(package): v2.0.7</li>
<li><a
href="0b4274e8cc"><code>0b4274e</code></a>
fix(filter): handle errors</li>
<li><a
href="1bd6dd578b"><code>1bd6dd5</code></a>
ci(github actions): add publish.yml</li>
<li>See full diff in <a
href="https://github.com/chimurai/http-proxy-middleware/compare/v2.0.6...v2.0.7">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=http-proxy-middleware&package-manager=npm_and_yarn&previous-version=2.0.6&new-version=2.0.7)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/zitadel/zitadel/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Max Peintner <max@caos.ch>
2024-11-18 10:20:03 +00:00
Silvan
fbebe0f183
docs: init benchmarks (#8894)
# Which Problems Are Solved

Adds initial benchmarks.

# How the Problems Are Solved

Added section `apis/benchmarks`

# Additional Changes

Update Makefile dependencies

# Additional Context

- Part of https://github.com/zitadel/zitadel/issues/8023
- Part of https://github.com/zitadel/zitadel/issues/8352
2024-11-15 23:44:22 +02:00
Florian Forster
45cf38e08f
chore: adding an adopters file for our community (#8909)
# Which Problems Are Solved

We want to give adopters a platform to show that they are using ZITADEL

# How the Problems Are Solved

Addding an ADOPTERS.md file

# Additional Changes

none

# Additional Context

none
2024-11-15 15:48:11 +00:00
Zach Hirschtritt
7ba797b872
fix: use correct check for user existing on import (#8907)
# Which Problems Are Solved

- ImportHuman was not checking for a `UserStateDeleted` state on import,
resulting in "already existing" errors when attempting to delete and
re-import a user with the same id

# How the Problems Are Solved

Use the `Exists` helper method to check for both `UserStateUnspecified`
and `UserStateDeleted` states on import

# Additional Changes

N/A

# Additional Context

N/A

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-11-15 07:46:33 +01:00
Livio Spring
374b9a7f66
fix(saml): provide option to get internal as default ACS (#8888)
# Which Problems Are Solved

Some SAML IdPs including Google only allow to configure a single
AssertionConsumerService URL.
Since the current metadata provides multiple and the hosted login UI is
not published as neither the first nor with `isDefault=true`, those IdPs
take another and then return an error on sign in.

# How the Problems Are Solved

Allow to reorder the ACS URLs using a query parameter
(`internalUI=true`) when retrieving the metadata endpoint.
This will list the `ui/login/login/externalidp/saml/acs` first and also
set the `isDefault=true`.

# Additional Changes

None

# Additional Context

Reported by a customer
2024-11-15 07:19:43 +01:00
Livio Spring
85bdf01505
fix(actions): preserve order of execution (#8895)
# Which Problems Are Solved

The order of actions on a trigger was not respected on the execution and
not correctly returned when retrieving the flow, for example in Console.
The supposed correction of the order (e.g. in the UI) would then return
a "no changes" error since the order was already as desired.

# How the Problems Are Solved

- Correctly order the actions of a trigger based on their configuration
(`trigger_sequence`).

# Additional Changes

- replaced a `reflect.DeepEqual` with `slices.Equal` for checking the
action list

# Additional Context

- reported by a customer
- requires backports
2024-11-14 14:04:39 +00:00
Tim Möhlmann
b77901cb4b
fix(cache): unset client and user names in defaults (#8901)
# Which Problems Are Solved

By having default entries in the `Username` and `ClientName` fields, it
was not possible to unset there parameters. Unsetting them is required
for GCP connections

# How the Problems Are Solved

Set the fields to empty strings.

# Additional Changes

- none

# Additional Context

- none
2024-11-13 21:18:47 +00:00
Tim Möhlmann
3b7b0c69e6
feat(cache): redis circuit breaker (#8890)
# Which Problems Are Solved

If a redis cache has connection issues or any other type of permament
error,
it tanks the responsiveness of ZITADEL.
We currently do not support things like Redis cluster or sentinel. So
adding a simple redis cache improves performance but introduces a single
point of failure.

# How the Problems Are Solved

Implement a [circuit
breaker](https://learn.microsoft.com/en-us/previous-versions/msp-n-p/dn589784(v=pandp.10)?redirectedfrom=MSDN)
as
[`redis.Limiter`](https://pkg.go.dev/github.com/redis/go-redis/v9#Limiter)
by wrapping sony's [gobreaker](https://github.com/sony/gobreaker)
package. This package is picked as it seems well maintained and we
already use their `sonyflake` package

# Additional Changes

- The unit tests constructed an unused `redis.Client` and didn't cleanup
the connector. This is now fixed.

# Additional Context

Closes #8864
2024-11-13 19:11:48 +01:00
Livio Spring
ecbf0db15b
fix(setup): improve search query to use index (#8898)
# Which Problems Are Solved

The setup filter for previous steps and kept getting slower. This is due
to the filter, which did not provide any instanceID and thus resulting
in a full table scan.

# How the Problems Are Solved

- Added an empty instanceID filter (since it's on system level)

# Additional Changes

None

# Additional Context

Noticed internally and during migrations on some regions
2024-11-13 07:50:23 +00:00
chris-1o
a09c772b03
fix(mirror): Fix instance_id check for tables without (#8852)
# Which Problems Are Solved

Fixes 'column "instance_id" does not exist' errors from #8558.

# How the Problems Are Solved

The instanceClause / WHERE clause in the query for the respective tables
is excluded.

I have successfully created a mirror with this change.
2024-11-12 16:03:41 +00:00
Tim Möhlmann
778b4041ca
fix(oidc): do not return access token for response type id_token (#8777)
# Which Problems Are Solved

Do not return an access token for implicit flow from v1 login, if the
`response_type` is `id_token`

# How the Problems Are Solved

Do not create the access token event if if the `response_type` is
`id_token`.

# Additional Changes

Token endpoint calls without auth request, such as machine users, token
exchange and refresh token, do not have a `response_type`. For such
calls the `OIDCResponseTypeUnspecified` enum is added at a `-1` offset,
in order not to break existing client configs.

# Additional Context

- https://discord.com/channels/927474939156643850/1294001717725237298
- Fixes https://github.com/zitadel/zitadel/issues/8776
2024-11-12 15:20:48 +00:00
chuangjinglu
69e9926bcc
fix: fix slice init length (#8707)
# Which Problems Are Solved

The intention here should be to initialize a slice with a capacity of
len(queriedOrgs.Orgs) rather than initializing the length of this slice.

the online demo: https://go.dev/play/p/vNUPNjdb2gJ


# How the Problems Are Solved

use `processedOrgs := make([]string, 0, len(queriedOrgs.Orgs))`

# Additional Changes

None

# Additional Context

None

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
2024-11-12 14:41:18 +00:00
Titouan-joseph Cicorella
f45f52ea0d
docs(v2): fix duplicate section of user, session, oidc and settings services (#8889)
# Which Problems Are Solved

Duplicate section in the doc

![image](https://github.com/user-attachments/assets/b9d31f87-9158-443f-8f76-1bae31fb7ee8)


# How the Problems Are Solved

Change the category link source to add a introduction section

![image](https://github.com/user-attachments/assets/562843e6-e8b9-4125-a3f7-8e4d2a24522d)
2024-11-12 14:14:17 +00:00
Jonathon Taylor
04a166f2d2
fix(translations): typo in VerifyEmail body (#8872)
# Which Problems Are Solved

Fixes small typo in email body during user creation & verification. The
change also includes the removal of some unnecessary white space in the
same yaml file.

# How the Problems Are Solved

Replaces din't with didn't. 

![image](https://github.com/user-attachments/assets/48abf38b-4deb-42b7-a85b-91009e19f27f)

Co-authored-by: jtaylor@dingo.com <jtaylor@dingo.com>
Co-authored-by: Silvan <silvan.reusser@gmail.com>
2024-11-11 12:03:15 +00:00
Livio Spring
fb6579e456
fix(milestones): use previous spelling for milestone types (#8886)
# Which Problems Are Solved

https://github.com/zitadel/zitadel/pull/8788 accidentally changed the
spelling of milestone types from PascalCase to snake_case. This breaks
systems where `milestone.pushed` events already exist.

# How the Problems Are Solved

- Use PascalCase again
- Prefix event types with v2. (Previous pushed event type was anyway
ignored).
- Create `milstones3` projection

# Additional Changes

None

# Additional Context

relates to #8788
2024-11-11 11:28:27 +00:00
Silvan
4a354a568d
docs: update currently available regions of zitadel.cloud (#7230)
Removes the available regions section in legal docs.

---------

Co-authored-by: mffap <mpa@zitadel.com>
2024-11-10 15:11:33 +00:00
Tim Möhlmann
250f2344c8
feat(cache): redis cache (#8822)
# Which Problems Are Solved

Add a cache implementation using Redis single mode. This does not add
support for Redis Cluster or sentinel.

# How the Problems Are Solved

Added the `internal/cache/redis` package. All operations occur
atomically, including setting of secondary indexes, using LUA scripts
where needed.

The [`miniredis`](https://github.com/alicebob/miniredis) package is used
to run unit tests.

# Additional Changes

- Move connector code to `internal/cache/connector/...` and remove
duplicate code from `query` and `command` packages.
- Fix a missed invalidation on the restrictions projection

# Additional Context

Closes #8130
2024-11-04 10:44:51 +00:00
Silvan
9c3e5e467b
perf(query): remove transactions for queries (#8614)
# Which Problems Are Solved

Queries currently execute 3 statements, begin, query, commit

# How the Problems Are Solved

remove transaction handling from query methods in database package

# Additional Changes

- Bump versions of `core_grpc_dependencies`-receipt in Makefile

# Additional info

During load tests we saw a lot of idle transactions of `zitadel_queries`
application name which is the connection pool used to query data in
zitadel. Executed query:

`select query_start - xact_start, pid, application_name, backend_start,
xact_start, query_start, state_change, wait_event_type,
wait_event,substring(query, 1, 200) query from pg_stat_activity where
datname = 'zitadel' and state <> 'idle';`

Mostly the last query executed was `begin isolation level read committed
read only`.

example: 

```
    ?column?     |  pid  |      application_name      |         backend_start         |          xact_start           |          query_start          |         state_change          | wait_event_type |  wait_event  |                                                                                                  query                                                                                                   
-----------------+-------+----------------------------+-------------------------------+-------------------------------+-------------------------------+-------------------------------+-----------------+--------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 00:00:00        | 33030 | zitadel_queries            | 2024-10-16 16:25:53.906036+00 | 2024-10-16 16:30:19.191661+00 | 2024-10-16 16:30:19.191661+00 | 2024-10-16 16:30:19.19169+00  | Client          | ClientRead   | begin isolation level read committed read only
 00:00:00        | 33035 | zitadel_queries            | 2024-10-16 16:25:53.909629+00 | 2024-10-16 16:30:19.19179+00  | 2024-10-16 16:30:19.19179+00  | 2024-10-16 16:30:19.191805+00 | Client          | ClientRead   | begin isolation level read committed read only
 00:00:00.00412  | 33028 | zitadel_queries            | 2024-10-16 16:25:53.904247+00 | 2024-10-16 16:30:19.187734+00 | 2024-10-16 16:30:19.191854+00 | 2024-10-16 16:30:19.191964+00 | Client          | ClientRead   | SELECT created_at, event_type, "sequence", "position", payload, creator, "owner", instance_id, aggregate_type, aggregate_id, revision FROM eventstore.events2 WHERE instance_id = $1 AND aggregate_type 
 00:00:00.084662 | 33134 | zitadel_es_pusher          | 2024-10-16 16:29:54.979692+00 | 2024-10-16 16:30:19.178578+00 | 2024-10-16 16:30:19.26324+00  | 2024-10-16 16:30:19.263267+00 | Client          | ClientRead   | RELEASE SAVEPOINT cockroach_restart
 00:00:00.084768 | 33139 | zitadel_es_pusher          | 2024-10-16 16:29:54.979585+00 | 2024-10-16 16:30:19.180762+00 | 2024-10-16 16:30:19.26553+00  | 2024-10-16 16:30:19.265531+00 | LWLock          | WALWriteLock | commit
 00:00:00.077377 | 33136 | zitadel_es_pusher          | 2024-10-16 16:29:54.978582+00 | 2024-10-16 16:30:19.187883+00 | 2024-10-16 16:30:19.26526+00  | 2024-10-16 16:30:19.265431+00 | Client          | ClientRead   | WITH existing AS (                                                                                                                                                                                      +
                 |       |                            |                               |                               |                               |                               |                 |              |     (SELECT instance_id, aggregate_type, aggregate_id, "sequence" FROM eventstore.events2 WHERE instance_id = $1 AND aggregate_type = $2 AND aggregate_id = $3 ORDER BY "sequence" DE
 00:00:00.012309 | 33123 | zitadel_es_pusher          | 2024-10-16 16:29:54.963484+00 | 2024-10-16 16:30:19.175066+00 | 2024-10-16 16:30:19.187375+00 | 2024-10-16 16:30:19.187376+00 | IO              | WalSync      | commit
 00:00:00        | 33034 | zitadel_queries            | 2024-10-16 16:25:53.90791+00  | 2024-10-16 16:30:19.262921+00 | 2024-10-16 16:30:19.262921+00 | 2024-10-16 16:30:19.263133+00 | Client          | ClientRead   | begin isolation level read committed read only
 00:00:00        | 33039 | zitadel_queries            | 2024-10-16 16:25:53.914106+00 | 2024-10-16 16:30:19.191676+00 | 2024-10-16 16:30:19.191676+00 | 2024-10-16 16:30:19.191687+00 | Client          | ClientRead   | begin isolation level read committed read only
 00:00:00.24539  | 33083 | zitadel_projection_spooler | 2024-10-16 16:27:49.895548+00 | 2024-10-16 16:30:19.020058+00 | 2024-10-16 16:30:19.265448+00 | 2024-10-16 16:30:19.26546+00  | Client          | ClientRead   | SAVEPOINT exec_stmt
 00:00:00        | 33125 | zitadel_es_pusher          | 2024-10-16 16:29:54.963859+00 | 2024-10-16 16:30:19.191715+00 | 2024-10-16 16:30:19.191715+00 | 2024-10-16 16:30:19.191729+00 | Client          | ClientRead   | begin
 00:00:00.004292 | 33032 | zitadel_queries            | 2024-10-16 16:25:53.906624+00 | 2024-10-16 16:30:19.187713+00 | 2024-10-16 16:30:19.192005+00 | 2024-10-16 16:30:19.192062+00 | Client          | ClientRead   | SELECT created_at, event_type, "sequence", "position", payload, creator, "owner", instance_id, aggregate_type, aggregate_id, revision FROM eventstore.events2 WHERE instance_id = $1 AND aggregate_type 
 00:00:00        | 33031 | zitadel_queries            | 2024-10-16 16:25:53.906422+00 | 2024-10-16 16:30:19.191625+00 | 2024-10-16 16:30:19.191625+00 | 2024-10-16 16:30:19.191645+00 | Client          | ClientRead   | begin isolation level read committed read only

```

The amount of idle transactions is significantly less if the query
transactions are removed:

example: 

```
    ?column?     |  pid  |      application_name      |         backend_start         |          xact_start           |          query_start          |         state_change          | wait_event_type | wait_event |                                                                                                  query                                                                                                   
-----------------+-------+----------------------------+-------------------------------+-------------------------------+-------------------------------+-------------------------------+-----------------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 00:00:00.000094 | 32741 | zitadel_queries            | 2024-10-16 16:23:49.73935+00  | 2024-10-16 16:24:59.785589+00 | 2024-10-16 16:24:59.785683+00 | 2024-10-16 16:24:59.785684+00 |                 |            | SELECT created_at, event_type, "sequence", "position", payload, creator, "owner", instance_id, aggregate_type, aggregate_id, revision FROM eventstore.events2 WHERE instance_id = $1 AND aggregate_type 
 00:00:00        | 32762 | zitadel_es_pusher          | 2024-10-16 16:24:02.275136+00 | 2024-10-16 16:24:59.784586+00 | 2024-10-16 16:24:59.784586+00 | 2024-10-16 16:24:59.784607+00 | Client          | ClientRead | begin
 00:00:00.000167 | 32742 | zitadel_queries            | 2024-10-16 16:23:49.740489+00 | 2024-10-16 16:24:59.784274+00 | 2024-10-16 16:24:59.784441+00 | 2024-10-16 16:24:59.784442+00 |                 |            | with usr as (                                                                                                                                                                                           +
                 |       |                            |                               |                               |                               |                               |                 |            |         select u.id, u.creation_date, u.change_date, u.sequence, u.state, u.resource_owner, u.username, n.login_name as preferred_login_name                                                            +
                 |       |                            |                               |                               |                               |                               |                 |            |         from projections.users13 u                                                                                                                                                                      +
                 |       |                            |                               |                               |                               |                               |                 |            |         left join projections.l
 00:00:00.256014 | 32759 | zitadel_projection_spooler | 2024-10-16 16:24:01.418429+00 | 2024-10-16 16:24:59.52959+00  | 2024-10-16 16:24:59.785604+00 | 2024-10-16 16:24:59.785649+00 | Client          | ClientRead | UPDATE projections.milestones SET reached_date = $1 WHERE (instance_id = $2) AND (type = $3) AND (reached_date IS NULL)
 00:00:00.014199 | 32773 | zitadel_es_pusher          | 2024-10-16 16:24:02.320404+00 | 2024-10-16 16:24:59.769509+00 | 2024-10-16 16:24:59.783708+00 | 2024-10-16 16:24:59.783709+00 | IO              | WalSync    | commit
 00:00:00        | 32765 | zitadel_es_pusher          | 2024-10-16 16:24:02.28173+00  | 2024-10-16 16:24:59.780413+00 | 2024-10-16 16:24:59.780413+00 | 2024-10-16 16:24:59.780426+00 | Client          | ClientRead | begin
 00:00:00.012729 | 32777 | zitadel_es_pusher          | 2024-10-16 16:24:02.339737+00 | 2024-10-16 16:24:59.767432+00 | 2024-10-16 16:24:59.780161+00 | 2024-10-16 16:24:59.780195+00 | Client          | ClientRead | RELEASE SAVEPOINT cockroach_restart
```

---------

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Max Peintner <max@caos.ch>
Co-authored-by: Elio Bischof <elio@zitadel.com>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
Co-authored-by: Miguel Cabrerizo <30386061+doncicuto@users.noreply.github.com>
Co-authored-by: Joakim Lodén <Loddan@users.noreply.github.com>
Co-authored-by: Yxnt <Yxnt@users.noreply.github.com>
Co-authored-by: Stefan Benz <stefan@caos.ch>
Co-authored-by: Harsha Reddy <harsha.reddy@klaviyo.com>
Co-authored-by: Zach H <zhirschtritt@gmail.com>
2024-11-04 10:06:14 +01:00
Stefan Benz
9422766e17
chore: remove some integration test flakiness (#8818)
Remove some integration test flakiness.

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-10-31 15:34:20 +00:00
Livio Spring
041af26917
feat(OIDC): add back channel logout (#8837)
# Which Problems Are Solved

Currently ZITADEL supports RP-initiated logout for clients. Back-channel
logout ensures that user sessions are terminated across all connected
applications, even if the user closes their browser or loses
connectivity providing a more secure alternative for certain use cases.

# How the Problems Are Solved

If the feature is activated and the client used for the authentication
has a back_channel_logout_uri configured, a
`session_logout.back_channel` will be registered. Once a user terminates
their session, a (notification) handler will send a SET (form POST) to
the registered uri containing a logout_token (with the user's ID and
session ID).

- A new feature "back_channel_logout" is added on system and instance
level
- A `back_channel_logout_uri` can be managed on OIDC applications
- Added a `session_logout` aggregate to register and inform about sent
`back_channel` notifications
- Added a `SecurityEventToken` channel and `Form`message type in the
notification handlers
- Added `TriggeredAtOrigin` fields to `HumanSignedOut` and
`TerminateSession` events for notification handling
- Exported various functions and types in the `oidc` package to be able
to reuse for token signing in the back_channel notifier.
- To prevent that current existing session termination events will be
handled, a setup step is added to set the `current_states` for the
`projections.notifications_back_channel_logout` to the current position

- [x] requires https://github.com/zitadel/oidc/pull/671

# Additional Changes

- Updated all OTEL dependencies to v1.29.0, since OIDC already updated
some of them to that version.
- Single Session Termination feature is correctly checked (fixed feature
mapping)

# Additional Context

- closes https://github.com/zitadel/zitadel/issues/8467
- TODO:
  - Documentation
  - UI to be done: https://github.com/zitadel/zitadel/issues/8469

---------

Co-authored-by: Hidde Wieringa <hidde@hiddewieringa.nl>
2024-10-31 15:57:17 +01:00
Tim Möhlmann
9cf67f30b8
fix(milestones): offset the type enum (#8849)
# Which Problems Are Solved

Migration of milestones failed on our QA due to the new milestone Type
enum being 0-indexed. The valid range was 0 till 5, inclusive. While on
the previous zitadel version this was 1 till 6, inclusive.

# How the Problems Are Solved

Offset the first constant with `1`.

# Additional Changes

- none

# Additional Context

Introduced in https://github.com/zitadel/zitadel/pull/8788
2024-10-31 12:03:40 +01:00
Max Peintner
692c9b7aa8
fix(login): org register script references (#8842)
Closes #8838 

This fixes a bug of the `/register/org` page where scripts where not
referenced correctly

Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2024-10-31 10:16:30 +00:00
Fabi
aa211489ee
docs: change recommendation from cockroachDB to postgreSQL (#8844)
# Which Problems Are Solved

As we switched to PostgreSQL with our cloud zitadel, we now want to
recommend it instead of CockroachDB.

# How the Problems Are Solved

Replaced the recommend section with postgresql
2024-10-31 10:24:45 +01:00
Stefan Benz
6780c5a07c
fix: add resourceowner to check for project in project grant (#8785)
# Which Problems Are Solved

Resource owner can be different than expected if the provided
x-zitadel-orgid header is provided.

# How the Problems Are Solved

Check that the project is only checked with the correct resource owner
to avoid unexpected situations.

# Additional Changes

None

# Additional Context

Closes #8685

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-10-30 08:53:00 +00:00
Elio Bischof
cff4fe5dfd
docs: fix and harmonize docker compose files (#8839)
# Which Problems Are Solved

1. Postgres spams FATAL: role "root" does not exist as mentioned in
https://github.com/zitadel/zitadel/discussions/7832 (even with -U)

2. The compose commands for a ZITADEL deployment with initial service
account key don't work out-of-the box with a non-root user, because
docker creates non-existing directories to bind-mount with root
ownership.


![image](https://github.com/user-attachments/assets/f2fc92d5-2ff4-47a4-bf4d-e9657aa2bb94)

```
time="2024-10-29T09:37:13Z" level=error msg="migration failed" caller="/home/runner/work/zitadel/zitadel/internal/migration/migration.go:68" error="open /machinekey/zitadel-admin-sa.json: permission denied" name=03_default_instance
time="2024-10-29T09:37:13Z" level=fatal msg="migration failed" caller="/home/runner/work/zitadel/zitadel/cmd/setup/setup.go:248" error="open /machinekey/zitadel-admin-sa.json: permission denied" name=03_default_instance
```

# How the Problems Are Solved

1. The branch bases on https://github.com/zitadel/zitadel/pull/8826. The
env vars are cleaned up and prettified across compose files.

2. A command is added to the docs that creates the directory with the
current users permission. The ZITADEL container runs with the current
users ID.

# Additional Context

- Replaces https://github.com/zitadel/zitadel/pull/8826
- Discussion https://github.com/zitadel/zitadel/discussions/7832
- Closes https://github.com/zitadel/zitadel/issues/7725

---------

Co-authored-by: m4tu4g <71326926+m4tu4g@users.noreply.github.com>
2024-10-29 20:02:04 +01:00
ChandanChainani
1426823d40
fix(mac): date command options not found #8757 (#8758)
# Which Problems Are Solved
Closes: #8757 

Problem:
```sh
➜  ~ date --rfc-3339=seconds | sed 's/ /T/'
date: illegal option -- -
usage: date [-jnRu] [-I[date|hours|minutes|seconds]] [-f input_fmt]
            [-r filename|seconds] [-v[+|-]val[y|m|w|d|H|M|S]]
            [[[[mm]dd]HH]MM[[cc]yy][.SS] | new_date] [+output_fmt]
```

# How the Problems Are Solved
```sh
➜  date "+%Y-%m-%dT%T%z" | sed -E 's/.([0-9]{2})([0-9]{2})$/-\1:\2/'
2024-10-10T19:09:53-05:30
➜  TZ=America/Los_Angeles date "+%Y-%m-%dT%T%z" | sed -E 's/.([0-9]{2})([0-9]{2})$/-\1:\2/'
2024-10-10T06:39:41-07:00
```

- `Mac` support while compiling the source code locally

Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2024-10-28 13:36:57 +01:00
Tim Möhlmann
32bad3feb3
perf(milestones): refactor (#8788)
# Which Problems Are Solved

Milestones used existing events from a number of aggregates. OIDC
session is one of them. We noticed in load-tests that the reduction of
the oidc_session.added event into the milestone projection is a costly
business with payload based conditionals. A milestone is reached once,
but even then we remain subscribed to the OIDC events. This requires the
projections.current_states to be updated continuously.


# How the Problems Are Solved

The milestone creation is refactored to use dedicated events instead.
The command side decides when a milestone is reached and creates the
reached event once for each milestone when required.

# Additional Changes

In order to prevent reached milestones being created twice, a migration
script is provided. When the old `projections.milestones` table exist,
the state is read from there and `v2` milestone aggregate events are
created, with the original reached and pushed dates.

# Additional Context

- Closes https://github.com/zitadel/zitadel/issues/8800
2024-10-28 08:29:34 +00:00
dependabot[bot]
54f1c0bc50
chore(deps): bump http-proxy-middleware from 2.0.6 to 2.0.7 in /docs (#8821)
Bumps
[http-proxy-middleware](https://github.com/chimurai/http-proxy-middleware)
from 2.0.6 to 2.0.7.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/chimurai/http-proxy-middleware/releases">http-proxy-middleware's
releases</a>.</em></p>
<blockquote>
<h2>v2.0.7</h2>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/chimurai/http-proxy-middleware/compare/v2.0.6...v2.0.7">https://github.com/chimurai/http-proxy-middleware/compare/v2.0.6...v2.0.7</a></p>
<h2>v2.0.7-beta.1</h2>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/chimurai/http-proxy-middleware/compare/v2.0.7-beta.0...v2.0.7-beta.1">https://github.com/chimurai/http-proxy-middleware/compare/v2.0.7-beta.0...v2.0.7-beta.1</a></p>
<h2>v2.0.7-beta.0</h2>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/chimurai/http-proxy-middleware/compare/v2.0.6...v2.0.7-beta.0">https://github.com/chimurai/http-proxy-middleware/compare/v2.0.6...v2.0.7-beta.0</a></p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/chimurai/http-proxy-middleware/blob/v2.0.7/CHANGELOG.md">http-proxy-middleware's
changelog</a>.</em></p>
<blockquote>
<h2><a
href="https://github.com/chimurai/http-proxy-middleware/releases/tag/v2.0.7">v2.0.7</a></h2>
<ul>
<li>ci(github actions): add publish.yml</li>
<li>fix(filter): handle errors</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="1e92339098"><code>1e92339</code></a>
ci(github-actions): fix npm tag</li>
<li><a
href="90afb7c9a6"><code>90afb7c</code></a>
chore(package): v2.0.7</li>
<li><a
href="0b4274e8cc"><code>0b4274e</code></a>
fix(filter): handle errors</li>
<li><a
href="1bd6dd578b"><code>1bd6dd5</code></a>
ci(github actions): add publish.yml</li>
<li>See full diff in <a
href="https://github.com/chimurai/http-proxy-middleware/compare/v2.0.6...v2.0.7">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=http-proxy-middleware&package-manager=npm_and_yarn&previous-version=2.0.6&new-version=2.0.7)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/zitadel/zitadel/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Florian Forster <florian@zitadel.com>
2024-10-25 20:12:27 +00:00
Livio Spring
0b5079c11f
fix: correctly search for verified domain (#8820)
# Which Problems Are Solved

Searching orgs by domain currently only looked for the primary domain,
but should be possible with all verified domains (as documented)

# How the Problems Are Solved

- fixed the search query

# Additional Changes

None

# Additional Context

- closes https://github.com/zitadel/zitadel/issues/8749
2024-10-25 17:51:44 +02:00
Mostafa Galal
4eeb2be36a
fix: Negative values allowed by spinners of Login Lifetimes inputs (#8694)
# Which Problems Are Solved

Previously, the login lifetime input fields allowed negative values and,
in some cases, zero values, which were not valid according to the
business rules.

# How the Problems Are Solved

The issue was resolved by adding min and step properties to the relevant
HTML input fields. This ensures that only valid values are entered,
adhering to the specific requirements for each field.

Co-authored-by: Max Peintner <max@caos.ch>
2024-10-25 10:44:15 +02:00
Stefan Benz
32d958ea43
chore: add await for project to oidc integration tests (#8809)
# Which Problems Are Solved

In integration tests there is waiting for the application, but the
project is also included if the token can be created.

# How the Problems Are Solved

Wait for project not only for the application in the integration tests.

# Additional Changes

Some more corrections in integration tests.

# Additional Context

None

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-10-23 07:36:50 +00:00
karatekaneen
d696d15a1c
docs: update logger example for action modules (#8813)
# Which Problems Are Solved

Updated the example to fit with the actual logger signature. Solves
#8811.
2024-10-23 07:40:43 +02:00
Mark Stosberg
70449caafb
docs: standardize multi-factor spelling and related string updates (#8752)
- **docs: s/Secondfactor/Second factor/**
- **docs: s/IDP/IdP/**
- **docs: s/Hardwaretokens/Hardware tokens/**
- **docs: standardize multi-factor vs multi factor vs multifactor**

# Which Problems Are Solved

 - English strings are improved

# How the Problems Are Solved

 - With better strings

---------

Co-authored-by: Fabi <fabienne@zitadel.com>
2024-10-22 14:59:16 +00:00
Livio Spring
79fb4cc1cc
fix: correctly check denied domains and ips for actions (#8810)
# Which Problems Are Solved

System administrators can block hosts and IPs for HTTP calls in actions.
Using DNS, blocked IPs could be bypassed.

# How the Problems Are Solved

- Hosts are resolved (DNS lookup) to check whether their corresponding
IP is blocked.

# Additional Changes

- Added complete lookup ip address range and "unspecified" address to
the default `DenyList`
2024-10-22 16:16:44 +02:00
Stefan Benz
fca6b28a97
chore: correct require usage to assert for eventual consistency (#8795)
# Which Problems Are Solved

Eventual consistency is handled wrongly in the newly improved
integration tests.

# How the Problems Are Solved

Correct the usage of the require package with the assert package where
necessary, to remove the panics where the EventuallyWithT functions can
rerun.

# Additional Changes

Modify the timeout values for some EventuallyWithT which can vary when a
instance is freshly setup.

# Additional Context

None
2024-10-21 19:15:02 +00:00
Laust Rud Jacobsen
11782cf422
docs: typo repair (#8796)
# Which Problems Are Solved

Saw a typo, fixed it.
2024-10-21 12:34:09 +02:00
Fabi
0de2f92d2e
docs: fix wrong examples and links (#8780)
# Which Problems Are Solved

- The addorgmember request shows a wrong example in the api
documentation
- Broken Links on actions feature description

# How the Problems Are Solved

- Change example of AddOrgMember API Docs
- Point towards correct links

---------

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
2024-10-18 11:12:23 +03:00
Stefan Benz
8d97363642
chore: improve integration tests (#8727)
Improve integration tests:
- spliting the tests in TokenExchange to isolated instances and in
parallel
- corrected some test structure so that the check for Details is no done
anymore if the test already failed
- replace required-calls with assert-calls to not stop the testing
- add gofakeit for application, project and usernames(emails)
- add eventually checks for testing in actions v2, so the request only
get called when the execution is defined
- check for length of results in list/search endpoints to avoid index
errors
2024-10-17 21:20:57 +00:00
Silvan
cc8d4fe17c
chore(load-test): add csv output (#8783)
# Which Problems Are Solved

Load tests currently do not output details about the data messured.

# How the Problems Are Solved

Added the `--out` flag to all load tests
2024-10-17 22:15:55 +03:00
Stefan Benz
c21e171519
fix: add allowRegister check for register handling in login (#8782)
# Which Problems Are Solved

There is currently the possibility that you can jump to the register
path, even if register is disallowed through the settings.

# How the Problems Are Solved

Check before handling the HTTP requests if register is allowed.

# Additional Changes

Function to determine the resourceowner for all register related
functionality in the login.

# Additional Context

closes #8123
2024-10-16 13:09:32 +00:00
Tim Möhlmann
4ebc23aa1f
fix(load-test): correct k6 command (#8760) 2024-10-15 14:26:16 +02:00
Fabi
3c4a92a981
chore(github): add type to issue templates (#8775)
# Which Problems Are Solved

Github introduced the new issue types, which we want to add to our
issues. Starting point is to change the templates, so we can add the
right types.
2024-10-15 09:42:16 +00:00
Shubham Singh Sugara
7eb54e4c7b
fix: Update Defaults.yaml (#8731)
# Which Problems Are Solved
The primary issue addressed in this PR is that the defaults.yaml file
contains escaped characters (like `&lt;` for < and `&gt;` for >) in
message texts, which prevents valid HTML rendering in certain parts of
the Zitadel platform.
These escaped characters are used in user-facing content (e.g., email
templates or notifications), resulting in improperly displayed text,
where the HTML elements like line breaks or bold text don't render
correctly.

# How the Problems Are Solved
The solution involves replacing the escaped characters with their
corresponding HTML tags in the defaults.yaml file, ensuring that the
HTML renders correctly in the emails or user interfaces where these
messages are displayed.
This update ensures that:
- The HTML in these message templates is rendered properly, improving
the user experience.
- The content looks professional and adheres to web standards for
displaying HTML content.
    
# Additional Changes
N/A

# Additional Context
N/A

- Closes #8531

Co-authored-by: Max Peintner <max@caos.ch>
2024-10-14 07:42:08 +00:00
TrueQAP
c455b8b1eb
feat(i18n): Add Hungarian language support to ZITADEL (#8645)
- Fully translated all UI elements, documentation, and error messages
- Added Hungarian as a new supported language option
- Updated language selection menus and related configuration files
- Ensured consistency across all translated content

# Which Problems Are Solved

- ZITADEL was not accessible for Hungarian-speaking users due to lack of
language support
- Hungarian users had to rely on English or other languages to use the
platform
- Potential user base was limited due to language barrier

# How the Problems Are Solved

- Translated all user interface elements, including console and login
interfaces
- Translated all documentation files to Hungarian
- Added Hungarian translations for all error messages and notifications
- Implemented Hungarian as a selectable language option in the system

# Additional Changes

- Updated language selection menus to include Hungarian
- Modified configuration files to support Hungarian language
- Ensured consistent terminology and style across all translated content
- Added Hungarian language option to relevant dropdown menus and
settings

# Additional Context

- Relates to the ongoing internationalization efforts of ZITADEL
- Enhances accessibility for Hungarian-speaking developers and users
- Expands ZITADEL's potential user base in Hungary and
Hungarian-speaking regions

---------

Co-authored-by: Max Peintner <max@caos.ch>
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2024-10-11 09:00:50 +00:00
Elio Bischof
464ca0bd00
fix: always create SAML with metadata (#8696)
# Which Problems Are Solved

Adding a SAML IDPs with an empty metadata XML and URL leads to failed
projection events. The main IDP template projection succeeds, but the
subtable projection for SAML template fails, because the metadata field
is not nullable in that table. The SAML IDP shows up in list queries,
because the list method only queries the main template projection.
However, getting a SAML IDP created without metadata by ID misses the
SAML specific IDP data.

# How the Problems Are Solved

- The command for adding a SAML IDP properly ensures that non-empty
metadata is either given by XML or resolved by URL.
- The console doesn't send requests with empty metadata anymore. This
works by explicitly setting a single oneof option for either XML or URL
and emptying the other one.

# Additional Context

Closes #8443
2024-10-11 08:09:51 +00:00