Commit Graph

332 Commits

Author SHA1 Message Date
Livio Spring
041af26917
feat(OIDC): add back channel logout (#8837)
# Which Problems Are Solved

Currently ZITADEL supports RP-initiated logout for clients. Back-channel
logout ensures that user sessions are terminated across all connected
applications, even if the user closes their browser or loses
connectivity providing a more secure alternative for certain use cases.

# How the Problems Are Solved

If the feature is activated and the client used for the authentication
has a back_channel_logout_uri configured, a
`session_logout.back_channel` will be registered. Once a user terminates
their session, a (notification) handler will send a SET (form POST) to
the registered uri containing a logout_token (with the user's ID and
session ID).

- A new feature "back_channel_logout" is added on system and instance
level
- A `back_channel_logout_uri` can be managed on OIDC applications
- Added a `session_logout` aggregate to register and inform about sent
`back_channel` notifications
- Added a `SecurityEventToken` channel and `Form`message type in the
notification handlers
- Added `TriggeredAtOrigin` fields to `HumanSignedOut` and
`TerminateSession` events for notification handling
- Exported various functions and types in the `oidc` package to be able
to reuse for token signing in the back_channel notifier.
- To prevent that current existing session termination events will be
handled, a setup step is added to set the `current_states` for the
`projections.notifications_back_channel_logout` to the current position

- [x] requires https://github.com/zitadel/oidc/pull/671

# Additional Changes

- Updated all OTEL dependencies to v1.29.0, since OIDC already updated
some of them to that version.
- Single Session Termination feature is correctly checked (fixed feature
mapping)

# Additional Context

- closes https://github.com/zitadel/zitadel/issues/8467
- TODO:
  - Documentation
  - UI to be done: https://github.com/zitadel/zitadel/issues/8469

---------

Co-authored-by: Hidde Wieringa <hidde@hiddewieringa.nl>
2024-10-31 15:57:17 +01:00
Mark Stosberg
70449caafb
docs: standardize multi-factor spelling and related string updates (#8752)
- **docs: s/Secondfactor/Second factor/**
- **docs: s/IDP/IdP/**
- **docs: s/Hardwaretokens/Hardware tokens/**
- **docs: standardize multi-factor vs multi factor vs multifactor**

# Which Problems Are Solved

 - English strings are improved

# How the Problems Are Solved

 - With better strings

---------

Co-authored-by: Fabi <fabienne@zitadel.com>
2024-10-22 14:59:16 +00:00
Fabi
0de2f92d2e
docs: fix wrong examples and links (#8780)
# Which Problems Are Solved

- The addorgmember request shows a wrong example in the api
documentation
- Broken Links on actions feature description

# How the Problems Are Solved

- Change example of AddOrgMember API Docs
- Point towards correct links

---------

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
2024-10-18 11:12:23 +03:00
Stefan Benz
4d593dace2
fix: add domain as attribute to list user auth methods (#8718)
# Which Problems Are Solved

There is no option to only query auth methods related to specific
domains.

# How the Problems Are Solved

Add domain as attribute to the ListAuthenticationMethodTypes request.

# Additional Changes

OwnerRemoved column removed from the projection.

# Additional Context

Closes #8615

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-10-10 16:50:53 +00:00
Livio Spring
16171ce3b9
fix: pass sessionID to OTP email link (#8745)
# Which Problems Are Solved

OTP Email links currently could not use / include the sessionID they
belong to. This prevents an easy use for redirecting and handling OTP
via email through the session API.

# How the Problems Are Solved

Added the sessionID as placeholder for the OTP Email link template.

# Additional Changes

List all available placeholders in the url_templates of V2 endpoints.

# Additional Context

- discussed in a customer meeting
2024-10-10 13:53:32 +00:00
Stefan Benz
d03ad62edd
fix: add default organization as query for org v2 list (#8719)
# Which Problems Are Solved

To get the default organization we have no direct solution.

# How the Problems Are Solved

Add default organization as query to the org v2 List.

# Additional Changes

None

# Additional Context

Closes #8616

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-10-07 09:28:39 +00:00
Tim Möhlmann
63d733b3a2
perf(oidc): disable push of user token meta-event (#8691)
# Which Problems Are Solved

When executing many concurrent authentication requests on a single
machine user, there were performance issues. As the same aggregate is
being searched and written to concurrently, we traced it down to a
locking issue on the used index.
We already optimized the token endpoint by creating a separate OIDC
aggregate.

At the time we decided to push a single event to the user aggregate, for
the user audit log. See [technical advisory
10010](https://zitadel.com/docs/support/advisory/a10010) for more
details.

However, a recent security fix introduced an additional search query on
the user aggregate, causing the locking issue we found.

# How the Problems Are Solved

Add a feature flag which disables pushing of the `user.token.v2.added`.
The event has no importance and was only added for informational
purposes on the user objects. The `oidc_session.access_token.added` is
the actual payload event and is pushed on the OIDC session aggregate and
can still be used for audit trail.

# Additional Changes

- Fix an event mapper type for
`SystemOIDCSingleV1SessionTerminationEventType`

# Additional Context

- Reported by support request
- https://github.com/zitadel/zitadel/pull/7822 changed the token
aggregate
- https://github.com/zitadel/zitadel/pull/8631 introduced user state
check

Load test trace graph with `user.token.v2.added` **enabled**. Query
times are steadily increasing:


![image](https://github.com/user-attachments/assets/4aa25055-8721-4e93-b695-625560979909)

Load test trace graph with `user.token.v2.added` **disabled**. Query
times constant:


![image](https://github.com/user-attachments/assets/a7657f6c-0c55-401b-8291-453da5d5caf9)

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-09-26 13:55:41 +00:00
Livio Spring
14e2aba1bc
feat: Add Twilio Verification Service (#8678)
# Which Problems Are Solved
Twilio supports a robust, multi-channel verification service that
notably supports multi-region SMS sender numbers required for our use
case. Currently, Zitadel does much of the work of the Twilio Verify (eg.
localization, code generation, messaging) but doesn't support the pool
of sender numbers that Twilio Verify does.

# How the Problems Are Solved
To support this API, we need to be able to store the Twilio Service ID
and send that in a verification request where appropriate: phone number
verification and SMS 2FA code paths.

This PR does the following: 
- Adds the ability to use Twilio Verify of standard messaging through
Twilio
- Adds support for international numbers and more reliable verification
messages sent from multiple numbers
- Adds a new Twilio configuration option to support Twilio Verify in the
admin console
- Sends verification SMS messages through Twilio Verify
- Implements Twilio Verification Checks for codes generated through the
same

# Additional Changes

# Additional Context
- base was implemented by @zhirschtritt in
https://github.com/zitadel/zitadel/pull/8268 ❤️
- closes https://github.com/zitadel/zitadel/issues/8581

---------

Co-authored-by: Zachary Hirschtritt <zachary.hirschtritt@klaviyo.com>
Co-authored-by: Joey Biscoglia <joey.biscoglia@klaviyo.com>
2024-09-26 09:14:33 +02:00
Stefan Benz
62cdec222e
feat: user v3 contact email and phone (#8644)
# Which Problems Are Solved

Endpoints to maintain email and phone contact on user v3 are not
implemented.

# How the Problems Are Solved

Add 3 endpoints with SetContactEmail, VerifyContactEmail and
ResendContactEmailCode.
Add 3 endpoints with SetContactPhone, VerifyContactPhone and
ResendContactPhoneCode.
Refactor the logic how contact is managed in the user creation and
update.

# Additional Changes

None

# Additional Context

- part of https://github.com/zitadel/zitadel/issues/6433

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-09-25 13:31:31 +00:00
Livio Spring
ca1914e235
fix: user grants deactivation (#8634)
# Which Problems Are Solved

ZITADEL's user grants deactivation mechanism did not work correctly.
Deactivated user grants were still provided in token, which could lead
to unauthorized access to applications and resources.
Additionally, the management and auth API always returned the state as
active or did not provide any information about the state.

# How the Problems Are Solved

- Correctly check the user grant state on active for tokens and user
information (userinfo, introspection, saml attributes)
- Map state in API and display in Console
2024-09-17 12:18:29 +00:00
Stefan Benz
4ac722d934
fix: use body for update user on user v2 API (#8635)
Use body for update user endpoint on user v2 API.
2024-09-17 11:53:43 +00:00
Stefan Benz
5fdad7b8f4
feat: user v3 api update (#8582)
# Which Problems Are Solved

Users are not yet able to update their information an status in user API
v3.

# How the Problems Are Solved

Add endpoints and functionality to update users and their status in user
API v3.

# Additional Changes

Aggregate_type and event_types are updated with "userschema" to avoid
conflicts with old events.

# Additional Context

closes #7898
2024-09-17 08:27:48 +00:00
Stefan Benz
c297a62c4f
docs: notification provider docs (#8626)
# Which Problems Are Solved

There is no documentation on how to use the HTTP providers for SMS and
Email.

# How the Problems Are Solved

Add documentation and correct the tags in the proto files to correctly
group the calls for the API docs.

# Additional Changes

None

# Additional Context

None

---------

Co-authored-by: Fabi <fabienne@zitadel.com>
2024-09-16 13:28:29 +02:00
Stefan Benz
21c38b061d
feat: add http as smtp provider (#8545)
# Which Problems Are Solved

Send Email messages as a HTTP call to a relay, for own logic on handling
different Email providers

# How the Problems Are Solved

Create endpoints under Email provider to manage SMTP and HTTP in the
notification handlers.

# Additional Changes

Clean up old logic in command and query side to handle the general Email
providers with deactivate, activate and remove.

# Additional Context

Partially closes #8270

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-09-12 06:27:29 +02:00
Stefan Benz
1a75414315
fix: add apple as identity provider type to settings v2 api (#8472)
# Which Problems Are Solved

Apple is not availabe as IDP type in the settings v2 API.

# How the Problems Are Solved

Add Apple as type.

# Additional Changes

None

# Additional Context

None

Co-authored-by: Max Peintner <max@caos.ch>
2024-09-11 14:26:28 +00:00
Livio Spring
a07b2f4677
feat: invite user link (#8578)
# Which Problems Are Solved

As an administrator I want to be able to invite users to my application
with the API V2, some user data I will already prefil, the user should
add the authentication method themself (password, passkey, sso).

# How the Problems Are Solved

- A user can now be created with a email explicitly set to false.
- If a user has no verified email and no authentication method, an
`InviteCode` can be created through the User V2 API.
  - the code can be returned or sent through email
- additionally `URLTemplate` and an `ApplicatioName` can provided for
the email
- The code can be resent and verified through the User V2 API
- The V1 login allows users to verify and resend the code and set a
password (analog user initialization)
- The message text for the user invitation can be customized

# Additional Changes

- `verifyUserPasskeyCode` directly uses `crypto.VerifyCode` (instead of
`verifyEncryptedCode`)
- `verifyEncryptedCode` is removed (unnecessarily queried for the code
generator)

# Additional Context

- closes #8310
- TODO: login V2 will have to implement invite flow:
https://github.com/zitadel/typescript/issues/166
2024-09-11 10:53:55 +00:00
Tim Möhlmann
3aba942162
feat: add debug events API (#8533)
# Which Problems Are Solved

Add a debug API which allows pushing a set of events to be reduced in a
dedicated projection.
The events can carry a sleep duration which simulates a slow query
during projection handling.

# How the Problems Are Solved

- `CreateDebugEvents` allows pushing multiple events which simulate the
lifecycle of a resource. Each event has a `projectionSleep` field, which
issues a `pg_sleep()` statement query in the projection handler :
  - Add
  - Change
  - Remove
- `ListDebugEventsStates` list the current state of the projection,
optionally with a Trigger
- `GetDebugEventsStateByID` get the current state of the aggregate ID in
the projection, optionally with a Trigger


# Additional Changes

- none

# Additional Context

-  Allows reproduction of https://github.com/zitadel/zitadel/issues/8517
2024-09-11 08:24:00 +00:00
Fabi
ccf222f0f6
docs(api): fix title of project grant member list (#8548)
# Which Problems Are Solved

The title of the project grant member list was wrong

# How the Problems Are Solved

Added right title
2024-09-09 12:09:34 +00:00
Stefan Benz
5bdf1a4547
feat: add http as sms provider (#8540)
# Which Problems Are Solved

Send SMS messages as a HTTP call to a relay, for own logic on handling
different SMS providers.

# How the Problems Are Solved

Add HTTP as SMS provider type and handling of webhook messages in the
notification handlers.

# Additional Changes

Clean up old Twilio events, which were supposed to handle the general
SMS providers with deactivate, activate and remove.

# Additional Context

Partially closes #8270

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-09-06 13:11:36 +00:00
Livio Spring
25bce59119
fix(api): pass trusted domain in body instead of param (#8544)
# Which Problems Are Solved

The trusted domain cannot be sent in the request body.

# How the Problems Are Solved

Added missing mapping

# Additional Changes

None

# Additional Context

None
2024-09-04 11:34:26 +00:00
Livio Spring
382a97c30f
feat(oidc): end session by id_token_hint and without cookie (#8542)
# Which Problems Are Solved

The end_session_endpoint currently always requires the userAgent cookie
to be able to terminate session created through the hosted login UI.
Only tokens issued through the Login V2 can be used to directly
terminate a specific session and without the need of a cookie.
This PR adds the possibility to terminate a single V1 session or all V1
sessions belonging to the same user agent without the need of the
userAgent cookie by providing an id_token as `id_token_hint` which
contains the id of a V1 session as `sid`.

# How the Problems Are Solved

- #8525 added the `sid` claim for id_tokens issued through the login UI
- The `sid` can now be checked for the `V1_` prefix and queries for
either the userAgentID and depending on the
`OIDCSingleV1SessionTermination` flag all userIDs of active session from
the same user agent id
- The `OIDCSingleV1SessionTermination` flag is added with default value
false to keep the existing behavior of terminating all sessions even in
case of providing an id_token_hint

# Additional Changes

- pass `context.Context` into session view functions for querying the
database with that context

# Additional Context

- relates to #8499 
- closes #8501
2024-09-04 10:14:50 +00:00
Livio Spring
c26a07210c
fix: update IDP Success and Failure URLs to accept up to 2048 characters again (#8543)
# Which Problems Are Solved

Durning the review of #8540, I noticed that the max length of the
success and failure url was set back to 200 instead of 2048 as changed
in #8327.
Looks like this was accidentally changed during the v2 GA release and
migration to v3 resources

# How the Problems Are Solved

Reapplied the change to 2048 max length

# Additional Changes

None

# Additional Context

- relates to #8327
- relates to #8343
  - requires backport until 2.58.x
- relates to #8494
  - requires backport to 2.60.x
2024-09-04 09:25:40 +00:00
Elio Bischof
41b7bf976d
fix: map gRPC Gateway paths correctly (#8536)
# Which Problems Are Solved

Alpha APIs are not available accessible using REST.

# How the Problems Are Solved

The base_path property for the swagger definition has nothing to do with
the reverse proxy generation. Reusing it and omitting its segments in
the individual methods leads to a wrong path mapping. To fix this, the
full paths are added to all the GW mappings.

# Additional Context

Replace this example with links to related issues, discussions, discord
threads, or other sources with more context.
Use the Closing #issue syntax for issues that are resolved with this PR.
- Closes #8507
2024-09-03 14:32:42 +02:00
Stefan Benz
a9eb50321c
fix: update userschema api to v3alpha definition (#8511)
# Which Problems Are Solved

UserSchema API is currently not completely as defined for v3alpha.

# How the Problems Are Solved

Update the protos and integration tests.

# Additional Changes

None

# Additional Context

None
2024-09-02 11:24:15 +00:00
Stefan Benz
41ae35f2ef
feat: add schema user create and remove (#8494)
# Which Problems Are Solved

Added functionality that user with a userschema can be created and
removed.

# How the Problems Are Solved

Added logic and moved APIs so that everything is API v3 conform.

# Additional Changes

- move of user and userschema API to resources folder
- changed testing and parameters
- some renaming

# Additional Context

closes #7308

---------

Co-authored-by: Elio Bischof <elio@zitadel.com>
2024-08-28 19:46:45 +00:00
Livio Spring
ca8f82423a
fix: automatically link user without prompt (#8487)
# Which Problems Are Solved

There were UX issue with the autolinking prompt page and users were not
able to link their account or would not understand what to do. Since the
trust to the IdP is already bound by the configuration, the user can
directly be linked without any user input.

# How the Problems Are Solved

- remove the prompt page and directly link the user if possible
- remove corresponding customization texts from the API and Console

# Additional Changes

None

# Additional Context

- relates to https://github.com/zitadel/zitadel/issues/7977
- discussed with customers
- created as a `fix` to be able to backport

---------

Co-authored-by: Max Peintner <max@caos.ch>
2024-08-28 05:33:20 +00:00
Tim Möhlmann
fd0c15dd4f
feat(oidc): use web keys for token signing and verification (#8449)
# Which Problems Are Solved

Use web keys, managed by the `resources/v3alpha/web_keys` API, for OIDC
token signing and verification,
as well as serving the public web keys on the jwks / keys endpoint.
Response header on the keys endpoint now allows caching of the response.
This is now "safe" to do since keys can be created ahead of time and
caches have sufficient time to pickup the change before keys get
enabled.

# How the Problems Are Solved

- The web key format is used in the `getSignerOnce` function in the
`api/oidc` package.
- The public key cache is changed to get and store web keys.
- The jwks / keys endpoint returns the combined set of valid "legacy"
public keys and all available web keys.
- Cache-Control max-age default to 5 minutes and is configured in
`defaults.yaml`.

When the web keys feature is enabled, fallback mechanisms are in place
to obtain and convert "legacy" `query.PublicKey` as web keys when
needed. This allows transitioning to the feature without invalidating
existing tokens. A small performance overhead may be noticed on the keys
endpoint, because 2 queries need to be run sequentially. This will
disappear once the feature is stable and the legacy code gets cleaned
up.

# Additional Changes

- Extend legacy key lifetimes so that tests can be run on an existing
database with more than 6 hours apart.
- Discovery endpoint returns all supported algorithms when the Web Key
feature is enabled.

# Additional Context

- Closes https://github.com/zitadel/zitadel/issues/8031
- Part of https://github.com/zitadel/zitadel/issues/7809
- After https://github.com/zitadel/oidc/pull/637
- After https://github.com/zitadel/oidc/pull/638
2024-08-23 14:43:46 +02:00
Tim Möhlmann
5fd2061770
feat(oidc): allow returning of parent errors to client (#8376)
# Which Problems Are Solved

Currently the OIDC API of ZITADEL only prints parent errors to the logs.
Where 4xx status are typically warn level and 5xx error level. This
makes it hard to debug certain errors for client in multi-instance
environments like ZITADEL cloud, where there is no direct access to
logs. In case of support requests we often can't correlate past
log-lines to the error that was reported.

This change adds the possibility to return the parent error in the
response to the OIDC client. For the moment this only applies to JSON
body responses, not error redirects to the RP.

# How the Problems Are Solved

- New instance-level feature flag: `debug_oidc_parent_error`
- Use the new `WithReturnParentToClient()` function from the oidc lib
introduced in https://github.com/zitadel/oidc/pull/629 for all cases
where `WithParent` was already used and the request context is
available.

# Additional Changes

none

# Additional Context

- Depends on: https://github.com/zitadel/oidc/pull/629
- Related to: https://github.com/zitadel/zitadel/issues/8362

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-08-20 06:45:24 +00:00
Tim Möhlmann
ab4daa0ebb
fix(webkeys): remove include private key from projection index (#8436)
# Which Problems Are Solved

When Web keys with a large RSA Bitsize were generated, the new web key
did not get projected.
Zitadel logs printed the following errors:

```
time="2024-08-15T09:18:47Z" level=debug msg="execution of statements failed" caller="/home/tim/Repositories/zitadel/zitadel/internal/eventstore/handler/v2/handler.go:518" error="statement failed: ID=CRDB-pKtsr Message=exec failed Parent=(ERROR: index row size 3400 exceeds btree version 4 maximum 2704 for index \"web_keys_web_key_state_idx\" (SQLSTATE 54000))" lastProcessedIndex=-1 projection=projections.web_keys
```


# How the Problems Are Solved

Remove the the `WithInclude` option with the Private key from the
`web_keys` projection index and bump the projection version.

# Additional Changes

- Correct RPC summaries for documentation
- Fix docs slug in sidebar.js

# Additional Context

- Found during release review.
2024-08-16 11:41:09 +00:00
Stefan Benz
5fab533e37
feat: org v2 ListOrganizations (#8411)
# Which Problems Are Solved

Org v2 service does not have a ListOrganizations endpoint.

# How the Problems Are Solved

Implement ListOrganizations endpoint.

# Additional Changes

- moved descriptions in the protos to comments
- corrected the RemoveNoPermissions for the ListUsers, to get the
correct TotalResults

# Additional Context

For new typescript login
2024-08-15 06:37:06 +02:00
Stefan Benz
3e3d46ac0d
feat: idp v2 api GetIDPByID (#8425)
# Which Problems Are Solved

GetIDPByID as endpoint in the API v2 so that it can be available for the
new login.

# How the Problems Are Solved

Create GetIDPByID endpoint with IDP v2 API, throught the GetProviderByID
implementation from admin and management API.

# Additional Changes

- Remove the OwnerType attribute from the response, as the information
is available through the resourceOwner.
- correct refs to messages in proto which are used for doc generation
- renaming of elements for API v3

# Additional Context

Closes #8337

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-08-14 18:18:29 +00:00
Tim Möhlmann
64a3bb3149
feat(v3alpha): web key resource (#8262)
# Which Problems Are Solved

Implement a new API service that allows management of OIDC signing web
keys.
This allows users to manage rotation of the instance level keys. which
are currently managed based on expiry.

The API accepts the generation of the following key types and
parameters:

- RSA keys with 2048, 3072 or 4096 bit in size and:
  - Signing with SHA-256 (RS256)
  - Signing with SHA-384 (RS384)
  - Signing with SHA-512 (RS512)
- ECDSA keys with
  - P256 curve
  - P384 curve
  - P512 curve
- ED25519 keys

# How the Problems Are Solved

Keys are serialized for storage using the JSON web key format from the
`jose` library. This is the format that will be used by OIDC for
signing, verification and publication.

Each instance can have a number of key pairs. All existing public keys
are meant to be used for token verification and publication the keys
endpoint. Keys can be activated and the active private key is meant to
sign new tokens. There is always exactly 1 active signing key:

1. When the first key for an instance is generated, it is automatically
activated.
2. Activation of the next key automatically deactivates the previously
active key.
3. Keys cannot be manually deactivated from the API
4. Active keys cannot be deleted

# Additional Changes

- Query methods that later will be used by the OIDC package are already
implemented. Preparation for #8031
- Fix indentation in french translation for instance event
- Move user_schema translations to consistent positions in all
translation files

# Additional Context

- Closes #8030
- Part of #7809

---------

Co-authored-by: Elio Bischof <elio@zitadel.com>
2024-08-14 14:18:14 +00:00
Livio Spring
e2e1100124
feat(idp): provide auto only options (#8420)
# Which Problems Are Solved

As of now, **automatic creation** and **automatic linking options** were
only considered if the corresponding **allowed option** (account
creation / linking allowed) was enabled.

With this PR, this is no longer needed and allows administrators to
address cases, where only an **automatic creation** is allowed, but
users themselves should not be allowed to **manually** create new
accounts using an identity provider or edit the information during the
process.
Also, allowing users to only link to the proposed existing account is
now possible with an enabled **automatic linking option**, while
disabling **account linking allowed**.

# How the Problems Are Solved

- Check for **automatic** options without the corresponding **allowed**
option.
- added technical advisory to notify about the possible behavior change

# Additional Changes

- display the error message on the IdP linking step in the login UI (in
case there is one)
- display an error in case no option is possible
- exchanged deprecated `eventstoreExpect` with `expectEventstore` in
touched test files

# Additional Context

closes https://github.com/zitadel/zitadel/issues/7393

---------

Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2024-08-14 13:04:26 +00:00
Elio Bischof
042c438813
feat(v3alpha): read actions (#8357)
# Which Problems Are Solved

The current v3alpha actions APIs don't exactly adhere to the [new
resources API
design](https://zitadel.com/docs/apis/v3#standard-resources).

# How the Problems Are Solved

- **Improved ID access**: The aggregate ID is added to the resource
details object, so accessing resource IDs and constructing proto
messages for resources is easier
- **Explicit Instances**: Optionally, the instance can be explicitly
given in each request
- **Pagination**: A default search limit and a max search limit are
added to the defaults.yaml. They apply to the new v3 APIs (currently
only actions). The search query defaults are changed to ascending by
creation date, because this makes the pagination results the most
deterministic. The creation date is also added to the object details.
The bug with updated creation dates is fixed for executions and targets.
- **Removed Sequences**: Removed Sequence from object details and
ProcessedSequence from search details

# Additional Changes

Object details IDs are checked in unit test only if an empty ID is
expected. Centralizing the details check also makes this internal object
more flexible for future evolutions.

# Additional Context

- Closes #8169 
- Depends on https://github.com/zitadel/zitadel/pull/8225

---------

Co-authored-by: Silvan <silvan.reusser@gmail.com>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2024-08-12 22:32:01 +02:00
Livio Spring
3d071fc505
feat: trusted (instance) domains (#8369)
# Which Problems Are Solved

ZITADEL currently selects the instance context based on a HTTP header
(see https://github.com/zitadel/zitadel/issues/8279#issue-2399959845 and
checks it against the list of instance domains. Let's call it instance
or API domain.
For any context based URL (e.g. OAuth, OIDC, SAML endpoints, links in
emails, ...) the requested domain (instance domain) will be used. Let's
call it the public domain.
In cases of proxied setups, all exposed domains (public domains) require
the domain to be managed as instance domain.
This can either be done using the "ExternalDomain" in the runtime config
or via system API, which requires a validation through CustomerPortal on
zitadel.cloud.

# How the Problems Are Solved

- Two new headers / header list are added:
- `InstanceHostHeaders`: an ordered list (first sent wins), which will
be used to match the instance.
(For backward compatibility: the `HTTP1HostHeader`, `HTTP2HostHeader`
and `forwarded`, `x-forwarded-for`, `x-forwarded-host` are checked
afterwards as well)
- `PublicHostHeaders`: an ordered list (first sent wins), which will be
used as public host / domain. This will be checked against a list of
trusted domains on the instance.
- The middleware intercepts all requests to the API and passes a
`DomainCtx` object with the hosts and protocol into the context
(previously only a computed `origin` was passed)
- HTTP / GRPC server do not longer try to match the headers to instances
themself, but use the passed `http.DomainContext` in their interceptors.
- The `RequestedHost` and `RequestedDomain` from authz.Instance are
removed in favor of the `http.DomainContext`
- When authenticating to or signing out from Console UI, the current
`http.DomainContext(ctx).Origin` (already checked by instance
interceptor for validity) is used to compute and dynamically add a
`redirect_uri` and `post_logout_redirect_uri`.
- Gateway passes all configured host headers (previously only did
`x-zitadel-*`)
- Admin API allows to manage trusted domain

# Additional Changes

None

# Additional Context

- part of #8279 
- open topics: 
  - "single-instance" mode
  - Console UI
2024-07-31 18:00:38 +03:00
Elio Bischof
cc3ec1e2a7
feat(v3alpha): write actions (#8225)
# Which Problems Are Solved

The current v3alpha actions APIs don't exactly adhere to the [new
resources API
design](https://zitadel.com/docs/apis/v3#standard-resources).

# How the Problems Are Solved

- **Breaking**: The current v3alpha actions APIs are removed. This is
breaking.
- **Resource Namespace**: New v3alpha actions APIs for targets and
executions are added under the namespace /resources.
- **Feature Flag**: New v3alpha actions APIs still have to be activated
using the actions feature flag
- **Reduced Executions Overhead**: Executions are managed similar to
settings according to the new API design: an empty list of targets
basically makes an execution a Noop. So a single method, SetExecution is
enough to cover all use cases. Noop executions are not returned in
future search requests.
- **Compatibility**: The executions created with previous v3alpha APIs
are still available to be managed with the new executions API.

# Additional Changes

- Removed integration tests which test executions but rely on readable
targets. They are added again with #8169

# Additional Context

Closes #8168
2024-07-31 14:42:12 +02:00
Stefan Benz
7d2d85f57c
feat: api v2beta to api v2 (#8283)
# Which Problems Are Solved

The v2beta services are stable but not GA.

# How the Problems Are Solved

The v2beta services are copied to v2. The corresponding v1 and v2beta
services are deprecated.

# Additional Context

Closes #7236

---------

Co-authored-by: Elio Bischof <elio@zitadel.com>
2024-07-26 22:39:55 +02:00
Stefan Benz
bc16962aac
feat: api v2beta to api v2 protos (#8343)
# Which Problems Are Solved

The go linter can't limit the checks to the diff in
https://github.com/zitadel/zitadel/pull/8283 because it's too large

# How the Problems Are Solved

The protos from https://github.com/zitadel/zitadel/pull/8283 are merged
separately

# Additional Context

Contributes to #7236

---------

Co-authored-by: Elio Bischof <elio@zitadel.com>
2024-07-26 12:28:28 +02:00
Joey Biscoglia
f34897a8c8
feat: add delete phone endpoint to v2 api (#8321)
# Which Problems Are Solved

- Adds delete phone endpoint to v2 api

# How the Problems Are Solved

- Adds new endpoint with DELETE method to /v2beta/users/:userId/phone
which removes currently set phone number

# Additional Changes

- Added integration test for new endpoint.

# Additional Context

- Solves
https://discord.com/channels/927474939156643850/1255557862286032996
2024-07-24 13:46:07 +00:00
Harsha Reddy
d90db6693c
fix: update IDP Success and Failure URLs to accept up to 2048 characters (#8327)
# Which Problems Are Solved

The success and failure URLs post IDP intent are limited to 200
characters. This is very low given the standard for URL lengths is much
higher
https://www.baeldung.com/cs/max-url-length#maximum-length-of-a-url-in-different-browsers


Name of the Browser | URL Length
-- | --
Google Chrome | Maximum of 2048 characters for a URL
Mozilla Firefox | The address bar no longer shows the URL after 65536
characters
Internet Explorer | Maximum length of the URL is 2083 characters
Safari | Supports up to 80000 characters for a URL
Opera | Can display up to 190000 characters in its address bar
Apache | Supports a maximum of 4000 characters



The user post SSO login w/ Zitadel, sees deep links to out app e.g.
`www.mycompany.com/idp/success?deeplink=app/mypage/id/test#thing=abcdef`
and these are usually greater than 200 characters.

# How the Problems Are Solved

Replace this example text with a concise list of changes that this PR
introduces.
- Update validation check for SuccessURL length on v2 and v3 APIs to a
max of 2048 characters
- Update validation check for FailureURL length on v2 and v3 APIs to a
max of 2048 characters

I didn't find any docs t update reflecting this change in length

# Additional Context
Fixes the error

```<PreparedRequest [POST]>, status: 400, reason: Bad Request, response: {"code":3,"message":"invalid StartIdentityProviderIntentRequest.Urls: embedded message failed validation | caused by: invalid RedirectURLs.SuccessUrl: value length must be between 1 and 200 runes, inclusive"}```
2024-07-19 12:27:28 +00:00
Stefan Benz
19561a092c
fix: add OrgIDQuery to the queries for organizations (#8312)
# Which Problems Are Solved

ListOrgs has no option to select for organizations specific to Ids.

# How the Problems Are Solved

Add OrgIDQuery to ListOrgs.

# Additional Changes

Clean up double mapping for the OrgQueries.

# Additional Context

- noted internally while checking performance issues (in Console)
2024-07-16 13:12:19 +00:00
borisroman
7d0c7e5b54
feat: allow machine user id to be set during creation (#8265)
# Which Problems Are Solved
Solves the problem described in #8264.

# How the Problems Are Solved

Added a UserID field which can be set during Machine User creation.

# Additional Changes

Added addition unit and integration tests to cover the cases where a
UserID field is present.

# Additional Context

- Closes #8264

Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2024-07-16 09:27:37 +02:00
Fabi
86c124955d
fix: multiple docs issues (#8244)
# Which Problems Are Solved

- `<details>` tag is rendered in some cases in the docs, instead of
rendering a detail section which can be expanded
- New API V2 and V3 services where not rendered correctly
- The plugin which made it possible to integrate external code files and
show them on the docs didn't work anymore

# How the Problems Are Solved

- remove / from details tag, so it is properly rendered
- changing link source from tag to auto
- Someone already forked the repository and made it available for
docusaurus v3, we integrated the forked version

---------

Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2024-07-11 07:54:04 +02:00
Stefan Benz
fb2d4545b9
fix: add details to ListUsers for user results (#8255)
# Which Problems Are Solved

In User v2 API, the ListUsers endpoint doesn't provide the information
to which organization the user belongs to.

# How the Problems Are Solved

Add the details to the user results from the ListUsers endpoint, so that
the OrgID is also included as ResourceOwner.

# Additional Changes
 
None

# Additional Context

Closes #8172
2024-07-10 17:49:35 +02:00
Stefan Benz
19a8ab02ad
fix: add missing totp remove endpoint in user v2 API (#8256)
# Which Problems Are Solved

TOTP remove endpoint available in management API, not in user v2 API.

# How the Problems Are Solved

Add endpoint RemoveTOTP to user v2 API.

# Additional Changes

None

# Additional Context

close #6605

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-07-10 12:31:28 +00:00
Tim Möhlmann
7967e6f98b
perf(import): optimize search for domains claimed by other organizations (#8200)
# Which Problems Are Solved

Improve the performance of human imports by optimizing the query that
finds domains claimed by other organizations.

# How the Problems Are Solved

Use the fields search table introduced in
https://github.com/zitadel/zitadel/pull/8191 by storing each
organization domain as Object ID and the verified status as field value.

# Additional Changes

- Feature flag for this optimization

# Additional Context

- Performance improvements for import are evaluated and acted upon
internally at the moment

---------

Co-authored-by: adlerhurst <silvan.reusser@gmail.com>
2024-07-05 09:36:00 +02:00
Tim Möhlmann
ecfb9d0d6d
perf(command): user grant pre-condition check using the search table (#8230)
# Which Problems Are Solved

Imporve the performance of user grant addition, especially for import.

# How the Problems Are Solved

Use the search table to query for the project grant state. 
This could easily be done by making the search used in
`checkProjectGrantPreCondition` reusable.

# Additional Changes

Chanded event declerations to `const` in the
`internal/repository/project` package.

# Additional Context

- Performance improvements for import are evaluated and acted upon
internally at the moment

---------

Co-authored-by: adlerhurst <silvan.reusser@gmail.com>
Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-07-04 16:18:43 +00:00
Silvan
1d84635836
feat(eventstore): add search table (#8191)
# Which Problems Are Solved

To improve performance a new table and method is implemented on
eventstore. The goal of this table is to index searchable fields on
command side to use it on command and query side.

The table allows to store one primitive value (numeric, text) per row.

The eventstore framework is extended by the `Search`-method which allows
to search for objects.
The `Command`-interface is extended by the `SearchOperations()`-method
which does manipulate the the `search`-table.

# How the Problems Are Solved

This PR adds the capability of improving performance for command and
query side by using the `Search`-method of the eventstore instead of
using one of the `Filter`-methods.

# Open Tasks

- [x] Add feature flag
- [x] Unit tests
- [ ] ~~Benchmarks if needed~~
- [x] Ensure no behavior change
- [x] Add setup step to fill table with current data
- [x] Add projection which ensures data added between setup and start of
the new version are also added to the table

# Additional Changes

The `Search`-method is currently used by `ProjectGrant`-command side.

# Additional Context

- Closes https://github.com/zitadel/zitadel/issues/8094
2024-07-03 15:00:56 +00:00
Miguel Cabrerizo
cd6b452fc9
fix: add missing SAML type for GetActiveIdentityProviders (#8186)
# Which Problems Are Solved

- When the endpoint http://{CUSTOM-DOMAIN}/v2beta/settings/login/idps is
called the type for an activated SAML provider is not sent.
- The IDENTITY_PROVIDER_TYPE_SAML is missing

# How the Problems Are Solved

- Adds the missing IDENTITY_PROVIDER_TYPE_SAML to the
IdentityProviderType proto definition
- Adds the missing case for idpTypeToPb 
- Adds the missing test case for idpTypeToPb

Here's a screenshot showing the endpoint response:


![image](https://github.com/zitadel/zitadel/assets/30386061/6e3e9c41-543c-472e-96ab-3d40736a2699)

# Additional Context

- Closes #7885

Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2024-06-28 15:00:56 +00:00
Miguel Cabrerizo
3635320ce8
feat: send test mail (#7968)
# Which Problems Are Solved

- Zitadel doesn't have a way to test SMTP settings either before
creating a new provider or once the SMTP provider has been created.
- Zitadel SMTP messages can be more informative for usual errors

# How the Problems Are Solved

- A new step is added to the new/update SMTP provider wizard that allows
us to test a configuration. The result is shown in a text area.
- From the table of SMTP providers you can test your settings too.
- The email address to send the email is by default the email address
for the logged in user as suggested.
- Some of the SMTP error messages have been changed to give more
information about the possible situation. For example: could not contact
with the SMTP server, check the port, firewall issues... instead of
could not dial

Here's a video showing this new option in action:


https://github.com/zitadel/zitadel/assets/30386061/50128ba1-c9fa-4481-8eec-e79a3ca69bda

# Additional Changes

Replace this example text with a concise list of additional changes that
this PR introduces, that are not directly solving the initial problem
but are related.
For example:
- The docs explicitly describe that the property XY is mandatory
- Adds missing translations for validations.

# Additional Context

- Closes #4504
2024-06-20 19:51:42 +00:00