Commit Graph

78 Commits

Author SHA1 Message Date
Tim Möhlmann
10d5fc6184
test: no location header in response (#8441)
# Which Problems Are Solved

Sometimes integrations tests are failing with an error `http: no
location header in response`. The underlying cause was hidden, as in
some tests we assumed a 3xx range response but got a 4xx response
instead. No assertion on the status code was made, resulting in the
above error message on calling `resp.Location()`.

The underlying issue, the application not found in the projection, is
also fixed.

# How the Problems Are Solved

This change adds a check for the status code and returns the response
body if the response is not in the 3xx status code range.

Helper function that create applications now do an additional
`GetAppByID` in a retry loop to ensure consitency in the projection
before proceeding with tests.

# Additional Changes

- none

# Additional Context

- Pipline failures were observed, no issue was created
- Cherry-picked form WIP #8407

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-08-17 14:48:06 +00:00
Stefan Benz
5fab533e37
feat: org v2 ListOrganizations (#8411)
# Which Problems Are Solved

Org v2 service does not have a ListOrganizations endpoint.

# How the Problems Are Solved

Implement ListOrganizations endpoint.

# Additional Changes

- moved descriptions in the protos to comments
- corrected the RemoveNoPermissions for the ListUsers, to get the
correct TotalResults

# Additional Context

For new typescript login
2024-08-15 06:37:06 +02:00
Stefan Benz
3e3d46ac0d
feat: idp v2 api GetIDPByID (#8425)
# Which Problems Are Solved

GetIDPByID as endpoint in the API v2 so that it can be available for the
new login.

# How the Problems Are Solved

Create GetIDPByID endpoint with IDP v2 API, throught the GetProviderByID
implementation from admin and management API.

# Additional Changes

- Remove the OwnerType attribute from the response, as the information
is available through the resourceOwner.
- correct refs to messages in proto which are used for doc generation
- renaming of elements for API v3

# Additional Context

Closes #8337

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-08-14 18:18:29 +00:00
Tim Möhlmann
64a3bb3149
feat(v3alpha): web key resource (#8262)
# Which Problems Are Solved

Implement a new API service that allows management of OIDC signing web
keys.
This allows users to manage rotation of the instance level keys. which
are currently managed based on expiry.

The API accepts the generation of the following key types and
parameters:

- RSA keys with 2048, 3072 or 4096 bit in size and:
  - Signing with SHA-256 (RS256)
  - Signing with SHA-384 (RS384)
  - Signing with SHA-512 (RS512)
- ECDSA keys with
  - P256 curve
  - P384 curve
  - P512 curve
- ED25519 keys

# How the Problems Are Solved

Keys are serialized for storage using the JSON web key format from the
`jose` library. This is the format that will be used by OIDC for
signing, verification and publication.

Each instance can have a number of key pairs. All existing public keys
are meant to be used for token verification and publication the keys
endpoint. Keys can be activated and the active private key is meant to
sign new tokens. There is always exactly 1 active signing key:

1. When the first key for an instance is generated, it is automatically
activated.
2. Activation of the next key automatically deactivates the previously
active key.
3. Keys cannot be manually deactivated from the API
4. Active keys cannot be deleted

# Additional Changes

- Query methods that later will be used by the OIDC package are already
implemented. Preparation for #8031
- Fix indentation in french translation for instance event
- Move user_schema translations to consistent positions in all
translation files

# Additional Context

- Closes #8030
- Part of #7809

---------

Co-authored-by: Elio Bischof <elio@zitadel.com>
2024-08-14 14:18:14 +00:00
Elio Bischof
042c438813
feat(v3alpha): read actions (#8357)
# Which Problems Are Solved

The current v3alpha actions APIs don't exactly adhere to the [new
resources API
design](https://zitadel.com/docs/apis/v3#standard-resources).

# How the Problems Are Solved

- **Improved ID access**: The aggregate ID is added to the resource
details object, so accessing resource IDs and constructing proto
messages for resources is easier
- **Explicit Instances**: Optionally, the instance can be explicitly
given in each request
- **Pagination**: A default search limit and a max search limit are
added to the defaults.yaml. They apply to the new v3 APIs (currently
only actions). The search query defaults are changed to ascending by
creation date, because this makes the pagination results the most
deterministic. The creation date is also added to the object details.
The bug with updated creation dates is fixed for executions and targets.
- **Removed Sequences**: Removed Sequence from object details and
ProcessedSequence from search details

# Additional Changes

Object details IDs are checked in unit test only if an empty ID is
expected. Centralizing the details check also makes this internal object
more flexible for future evolutions.

# Additional Context

- Closes #8169 
- Depends on https://github.com/zitadel/zitadel/pull/8225

---------

Co-authored-by: Silvan <silvan.reusser@gmail.com>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2024-08-12 22:32:01 +02:00
Livio Spring
3d071fc505
feat: trusted (instance) domains (#8369)
# Which Problems Are Solved

ZITADEL currently selects the instance context based on a HTTP header
(see https://github.com/zitadel/zitadel/issues/8279#issue-2399959845 and
checks it against the list of instance domains. Let's call it instance
or API domain.
For any context based URL (e.g. OAuth, OIDC, SAML endpoints, links in
emails, ...) the requested domain (instance domain) will be used. Let's
call it the public domain.
In cases of proxied setups, all exposed domains (public domains) require
the domain to be managed as instance domain.
This can either be done using the "ExternalDomain" in the runtime config
or via system API, which requires a validation through CustomerPortal on
zitadel.cloud.

# How the Problems Are Solved

- Two new headers / header list are added:
- `InstanceHostHeaders`: an ordered list (first sent wins), which will
be used to match the instance.
(For backward compatibility: the `HTTP1HostHeader`, `HTTP2HostHeader`
and `forwarded`, `x-forwarded-for`, `x-forwarded-host` are checked
afterwards as well)
- `PublicHostHeaders`: an ordered list (first sent wins), which will be
used as public host / domain. This will be checked against a list of
trusted domains on the instance.
- The middleware intercepts all requests to the API and passes a
`DomainCtx` object with the hosts and protocol into the context
(previously only a computed `origin` was passed)
- HTTP / GRPC server do not longer try to match the headers to instances
themself, but use the passed `http.DomainContext` in their interceptors.
- The `RequestedHost` and `RequestedDomain` from authz.Instance are
removed in favor of the `http.DomainContext`
- When authenticating to or signing out from Console UI, the current
`http.DomainContext(ctx).Origin` (already checked by instance
interceptor for validity) is used to compute and dynamically add a
`redirect_uri` and `post_logout_redirect_uri`.
- Gateway passes all configured host headers (previously only did
`x-zitadel-*`)
- Admin API allows to manage trusted domain

# Additional Changes

None

# Additional Context

- part of #8279 
- open topics: 
  - "single-instance" mode
  - Console UI
2024-07-31 18:00:38 +03:00
Elio Bischof
cc3ec1e2a7
feat(v3alpha): write actions (#8225)
# Which Problems Are Solved

The current v3alpha actions APIs don't exactly adhere to the [new
resources API
design](https://zitadel.com/docs/apis/v3#standard-resources).

# How the Problems Are Solved

- **Breaking**: The current v3alpha actions APIs are removed. This is
breaking.
- **Resource Namespace**: New v3alpha actions APIs for targets and
executions are added under the namespace /resources.
- **Feature Flag**: New v3alpha actions APIs still have to be activated
using the actions feature flag
- **Reduced Executions Overhead**: Executions are managed similar to
settings according to the new API design: an empty list of targets
basically makes an execution a Noop. So a single method, SetExecution is
enough to cover all use cases. Noop executions are not returned in
future search requests.
- **Compatibility**: The executions created with previous v3alpha APIs
are still available to be managed with the new executions API.

# Additional Changes

- Removed integration tests which test executions but rely on readable
targets. They are added again with #8169

# Additional Context

Closes #8168
2024-07-31 14:42:12 +02:00
Stefan Benz
7d2d85f57c
feat: api v2beta to api v2 (#8283)
# Which Problems Are Solved

The v2beta services are stable but not GA.

# How the Problems Are Solved

The v2beta services are copied to v2. The corresponding v1 and v2beta
services are deprecated.

# Additional Context

Closes #7236

---------

Co-authored-by: Elio Bischof <elio@zitadel.com>
2024-07-26 22:39:55 +02:00
Joey Biscoglia
f34897a8c8
feat: add delete phone endpoint to v2 api (#8321)
# Which Problems Are Solved

- Adds delete phone endpoint to v2 api

# How the Problems Are Solved

- Adds new endpoint with DELETE method to /v2beta/users/:userId/phone
which removes currently set phone number

# Additional Changes

- Added integration test for new endpoint.

# Additional Context

- Solves
https://discord.com/channels/927474939156643850/1255557862286032996
2024-07-24 13:46:07 +00:00
Silvan
99c645cc60
refactor(database): exchange connection pool (#8325)
# Which Problems Are Solved

The connection pool of go uses a high amount of database connections.

# How the Problems Are Solved

The standard lib connection pool was replaced by `pgxpool.Pool`

# Additional Changes

The `db.BeginTx`-spans are removed because they cause to much noise in
the traces.

# Additional Context

- part of https://github.com/zitadel/zitadel/issues/7639
2024-07-17 15:16:02 +00:00
Stefan Benz
fb2d4545b9
fix: add details to ListUsers for user results (#8255)
# Which Problems Are Solved

In User v2 API, the ListUsers endpoint doesn't provide the information
to which organization the user belongs to.

# How the Problems Are Solved

Add the details to the user results from the ListUsers endpoint, so that
the OrgID is also included as ResourceOwner.

# Additional Changes
 
None

# Additional Context

Closes #8172
2024-07-10 17:49:35 +02:00
Tim Möhlmann
669ac6bda2
perf(import): do not check for existing grant ID (#8164)
# Which Problems Are Solved

Improve the performance of the `admin/v1/import` API endpoint.
Specifaclly the import of large amount of project grants.

# How the Problems Are Solved

`AddProjectGrantWithID` and `AddProjectGrantMember` methods of
`Commands` used to get the current state of the Writemodel to check if
the current GrantID or the combination of GrantID & UserID wasn't
already used. However, the Added events already have protection against
duplication by the `UniqueConstaint` methods.

The queries become very slow when there is a great amount of project
grants. Because all the events are pushed to the aggregate ID of the
project, we had to obtain all related project events, including events
of grantIDs we do not care about. This O(n) duration for bached import
jobs adding many organization granted to a single project.

This change removes the unnecesary state query to improve performance.

# Additional Changes

- Add integration tests for import

# Additional Context

- reported internally
2024-06-20 13:31:58 +00:00
Livio Spring
fb8cd18f93
feat: password age policy (#8132)
# Which Problems Are Solved

Some organizations / customers have the requirement, that there users
regularly need to change their password.
ZITADEL already had the possibility to manage a `password age policy` (
thought the API) with the maximum amount of days a password should be
valid, resp. days after with the user should be warned of the upcoming
expiration.
The policy could not be managed though the Console UI and was not
checked in the Login UI.

# How the Problems Are Solved

- The policy can be managed in the Console UI's settings sections on an
instance and organization level.
- During an authentication in the Login UI, if a policy is set with an
expiry (>0) and the user's last password change exceeds the amount of
days set, the user will be prompted to change their password.
- The prompt message of the Login UI can be customized in the Custom
Login Texts though the Console and API on the instance and each
organization.
- The information when the user last changed their password is returned
in the Auth, Management and User V2 API.
- The policy can be retrieved in the settings service as `password
expiry settings`.

# Additional Changes

None.

# Additional Context

- closes #8081

---------

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
2024-06-18 11:27:44 +00:00
Livio Spring
b6c10c4c83
fix: consider oidc session events for authN milestones (#8089)
# Which Problems Are Solved

After migrating the access token events in #7822, milestones based on
authentication, resp. theses events would not be reached.

# How the Problems Are Solved

Additionally use the `oidc_session.Added` event to check for
`milestone.AuthenticationSucceededOnInstance` and
`milestone.AuthenticationSucceededOnApplication`.

# Additional Changes

None.

# Additional Context

- relates to #7822 
- noticed internally
2024-06-12 06:49:14 +02:00
Livio Spring
f065b42a97
fix(oidc): respect role assertion and idTokenInfo flags and trigger preAccessToken trigger (#8046)
# Which Problems Are Solved

After deployment of 2.53.x, customers noted that the roles claims where
always present in the tokens even if the corresponding option on the
client (accessTokenRoleAssertion, idTokenRoleAsseriton) was disabled.
Only the project flag (assertRolesOnAuthentication) would be considered.

Further it was noted, that the action on the preAccessTokenCreation
trigger would not be executed.

Additionally, while testing those issues we found out, that the user
information (name, givenname, family name, ...) where always present in
the id_token even if the option (idTokenUserInfo) was not enabled.

# How the Problems Are Solved

- The `getUserinfoOnce` which was used for access and id_tokens is
refactored to `getUserInfo` and no longer only queries the info once
from the database, but still provides a mechanism to be reused for
access and id_token where the corresponding `roleAssertion` and action
`triggerType` can be passed.
- `userInfo` on the other hand now directly makes sure the information
is only queried once from the database. Role claims are asserted every
time and action triggers are executed on every call.
- `userInfo` now also checks if the profile information need to be
returned.

# Additional Changes

None.

# Additional Context

- relates to #7822 
- reported by customers
2024-05-31 10:10:18 +00:00
Livio Spring
e57a9b57c8
feat(saml): allow setting nameid-format and alternative mapping for transient format (#7979)
# Which Problems Are Solved

ZITADEL currently always uses
`urn:oasis:names:tc:SAML:2.0:nameid-format:persistent` in SAML requests,
relying on the IdP to respect that flag and always return a peristent
nameid in order to be able to map the external user with an existing
user (idp link) in ZITADEL.
In case the IdP however returns a
`urn:oasis:names:tc:SAML:2.0:nameid-format:transient` (transient)
nameid, the attribute will differ between each request and it will not
be possible to match existing users.

# How the Problems Are Solved

This PR adds the following two options on SAML IdP:
- **nameIDFormat**: allows to set the nameid-format used in the SAML
Request
- **transientMappingAttributeName**: allows to set an attribute name,
which will be used instead of the nameid itself in case the returned
nameid-format is transient

# Additional Changes

To reduce impact on current installations, the `idp_templates6_saml`
table is altered with the two added columns by a setup job. New
installations will automatically get the table with the two columns
directly.
All idp unit tests are updated to use `expectEventstore` instead of the
deprecated `eventstoreExpect`.

# Additional Context

Closes #7483
Closes #7743

---------

Co-authored-by: peintnermax <max@caos.ch>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2024-05-23 05:04:07 +00:00
Tim Möhlmann
5b1160de1e
feat(session): allow update of session without token (#7963)
# Which Problems Are Solved

The session update requires the current session token as argument.
Since this adds extra complexity but no real additional security and
prevents case like magic links, we want to remove this requirement.

We still require the session token on other resouces / endpoints, e.g.
for finalizing the auth request or on idp intents.

# How the Problems Are Solved

- Removed the session token verifier in the Update Session GRPc call.
- Removed the session token from login UI examples session update calls

# Additional Changes

- none

# Additional Context

- Closes #7883
2024-05-22 05:56:11 +00:00
Tim Möhlmann
8e0c8393e9
perf(oidc): optimize token creation (#7822)
* implement code exchange

* port tokenexchange to v2 tokens

* implement refresh token

* implement client credentials

* implement jwt profile

* implement device token

* cleanup unused code

* fix current unit tests

* add user agent unit test

* unit test domain package

* need refresh token as argument

* test commands create oidc session

* test commands device auth

* fix device auth build error

* implicit for oidc session API

* implement authorize callback handler for legacy implicit mode

* upgrade oidc module to working draft

* add missing auth methods and time

* handle all errors in defer

* do not fail auth request on error

the oauth2 Go client automagically retries on any error. If we fail the auth request on the first error, the next attempt will always fail with the Errors.AuthRequest.NoCode, because the auth request state is already set to failed.
The original error is then already lost and the oauth2 library does not return the original error.

Therefore we should not fail the auth request.

Might be worth discussing and perhaps send a bug report to Oauth2?

* fix code flow tests by explicitly setting code exchanged

* fix unit tests in command package

* return allowed scope from client credential client

* add device auth done reducer

* carry nonce thru session into ID token

* fix token exchange integration tests

* allow project role scope prefix in client credentials client

* gci formatting

* do not return refresh token in client credentials and jwt profile

* check org scope

* solve linting issue on authorize callback error

* end session based on v2 session ID

* use preferred language and user agent ID for v2 access tokens

* pin oidc v3.23.2

* add integration test for jwt profile and client credentials with org scopes

* refresh token v1 to v2

* add user token v2 audit event

* add activity trigger

* cleanup and set panics for unused methods

* use the encrypted code for v1 auth request get by code

* add missing event translation

* fix pipeline errors (hopefully)

* fix another test

* revert pointer usage of preferred language

* solve browser info panic in device auth

* remove duplicate entries in AMRToAuthMethodTypes to prevent future `mfa` claim

* revoke v1 refresh token to prevent reuse

* fix terminate oidc session

* always return a new refresh toke in refresh token grant

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-05-16 07:07:56 +02:00
Stefan Benz
0e9ebed8d0
fix: import totp in add human user with secret (#7936)
* fix: import totp in add human user with secret

* fix: import totp in add human user with secret

* fix: import totp in add human user with secret

* fix: review comment changes
2024-05-14 09:20:31 +02:00
Stefan Benz
72c5b057f1
fix: allow start and retrieve IdP intents with IdPs from other organizations (#7871)
* fix: correct resourceowner of intent to instance

* fix: correct resourceowner of intent to instance

* fix: correct resourceowner of intent to instance

* fix: correct resourceowner of intent to instance

* fix: correct resourceowner of intent to instance

* docs: expand the login example with org specific parameters

* fix: existence of idp is not checked through resourceowner

* fix: existence of idp is not checked through resourceowner

* fix: existence of idp is not checked through resourceowner

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-05-07 06:11:20 +00:00
Stefan Benz
1c5ecba42a
feat: add action v2 execution on requests and responses (#7637)
* feat: add execution of targets to grpc calls

* feat: add execution of targets to grpc calls

* feat: add execution of targets to grpc calls

* feat: add execution of targets to grpc calls

* feat: add execution of targets to grpc calls

* feat: add execution of targets to grpc calls

* feat: add execution of targets to grpc calls

* feat: split request and response logic to handle the different context information

* feat: split request and response logic to handle the different context information

* fix: integration test

* fix: import alias

* fix: refactor execution package

* fix: refactor execution interceptor integration and unit tests

* fix: refactor execution interceptor integration and unit tests

* fix: refactor execution interceptor integration and unit tests

* fix: refactor execution interceptor integration and unit tests

* fix: refactor execution interceptor integration and unit tests

* docs: basic documentation for executions and targets

* fix: change order for interceptors

* fix: merge back origin/main

* fix: change target definition command and query side (#7735)

* fix: change target definition command and query side

* fix: correct refactoring name changes

* fix: correct refactoring name changes

* fix: changing execution defintion with target list and type

* fix: changing execution definition with target list and type

* fix: add back search queries for target and include

* fix: projections change for execution with targets suffix table

* fix: projections change for execution with targets suffix table

* fix: projections change for execution with targets suffix table

* fix: projections change for execution with targets suffix table

* fix: projections change for execution with targets suffix table

* fix: projections change for execution with targets suffix table

* fix: projections change for execution with targets suffix table

* docs: add example to actions v2

* docs: add example to actions v2

* fix: correct integration tests on query for executions

* fix: add separate event for execution v2 as content changed

* fix: add separate event for execution v2 as content changed

* fix: added review comment changes

* fix: added review comment changes

* fix: added review comment changes

---------

Co-authored-by: adlerhurst <silvan.reusser@gmail.com>

* fix: added review comment changes

* fix: added review comment changes

* Update internal/api/grpc/server/middleware/execution_interceptor.go

Co-authored-by: Silvan <silvan.reusser@gmail.com>

* fix: added review comment changes

* fix: added review comment changes

* fix: added review comment changes

* fix: added review comment changes

* fix: added review comment changes

* fix: added review comment changes

---------

Co-authored-by: adlerhurst <silvan.reusser@gmail.com>
Co-authored-by: Elio Bischof <elio@zitadel.com>
2024-05-04 11:55:57 +02:00
Elio Bischof
42bd636d21
test(e2e): fix events flakiness (#7829) 2024-04-23 09:20:11 +02:00
Tim Möhlmann
9ccbbe05bc
fix(oidc): roles in userinfo for client credentials token (#7763)
* fix(oidc): roles in userinfo for client credentials token

When tokens were obtained using the client credentials grant,
with audience and role scopes, userinfo would not return the role claims. This had multiple causes:

1. There is no auth request flow, so for legacy userinfo project data was never attached to the token
2. For optimized userinfo, there is no client ID that maps to an application. The client ID for client credentials is the machine user's name. There we can't obtain a project ID. When the project ID remained empty, we always ignored the roleAudience.

This PR fixes situation 2, by always taking the roleAudience into account, even when the projectID is empty. The code responsible for the bug is also refactored to be more readable and understandable, including additional godoc.

The fix only applies to the optimized userinfo code introduced in #7706 and released in v2.50 (currently in RC). Therefore it can't be back-ported to earlier versions.

Fixes #6662

* chore(deps): update all go deps (#7764)

This change updates all go modules, including oidc, a major version of go-jose and the go 1.22 release.

* Revert "chore(deps): update all go deps" (#7772)

Revert "chore(deps): update all go deps (#7764)"

This reverts commit 6893e7d060.

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-04-16 13:02:38 +00:00
Stefan Benz
6dcdef0268
fix: add action v2 execution to features (#7597)
* fix: add action v2 execution to features

* fix: add action v2 execution to features

* fix: add action v2 execution to features

* fix: update internal/command/instance_features_model.go

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>

* fix: merge back main

* fix: merge back main

* fix: rename feature and service

* fix: rename feature and service

* fix: review changes

* fix: review changes

---------

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
2024-04-09 20:21:21 +03:00
Stefan Benz
217703395e
feat: add user v2 pw change required information on query (#7603)
* fix: add resource owner as query for user v2 ListUsers and clean up deprecated attribute

* fix: add resource owner as query for user v2 ListUsers and clean up deprecated attribute

* fix: add resource owner as query for user v2 ListUsers and clean up deprecated attribute

* fix: review changes

* fix: review changes

* fix: review changes

* fix: review changes

* fix: add password change required to user v2 get and list

* fix: update unit tests for query side with new column and projection

* fix: change projection in setup steps

* fix: change projection in setup steps

* fix: remove setup step 25

* fix: add password_change_required into ListUsers response

* fix: correct SetUserPassword parameters

* fix: rollback to change setup instead of projection directly

* fix: rollback to change setup instead of projection directly

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-03-28 06:21:21 +00:00
Livio Spring
7494a7b6d9
feat(api): add possibility to retrieve user schemas (#7614)
This PR extends the user schema service (V3 API) with the possibility to ListUserSchemas and GetUserSchemaByID.
The previously started guide is extended to demonstrate how to retrieve the schema(s) and notes the generated revision property.
2024-03-22 13:26:13 +00:00
Tim Möhlmann
6398349c24
feat(oidc): token exchange impersonation (#7516)
* add token exchange feature flag

* allow setting reason and actor to access tokens

* impersonation

* set token types and scopes in response

* upgrade oidc to working draft state

* fix tests

* audience and scope validation

* id toke and jwt as input

* return id tokens

* add grant type  token exchange to app config

* add integration tests

* check and deny actors in api calls

* fix instance setting tests by triggering projection on write and cleanup

* insert sleep statements again

* solve linting issues

* add translations

* pin oidc v3.15.0

* resolve comments, add event translation

* fix refreshtoken test

* use ValidateAuthReqScopes from oidc

* apparently the linter can't make up its mind

* persist actor thru refresh tokens and check in tests

* remove unneeded triggers
2024-03-20 10:18:46 +00:00
Stefan Benz
fb3c6f791b
feat: query side for executions and targets for actions v2 (#7524)
* feat: add projections and query side to executions and targets

* feat: add list and get endpoints for targets

* feat: add integration tests for query endpoints target and execution

* fix: linting

* fix: linting

* fix: review changes, renames and corrections

* fix: review changes, renames and corrections

* fix: review changes, renames and corrections

* fix: review changes, renames and corrections

* fix: review changes, renames and corrections

* fix: review changes, renames and corrections

* fix: remove position from list details
2024-03-14 09:56:23 +00:00
Livio Spring
0e181b218c
feat: implement user schema management (#7416)
This PR adds the functionality to manage user schemas through the new user schema service.
It includes the possibility to create a basic JSON schema and also provides a way on defining permissions (read, write) for owner and self context with an annotation.

Further annotations for OIDC claims and SAML attribute mappings will follow.

A guide on how to create a schema and assign permissions has been started. It will be extended though out the process of implementing the schema and users based on those.

Note:
This feature is in an early stage and therefore not enabled by default. To test it out, please enable the UserSchema feature flag on your instance / system though the feature service.
2024-03-12 13:50:13 +00:00
Stefan Benz
9f72fc63ac
fix: add additional permission tests to user v2 query endpoints (#7382)
Add additional permission integration tests to the user v2 query endpoints including some fixes to correctly check the permissions after the data is known which you want to query.
2024-03-08 08:37:23 +00:00
Tim Möhlmann
062d153cfe
feat: impersonation roles (#7442)
* partial work done

* test IAM membership roles

* org membership tests

* console :(, translations and docs

* fix integration test

* fix tests

* add EnableImpersonation to security policy API

* fix integration test timestamp checking

* add security policy tests and fix projections

* add impersonation setting in console

* add security settings to the settings v2 API

* fix typo

* move impersonation to instance

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-02-28 10:21:11 +00:00
Tim Möhlmann
26d1563643
feat(api): feature flags (#7356)
* feat(api): feature API proto definitions

* update proto based on discussion with @livio-a

* cleanup old feature flag stuff

* authz instance queries

* align defaults

* projection definitions

* define commands and event reducers

* implement system and instance setter APIs

* api getter implementation

* unit test repository package

* command unit tests

* unit test Get queries

* grpc converter unit tests

* migrate the V1 features

* migrate oidc to dynamic features

* projection unit test

* fix instance by host

* fix instance by id data type in sql

* fix linting errors

* add system projection test

* fix behavior inversion

* resolve proto file comments

* rename SystemDefaultLoginInstanceEventType to SystemLoginDefaultOrgEventType so it's consistent with the instance level event

* use write models and conditional set events

* system features integration tests

* instance features integration tests

* error on empty request

* documentation entry

* typo in feature.proto

* fix start unit tests

* solve linting error on key case switch

* remove system defaults after discussion with @eliobischof

* fix system feature projection

* resolve comments in defaults.yaml

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-02-28 10:55:54 +02:00
Stefan Benz
2731099db3
feat: add executions for actions v2 (#7433)
* feat: add events for execution

* feat: add events for execution and command side

* feat: add events for execution and command side

* feat: add api endpoints for set and delete executions with integration tests

* feat: add integration and unit tests and more existence checks

* feat: add integration and unit tests and more existence checks

* feat: unit tests for includes in executions

* feat: integration tests for includes in executions

* fix: linting

* fix: update internal/api/api.go

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>

* fix: update internal/command/command.go

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>

* fix: apply suggestions from code review

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>

* fix: change api return

* fix: change aggregateID with prefix of execution type and add to documentation

* fix: change body in proto for documentation and correct linting

* fix: changed existing check to single query in separate writemodel

* fix: linter changes and list endpoints for conditions in executions

* fix: remove writemodel query on exeuction set as state before is irrelevant

* fix: testing for exists write models and correction

* fix: translations for errors and event types

---------

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
2024-02-26 12:49:43 +02:00
Elio Bischof
585988bd83
chore(ci): merge test runs (#7388)
* chore(ci): merge test runs

* docs: improve CONTRIBUTING.md

* expect failure

* expect failure unit

* cleanup

* fix: correct binding in integration tests

* fix: correct binding in integration tests

---------

Co-authored-by: Stefan Benz <stefan@caos.ch>
2024-02-19 07:50:37 +01:00
Stefan Benz
198bc017b8
feat: actions v2 execution targets command side (#7384)
Adds the API to create, update, delete targets for execution in a new ExecutionService (v3alpha)
2024-02-15 05:39:10 +00:00
Silvan
17953e9040
fix(setup): init projections (#7194)
Even though this is a feature it's released as fix so that we can back port to earlier revisions.

As reported by multiple users startup of ZITADEL after leaded to downtime and worst case rollbacks to the previously deployed version.

The problem starts rising when there are too many events to process after the start of ZITADEL. The root cause are changes on projections (database tables) which must be recomputed. This PR solves this problem by adding a new step to the setup phase which prefills the projections. The step can be enabled by adding the `--init-projections`-flag to `setup`, `start-from-init` and `start-from-setup`. Setting this flag results in potentially longer duration of the setup phase but reduces the risk of the problems mentioned in the paragraph above.
2024-01-25 17:28:20 +01:00
Elio Bischof
ed0bc39ea4
feat: block instances (#7129)
* docs: fix init description typos

* feat: block instances using limits

* translate

* unit tests

* fix translations

* redirect /ui/login

* fix http interceptor

* cleanup

* fix http interceptor

* fix: delete cookies on gateway 200

* add integration tests

* add command test

* docs

* fix integration tests

* add bulk api and integration test

* optimize bulk set limits

* unit test bulk limits

* fix broken link

* fix assets middleware

* fix broken link

* validate instance id format

* Update internal/eventstore/search_query.go

Co-authored-by: Livio Spring <livio.a@gmail.com>

* remove support for owner bulk limit commands

* project limits to instances

* migrate instances projection

* Revert "migrate instances projection"

This reverts commit 214218732a.

* join limits, remove owner

* remove todo

* use optional bool

* normally validate instance ids

* use 302

* cleanup

* cleanup

* Update internal/api/grpc/system/limits_converter.go

Co-authored-by: Livio Spring <livio.a@gmail.com>

* remove owner

* remove owner from reset

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-01-17 10:16:48 +00:00
Stefan Benz
d9d376a275
feat: user v2 service query (#7095)
* feat: add query endpoints for user v2 api

* fix: correct integration tests

* fix: correct linting

* fix: correct linting

* fix: comment out permission check on user get and list

* fix: permission check on user v2 query

* fix: merge back origin/main

* fix: add search query in user emails

* fix: reset count for SearchUser if users are removed due to permissions

* fix: reset count for SearchUser if users are removed due to permissions

---------

Co-authored-by: Elio Bischof <elio@zitadel.com>
2024-01-17 10:00:10 +01:00
Tim Möhlmann
45ccdcfa99
fix(oidc): nil check for client secret (#7115)
This fixes a nil pointer panic when client basic auth is attempted on a client without secret in introspection.
2023-12-28 13:31:41 +00:00
Yordis Prieto
9d5d1cf3ea
feat: allow glob redirects (#7091)
fixes #5110
2023-12-28 11:25:18 +02:00
Stefan Benz
a0a82b59e1
feat: user service v2 create, update and remove (#6996)
* feat: user service v2 remove user

* feat: user service v2 add user human

* feat: user service v2 change user human

* feat: user service v2 change user human unit tests

* feat: user service v2 reactivate, deactivate, lock, unlock user

* feat: user service v2 integration tests

* fix: merge back origin/main

* lint: linter corrections

* fix: move permission check for isVerfied and password change

* fix: add deprecated notices and other review comments

* fix: consistent naming in proto

* fix: errors package renaming

* fix: remove / delete user renaming in integration test

* fix: machine user status changes through user v2 api

* fix: linting changes

* fix: linting changes

* fix: changes from review

* fix: changes from review

* fix: changes from review

* fix: changes from review

* fix: changes from review

---------

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-12-21 10:03:37 +01:00
Tim Möhlmann
f680dd934d
refactor: rename package errors to zerrors (#7039)
* chore: rename package errors to zerrors

* rename package errors to gerrors

* fix error related linting issues

* fix zitadel error assertion

* fix gosimple linting issues

* fix deprecated linting issues

* resolve gci linting issues

* fix import structure

---------

Co-authored-by: Elio Bischof <elio@zitadel.com>
2023-12-08 15:30:55 +01:00
Elio Bischof
8c85318fbd
fix: restrict languages in console (#6964)
* feat: return 404 or 409 if org reg disallowed

* fix: system limit permissions

* feat: add iam limits api

* feat: disallow public org registrations on default instance

* add integration test

* test: integration

* fix test

* docs: describe public org registrations

* avoid updating docs deps

* fix system limits integration test

* silence integration tests

* fix linting

* ignore strange linter complaints

* review

* improve reset properties naming

* redefine the api

* use restrictions aggregate

* test query

* simplify and test projection

* test commands

* fix unit tests

* move integration test

* support restrictions on default instance

* also test GetRestrictions

* self review

* lint

* abstract away resource owner

* fix tests

* configure supported languages

* fix allowed languages

* fix tests

* default lang must not be restricted

* preferred language must be allowed

* change preferred languages

* check languages everywhere

* lint

* test command side

* lint

* add integration test

* add integration test

* restrict supported ui locales

* lint

* lint

* cleanup

* lint

* allow undefined preferred language

* fix integration tests

* update main

* fix env var

* ignore linter

* ignore linter

* improve integration test config

* reduce cognitive complexity

* compile

* fix(console): switch back to saved language

* feat(API): get allowed languages

* fix(console): only make allowed languages selectable

* warn when editing not allowed languages

* check for duplicates

* remove useless restriction checks

* review

* revert restriction renaming

* fix language restrictions

* lint

* generate

* allow custom texts for supported langs for now

* fix tests

* cleanup

* cleanup

* cleanup

* lint

* unsupported preferred lang is allowed

* fix integration test

* allow unsupported preferred languages

* lint

* load languages for tests

* cleanup

* lint

* cleanup

* get allowed only on admin

* cleanup

* reduce flakiness on very limited postgres

* simplify langSvc

* refactor according to suggestions in pr

* lint

* set first allowed language as default

* selectionchange for language in msg texts

* initialize login texts

* init message texts

* lint

---------

Co-authored-by: peintnermax <max@caos.ch>
2023-12-07 08:43:23 +00:00
Tim Möhlmann
ec03340b67
perf(oidc): optimize client verification (#6999)
* fix some spelling errors

* client credential auth

* implementation of client auth

* improve error handling

* unit test command package

* unit test database package

* unit test query package

* cleanup unused tracing func

* fix integration tests

* errz to zerrors

* fix linting and import issues

* fix another linting error

* integration test with client secret

* Revert "integration test with client secret"

This reverts commit 0814ba522f.

* add integration tests

* client credentials integration test

* resolve comments

* pin oidc v3.5.0
2023-12-05 17:01:03 +00:00
Elio Bischof
dd33538c0a
feat: restrict languages (#6931)
* feat: return 404 or 409 if org reg disallowed

* fix: system limit permissions

* feat: add iam limits api

* feat: disallow public org registrations on default instance

* add integration test

* test: integration

* fix test

* docs: describe public org registrations

* avoid updating docs deps

* fix system limits integration test

* silence integration tests

* fix linting

* ignore strange linter complaints

* review

* improve reset properties naming

* redefine the api

* use restrictions aggregate

* test query

* simplify and test projection

* test commands

* fix unit tests

* move integration test

* support restrictions on default instance

* also test GetRestrictions

* self review

* lint

* abstract away resource owner

* fix tests

* configure supported languages

* fix allowed languages

* fix tests

* default lang must not be restricted

* preferred language must be allowed

* change preferred languages

* check languages everywhere

* lint

* test command side

* lint

* add integration test

* add integration test

* restrict supported ui locales

* lint

* lint

* cleanup

* lint

* allow undefined preferred language

* fix integration tests

* update main

* fix env var

* ignore linter

* ignore linter

* improve integration test config

* reduce cognitive complexity

* compile

* check for duplicates

* remove useless restriction checks

* review

* revert restriction renaming

* fix language restrictions

* lint

* generate

* allow custom texts for supported langs for now

* fix tests

* cleanup

* cleanup

* cleanup

* lint

* unsupported preferred lang is allowed

* fix integration test

* finish reverting to old property name

* finish reverting to old property name

* load languages

* refactor(i18n): centralize translators and fs

* lint

* amplify no validations on preferred languages

* fix integration test

* lint

* fix resetting allowed languages

* test unchanged restrictions
2023-12-05 11:12:01 +00:00
Elio Bischof
60688757fa
test(postgres): always test against latest release (#6972)
* test(postgres): always test against latest

* Update CONTRIBUTING.md

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>

* Update internal/integration/config/docker-compose.yaml

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>

---------

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
2023-11-24 13:56:58 +00:00
Tim Möhlmann
72bc3ffe14
fix(oidc): add missing fields to introspection (#6967)
during QA I found some user info and org ID was missing.
This change adds those missing fields.
2023-11-23 16:17:50 +02:00
Elio Bischof
76fe032b5f
feat: option to disallow public org registration (#6917)
* feat: return 404 or 409 if org reg disallowed

* fix: system limit permissions

* feat: add iam limits api

* feat: disallow public org registrations on default instance

* add integration test

* test: integration

* fix test

* docs: describe public org registrations

* avoid updating docs deps

* fix system limits integration test

* silence integration tests

* fix linting

* ignore strange linter complaints

* review

* improve reset properties naming

* redefine the api

* use restrictions aggregate

* test query

* simplify and test projection

* test commands

* fix unit tests

* move integration test

* support restrictions on default instance

* also test GetRestrictions

* self review

* lint

* abstract away resource owner

* fix tests

* lint
2023-11-22 09:29:38 +00:00
Tim Möhlmann
ba9b807854
perf(oidc): optimize the introspection endpoint (#6909)
* get key by id and cache them

* userinfo from events for v2 tokens

* improve keyset caching

* concurrent token and client checks

* client and project in single query

* logging and otel

* drop owner_removed column on apps and authN tables

* userinfo and project roles in go routines

* get  oidc user info from projections and add actions

* add avatar URL

* some cleanup

* pull oidc work branch

* remove storage from server

* add config flag for experimental introspection

* legacy introspection flag

* drop owner_removed column on user projections

* drop owner_removed column on useer_metadata

* query userinfo unit test

* query introspection client test

* add user_grants to the userinfo query

* handle PAT scopes

* bring triggers back

* test instance keys query

* add userinfo unit tests

* unit test keys

* go mod tidy

* solve some bugs

* fix missing preferred login name

* do not run triggers in go routines, they seem to deadlock

* initialize the trigger handlers late with a sync.OnceValue

* Revert "do not run triggers in go routines, they seem to deadlock"

This reverts commit 2a03da2127.

* add missing translations

* chore: update go version for linting

* pin oidc version

* parse a global time location for query test

* fix linter complains

* upgrade go lint

* fix more linting issues

---------

Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2023-11-21 13:11:38 +01:00
Livio Spring
f3b8a3aece
feat: add possibility to set an expiration to a session (#6851)
* add lifetime to session api

* extend session with lifetime

* check session token expiration

* fix typo

* integration test to check session token expiration

* integration test to check session token expiration

* i18n

* cleanup

* improve tests

* prevent negative lifetime

* fix error message

* fix lifetime check
2023-11-06 10:48:28 +01:00