feat: add UUID support#7962
Conversation
COMPARE TO
|
| Name | Diff |
|---|---|
| packages/cli/src/commands/database/seed/index.ts | 📈 +1.13 KB |
| packages/cli/src/commands/database/seed/tables.ts | 📈 +3.28 KB |
| packages/cli/src/commands/database/seed/utils.ts | 📈 +1.56 KB |
| packages/cli/src/commands/install/utils.ts | 📈 +201 Bytes |
| packages/core/src/libraries/id-format.test.ts | 📈 +7.42 KB |
| packages/core/src/libraries/id-format.ts | 📈 +3.51 KB |
| packages/core/src/libraries/user.ts | 📈 +223 Bytes |
| packages/core/src/queries/tenant-id-config.test.ts | 📈 +3.14 KB |
| packages/core/src/queries/tenant-id-config.ts | 📈 +1.62 KB |
| packages/core/src/routes/applications/application.ts | 📈 +30 Bytes |
| packages/core/src/routes/organization-role/index.ts | 📈 +58 Bytes |
| packages/core/src/routes/organization/index.ts | 📈 +76 Bytes |
| packages/core/src/routes/role.ts | 📈 +27 Bytes |
| packages/core/src/routes/saml-application/index.ts | 📈 +18 Bytes |
| packages/core/src/tenants/Libraries.ts | 📈 +151 Bytes |
| packages/core/src/tenants/Queries.ts | 📈 +135 Bytes |
| packages/core/src/utils/SchemaRouter.ts | 📈 +490 Bytes |
| packages/schemas/alterations/next-1762400000.1-add-uuid-support-tenant-config.ts | 📈 +3.06 KB |
| packages/schemas/alterations/next-1762400000.2-add-uuid-support-primary-tables.ts | 📈 +2.07 KB |
| packages/schemas/alterations/next-1762400000.3-add-uuid-support-user-fks.ts | |
| packages/schemas/alterations/next-1762400000.4-add-uuid-support-org-role-fks.ts | |
| packages/schemas/alterations/next-1762400000.5-add-uuid-support-app-fks.ts | |
| packages/schemas/src/seeds/index.ts | 📈 +39 Bytes |
| packages/schemas/src/seeds/tenant-id-config.ts | 📈 +705 Bytes |
| packages/schemas/tables/application_secrets.sql | 0 Bytes |
| packages/schemas/tables/application_sign_in_experiences.sql | 0 Bytes |
| packages/schemas/tables/application_user_consent_organization_resource_scopes.sql | 0 Bytes |
| packages/schemas/tables/application_user_consent_organization_scopes.sql | 0 Bytes |
| packages/schemas/tables/application_user_consent_organizations.sql | 0 Bytes |
| packages/schemas/tables/application_user_consent_resource_scopes.sql | 0 Bytes |
| packages/schemas/tables/application_user_consent_user_scopes.sql | 0 Bytes |
| packages/schemas/tables/applications.sql | 0 Bytes |
| packages/schemas/tables/applications_roles.sql | 0 Bytes |
| packages/schemas/tables/daily_active_users.sql | 0 Bytes |
| packages/schemas/tables/organization_application_relations.sql | 0 Bytes |
| packages/schemas/tables/organization_invitation_role_relations.sql | 0 Bytes |
| packages/schemas/tables/organization_invitations.sql | 0 Bytes |
| packages/schemas/tables/organization_jit_email_domains.sql | 0 Bytes |
| packages/schemas/tables/organization_jit_roles.sql | 0 Bytes |
| packages/schemas/tables/organization_jit_sso_connectors.sql | 0 Bytes |
| packages/schemas/tables/organization_role_application_relations.sql | 0 Bytes |
| packages/schemas/tables/organization_role_resource_scope_relations.sql | 0 Bytes |
| packages/schemas/tables/organization_role_scope_relations.sql | 0 Bytes |
| packages/schemas/tables/organization_role_user_relations.sql | 0 Bytes |
| packages/schemas/tables/organization_roles.sql | 0 Bytes |
| packages/schemas/tables/organization_user_relations.sql | 0 Bytes |
| packages/schemas/tables/organizations.sql | 0 Bytes |
| packages/schemas/tables/personal_access_tokens.sql | 0 Bytes |
| packages/schemas/tables/roles.sql | 0 Bytes |
| packages/schemas/tables/roles_scopes.sql | 0 Bytes |
| packages/schemas/tables/saml_application_configs.sql | 0 Bytes |
| packages/schemas/tables/saml_application_secrets.sql | 0 Bytes |
| packages/schemas/tables/saml_application_sessions.sql | 0 Bytes |
| packages/schemas/tables/secrets.sql | 0 Bytes |
| packages/schemas/tables/sso_connector_idp_initiated_auth_configs.sql | 0 Bytes |
| packages/schemas/tables/subject_tokens.sql | 0 Bytes |
| packages/schemas/tables/tenant_id_config.sql | 📈 +608 Bytes |
| packages/schemas/tables/user_sso_identities.sql | 0 Bytes |
| packages/schemas/tables/users.sql | 0 Bytes |
| packages/schemas/tables/users_roles.sql | 0 Bytes |
| packages/schemas/tables/verification_records.sql | 0 Bytes |
| packages/schemas/tables/verification_statuses.sql | 0 Bytes |
| packages/shared/package.json | 📈 +23 Bytes |
| packages/shared/src/node/env/GlobalValues.ts | 📈 +335 Bytes |
| packages/shared/src/utils/id.test.ts | 📈 +1.85 KB |
| packages/shared/src/utils/id.ts | 📈 +1.11 KB |
| pnpm-lock.yaml | 📈 +349 Bytes |
|
I created this feature because it's the only blocker for us to use logto in our environment. We use UUIDs across our rest APIs. Implementing nanoid would require mapping nanoid to uuid and it feels like a hacky solution. PostgreSQL 18 will also have improved UUIDv7 support: https://www.thenile.dev/blog/uuidv7 Backwards compatible and selectable if you prefer to use nanoid. Let me know what you think. |
|
@Zyles We’re also planning to upgrade the Postgres version. I’ll get back to you once I have a final answer from the team about this. |
|
Hi @Zyles Thanks for the clear write-up, the idea makes sense. If you need it, forking and extending might be the better path. Appreciate the contribution! 🙌 |
I would prefer not having to fork and patch. What type of isolation do you have problems with? I could make it so that this is only applicable on new deployments, so you will need to explicitly set the ID generation on project installation and it will migrate either varchar or UUID for the column. This will keep the old varchar lengths vs UUID type in DB. Nothing really changes in currently running projects. |
|
This pattern also opens up for using other ID generation libraries if someone else is using something of the 20+ libraries out there since you are just working against one function (generateId). |
|
Hi @Zyles, if the following conditions are met, we can accept this PR:
|
|
This PR is stale because it has been open for 30 days with no activity. |
|
Hi, @Zyles any updates on this? |
I will look at it in February. |
|
@wangsijie How do you envision the ENV variable? This current solution makes it so that each tenant can have their own config/ID generation method. An ENV variable would force all tenants on the installation to use the same ID generation method? |
I think this should be instance-level control |
Zyles
force-pushed
the
feat/uuid-support
branch
from
9ce81e6 to
c80788e
Compare
Zyles
force-pushed
the
feat/uuid-support
branch
2 times, most recently
from
ae19d73 to
8396c38
Compare
Zyles
force-pushed
the
feat/uuid-support
branch
from
1acad7a to
25c00a1
Compare
|
@wangsijie Check this out. Did a bunch of refactoring and better type support. The flow is now:
|
|
Hi, @Zyles, thanks for the contribution, it looks good to me, and now the team is reviewing this PR, let's wait for a short time |
There was a problem hiding this comment.
Pull request overview
Adds an instance-wide, configurable entity ID format (default nanoid, optional UUID v7), updates shared ID utilities, database table definitions, seed flows, and runtime preconditions to support format-aware IDs and persistent ID-format locking via the systems table.
Changes:
- Introduce
IdFormat+ format-aware ID generation (UUID v7) and seed-ID conversion utilities in@logto/shared. - Update schema SQL to use an
${id_format}placeholder for entity ID columns and adjust schema/type generation to recognize and validate those columns. - Extend CLI install/seed flows to select/store
ID_FORMAT, resolve${id_format}at seed time, and validate env-vs-db format at runtime startup.
Reviewed changes
Copilot reviewed 79 out of 80 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| pnpm-lock.yaml | Adds uuid dependency resolution and lock updates. |
| packages/shared/package.json | Adds uuid dependency to shared package. |
| packages/shared/src/utils/id.ts | Adds IdFormat, UUID v7 generation, seed ID conversion, and format-aware generateId/generateStandardId. |
| packages/shared/src/utils/id.test.ts | Adds tests for UUID v7 and format-aware generateId. |
| packages/shared/src/node/env/GlobalValues.ts | Adds idFormat env parsing/validation to global env values. |
| packages/schemas/tables/verification_statuses.sql | Switches ID columns to ${id_format}. |
| packages/schemas/tables/verification_records.sql | Switches ID columns to ${id_format}. |
| packages/schemas/tables/users.sql | Switches users.id to ${id_format} and widens application_id to varchar(36). |
| packages/schemas/tables/users_roles.sql | Switches join IDs to ${id_format}. |
| packages/schemas/tables/user_sso_identities.sql | Switches IDs to ${id_format}. |
| packages/schemas/tables/subject_tokens.sql | Switches user_id to ${id_format}. |
| packages/schemas/tables/sso_connector_idp_initiated_auth_configs.sql | Widens default_application_id to varchar(36). |
| packages/schemas/tables/service_logs.sql | Switches id to ${id_format}. |
| packages/schemas/tables/sentinel_activities.sql | Switches id to ${id_format}. |
| packages/schemas/tables/secrets.sql | Switches IDs to ${id_format}. |
| packages/schemas/tables/secret_social_connector_relations.sql | Switches secret_id to ${id_format}. |
| packages/schemas/tables/secret_enterprise_sso_connector_relations.sql | Switches secret_id to ${id_format}. |
| packages/schemas/tables/scopes.sql | Widens id/resource_id to varchar(36). |
| packages/schemas/tables/saml_application_sessions.sql | Widens application_id to varchar(36). |
| packages/schemas/tables/saml_application_secrets.sql | Switches secret id to ${id_format}; widens application_id. |
| packages/schemas/tables/saml_application_configs.sql | Widens application_id to varchar(36). |
| packages/schemas/tables/roles.sql | Switches roles.id and function arg types to ${id_format}. |
| packages/schemas/tables/roles_scopes.sql | Switches IDs to ${id_format}; widens scope_id to varchar(36). |
| packages/schemas/tables/resources.sql | Widens resources.id to varchar(36). |
| packages/schemas/tables/personal_access_tokens.sql | Switches user_id to ${id_format}. |
| packages/schemas/tables/passcodes.sql | Switches id to ${id_format}. |
| packages/schemas/tables/organizations.sql | Switches organizations.id to ${id_format}. |
| packages/schemas/tables/organization_user_relations.sql | Switches org/user IDs to ${id_format}. |
| packages/schemas/tables/organization_scopes.sql | Widens id to varchar(36). |
| packages/schemas/tables/organization_roles.sql | Switches id and function arg types to ${id_format}. |
| packages/schemas/tables/organization_role_user_relations.sql | Switches org/role/user IDs to ${id_format}. |
| packages/schemas/tables/organization_role_scope_relations.sql | Switches organization_role_id to ${id_format}; widens scope IDs. |
| packages/schemas/tables/organization_role_resource_scope_relations.sql | Switches organization_role_id to ${id_format}; widens scope IDs. |
| packages/schemas/tables/organization_role_application_relations.sql | Switches org/role IDs to ${id_format}; widens application_id. |
| packages/schemas/tables/organization_jit_sso_connectors.sql | Switches organization_id to ${id_format}. |
| packages/schemas/tables/organization_jit_roles.sql | Switches org/role IDs to ${id_format}. |
| packages/schemas/tables/organization_jit_email_domains.sql | Switches organization_id to ${id_format}. |
| packages/schemas/tables/organization_invitations.sql | Switches invitation/org/user IDs to ${id_format}. |
| packages/schemas/tables/organization_invitation_role_relations.sql | Switches IDs to ${id_format}. |
| packages/schemas/tables/organization_application_relations.sql | Switches organization_id to ${id_format}; widens application_id. |
| packages/schemas/tables/one_time_tokens.sql | Switches id to ${id_format}. |
| packages/schemas/tables/oidc_session_extensions.sql | Switches account_id to ${id_format}. |
| packages/schemas/tables/logs.sql | Switches id to ${id_format}. |
| packages/schemas/tables/idp_initiated_saml_sso_sessions.sql | Switches id to ${id_format}. |
| packages/schemas/tables/hooks.sql | Switches id to ${id_format}. |
| packages/schemas/tables/email_templates.sql | Switches id to ${id_format}. |
| packages/schemas/tables/domains.sql | Switches id to ${id_format}. |
| packages/schemas/tables/daily_token_usage.sql | Switches id to ${id_format}. |
| packages/schemas/tables/daily_active_users.sql | Switches IDs to ${id_format}. |
| packages/schemas/tables/custom_profile_fields.sql | Switches id to ${id_format}. |
| packages/schemas/tables/custom_phrases.sql | Switches id to ${id_format}. |
| packages/schemas/tables/applications.sql | Widens applications.id and related function arg types to varchar(36). |
| packages/schemas/tables/applications_roles.sql | Switches join IDs to ${id_format}; widens application_id. |
| packages/schemas/tables/application_user_consent_user_scopes.sql | Widens application_id to varchar(36). |
| packages/schemas/tables/application_user_consent_resource_scopes.sql | Widens application_id/scope_id to varchar(36). |
| packages/schemas/tables/application_user_consent_organizations.sql | Widens application_id; switches org/user IDs to ${id_format}. |
| packages/schemas/tables/application_user_consent_organization_scopes.sql | Widens application_id/organization_scope_id to varchar(36). |
| packages/schemas/tables/application_user_consent_organization_resource_scopes.sql | Widens application_id/scope_id to varchar(36). |
| packages/schemas/tables/application_sign_in_experiences.sql | Widens application_id to varchar(36). |
| packages/schemas/tables/application_secrets.sql | Widens application_id to varchar(36). |
| packages/schemas/tables/_after_all.sql | Adds grant usage on schema public for tenant role after schema creation. |
| packages/schemas/src/types/tenant-organization.ts | Uses buildSeedId() for tenant-organization IDs and tenant role IDs. |
| packages/schemas/src/types/system.ts | Adds systems.idFormat key + guard/types for persisted ID format. |
| packages/schemas/src/types/mapi-proxy.ts | Makes proxy role/app IDs format-aware via buildSeedId(). |
| packages/schemas/src/seeds/management-api.ts | Adds createDefaultManagementApi() to generate format-aware seed data. |
| packages/schemas/src/seeds/application.ts | Adds format-aware getters for built-in application IDs and adjusts built-in logic. |
| packages/schemas/src/gen/utils.ts | Marks ${id_format} columns via /* @id_format */ during parsing. |
| packages/schemas/src/gen/types.ts | Adds Field.isIdFormat flag for schema generation. |
| packages/schemas/src/gen/schema.ts | Widens max length validations for ${id_format} columns (up to 36). |
| packages/schemas/src/gen/index.ts | Replaces ${id_format} with annotated default during type generation. |
| packages/core/src/utils/SchemaRouter.ts | Notes idLength is ignored for UUID format; refactors ID assignment. |
| packages/core/src/routes/organization-role/index.ts | Minor refactor of role insert call formatting. |
| packages/core/src/oidc/adapter.ts | Builds Admin Console metadata at runtime; uses format-aware built-in client IDs. |
| packages/core/src/libraries/user.ts | Switches user ID generation from short nanoid to format-aware standard IDs. |
| packages/core/src/env-set/preconditions.ts | Adds startup check to enforce env-vs-db idFormat consistency (and adopt DB value when env unset). |
| packages/console/src/containers/ConsoleContent/hooks.ts | Skips tenant-scope listening for OSS (cloud-only behavior). |
| packages/cli/src/commands/install/utils.ts | Adds ID format validation/prompt before seeding during install. |
| packages/cli/src/commands/database/seed/utils.ts | Adds interactive prompt for choosing ID format. |
| packages/cli/src/commands/database/seed/tables.ts | Resolves ${id_format} in SQL at seed-time; persists chosen format in systems. |
| packages/cli/src/commands/database/seed/index.ts | Adds --id-format option and resolves ID_FORMAT (CLI/env/prompt/default). |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
Comments suppressed due to low confidence (1)
packages/schemas/src/types/tenant-organization.ts:27
getTenantOrganizationId()now returnsbuildSeedId('t-${tenantId}'), which becomes a UUID v5 whenID_FORMAT=uuid. In that mode,getTenantIdFromOrganizationId()will always throw because UUIDs don’t start witht-, breaking tenant navigation/mapping (e.g. Console usesgetTenantIdFromOrganizationId(organizationId)). Consider keeping tenant-organization IDs reversible (e.g. storetenantIdseparately and stop parsing it fromorganizationId, or avoid UUID conversion for this specific derived ID scheme).
export const getTenantOrganizationId = (tenantId: string) => buildSeedId(`t-${tenantId}`);
/** Given an admin tenant organization ID, check the format and return the corresponding user tenant ID. */
export const getTenantIdFromOrganizationId = (organizationId: string) => {
if (!organizationId.startsWith('t-')) {
throw new Error(`Invalid admin tenant organization ID: ${organizationId}`);
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Zyles
force-pushed
the
feat/uuid-support
branch
from
25c00a1 to
2c0c598
Compare
Zyles
force-pushed
the
feat/uuid-support
branch
from
2c0c598 to
7a662db
Compare
|
@wangsijie The copilot issues have been resolved. |
Add support for UUID v7 as an alternative ID format, configurable per tenant.
Zyles
force-pushed
the
feat/uuid-support
branch
from
7a662db to
e71e906
Compare
|
There are new conflicts |
|
Hi @Zyles First of all, thank you so much for your hard work on this PR. We really appreciate the effort and thought you put into it. Also, apologies for the delayed response, as we took some time to review the changes internally with the team. After reviewing, we feel that while the UUID v7 support is a valuable direction, the current implementation mainly targets newly provisioned Logto open-source instances. For existing systems, it introduces certain migration risks and additional validation effort. Mostly, the current approach requires fairly intrusive changes to the existing schema/table definitions, which could increase long-term maintenance costs for future development. Especially given the fact that we may introduce new table definitions and DB alterations frequently. Given these considerations, we don’t think this implementation is suitable for merging into the main branch at the moment. That said, we do agree with the overall goal of supporting newer UUID formats. It’s just that, considering the balance between backward compatibility, maintenance overhead, and migration impact, this particular approach may not be the best fit. As a potential direction, it might be worth exploring a less intrusive approach, for example, introducing a post-create / post-alter table hook or function in the seeding and alteration script. With the env feature flag you have introduced, this could automatically apply UUID-related changes during seed jobs or after schema updates. This would help minimize changes to existing schema definitions while limiting the impact scope to newly initialized systems. Thanks again for the contribution. We really appreciate it, and we’d be happy to continue discussing alternative approaches if you’re interested. |
Okay, that is disappointing to hear. The solution is working and I have tested every possible scenario I came up with. The pattern follows the same pattern you already use in other SQL files with the variable tokens. I don't think I can come up with a cleaner solution than this. If you want to do post alteration that would require maintaining an array of tables that need to be updated instead and rely on matching the correct columns? How do you envision this to work? If you are worried by introducing a new id or foreign keys in new migration files that is not compatible you could have a git hook or similar that analyze the new migration SQL, to determine if ${id_format} is missing and error out before it is checked into the project to enforce the pattern. What is a realistic solution for you? |
|
You’re right that post-table-creation transformation is relatively straightforward, but post-alteration handling is much harder. In practice, it’s difficult to make ID-format conversion fully automatic and reliable for all future alteration scripts. We truly appreciate your contribution. If you’d like, please feel free to keep a fork with UUID support (or your own variant), or leave this PR open as a reference for future exploration. Suggestions are always welcome, and we’ll keep you posted if we come up with a better path forward. |
|
I will just drop this in here: https://better-auth.com/docs/reference/options#advanced This platform supports custom id generation and defaults with serial and uuid. I still think this would benefit this platform greatly. We opt-ed out to use your cloud offering because of nanoid. If you had uuid support in the cloud we would have already been a customer. This was a hard rule in our decision to not having to map new auth id numbers to our current infrastructure. |
|
Quick updates, we have added this to our Postgres DB updates project, and will address it by then. Thanks again for your great effort. I'll leave this PR open for future reference, until this issue has been addressed in Logto code base. |
feat: add UUID v7 support for entity IDs with per-tenant configuration
Add support for UUID v7 as an alternative ID format, configurable per tenant.
This allows tenants to choose between nanoid (existing default) and UUID v7 for
better compatibility with external systems and time-ordered ID requirements.
All entity ID columns expanded from varchar(12/21) to varchar(36) to
accommodate UUID v7 format. Existing nanoid IDs continue to work without migration.
PostgreSQL 18 will have improved support for UUID v7 natively using UUID column type allowing for faster indexing since UUID v7 is time-ordered.
Key Changes
Database Schema (packages/schemas)
tenant_id_configtable with unifiedid_formatcolumnCore Libraries (packages/core)
id-formatlibrary for tenant-specific ID format managementtenant-id-configqueries for configuration persistenceShared Utilities (packages/shared)
uuidpackagegenerateUuidV7()andgenerateId(format, size?)functionsIdFormattype: 'nanoid' | 'uuidv7'DEFAULT_ID_FORMATenvironment variable (defaults to 'nanoid')