Skip to content

plugins/duckdb: fold schema into Asset.Name to avoid MRN collisions (concrete fix, follow-up to #174) #177

Closed

Description

@antonioromero-pm

Context

Follow-up to #174, scoped specifically to plugins/duckdb. That issue reports the general bug (server reconstructs the stored MRN from bare Name, dropping whatever qualifier a plugin only put in the MRN string); this issue proposes the concrete fix for this one plugin, since it's a core plugin shipped with Marmot and the fix is small.

Current behavior

plugins/duckdb/duckdb/source.go builds two separate things per table/view:

// assetName is how this catalog addresses a table: one file holds many
// schemas, each able to hold the same table name. The name shown in the
// UI stays the table's own name; only the MRN carries the schema.
func assetName(schema, table string) string {
	return fmt.Sprintf("%s.%s", schema, table)
}

func assetMRN(assetType, schema, table string) string {
	return mrn.New(assetType, "DuckDB", assetName(schema, table))
}

and then:

a := pluginsdk.Asset{
	Name:      &tableName,   // bare, e.g. "orders"
	MRN:       &mrnValue,    // schema-qualified, e.g. mrn://table/duckdb/sales.orders
	...
}

Per #174, the server drops the qualifier from the stored MRN because it derives it from Name, not the submitted mrn value. Confirmed directly against a live catalog: DuckDB assets we ingested show bare stored MRNs like mrn://table/duckdb/studies, with no schema segment, regardless of what assetMRN constructed.

Proposed fix

Fold the schema into Name itself, matching what we did in our own SQLite plugin (which hit the identical issue — see the "Fix MRN collisions" commit in marmot-plugin-sqlite, sqlite/source.go's assetMRN doc comment and the displayName := s.alias + "." + name change):

displayName := assetName(schemaName, tableName) // "sales.orders" instead of bare "orders"

a := pluginsdk.Asset{
	Name:      &displayName,
	MRN:       &mrnValue,
	...
}

Since assetName already exists and is already used to build the MRN, this is a small, low-risk change — swap what gets passed as Name from tableName to assetName(schemaName, tableName). Same fix likely applies to plugins/mysql and plugins/postgresql, which look like they follow the same bare-Name convention.

Why this matters here specifically

DuckDB files commonly hold multiple schemas by design (that's the whole reason assetName exists — the comment literally says "one file holds many schemas, each able to hold the same table name"), so this plugin is exactly the case #174 describes: two tables named the same in different schemas of the same file, or across two different .duckdb files ingested together, currently collide silently with no error.

Activity

  1. bschaatsbergen commented on Aug 14, 2026

    @bschaatsbergen
    Member

    Thanks @antonioromero-pm, great analysis. We're working on a larger effort to make MRNs first-class unique identifiers instead of deriving them from Name.. that's the real fix for #174, but it's a longer leg of work. This stopgap makes sense meanwhile, on it 👍

  2. bschaatsbergen commented on Aug 26, 2026

    @bschaatsbergen
    Member

    /area plugin

  3. added this to the 0.11.0 milestone on Aug 27, 2026
  4. Itsdarkhere commented on Nov 5, 2026

    @Itsdarkhere

    @bschaatsbergen curious about the first-class MRN work. I'm hitting two kinds of collision:

    1. Schemas: if schema x and schema y both have an orders table, both get mrn://table/postgresql/orders and merge into one asset. Shouldn't the database and schema be part of the MRN?

    2. Servers: I've been using DataHub a lot lately. It has a "platform instance" for this: if two PG servers have identical databases, you give each one an instance (say, the service name) and they stop colliding. It also has a map from host to instance, so Kafka Connect lineage lands on the right server's tables.

    Any plans for something similar?

  5. charlie-haley commented on Nov 18, 2026

    @charlie-haley
    Member

    This has been fixed. MRNs are now more unique. There's now also supports for "Zones" within an MRN to add unique MRNs across envs https://marmotdata.io/docs/MRNs/zones

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions