You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
plugins/duckdb: fold schema into Asset.Name to avoid MRN collisions (concrete fix, follow-up to #174) #177
Follow-up to #174, scoped specifically to plugins/duckdb. That issue reports the general bug (server reconstructs the stored MRN from bare Name, dropping whatever qualifier a plugin only put in the MRN string); this issue proposes the concrete fix for this one plugin, since it's a core plugin shipped with Marmot and the fix is small.
Current behavior
plugins/duckdb/duckdb/source.go builds two separate things per table/view:
// assetName is how this catalog addresses a table: one file holds many// schemas, each able to hold the same table name. The name shown in the// UI stays the table's own name; only the MRN carries the schema.funcassetName(schema, tablestring) string {
returnfmt.Sprintf("%s.%s", schema, table)
}
funcassetMRN(assetType, schema, tablestring) string {
returnmrn.New(assetType, "DuckDB", assetName(schema, table))
}
and then:
a:= pluginsdk.Asset{
Name: &tableName, // bare, e.g. "orders"MRN: &mrnValue, // schema-qualified, e.g. mrn://table/duckdb/sales.orders...
}
Per #174, the server drops the qualifier from the stored MRN because it derives it from Name, not the submitted mrn value. Confirmed directly against a live catalog: DuckDB assets we ingested show bare stored MRNs like mrn://table/duckdb/studies, with no schema segment, regardless of what assetMRN constructed.
Proposed fix
Fold the schema into Name itself, matching what we did in our own SQLite plugin (which hit the identical issue — see the "Fix MRN collisions" commit in marmot-plugin-sqlite, sqlite/source.go's assetMRN doc comment and the displayName := s.alias + "." + name change):
displayName:=assetName(schemaName, tableName) // "sales.orders" instead of bare "orders"a:= pluginsdk.Asset{
Name: &displayName,
MRN: &mrnValue,
...
}
Since assetName already exists and is already used to build the MRN, this is a small, low-risk change — swap what gets passed as Name from tableName to assetName(schemaName, tableName). Same fix likely applies to plugins/mysql and plugins/postgresql, which look like they follow the same bare-Name convention.
Why this matters here specifically
DuckDB files commonly hold multiple schemas by design (that's the whole reason assetName exists — the comment literally says "one file holds many schemas, each able to hold the same table name"), so this plugin is exactly the case #174 describes: two tables named the same in different schemas of the same file, or across two different .duckdb files ingested together, currently collide silently with no error.
Thanks @antonioromero-pm, great analysis. We're working on a larger effort to make MRNs first-class unique identifiers instead of deriving them from Name.. that's the real fix for #174, but it's a longer leg of work. This stopgap makes sense meanwhile, on it 👍
@bschaatsbergen curious about the first-class MRN work. I'm hitting two kinds of collision:
Schemas: if schema x and schema y both have an orders table, both get mrn://table/postgresql/orders and merge into one asset. Shouldn't the database and schema be part of the MRN?
Servers: I've been using DataHub a lot lately. It has a "platform instance" for this: if two PG servers have identical databases, you give each one an instance (say, the service name) and they stop colliding. It also has a map from host to instance, so Kafka Connect lineage lands on the right server's tables.
This has been fixed. MRNs are now more unique. There's now also supports for "Zones" within an MRN to add unique MRNs across envs https://marmotdata.io/docs/MRNs/zones
Activity
bschaatsbergen commentedon Aug 14, 2026
Thanks @antonioromero-pm, great analysis. We're working on a larger effort to make MRNs first-class unique identifiers instead of deriving them from Name.. that's the real fix for #174, but it's a longer leg of work. This stopgap makes sense meanwhile, on it 👍
bschaatsbergen commentedon Aug 26, 2026
/area plugin
Itsdarkhere commentedon Nov 5, 2026
@bschaatsbergen curious about the first-class MRN work. I'm hitting two kinds of collision:
Schemas: if schema x and schema y both have an orders table, both get mrn://table/postgresql/orders and merge into one asset. Shouldn't the database and schema be part of the MRN?
Servers: I've been using DataHub a lot lately. It has a "platform instance" for this: if two PG servers have identical databases, you give each one an instance (say, the service name) and they stop colliding. It also has a map from host to instance, so Kafka Connect lineage lands on the right server's tables.
Any plans for something similar?
charlie-haley commentedon Nov 18, 2026
This has been fixed. MRNs are now more unique. There's now also supports for "Zones" within an MRN to add unique MRNs across envs https://marmotdata.io/docs/MRNs/zones