Skip to content

Export partition - Positional column matching might lead to incorrect partitioning #2123

Description

@arthurpassos

As of today, export partition will do positional column matching just like INSERT INTO dst SELECT * FROM src;. For example:

a Int32, b Int32 is exportable to b Int32, a Int32. This is handy because we don't need to bother about renames or different naming conventions in the destination table. It also matches the default and known behavior of insert select.

On the other hand, this is a problem for partition export in the following scenario:

src (a Int32, b Int32) partition by a;
dst (b Int32, a Int32) partition by a;

insert into src values (1, 1), (1, 2); -- src has a single partition with two rows

export src to dst;

In the above scenario, the partition expression is the same for both tables. It is partition by a. The problem is that since the columns are in different order, the src column b gets fed into the dst column a. The table dst ends up with two rows for the column a containing the values 1 and 2, which given the partition expression should live into two different partitions.

This is not a problem for the insert select idiom because it has some properties the export doesn't, these are:

  1. Insert select always repartition;
  2. Source columns can be tweaked (e.g, INSERT INTO dst SELECT b, a)

Solution is most likely to do column matching by name instead of position. We need to evaluate the pros and cons, and implement it.

Metadata

Metadata

Assignees

Labels

antalyaroadmapKey features and improvements for Antalya project

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions