fix(doris): re-raise genuine column DDL failures instead of swallowing them

[AmirF194]

Fixes #2358

Root cause

_apply_table_actions in python/cocoindex/connectors/doris/_target.py applied non-PK column ADD COLUMN/DROP COLUMN DDL inside a bare except Exception: pass. Because _TableHandler.reconcile() commits the tracking record from the desired schema regardless of whether the DDL succeeded, a genuine failure (a permission error, a schema-change job already in progress, a syntax error) was silently accepted as success: the next incremental run saw prev == desired and never retried, leaving the tracked schema permanently out of sync with the real table.

Fix

Extracted the per-column apply loop into _apply_column_actions and added _is_benign_column_ddl_error, which distinguishes an idempotent no-op (the column is already in the state the action wants) from a genuine failure. A benign error is still ignored; anything else is logged and re-raised, so the caller's reconcile() does not commit a tracking record for a change that never happened. This mirrors how the sqlite connector already handles the identical failure mode for its own ADD COLUMN.

Doris' DDL dialect does not appear to support ADD COLUMN IF NOT EXISTS / DROP COLUMN IF EXISTS (unlike postgres, which sidesteps this ambiguity entirely with those clauses), so the fix classifies the error message rather than changing the SQL, the same approach sqlite's connector takes for duplicate column name.

Verification

This repo's own Doris tests need a live cluster (DORIS_FE_HOST/DORIS_PASSWORD), which I don't have, so I added python/tests/connectors/test_doris_target_ddl_swallow.py, which exercises the real control flow without one: it calls _apply_column_actions directly, and an end-to-end test calls the actual _apply_table_actions entry point with a stubbed _execute_ddl_sync, ContextProvider, and ManagedConnection (no Doris connection ever opens).

• All 10 new tests fail against pre-fix main, the end-to-end one with DID NOT RAISE Exception (confirming the swallow, not just missing symbols), and pass against this branch. Ran both ways in the same container (python:3.11, matching this repo's own CI matrix's standard leg) against cocoindex==1.0.20's compiled core with this repo's Python source overlaid on top.
• ruff check and ruff format --check on both changed files: clean, pinned to this repo's own v0.12.0.
• mypy strict over the whole python/ tree (uv run mypy): clean, 281 files.
• Not verified: an actual Doris cluster returning these specific error strings for a real ADD/DROP COLUMN conflict. I don't have one available, and this repo's own connector tests are gated on DORIS_FE_HOST for the same reason. I did not attempt a local maturin develop build (this repo's real pytest CI hook) since a full Rust workspace build was not practical in my environment; the tests above ran against the officially published wheel's compiled core with this branch's Python source substituted in, exercising the exact code path unmodified.

[badmonster0]

@ZhiHanZ @tomz-alt Tom could you help take a look at this fix?

[AmirF194]

Bumping this in case it slipped by, no rush. Happy to answer questions on the DDL reconciliation fix or adjust the approach.

[tomz-alt]

Hi @AmirF194 , I think it is LGTM with some minor fix expected as follow up.
Verified the mechanism rather than just the diff, and it holds. The part I wanted to be sure of is that re-raising actually un-commits: sink raises → cleanup_pending_token (rust/core/src/engine/execution.rs:1814) → the item keeps its multi-state shape → next diff_composite trips len(grouped_state.prev) < len(t.prev) and re-emits. Against a build of this branch:

after rolled-back run -> column_actions {'col:col1': 'upsert'} # retried
steady state -> column_actions {'col:col1': None} # no churn

Also checked the matcher against a real Doris/VeloDB cluster instead of the strings in the tests, since those are circular by construction:

• ADD duplicate → Can not add column which already exists in base table: col1 → benign ✓
• DROP missing → Column does not exists: nosuchcol → benign ✓
• missing table → Unknown table '...' → re-raised ✓

I expected "not exist" to over-match the missing-table case; it doesn't.

One thing to fix before merge: these tests skip in CI. --group ci pulls ci-enabled-optional-deps (pydantic, asyncpg, neo4j), and pymysql/aiohttp only live in the doris extra, so DEPS_AVAILABLE=False and the module never runs — the green checks here don't cover it. Skipping is normal for python/tests/connectors/, but this is the first file there that doesn't need a live cluster, so adding pymysql + aiohttp to ci-enabled-optional-deps makes them real. (aiomysql is imported lazily in connect_async, so those two are enough.)

Two follow-ups, not blockers — the same bug class in paths this PR didn't touch:

1. "replace" still falls through _apply_column_actions with no DDL at all, while reconcile() commits the new tracking record. A VARCHAR(255) → VARCHAR(1024) change yields {'col:col1': 'replace'} and emits nothing — exactly the desync described here, reached by a column type change.
2. The DROP TABLE for main_action replace/delete is still except Exception: warning(...) with no re-raise.

Worth a line in the PR body that a genuine failure now aborts the rest of the batch rather than continuing silently. That's the right trade and rollback makes it safe, but it's a user-visible change.

[AmirF194]

Thanks for verifying the mechanism directly, and for catching the CI gap. Added pymysql and aiohttp to ci-enabled-optional-deps (aiomysql is only used lazily in connect_async, so those two are enough); confirmed locally that the suite skips all 10 without them and passes all 10 with them, matching what CI will now install.

Left the two follow-ups (replace falling through with no DDL, and the swallowed DROP TABLE exception) out of this PR since they're a different code path than the one this fix touches, happy to open a separate issue for those if that's useful.