Bug fixed for ceph RGW diff fix - #248
Conversation
|
Hi @sarwottamdev, thank you for pointing this out. The issue appears to be slightly wider and relates to how keys are treated by back end. In a short, some back ends respect byte-to-byte keys on put, some are trying to transform them.
We'll check how that can be addressed and will get back. |
Co-authored-by: Artem <torubarov.a.a@gmail.com>
Co-authored-by: Artem <torubarov.a.a@gmail.com>
LGTM!! Co-authored-by: Artem <torubarov.a.a@gmail.com>
|
Hi @sarwottamdev. Your concern has been addressed in #251 We've decided to drop off normalization completely, as it is apparently not addressing the problem it was intended for. |
arttor
left a comment
arttor
left a comment
There was a problem hiding this comment.
one of the log statements still using warn level.
| return nil | ||
| } | ||
| if isDiffFix { | ||
| logger.Warn().Int("from_ver", fromVer).Int("to_ver", toVer).Msg("diff fix copy: proceeding with copy despite version check") |
There was a problem hiding this comment.
this also should be info or debug.
| logger.Warn().Int("from_ver", fromVer).Int("to_ver", toVer).Msg("diff fix copy: proceeding with copy despite version check") | |
| logger.Info().Int("from_ver", fromVer).Int("to_ver", toVer).Msg("diff fix copy: proceeding with copy despite version check") |
| return fmt.Errorf("unable to get objects to copy: %w", err) | ||
| } | ||
|
|
||
| zerolog.Ctx(ctx).Info().Int("count", len(objectsToCopy)).Msg("EnsureObjectsDeleted: enqueuing copy tasks") |
There was a problem hiding this comment.
do we need 2 log statements per EnsureObjectsDeleted? should it be enough to have 1 on start or on succesful finish?
3 participants
Diff Fix for Ceph RGW
Description
This PR addresses issues identified during diff migrations against Ceph RGW and fixes cases where valid copy operations were not executed.
1. Object Key Normalization Fix
Updated the
normalizeOutputNameandnormalizeInputNamefunctions to stop prepending a leading/to object keys.Previously, object keys for non-MinIO providers (such as Ceph RGW) were normalized with a leading slash, causing S3 operations including:
StatObjectGetObjectPutObjectto fail with errors such as:
Object keys are now preserved as-is, ensuring compatibility with Ceph RGW object lookups.
2. Diff Pipeline Fix
File:
service/worker/handler/diff_handlers.goChanges:
len(objectsToRemove) == 0early return fromEnsureObjectsDeleted()fixCopySetStore.Drop()to clean up stale Redis state before creating new diff-fix copy tasksThis prevents scenarios where the diff-fix workflow exits prematurely and skips required copy operations.
3. Diff Fix Copy Version Check Bypass
File:
service/worker/handler/migration_obj_copy_handler.goAdded an
isDiffFixcheck to bypass the version validation below for diff-fix copy tasks:This prevents valid diff-fix copy operations from being skipped due to version comparison logic that is intended for normal migration workflows rather than repair or reconciliation tasks.
Checklist
Signed-off-byline to certify agreement with the pdated documentation in README.md if applicable.Note: By submitting this PR, you agree to license your contributions under the Apache 2.0 License _CONDUCT.md.