-
Notifications
You must be signed in to change notification settings - Fork 22
PBM-1727 Oracle Cloud Storage #377
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
rasika-chivate
wants to merge
62
commits into
2.15.0
Choose a base branch
from
PBM-1727-Oracle-Cloud-Storage
base: 2.15.0
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 52 commits
Commits
Show all changes
62 commits
Select commit
Hold shift + click to select a range
218b613
PBM-1727 Oracle Cloud Storage
rasika-chivate cb1ec3d
Update oci-storage.md
rasika-chivate 6780b88
Potential fix for pull request finding
rasika-chivate b6defbd
Potential fix for pull request finding
rasika-chivate 434ac3f
Potential fix for pull request finding
rasika-chivate 71732c0
Potential fix for pull request finding
rasika-chivate 47d828d
Potential fix for pull request finding
rasika-chivate 826abd4
Potential fix for pull request finding
rasika-chivate 2bc680b
Potential fix for pull request finding
rasika-chivate 7b0c1f2
Potential fix for pull request finding
rasika-chivate ecd613e
Potential fix for pull request finding
rasika-chivate 3a571ea
Potential fix for pull request finding
rasika-chivate 9306047
Potential fix for pull request finding
rasika-chivate 26df0f5
Potential fix for pull request finding
rasika-chivate 168ac45
Potential fix for pull request finding
rasika-chivate fd77dbd
Add OCI storage page to docs navigation
Copilot 5b4d548
Potential fix for pull request finding
rasika-chivate 9be198f
Potential fix for pull request finding
rasika-chivate 996f477
Potential fix for pull request finding
rasika-chivate f52f739
Document OCI config options in reference docs
Copilot ece2f50
Update oci-storage.md
rasika-chivate 373ceeb
added example
rasika-chivate 6a6c63d
Potential fix for pull request finding
rasika-chivate e503805
Potential fix for pull request finding
rasika-chivate 897a91e
Remove generated git-committers cache artifact
Copilot 4fdda27
Potential fix for pull request finding
rasika-chivate 7faa675
Align OCI auth list with documented methods
Copilot 125e38c
Update oci-storage.md
rasika-chivate 19b7fe5
Potential fix for pull request finding
rasika-chivate 25cebef
Update oci-storage.md
rasika-chivate a5b37d1
Merge branch 'PBM-1727-Oracle-Cloud-Storage' of https://github.com/pe…
rasika-chivate ea2d70b
Create oci-WIF.md
rasika-chivate 493dd3b
Potential fix for pull request finding
rasika-chivate 9c95874
Potential fix for pull request finding
rasika-chivate f6b89f6
Potential fix for pull request finding
rasika-chivate c9e3370
Added OCI with WIF section
rasika-chivate 919ba1c
Potential fix for pull request finding
rasika-chivate c174fdf
Potential fix for pull request finding
rasika-chivate d051e2a
docs: include okeWorkloadIdentity in OCI credentials type options
Copilot 799a81d
Rename OCI WIF doc filename to lowercase
Copilot 61a5065
Update mkdocs-base.yml
rasika-chivate 46721d3
Merge branch 'PBM-1727-Oracle-Cloud-Storage' of https://github.com/pe…
rasika-chivate 33a709f
Update oci-storage.md
rasika-chivate a054489
Update oci-storage.md
rasika-chivate 6c49d62
Update mkdocs-base.yml
rasika-chivate cf2755e
Update oci-storage.md
rasika-chivate ca57383
Update oci-wif.md
rasika-chivate 1e7ab2c
Update oci-wif.md
rasika-chivate dede907
Update oci-storage.md
rasika-chivate 022a45b
Update oci-storage.md
rasika-chivate c6322b8
Update oci-storage.md
rasika-chivate 96ba9e1
Update oci-storage.md
rasika-chivate bc0f74e
Potential fix for pull request finding
rasika-chivate 3209746
Potential fix for pull request finding
rasika-chivate a8b30d7
Potential fix for pull request finding
rasika-chivate f0eb95f
Potential fix for pull request finding
rasika-chivate 3484bf7
Potential fix for pull request finding
rasika-chivate 0bae2c9
Update oci-wif.md
rasika-chivate d58d307
Update oci-wif.md
rasika-chivate 5de65e6
Update oci-storage.md
rasika-chivate c07cad8
Update oci-storage.md
rasika-chivate eae84da
Update oci-storage.md
rasika-chivate File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -15,3 +15,6 @@ styles/ | |
| # Allow | ||
|
|
||
| !styles/config/vocabularies/Percona/** | ||
|
|
||
| # Generated plugin cache | ||
| .cache/plugin/git-committers/ | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,371 @@ | ||
| # Oracle Cloud Infrastructure Object Storage | ||
|
|
||
| Percona Backup for MongoDB (PBM) supports [OCI Object Storage](https://docs.oracle.com/en-us/iaas/Content/Object/Concepts/objectstorageoverview.htm) | ||
| as a remote backup destination through a dedicated OCI native | ||
| driver. PBM connects to OCI Object Storage using one of two | ||
| authentication types: | ||
|
|
||
| | **Authentication type** | **Use when** | | ||
| | --- | --- | | ||
| | `userPrincipal` | PBM runs anywhere; authenticates with OCI API signing keys | | ||
| | `instancePrincipal` | PBM runs on an OCI Compute instance | | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| Before configuring PBM, ensure that you have: | ||
|
|
||
| - An active OCI tenancy with at least one subscribed region | ||
| - The OCI CLI installed and configured (`oci setup config`). | ||
| See the [OCI CLI documentation](https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/cliinstall.htm) | ||
| for installation instructions | ||
| - An OCI user with permission to create compartments, buckets, | ||
| dynamic groups, and IAM policies in your tenancy | ||
| - An OCI API signing key pair: private key on the host running | ||
| PBM, public key uploaded to the OCI user | ||
|
|
||
|
|
||
| Initialize the OCI CLI configuration if you have not done so already: | ||
|
|
||
| ```sh | ||
| oci setup config | ||
| ``` | ||
|
|
||
| Use your tenancy home region as the default CLI region. If the setup generated a new API key, upload the public key | ||
| to your OCI user: | ||
|
|
||
| ```sh | ||
| cat ~/.oci/oci_api_key_public.pem | ||
| ``` | ||
|
|
||
| In the OCI Console, go to **User settings → Tokens and keys → | ||
| API keys → Add API key**, paste the public key, and confirm | ||
| the fingerprint matches `~/.oci/config`. | ||
|
|
||
| If needed, update local file permissions: | ||
|
|
||
| ```bash | ||
| oci setup repair-file-permissions --file ~/.oci/config | ||
| oci setup repair-file-permissions --file ~/.oci/oci_api_key.pem | ||
| ``` | ||
|
|
||
| ## Verify region access | ||
|
|
||
| Check the regions available to your tenancy: | ||
|
|
||
| ```bash | ||
| oci iam region-subscription list \ | ||
| --region <HOME_REGION> \ | ||
| --output table | ||
| ``` | ||
|
|
||
| !!! note | ||
| The region specified in the configuration must be enabled and subscribed to in your OCI tenancy. | ||
|
|
||
| ## Set up OCI resources | ||
|
|
||
| ### Export variables | ||
|
|
||
| Set the following variables before running any commands in this | ||
| section. All subsequent commands reference them. | ||
|
|
||
| ```sh | ||
| export HOME_REGION=<your-home-region> # e.g. us-ashburn-1 | ||
| export BUCKET_REGION=<your-bucket-region> # e.g. eu-frankfurt-1 | ||
| export COMPARTMENT_NAME=pbm-backup | ||
| export BUCKET_NAME=<your-bucket-name> | ||
| ``` | ||
| Retrieve and export the values PBM requires: | ||
|
|
||
| ```sh | ||
| export TENANCY_OCID=$( | ||
| oci iam tenancy get \ | ||
| --tenancy-id "$(awk -F= '/^tenancy=/{print $2}' ~/.oci/config)" \ | ||
| --region "$HOME_REGION" \ | ||
| --query 'data.id' \ | ||
| --raw-output | ||
| ) | ||
|
|
||
| export USER_OCID=$(awk -F= '/^user=/{print $2}' ~/.oci/config) | ||
| export FINGERPRINT=$(awk -F= '/^fingerprint=/{print $2}' ~/.oci/config) | ||
| export KEY_FILE=$(awk -F= '/^key_file=/{print $2}' ~/.oci/config) | ||
| export KEY_FILE="${KEY_FILE/#\~/$HOME}" | ||
| export NAMESPACE=$( | ||
| oci os ns get \ | ||
| --region "$BUCKET_REGION" \ | ||
| --query 'data' \ | ||
| --raw-output | ||
| ) | ||
|
|
||
| echo "TENANCY_OCID: $TENANCY_OCID" | ||
| echo "USER_OCID: $USER_OCID" | ||
| echo "FINGERPRINT: $FINGERPRINT" | ||
| echo "KEY_FILE: $KEY_FILE" | ||
| echo "NAMESPACE: $NAMESPACE" | ||
|
Copilot marked this conversation as resolved.
|
||
| ``` | ||
| Verify all five values are populated before continuing. An empty value means the OCI CLI is not authenticated or the variable was not set correctly. | ||
|
|
||
| ## Create a compartment | ||
|
|
||
| Create a compartment for PBM backup resources: | ||
|
|
||
| ```sh | ||
| oci iam compartment create \ | ||
| --region "$HOME_REGION" \ | ||
| --compartment-id "$TENANCY_OCID" \ | ||
| --name "$COMPARTMENT_NAME" \ | ||
| --description "PBM OCI Object Storage backup" | ||
| ``` | ||
| Wait until the compartment is active, then export its OCID: | ||
|
|
||
| ```sh | ||
| export COMPARTMENT_OCID=$( | ||
| oci iam compartment list \ | ||
| --region "$HOME_REGION" \ | ||
| --compartment-id "$TENANCY_OCID" \ | ||
| --compartment-id-in-subtree true \ | ||
| --all \ | ||
| --query "data[?name=='$COMPARTMENT_NAME' && \"lifecycle-state\"=='ACTIVE'].id | [0]" \ | ||
| --raw-output | ||
| ) | ||
|
|
||
| echo "COMPARTMENT_OCID: $COMPARTMENT_OCID" | ||
| ``` | ||
| ### Create an Object Storage bucket | ||
|
|
||
| Create the bucket: | ||
|
|
||
| ```sh | ||
| oci os bucket create \ | ||
| --region "$BUCKET_REGION" \ | ||
| --namespace-name "$NAMESPACE" \ | ||
| --compartment-id "$COMPARTMENT_OCID" \ | ||
| --name "$BUCKET_NAME" | ||
| ``` | ||
|
|
||
| Verify the bucket was created: | ||
|
|
||
| ```sh | ||
| oci os bucket get \ | ||
| --region "$BUCKET_REGION" \ | ||
| --namespace-name "$NAMESPACE" \ | ||
| --bucket-name "$BUCKET_NAME" \ | ||
| --query 'data.{name:name,namespace:namespace}' \ | ||
| --output table | ||
| ``` | ||
|
|
||
| ### Create IAM policies | ||
|
|
||
| PBM must be able to create, read, overwrite, and delete backup objects. | ||
|
|
||
| Two policies are required: | ||
|
|
||
| **User access policy** — grants your OCI user group permission | ||
| to manage objects in the PBM compartment. Replace | ||
| `<OCI_GROUP_NAME>` with the name of the group containing | ||
| your PBM user: | ||
|
|
||
| ```sh | ||
| oci iam policy create \ | ||
| --region "$HOME_REGION" \ | ||
| --compartment-id "$TENANCY_OCID" \ | ||
| --name pbm-user-access \ | ||
| --description "Allow PBM user group to manage backup objects" \ | ||
| --statements "[\"Allow group <OCI_GROUP_NAME> to manage object-family in compartment $COMPARTMENT_NAME\"]" | ||
| ``` | ||
|
|
||
| **Native copy policy** — grants the OCI Object Storage service | ||
| permission to copy objects internally. PBM requires this for | ||
| server-side copy operations: | ||
|
|
||
| ```sh | ||
| oci iam policy create \ | ||
| --region "$HOME_REGION" \ | ||
| --compartment-id "$TENANCY_OCID" \ | ||
| --name "pbm-native-copy-$BUCKET_REGION" \ | ||
| --description "Allow Object Storage service to copy PBM objects" \ | ||
| --statements "[\"Allow service objectstorage-$BUCKET_REGION to manage object-family \ | ||
| in compartment $COMPARTMENT_NAME where any { \ | ||
| request.permission='OBJECT_READ', \ | ||
| request.permission='OBJECT_INSPECT', \ | ||
| request.permission='OBJECT_CREATE', \ | ||
| request.permission='OBJECT_OVERWRITE', \ | ||
| request.permission='OBJECT_DELETE'}\"]" | ||
| ``` | ||
|
|
||
| !!! note | ||
| IAM policy changes can take up to 2 minutes to propagate. If PBM reports an authorization error immediately after creating the policies, wait 2 minutes and retry. | ||
|
|
||
| ## Configure PBM | ||
|
|
||
| ### userPrincipal | ||
|
|
||
| Use this when PBM runs outside OCI, or when you want to | ||
| authenticate with OCI API signing keys. | ||
|
|
||
| Generate the correctly indented private key before creating | ||
| the config file: | ||
|
|
||
| ```sh | ||
| sed 's/^/ /' "$KEY_FILE" | ||
| ``` | ||
| Create the configuration file: | ||
|
|
||
| ```yaml | ||
| storage: | ||
| type: oci | ||
| oci: | ||
| region: <BUCKET_REGION> | ||
| namespace: <NAMESPACE> | ||
| bucket: <BUCKET_NAME> | ||
| prefix: pbm | ||
|
rasika-chivate marked this conversation as resolved.
|
||
| credentials: | ||
| type: userPrincipal | ||
| userPrincipal: | ||
| tenancy: <TENANCY_OCID> | ||
| user: <USER_OCID> | ||
| fingerprint: <FINGERPRINT> | ||
| privateKey: | | ||
| -----BEGIN PRIVATE KEY----- | ||
| ... | ||
| -----END PRIVATE KEY----- | ||
| ``` | ||
| !!! warning | ||
| The `user` value must be a user OCID starting with | ||
| `ocid1.user.oc1`. A bucket or compartment OCID causes | ||
| a 401 authentication failure. | ||
|
|
||
| !!! tip | ||
| Indent the private key correctly before adding it to the configuration: | ||
|
|
||
| ```sh | ||
| sed 's/^/ /' "$KEY_FILE" | ||
| ``` | ||
|
|
||
| ### instancePrincipal | ||
|
|
||
| Use this when PBM runs on an OCI Compute instance. No API | ||
| keys are required in the configuration file. | ||
|
|
||
| 1. Create a dynamic group that includes the compute instance: | ||
|
|
||
| ```sh | ||
| oci iam dynamic-group create \ | ||
| --region "$HOME_REGION" \ | ||
| --compartment-id "$TENANCY_OCID" \ | ||
| --name pbm-instance-group \ | ||
| --description "PBM Compute instance principal" \ | ||
| --matching-rule "ANY {instance.id = '<INSTANCE_OCID>'}" | ||
| ``` | ||
|
|
||
| 2. Create a policy granting the dynamic group access to the bucket: | ||
|
|
||
| ```sh | ||
| oci iam policy create \ | ||
| --region "$HOME_REGION" \ | ||
| --compartment-id "$TENANCY_OCID" \ | ||
| --name pbm-instance-policy \ | ||
| --description "Allow PBM instance to access backup bucket" \ | ||
| --statements "[\"Allow dynamic-group pbm-instance-group to manage objects \ | ||
| in compartment pbm-backup where target.bucket.name = '<BUCKET_NAME>'\"]" | ||
| ``` | ||
|
|
||
| 3. Configure PBM: | ||
|
|
||
| ```yaml | ||
| storage: | ||
| type: oci | ||
| oci: | ||
| region: <BUCKET_REGION> | ||
| namespace: <NAMESPACE> | ||
| bucket: <BUCKET_NAME> | ||
| prefix: pbm | ||
| credentials: | ||
| type: instancePrincipal | ||
| ``` | ||
|
Copilot marked this conversation as resolved.
|
||
|
|
||
| Wait for a few minutes for IAM policy propagation before testing the configuration. | ||
|
|
||
| !!! note | ||
| IAM changes for dynamic groups can take 5 to 10 minutes to propagate. The native copy policy from the previous section is still required alongside the instance | ||
| principal policy. | ||
|
Copilot marked this conversation as resolved.
Outdated
|
||
|
|
||
| ## Apply the PBM configuration | ||
|
|
||
| Apply the configuration: | ||
|
|
||
| ```bash | ||
| pbm config --file /path/to/oci-config.yaml | ||
| ``` | ||
|
|
||
| Force PBM to resync the storage configuration: | ||
|
|
||
| ```sh | ||
| pbm config --force-resync | ||
| ``` | ||
|
|
||
| ## Verify the configuration | ||
|
|
||
| Verify all agents connected and storage initialized successfully: | ||
|
|
||
| ```sh | ||
| pbm status | ||
| ``` | ||
|
|
||
| ??? example "Output" | ||
|
|
||
| ```sh | ||
| $ pbm status | ||
| Cluster: | ||
| ======== | ||
| rs1: | ||
| - rs101:27017 [S]: pbm-agent [v2.15.0] OK | ||
| - rs102:27017 [S]: pbm-agent [v2.15.0] OK | ||
| - rs103:27017 [P]: pbm-agent [v2.15.0] OK | ||
|
|
||
| PITR incremental backup: | ||
| ======================== | ||
| Status [OFF] | ||
|
|
||
| Currently running: | ||
| ================== | ||
| (none) | ||
|
|
||
| Backups: | ||
| ======== | ||
| Main storage: | ||
| Type: OCI | ||
| Region: us-ashburn-1 | ||
| Path: oci://idvufsl0apl6/rasika-bucket/pbm | ||
| (no snapshots or PITR chunks) | ||
| ``` | ||
|
|
||
| Every node must show `pbm-agent` as `OK` and storage as `ok`. | ||
|
|
||
| Run a test backup to confirm end-to-end functionality: | ||
|
|
||
| ```sh | ||
| pbm backup | ||
| pbm list | ||
| ``` | ||
|
|
||
| ??? example "Output" | ||
|
|
||
| ```sh | ||
| $ pbm backup | ||
| Starting backup "2026-06-12T07:11:31Z"..... | ||
| Backup "2026-06-12T07:11:31Z" saved to remote store (path: "oci://idvufsl0apl6/rasika-bucket/pbm") | ||
| ``` | ||
|
|
||
| ```sh | ||
| $ pbm list | ||
| Backup snapshots: | ||
| NAME TYPE PROFILE SELECTIVE BASE RESTORE TIME | ||
| ----------------------------------------------------- | ||
| 2026-06-11T13:14:51Z logical no no 2026-06-11T13:15:07 | ||
| 2026-06-12T07:04:27Z logical no no 2026-06-12T07:04:42 | ||
| 2026-06-12T07:11:31Z logical no no 2026-06-12T07:11:46 | ||
|
|
||
| PITR <off>: | ||
| ``` | ||
|
|
||
| A backup with status `done` confirms the setup is complete. | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.