-
Notifications
You must be signed in to change notification settings - Fork 22
Add compatibility with Oracle Cloud Storage #371
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 4 commits
ef3bfee
9af2097
c23adea
8c8c67d
96b3cb4
0fb03d4
49d2a4b
ab87f89
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,293 @@ | ||
| # Configure Oracle Cloud Infrastructure Object Storage | ||
|
Check warning on line 1 in docs/details/oci-storage.md
|
||
|
|
||
| Percona Backup for MongoDB (PBM) can store backups in Oracle Cloud Infrastructure (OCI) Object Storage using the native OCI storage backend. | ||
|
Check warning on line 3 in docs/details/oci-storage.md
|
||
|
|
||
| OCI Object Storage is a cloud object storage service that stores data as objects in buckets. PBM uses this storage to save backup data, metadata, and Point-in-Time Recovery oplog chunks. You can configure PBM to authenticate to OCI by using either a user principal or an instance principal. | ||
|
|
||
| ## Before you start | ||
|
Comment on lines
+7
to
+8
|
||
|
|
||
| Ensure you have the following: | ||
|
|
||
| * An Oracle Cloud Infrastructure account | ||
|
Check warning on line 11 in docs/details/oci-storage.md
|
||
| * OCI CLI installed | ||
|
Check warning on line 12 in docs/details/oci-storage.md
|
||
| * Access to create compartments, buckets, and IAM policies | ||
| * A running PBM deployment | ||
| * The OCI region where you want to store backups | ||
| * Access to the OCI user or compute instance that PBM will use | ||
|
Check warning on line 16 in docs/details/oci-storage.md
|
||
|
|
||
| If the bucket region is different from the tenancy home region, subscribe the tenancy to the bucket region first. | ||
|
|
||
| ## Configure OCI CLI | ||
|
Check warning on line 20 in docs/details/oci-storage.md
|
||
|
|
||
| Initialize the OCI CLI configuration: | ||
|
Check warning on line 22 in docs/details/oci-storage.md
|
||
|
|
||
| ```bash | ||
| oci setup config | ||
| ``` | ||
|
|
||
| Use your tenancy home region as the default CLI region. | ||
|
Check warning on line 28 in docs/details/oci-storage.md
|
||
|
|
||
| If needed, repair local file permissions: | ||
|
|
||
| ```bash | ||
| oci setup repair-file-permissions --file ~/.oci/config | ||
| oci setup repair-file-permissions --file ~/.oci/oci_api_key.pem | ||
| ``` | ||
|
|
||
| If the setup generated a new API key, upload the public key to your OCI user: | ||
|
|
||
| ```bash | ||
| cat ~/.oci/oci_api_key_public.pem | ||
| ``` | ||
|
|
||
| In the OCI Console, go to **User settings** → **Tokens and keys** → **API keys** → **Add API key**, and paste the public key. | ||
|
|
||
| ## Verify region access | ||
|
|
||
| Check the regions available to your tenancy: | ||
|
|
||
| ```bash | ||
| oci iam region-subscription list \ | ||
| --region <HOME_REGION> \ | ||
| --output table | ||
| ``` | ||
|
|
||
| If the bucket region is not listed, subscribe to it in the OCI Console. Open the region menu, select **Manage regions**, find the target region, and click **Subscribe**. | ||
|
|
||
| Wait until the subscription is active before creating Object Storage resources. | ||
|
|
||
| ## Get required OCI values | ||
|
|
||
| Export the values required for the PBM configuration: | ||
|
|
||
| ```bash | ||
| export HOME_REGION=<HOME_REGION> | ||
| export BUCKET_REGION=<BUCKET_REGION> | ||
| export COMPARTMENT_NAME=<COMPARTMENT_NAME> | ||
| export BUCKET_NAME=<BUCKET_NAME> | ||
| export PBM_PREFIX=pbm | ||
| ``` | ||
|
|
||
| Get the tenancy OCID, user OCID, API key fingerprint, private key path, and Object Storage namespace: | ||
|
Check warning on line 71 in docs/details/oci-storage.md
|
||
|
|
||
| ```bash | ||
| export TENANCY_OCID=$(oci iam tenancy get \ | ||
| --tenancy-id "$(awk -F= '/^tenancy=/{print $2}' ~/.oci/config)" \ | ||
| --region "$HOME_REGION" \ | ||
| --query 'data.id' \ | ||
| --raw-output) | ||
|
|
||
| export USER_OCID=$(awk -F= '/^user=/{print $2}' ~/.oci/config) | ||
| export FINGERPRINT=$(awk -F= '/^fingerprint=/{print $2}' ~/.oci/config) | ||
| export KEY_FILE=$(awk -F= '/^key_file=/{print $2}' ~/.oci/config) | ||
|
|
||
|
Copilot marked this conversation as resolved.
Outdated
|
||
| export NAMESPACE=$(oci os ns get \ | ||
| --region "$BUCKET_REGION" \ | ||
| --query 'data' \ | ||
| --raw-output) | ||
| ``` | ||
|
|
||
| ## Create a compartment | ||
|
|
||
| Create a compartment for PBM backup resources: | ||
|
|
||
| ```bash | ||
| oci iam compartment create \ | ||
| --region "$HOME_REGION" \ | ||
| --compartment-id "$TENANCY_OCID" \ | ||
| --name "$COMPARTMENT_NAME" \ | ||
| --description "PBM OCI Object Storage" | ||
| ``` | ||
|
|
||
| Export the compartment OCID: | ||
|
|
||
| ```bash | ||
| export COMPARTMENT_OCID=$(oci iam compartment list \ | ||
| --region "$HOME_REGION" \ | ||
| --compartment-id "$TENANCY_OCID" \ | ||
| --compartment-id-in-subtree true \ | ||
| --all \ | ||
| --query "data[?name=='$COMPARTMENT_NAME' && \"lifecycle-state\"=='ACTIVE'].id | [0]" \ | ||
| --raw-output) | ||
| ``` | ||
|
|
||
| ## Create an Object Storage bucket | ||
|
Check warning on line 114 in docs/details/oci-storage.md
|
||
|
|
||
| Create the bucket: | ||
|
|
||
| ```bash | ||
| oci os bucket create \ | ||
| --region "$BUCKET_REGION" \ | ||
| --namespace-name "$NAMESPACE" \ | ||
| --compartment-id "$COMPARTMENT_OCID" \ | ||
| --name "$BUCKET_NAME" | ||
| ``` | ||
|
|
||
| Verify the bucket: | ||
|
|
||
| ```bash | ||
| oci os bucket get \ | ||
| --region "$BUCKET_REGION" \ | ||
| --namespace-name "$NAMESPACE" \ | ||
| --bucket-name "$BUCKET_NAME" | ||
| ``` | ||
|
|
||
| ## Configure IAM policies | ||
|
|
||
| PBM must be able to create, read, overwrite, and delete backup objects. | ||
|
|
||
| Create a policy that allows the PBM user group to manage objects in the bucket: | ||
|
|
||
| ```text | ||
| Allow group <OCI_GROUP_NAME> to manage object-family in compartment <COMPARTMENT_NAME> | ||
|
Copilot marked this conversation as resolved.
Outdated
|
||
| ``` | ||
|
|
||
| PBM also uses OCI native server-side copy operations. Add a policy for the regional Object Storage service: | ||
|
|
||
| ```text | ||
| Allow service objectstorage-<BUCKET_REGION> to manage object-family in compartment <COMPARTMENT_NAME> where any { | ||
| request.permission='OBJECT_READ', | ||
| request.permission='OBJECT_INSPECT', | ||
| request.permission='OBJECT_CREATE', | ||
| request.permission='OBJECT_OVERWRITE', | ||
| request.permission='OBJECT_DELETE' | ||
| } | ||
| ``` | ||
|
|
||
| Allow a few minutes for IAM policy changes to propagate. | ||
|
|
||
| ## Configure PBM with a user principal | ||
|
|
||
| Use user principal authentication when PBM runs outside OCI or when you want to authenticate with OCI API signing keys. | ||
|
|
||
| Create a PBM configuration file: | ||
|
|
||
| ```yaml | ||
| storage: | ||
| type: oci | ||
| oci: | ||
|
Comment on lines
+165
to
+168
|
||
| region: <BUCKET_REGION> | ||
| namespace: <NAMESPACE> | ||
| bucket: <BUCKET_NAME> | ||
| prefix: pbm | ||
| credentials: | ||
| type: userPrincipal | ||
| userPrincipal: | ||
| tenancy: <TENANCY_OCID> | ||
| user: <USER_OCID> | ||
| fingerprint: <FINGERPRINT> | ||
| privateKey: | | ||
| -----BEGIN PRIVATE KEY----- | ||
| ... | ||
| -----END PRIVATE KEY----- | ||
| ``` | ||
|
|
||
| Indent the private key correctly before adding it to the configuration: | ||
|
|
||
| ```bash | ||
| sed 's/^/ /' "$KEY_FILE" | ||
| ``` | ||
|
|
||
| ## Configure PBM with an instance principal | ||
|
|
||
| Use instance principal authentication when PBM runs on an OCI Compute instance. This method avoids storing API signing keys in the PBM configuration. | ||
|
|
||
| Create a dynamic group that includes the compute instance: | ||
|
|
||
| ```text | ||
| ANY {instance.id = '<INSTANCE_OCID>'} | ||
| ``` | ||
|
|
||
| Create a policy that allows the dynamic group to access the bucket: | ||
|
|
||
| ```text | ||
| Allow dynamic-group <DYNAMIC_GROUP_NAME> to manage objects in compartment <COMPARTMENT_NAME> where target.bucket.name = '<BUCKET_NAME>' | ||
| ``` | ||
|
|
||
| Then configure PBM: | ||
|
|
||
| ```yaml | ||
| storage: | ||
| type: oci | ||
| oci: | ||
| region: <BUCKET_REGION> | ||
| namespace: <NAMESPACE> | ||
| bucket: <BUCKET_NAME> | ||
| prefix: pbm | ||
| credentials: | ||
| type: instancePrincipal | ||
| ``` | ||
|
|
||
| Wait several minutes for IAM policy propagation before testing the configuration. | ||
|
|
||
| ## Configure server-side encryption with OCI KMS | ||
|
|
||
| You can use an OCI Vault key for server-side encryption. | ||
|
|
||
| Create or select a Vault key in the same region as the bucket. Then allow both the PBM principal and the Object Storage service to use the key: | ||
|
|
||
| ```text | ||
| Allow group <OCI_GROUP_NAME> to use keys in compartment <COMPARTMENT_NAME> | ||
| Allow service objectstorage-<BUCKET_REGION> to use keys in compartment <COMPARTMENT_NAME> | ||
| ``` | ||
|
|
||
| Add the key OCID to the PBM configuration: | ||
|
|
||
| ```yaml | ||
| storage: | ||
| type: oci | ||
| oci: | ||
| region: <BUCKET_REGION> | ||
| namespace: <NAMESPACE> | ||
| bucket: <BUCKET_NAME> | ||
| prefix: pbm | ||
| credentials: | ||
| type: userPrincipal | ||
| userPrincipal: | ||
| tenancy: <TENANCY_OCID> | ||
| user: <USER_OCID> | ||
| fingerprint: <FINGERPRINT> | ||
| privateKey: | | ||
| -----BEGIN PRIVATE KEY----- | ||
| ... | ||
| -----END PRIVATE KEY----- | ||
| serverSideEncryption: | ||
| kmsKeyID: <KMS_KEY_OCID> | ||
| ``` | ||
|
|
||
| Do not configure `kmsKeyID` and `sseCustomerKey` together. PBM treats OCI KMS and customer-provided encryption keys as mutually exclusive. | ||
|
|
||
| ## Apply the PBM configuration | ||
|
|
||
| Apply the configuration: | ||
|
|
||
| ```bash | ||
| pbm config --file /path/to/oci-config.yaml | ||
| ``` | ||
|
|
||
| Force PBM to resync the storage configuration: | ||
|
|
||
| ```bash | ||
| pbm config --force-resync --wait | ||
| ``` | ||
|
|
||
| ## Verify the configuration | ||
|
|
||
| Check that PBM created objects under the configured prefix: | ||
|
|
||
| ```bash | ||
| oci os object list \ | ||
| --region "$BUCKET_REGION" \ | ||
| --namespace-name "$NAMESPACE" \ | ||
| --bucket-name "$BUCKET_NAME" \ | ||
| --prefix "$PBM_PREFIX/" | ||
| ``` | ||
|
|
||
| Run a backup: | ||
|
|
||
| ```bash | ||
| pbm backup | ||
| ``` | ||
|
|
||
| Verify that backup files appear in the OCI bucket. | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| # Troubleshoot issues for OCI | ||
|
|
||
|
Copilot marked this conversation as resolved.
|
||
| ## Storage is not initialized | ||
|
Comment on lines
+3
to
+4
|
||
|
|
||
| If PBM reports that storage is not initialized, verify the following: | ||
|
|
||
| * The bucket exists. | ||
| * The bucket region is correct. | ||
| * The namespace is correct. | ||
|
Check warning on line 9 in docs/troubleshoot/oci.md
|
||
| * The PBM principal has access to the bucket. | ||
| * The Object Storage service copy policy exists. | ||
| * IAM policy propagation has completed. | ||
|
|
||
| ## KMS authorization errors | ||
|
Check warning on line 14 in docs/troubleshoot/oci.md
|
||
|
|
||
| If uploads fail with `NotAuthorizedOrFoundKmsKey`, verify the following: | ||
|
|
||
| * The KMS key exists in the bucket region. | ||
| * The PBM principal can use the key. | ||
| * The regional Object Storage service can use the key. | ||
| * The KMS policy has propagated. | ||
|
|
||
| ## Instance principal access fails | ||
|
|
||
| If instance principal authentication fails, verify the following: | ||
|
|
||
| * The compute instance is included in the dynamic group. | ||
| * The dynamic group policy grants access to the bucket. | ||
| * The instance and bucket are in the expected tenancy and compartment. | ||
| * IAM policy propagation has completed. | ||
Uh oh!
There was an error while loading. Please reload this page.