CRASM - 4247 ME report uploads and scoped S3 retention

docs/src/documentation-pages/dev/deployment.md

@@ -30,6 +30,80 @@ make plan
 make apply
 ```
 
+##### CyHy report sharing
+
+Report-sharing resources are in `infrastructure/s3.tf`, with inputs in
+`vars.tf`, environment settings in `stage-cd.tfvars`, and IAM/key-policy
+outputs in `output.tf`. They use the existing provider, backend, and deployment
+process. The existing `cisa-was-reports` bucket and manually created customer
+managed KMS key are referenced; they are not created or replaced.
+
+`stage-cd.tfvars` enables ME uploads and 14-day retention in the same change and
+contains the supplied CMK, two ME producer roles, and confirmed existing
+HTTPS-only bucket policy. The existing lifecycle configuration was confirmed
+empty for this change. Bucket ownership is unchanged and no ownership
+confirmation flag is required. ME uploads use
+`reader_role_arns = []`; analyst roles do not block producer access. The grouped
+input permits dependent settings to be validated with the Terraform 1.0.7
+version used by the existing pipeline. Other environments default to disabled.
+
+Before enabling it, preserve the complete current bucket policy in
+`existing_bucket_policy_json`. This is the **S3 bucket policy**, not the KMS key
+policy. Use an empty
+Statement array only after confirming there is no current policy, and retain
+the original policy as the input for later changes. This change does not modify
+ownership, ACLs, public-access settings, or the bucket's default encryption.
+Confirm that
+no other Terraform state owns its bucket policy or lifecycle configuration;
+adopt existing controls into the current environment's state where necessary.
+
+The new grants and CMK upload requirements cover only `cybex_scorecard/`,
+`cyhy_ad-hoc_notifications/`, `cyhy_vs_reports/`, `https_reports/`, and
+`trustymail_reports/`. They exclude existing `was_reports/` and `capacity/`
+objects, including customer CSV exports. All configured readers can access all
+five prefixes, so every reader must be approved for that scope. Existing grants
+are preserved rather than revoked. The new TLS object denial covers only ME
+objects; secure transport is required on the new bucket-level grants without
+adding a bucket-wide denial affecting WAS operations. Confirm the bucket
+account/region, ATO approval, audit coverage, and organizational restrictions
+before deployment.
+
+Producers must request `aws:kms` and the exact CMK ARN on uploads. The external
+role owners must attach the generated producer IAM policy. When analyst access
+is ready, add approved roles to `reader_role_arns`. Only then are analyst
+download/list grants and reader IAM/key-policy outputs generated. Analyst role
+owners and the CMK owner must review those policies. Append the reader
+key-policy statement to the existing key policy, preserving its administrator
+and producer statements. This configuration does not change the shared bucket's
+default encryption or re-encrypt existing objects.
+Ownership of newly uploaded ME objects follows the bucket's existing ownership
+mode. Upload access does not require changing that mode. Any later analyst
+download setup must account for the actual object ownership and permissions.
+
+`manage_retention` is enabled in `stage-cd.tfvars`, and
+`existing_lifecycle_configuration_reviewed_empty` records confirmation that no
+existing lifecycle rules are configured. Confirm no other Terraform state owns
+the configuration. If rules are subsequently added, preserve them in the
+complete lifecycle configuration before deployment.
+The retention rules use a separate fixed list containing only the five ME
+prefixes. No expiration, version cleanup, delete-marker cleanup, or incomplete
+upload cleanup rule added by this change matches `was_reports/` or `capacity/`.
+Changing access permissions does not expand the retention scope. Existing WAS
+lifecycle rules must not be removed or replaced; if any exist, the current
+ME-only lifecycle resource remains blocked until preservation is addressed.
+The optional rules expire reports after 14 days, noncurrent versions one day
+after becoming noncurrent, and incomplete uploads after one day, with separate
+delete-marker cleanup. Versioning, Object Lock, replication, backups, and
+asynchronous S3 processing mean this is not a strict 14-day permanent-deletion
+guarantee. Review old matching objects before activation.
+
+Terraform tests and validation tooling stay local, outside the repository.
+Do not access AWS, run a live plan, import resources, or deploy from the
+development environment. The existing workflow can apply this infrastructure
+after a merge; enablement requires deployment review. COOL organization access
+and possible long-term copying require separate source/destination details and
+are not configured by these report-sharing resources.
+
 #### Backend
 
 The backend API is managed by the Serverless Framework. To deploy, run:

infrastructure/output.tf

@@ -52,3 +52,87 @@ output "was_reporting_private_ip" {
   description = "Private IP address of the WAS reporting EC2 instance when enabled."
   value       = try(aws_instance.was_reporting[0].private_ip, null)
 }
+
+output "techops_reports_producer_iam_policy_json" {
+  description = "Identity policy for the external producer-role owners to review and attach."
+  value = var.techops_reports.enabled ? jsonencode({
+    Version = "2012-10-17"
+    Statement = [
+      {
+        Sid      = "UploadCyHyReports"
+        Effect   = "Allow"
+        Action   = local.techops_reports_producer_actions
+        Resource = local.techops_reports_object_arns
+      },
+      {
+        Sid      = "LocateReportsBucket"
+        Effect   = "Allow"
+        Action   = "s3:GetBucketLocation"
+        Resource = local.techops_reports_bucket_arn
+        Condition = {
+          Bool = { "aws:SecureTransport" = "true" }
+        }
+      },
+      {
+        Sid       = "UseReportKeyThroughS3"
+        Effect    = "Allow"
+        Action    = ["kms:GenerateDataKey", "kms:Decrypt"]
+        Resource  = var.techops_reports.kms_key_arn
+        Condition = local.techops_reports_kms_conditions
+      },
+    ]
+  }) : null
+}
+
+output "techops_reports_reader_iam_policy_json" {
+  description = "Identity policy for the approved analyst-role owners to review and attach."
+  value = var.techops_reports.enabled && length(var.techops_reports.reader_role_arns) > 0 ? jsonencode({
+    Version = "2012-10-17"
+    Statement = [
+      {
+        Sid      = "DownloadCyHyReports"
+        Effect   = "Allow"
+        Action   = "s3:GetObject"
+        Resource = local.techops_reports_object_arns
+      },
+      {
+        Sid      = "ListCyHyReportPrefixes"
+        Effect   = "Allow"
+        Action   = "s3:ListBucket"
+        Resource = local.techops_reports_bucket_arn
+        Condition = {
+          StringLike = { "s3:prefix" = [for prefix in local.techops_reports_prefixes : "${prefix}*"] }
+          Bool       = { "aws:SecureTransport" = "true" }
+        }
+      },
+      {
+        Sid      = "LocateReportsBucket"
+        Effect   = "Allow"
+        Action   = "s3:GetBucketLocation"
+        Resource = local.techops_reports_bucket_arn
+        Condition = {
+          Bool = { "aws:SecureTransport" = "true" }
+        }
+      },
+      {
+        Sid       = "DecryptReportsThroughS3"
+        Effect    = "Allow"
+        Action    = "kms:Decrypt"
+        Resource  = var.techops_reports.kms_key_arn
+        Condition = local.techops_reports_kms_conditions
+      },
+    ]
+  }) : null
+}
+
+output "techops_reports_reader_kms_key_policy_statement_json" {
+  description = "Statement for the key owner to append to the existing CMK policy, preserving administration and producer access."
+  value = var.techops_reports.enabled && length(var.techops_reports.reader_role_arns) > 0 ? jsonencode({
+    Sid       = "AllowCyHyReportReadersThroughS3"
+    Effect    = "Allow"
+    Principal = { AWS = sort(tolist(var.techops_reports.reader_role_arns)) }
+    Action    = "kms:Decrypt"
+    Resource  = "*"
+    Condition = local.techops_reports_kms_conditions
+  }) : null
+}

infrastructure/s3.tf

@@ -394,3 +394,187 @@ resource "aws_s3_bucket_logging" "django_env_bucket" {
   target_bucket = aws_s3_bucket.logging_bucket.id
   target_prefix = "django_env_bucket/"
 }
+
+# CyHy sharing adopts controls on the existing WAS bucket. It does not create
+# the bucket or change the encryption defaults used by existing WAS clients.
+locals {
+  techops_reports_bucket_arn = "arn:${var.aws_partition}:s3:::${var.was_reporting_reports_bucket_name}"
+  techops_reports_prefixes = [
+    "cybex_scorecard/",
+    "cyhy_ad-hoc_notifications/",
+    "cyhy_vs_reports/",
+    "https_reports/",
+    "trustymail_reports/",
+  ]
+  # Retention is intentionally independent of access permissions. Giving a
+  # future role access to another prefix must not schedule that data for deletion.
+  techops_reports_retention_days_by_prefix = {
+    "cybex_scorecard/"           = 14
+    "cyhy_ad-hoc_notifications/" = 14
+    "cyhy_vs_reports/"           = 14
+    "https_reports/"             = 14
+    "trustymail_reports/"        = 14
+  }
+  techops_reports_object_arns = [for prefix in local.techops_reports_prefixes : "${local.techops_reports_bucket_arn}/${prefix}*"]
+  techops_reports_producer_actions = [
+    "s3:AbortMultipartUpload",
+    "s3:ListMultipartUploadParts",
+    "s3:PutObject",
+  ]
+  techops_reports_existing_policy = jsondecode(
+    var.techops_reports.enabled ? var.techops_reports.existing_bucket_policy_json : "{\"Version\":\"2012-10-17\",\"Statement\":[]}"
+  )
+  techops_reports_policy = merge(local.techops_reports_existing_policy, {
+    Statement = concat(
+      flatten([local.techops_reports_existing_policy.Statement]),
+      [
+        {
+          Sid       = "TechOpsReportsRequireTLS"
+          Effect    = "Deny"
+          Principal = "*"
+          Action    = "s3:*"
+          Resource  = local.techops_reports_object_arns
+          Condition = {
+            Bool = {
+              "aws:SecureTransport"       = "false"
+              "aws:PrincipalIsAWSService" = "false"
+            }
+          }
+        },
+        {
+          Sid       = "TechOpsReportsRequireKmsEncryption"
+          Effect    = "Deny"
+          Principal = "*"
+          Action    = "s3:PutObject"
+          Resource  = local.techops_reports_object_arns
+          Condition = {
+            StringNotEquals = { "s3:x-amz-server-side-encryption" = "aws:kms" }
+          }
+        },
+        {
+          Sid       = "TechOpsReportsRequireApprovedKey"
+          Effect    = "Deny"
+          Principal = "*"
+          Action    = "s3:PutObject"
+          Resource  = local.techops_reports_object_arns
+          Condition = {
+            StringNotEquals = { "s3:x-amz-server-side-encryption-aws-kms-key-id" = var.techops_reports.kms_key_arn }
+          }
+        },
+        {
+          Sid       = "TechOpsReportsAllowProducers"
+          Effect    = "Allow"
+          Principal = { AWS = sort(tolist(var.techops_reports.producer_role_arns)) }
+          Action    = local.techops_reports_producer_actions
+          Resource  = local.techops_reports_object_arns
+        },
+      ],
+      # Analyst download grants are independent of producer uploads. Omit the
+      # statements entirely when no reader roles have been configured.
+      [for statement in [
+        {
+          Sid       = "TechOpsReportsAllowReaders"
+          Effect    = "Allow"
+          Principal = { AWS = sort(tolist(var.techops_reports.reader_role_arns)) }
+          Action    = "s3:GetObject"
+          Resource  = local.techops_reports_object_arns
+        },
+        {
+          Sid       = "TechOpsReportsAllowReaderListing"
+          Effect    = "Allow"
+          Principal = { AWS = sort(tolist(var.techops_reports.reader_role_arns)) }
+          Action    = "s3:ListBucket"
+          Resource  = local.techops_reports_bucket_arn
+          Condition = {
+            StringLike = { "s3:prefix" = [for prefix in local.techops_reports_prefixes : "${prefix}*"] }
+            Bool       = { "aws:SecureTransport" = "true" }
+          }
+        },
+      ] : statement if length(var.techops_reports.reader_role_arns) > 0],
+      [
+        {
+          Sid       = "TechOpsReportsAllowBucketLocation"
+          Effect    = "Allow"
+          Principal = { AWS = sort(tolist(setunion(var.techops_reports.producer_role_arns, var.techops_reports.reader_role_arns))) }
+          Action    = "s3:GetBucketLocation"
+          Resource  = local.techops_reports_bucket_arn
+          Condition = { Bool = { "aws:SecureTransport" = "true" } }
+        },
+      ]
+    )
+  })
+  techops_reports_kms_conditions = {
+    StringEquals = { "kms:ViaService" = "s3.us-east-1.amazonaws.com" }
+    ArnLike = {
+      "kms:EncryptionContext:aws:s3:arn" = [local.techops_reports_bucket_arn, "${local.techops_reports_bucket_arn}/*"]
+    }
+  }
+}
+
+resource "aws_s3_bucket_policy" "techops_reports" {
+  provider = aws.main
+  count    = var.techops_reports.enabled ? 1 : 0
+  bucket   = var.was_reporting_reports_bucket_name
+  policy   = jsonencode(local.techops_reports_policy)
+
+  lifecycle {
+    prevent_destroy = true
+  }
+}
+
+# S3 stores one complete lifecycle configuration per bucket. The input guard
+# prevents this ME-only resource from replacing unreviewed existing WAS rules.
+resource "aws_s3_bucket_lifecycle_configuration" "techops_reports" {
+  provider              = aws.main
+  count                 = var.techops_reports.enabled && var.techops_reports.manage_retention ? 1 : 0
+  bucket                = var.was_reporting_reports_bucket_name
+  expected_bucket_owner = "957221700844"
+
+  dynamic "rule" {
+    for_each = local.techops_reports_retention_days_by_prefix
+
+    content {
+      id     = "techops-expire-${trimsuffix(rule.key, "/")}"
+      status = "Enabled"
+
+      filter {
+        prefix = rule.key
+      }
+
+      expiration {
+        days = rule.value
+      }
+
+      # In versioned buckets, current-version expiration leaves a noncurrent
+      # copy. It becomes eligible for permanent cleanup one day later.
+      noncurrent_version_expiration {
+        noncurrent_days = 1
+      }
+
+      abort_incomplete_multipart_upload {
+        days_after_initiation = 1
+      }
+    }
+  }
+
+  dynamic "rule" {
+    for_each = local.techops_reports_retention_days_by_prefix
+
+    content {
+      id     = "techops-markers-${trimsuffix(rule.key, "/")}"
+      status = "Enabled"
+
+      filter {
+        prefix = rule.key
+      }
+
+      expiration {
+        expired_object_delete_marker = true
+      }
+    }
+  }
+
+  lifecycle {
+    prevent_destroy = true
+  }
+}