diff --git a/README.md b/README.md index 1c7460a..cce6ad1 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@ --> # Manta: a scalable, distributed object store @@ -93,11 +93,9 @@ The front door services respond to requests from the internet at large: service * [muskie](https://github.com/joyent/manta-muskie): the node.js-based "webapi" service, this is Manta's "Directory API" -* [buckets-api](https://github.com/joyent/manta-buckets-api): Node.js-based - "buckets-api" service, this is Manta's "Buckets API" -The metadata tiers for the Directory and Buckets APIs store the entire object -namespace (not object data) as well as backend storage system capacity: +The metadata tier APIs store the entire object namespace (not object data) as +well as backend storage system capacity: * [manatee](https://github.com/joyent/manatee): the "postgres" service, a high-availability postgres cluster using synchronous replication and automatic @@ -108,11 +106,6 @@ namespace (not object data) as well as backend storage system capacity: * [electric-moray](https://github.com/joyent/electric-moray): Node-based service that provides the same interface as Moray, but which directs requests to one or more Moray+Manatee *shards* based on hashing the Moray key. -* [buckets-mdapi](https://github.com/joyent/manta-buckets-mdapi): a Rust-based - API for managing all metadata for the Buckets API -* [buckets-mdplacement](https://github.com/joyent/manta-buckets-mdplacement): a - Rust-based API for handling routing of Buckets API objects to appropriate - nodes in the storage tier. The storage tier is responsible for actually storing bits on disk: diff --git a/docs/developer-guide/README.md b/docs/developer-guide/README.md index d03e8eb..4187514 100644 --- a/docs/developer-guide/README.md +++ b/docs/developer-guide/README.md @@ -31,7 +31,6 @@ nitty-gritty content than makes sense for the README. - [Configuration](#configuration) - [Configuration Updates](#configuration-updates) - [Directory API Shard Management](#directory-api-shard-management) - - [Buckets API Shard Management](#buckets-api-shard-management) @@ -405,23 +404,3 @@ find yourself adding additional capacity, deploy the new shard first, coordinate with all existing shards, then use manta-shardadm to add the shard to list of shards for the indexing tier. - -## Buckets API Shard Management - -Buckets API is an experimental feature that serves similar functions as the -Directory API but has a different paradigm for object organization. As opposed -to the hierarchical object support provided by Directory API that comes with -a limit on the maximum number of objects per directory, Buckets API offers a -simpler structure for storing an unlimited number of objects in groups. The -two-level object structure incurs a much smaller overhead in request processing -and is more cost-efficient from a metadata perspective. It is a better option -when the objects in the same bucket are loosely related and do not need a -finer categorization. - -Buckets shards are defined and stored in the same way as directory shards -but are separate entities altogether. They are also managed with -`manta-shardadm` which generates the `BUCKETS_MORAY_SHARDS` and -`BUCKETS_HASH_RING_IMAGE` configurations in SAPI metadata. - -Note: The "buckets" that Buckets API manages are not to be confused with moray -"buckets" which represent the object namespaces for the data stored in Postgres. diff --git a/docs/mantav2.md b/docs/mantav2.md index 7f7adfa..f6f6f45 100644 --- a/docs/mantav2.md +++ b/docs/mantav2.md @@ -1,15 +1,21 @@ # Mantav2 -Starting November 2019, there will be two separate active versions of Manta: +Since November 2019, there have been two separate active versions of Manta: - "**mantav1**" - a long term support branch of Manta that maintains current Manta features. -- "**mantav2**" - a new major version of Manta that adds (Buckets API, storage - "rebalancer" service) and removes (jobs, snaplinks, etc.) some major - features, and becomes the focus of future Manta development. +- "**mantav2**" - a new major version of Manta that removes some features (jobs, + snaplinks, MPU, etc.) and makes some major improvements to things like garbage + collection. -At this time, mantav1 is the recommended version for production usage, but -that is expected to change to mantav2 during 2020. +As of 2021H1, mantav2 is the recommended version for new installations. + +Originally mantav2 was intended to include a new "Buckets API" but this work was +abandoned in early 2020. There may still be references in documentation and code +to "buckets" but these features should not be deployed or used. Additionally +there was originally mention of online rebalancing of objects. These features +were never completed to the point they could be considered part of the product +and this feature should also be considered abandoned. ## What is mantav2? @@ -29,11 +35,6 @@ Significant changes are: - snaplinks - metering data under `//reports/...` Otherwise the Directory API remains a part of Manta. -- A new "Buckets API" (S3-like) is added. This is the API for which latency - improvements are being made. -- A "rebalancer" system is added for storage tier maintenance. -- The garbage collection (GC) system is improved for larger scale. -- Improved per-account usage data for operators. The "master" branch of Manta-related git repos is for mantav2. Mantav1 development has moved to "mantav1" branches. @@ -78,12 +79,11 @@ The Node.js Manta client is developed in the - mantav1: Currently on the ["master" branch](https://github.com/joyent/node-manta/tree/master/) of joyent/node-manta, and published to npm as ["manta"](https://www.npmjs.com/package/manta) -- i.e. `npm install manta`. -- mantav2: Currently on the ["buckets" branch](https://github.com/joyent/node-manta/tree/buckets/) - of joyent/node-manta. It is not yet published to npm. +- mantav2: The mantav1 client should work with mantav2. -*(The intent is to eventually move mantav2 to the "master" branch and publish it -to npm as "mantav2". Mantav1 dev would move to the "mantav1" branch and continue -to publish to npm as "manta".)* +*(The intent is to eventually move a mantav2 version to the "master" branch and +publish it to npm as "mantav2". Mantav1 dev would move to the "mantav1" branch +and continue to publish to npm as "manta".)* #### Java client @@ -93,11 +93,10 @@ The Java Manta client is developed in the - mantav1: Currently on the ["master" branch](https://github.com/joyent/java-manta/tree/master/) of joyent/java-manta. Current release versions are 3.x. -- mantav2: Currently on the ["buckets-m1" branch](https://github.com/joyent/java-manta/tree/buckets-m1/) - of joyent/java-manta. +- mantav2: The mantav1 client should work with mantav2. -*(The intent is to eventually move mantav2 to the "master" branch and release it -as a new "4.x" major version. Mantav1 dev would move to the "3.x" branch and +*(The intent is to eventually move a mantav2 to the "master" branch and release +it as a new "4.x" major version. Mantav1 dev would move to the "3.x" branch and continue to release as 3.x versions.)* diff --git a/docs/operator-guide/README.md b/docs/operator-guide/README.md index b2729dc..fb1ef13 100644 --- a/docs/operator-guide/README.md +++ b/docs/operator-guide/README.md @@ -20,11 +20,9 @@ essentially: * A separate filesystem-like namespace *Directory API*, with directories and objects, accessible over HTTP. -* A *Buckets API* (an experimental feature similar to S3) with objects, - accessible over HTTP. * *Objects* are arbitrary-size blobs of data * Users can use standard HTTP `PUT`/`GET`/`DELETE` verbs to create, list, and - remove buckets, directories, and objects. + remove directories, and objects. * Users can fetch arbitrary ranges of an object, but may not *modify* an object except by replacing it. diff --git a/docs/operator-guide/architecture.md b/docs/operator-guide/architecture.md index 4775e67..179e049 100644 --- a/docs/operator-guide/architecture.md +++ b/docs/operator-guide/architecture.md @@ -24,12 +24,9 @@ This section discusses the basics of the Manta architecture. - [Postgres, replication, and sharding](#postgres-replication-and-sharding) - [Other shards](#other-shards) - [Moray](#moray) - - [Buckets-mdapi](#buckets-mdapi) - [Electric-moray](#electric-moray) - - [Buckets-mdplacement](#buckets-mdplacement) - [The front door](#the-front-door) - [Objects and directories](#objects-and-directories) -- [Multipart uploads](#multipart-uploads) - [Garbage Collection](#garbage-collection) - [Metering](#metering) - [Manta Scalability](#manta-scalability) @@ -133,23 +130,15 @@ services. | Service | Consensus | nameservice | Service discovery | ZooKeeper, [binder](https://github.com/joyent/binder) (DNS) | | Service | Front door | loadbalancer | SSL termination and load balancing | haproxy, [muppet](https://github.com/joyent/muppet) | | Service | Front door | webapi | Manta HTTP Directory API server | [muskie](https://github.com/joyent/manta-muskie) | -| Service | Front door | buckets-api\* | Manta HTTP Buckets API server | [buckets-api](https://github.com/joyent/manta-buckets-api) | | Service | Front door | authcache | Authentication cache | [mahi](https://github.com/joyent/mahi) (redis) | -| Service | Garbage Collection | garbage-buckets-consumer | Manta Buckets API garbage collection | [garbage-buckets-consumer (bin)](https://github.com/joyent/manta-garbage-collector/blob/master/bin/garbage-buckets-consumer.js), [garbage-buckets-consumer (lib)](https://github.com/joyent/manta-garbage-collector/blob/master/lib/garbage-buckets-consumer.js) | | Service | Garbage Collection | garbage-deleter | Deleting storage for objects | [garbage-deleter (bin)](https://github.com/joyent/manta-mako/blob/master/bin/garbage-deleter.js), [garbage-deleter (lib)](https://github.com/joyent/manta-mako/blob/master/lib/garbage-deleter.js) | | Service | Garbage Collection | garbage-dir-consumer | Manta Directory API garbage collection | [garbage-dir-consumer (bin)](https://github.com/joyent/manta-garbage-collector/blob/master/bin/garbage-dir-consumer.js), [garbage-dir-consumer (lib)](https://github.com/joyent/manta-garbage-collector/blob/master/lib/garbage-dir-consumer.js) | -| Service | Garbage Collection | garbage-mpu-cleaner | MPU garbage collection | [garbage-mpu-cleaner (bin)](https://github.com/joyent/manta-garbage-collector/blob/master/bin/garbage-mpu-cleaner.js), [garbage-mpu-cleaner (lib)](https://github.com/joyent/manta-garbage-collector/blob/master/lib/garbage-mpu-cleaner.js) | | Service | Garbage Collection | garbage-uploader | Send GC instructions to storage zones | [garbage-uploader (bin)](https://github.com/joyent/manta-garbage-collector/blob/master/bin/garbage-uploader.js), [garbage-uploader (lib)](https://github.com/joyent/manta-garbage-collector/blob/master/lib/garbage-uploader.js) | | Service | Metadata | postgres | Directory metadata storage | postgres, [manatee](https://github.com/joyent/manta-manatee) | -| Service | Metadata | buckets-postgres\* | Buckets metadata storage | postgres, [manatee](https://github.com/joyent/manta-manatee) | | Service | Metadata | moray | Directory key-value store | [moray](https://github.com/joyent/moray) | -| Service | Metadata | buckets-mdapi\* | Buckets key-value store | [buckets-mdapi](https://github.com/joyent/manta-buckets-mdapi) | | Service | Metadata | electric-moray | Directory consistent hashing (sharding)| [electric-moray](https://github.com/joyent/electric-moray) | -| Service | Metadata | buckets-mdplacement\* | Buckets consistent hashing (sharding) | [buckets-mdplacement](https://github.com/joyent/manta-buckets-mdplacement) | -| Service | Metadata | reshard\* | Metadata reshard tool | [reshard](https://github.com/joyent/manta-reshard) | | Service | Metadata | storinfo | Storage metadata cache and picker | [storinfo](https://github.com/joyent/manta-storinfo) | -| Service | Storage | storage | Object storage and capacity reporting | [mako](https://github.com/joyent/manta-mako) (nginx), [minnow](https://github.com/joyent/manta-minnow), [rebalancer-agent](https://github.com/joyent/manta-rebalancer/tree/master/agent) | -| Service | Storage | rebalancer | Storage zone evacuation/rebalancing | [rebalancer](https://github.com/joyent/manta-rebalancer) | +| Service | Storage | storage | Object storage and capacity reporting | [mako](https://github.com/joyent/manta-mako) (nginx), [minnow](https://github.com/joyent/manta-minnow) | | Service | Operations | madtom | Web-based Manta monitoring | [madtom](https://github.com/joyent/manta-madtom) | | Service | Operations | ops | Operator workspace | | @@ -209,25 +198,15 @@ runs: * **minnow**: a small Node service that periodically reports storage capacity data into the metadata tier so that the front door knows how much capacity each storage node has. -* **rebalancer-agent**: a small Rust service that processes object copy - requests coming from the rebalancer. - -The **rebalancer** or **rebalancer manager** is a separate service running -outside of the storage zones which orchestrates the migration of objects -between storage zones. The service allows an operator to evacuate an entire -storage zone in the event of hardware outage or planned replacement. ## Metadata tier The metadata tier is itself made up of three levels: -* "postgres" and "buckets-postgres" zones, which run instances of the - postgresql database -* "moray" and "buckets-mdapi" zones, which run key-value stores on top of - postgres -* "electric-moray" and "buckets-mdplacement" zones, which handle sharding of - metadata requests +* "postgres" zones, which run instances of the postgresql database +* "moray" zones, which run key-value stores on top of postgres +* "electric-moray" zones, which handle sharding of metadata requests ### Postgres, replication, and sharding @@ -292,12 +271,6 @@ There are actually two kinds of metadata in Manta: above) and all lives on one shard. This is extremely low-volume: a couple of writes per storage node per minute. -Manta supports the **resharding** of directory object metadata, a process which -would typically be used to add additional shards (for horizontal scaling of -metadata capacity). This operation is handled by the reshard service which -currently supports only the doubling of shards and incurs object write downtime -during the hash ring update step in the process. - ### Moray For each metadata shard (which we said above consists of three PostgreSQL @@ -312,11 +285,6 @@ This way, clients don't need to know about the replication topology. Like Postgres, each Moray instance is tied to a particular shard. These are typically referred to as "1.moray", "2.moray", and so on. -### Buckets-mdapi - -Whereas moray handles the metadata requests for directory objects, buckets-mdapi -performs the same function for buckets objects. - ### Electric-moray The electric-moray service sits in front of the sharded Moray instances and @@ -325,34 +293,20 @@ metadata for `/mark/stor/foo`, electric-moray will hash `/mark/stor` to find the right shard and then proxy the request to one of the Moray instances operating that shard. -### Buckets-mdplacement - -Buckets-mdplacement performs the same sharding function as electric-moray for -buckets objects. But unlike electric-moray, it is not on the data path of -service requests. The hash ring information is cached in buckets-api upon -service startup. - ## The front door -The front door consists of "loadbalancer", "webapi" and "buckets-api" zones. +The front door consists of "loadbalancer" and "webapi" zones. "loadbalancer" zones run haproxy for both SSL termination and load balancing -across the available "webapi" and "buckets-api" instances. "haproxy" is -managed by a component called "muppet" that uses the DNS-based service discovery -mechanism to keep haproxy's list of backends up-to-date. +across the available "webapi" instances. "haproxy" is managed by a component +called "muppet" that uses the DNS-based service discovery mechanism to keep +haproxy's list of backends up-to-date. "webapi" zones run the Manta-specific API server, called **muskie**. Muskie handles PUT/GET/DELETE requests to the front door, including requests to: * create and delete objects * create, list, and delete directories -* create multipart uploads, upload parts, fetch multipart upload state, commit - multipart uploads, and abort multipart uploads - -"buckets-api" zones run an alternative API server which handles S3-like -PUT/GET/DELETE requests for objects with a different organization paradigm. -The feature is still considered experimental, with limited support and -unpublished API clients at this time. ### Objects and directories @@ -361,14 +315,13 @@ Requests for objects and directories involve: * validating the request * authenticating the user (via mahi, the auth cache) -* looking up the requested object's metadata (via electric-moray or buckets- - mdplacement) +* looking up the requested object's metadata (via electric-moray) * authorizing the user for access to the specified resource For requests on directories and zero-byte objects, the last step is to update or return the right metadata. -For write requests on objects, muskie or buckets-api then: +For write requests on objects, muskie then: * Constructs a set of candidate storage nodes that will be used to store the object's data, where each storage node is located in a different datacenter @@ -382,66 +335,29 @@ For write requests on objects, muskie or buckets-api then: streamed to all nodes. Upon completion, there should be a 204 response from each storage node. * Once the data is safely written to all nodes, the metadata tier is updated - (using a PUT to electric-moray or to buckets-mdapi), and a 204 is returned to - the client. At this point, the object's data is recorded persistently on the - requested number of storage nodes, and the metadata is replicated on at least - two index nodes. - -For read requests on objects, muskie or buckets-api instead contacts each of -the storage nodes hosting the data and streams data from whichever one responds -first to the client. - - -## Multipart uploads - -Multipart uploads, a.k.a. MPU, provide an alternate way for users to upload -Manta objects (only for the Directory API). The user creates the multipart -upload, uploads the object in parts, and exposes the object in Manta by -committing the multipart upload. Generally, these operations are implemented -using existing Manta constructs: - -* Parts are normal Manta objects, with a few key differences. Users cannot use - the GET, POST or DELETE HTTP methods on parts. Additionally, all parts are - co-located on the same set of storage nodes, which are selected when the - multipart upload is created. -* All parts for a given multipart upload are stored in a parts directory, which - is a normal Manta directory. -* Part directories are stored in the top-level `/$MANTA_USER/uploads` directory - tree. - -Most of the logic for multipart uploads is performed by Muskie, but there are -some additional features of the system only used for multipart uploads: - -* the **manta_uploads** bucket in Moray stores **finalizing records** for a - given shard. A finalizing record is inserted atomically with the target - object record when a multipart upload is committed. -* the mako zones have a custom **mako-finalize** operation invoked by muskie - when a multipart upload is committed. This operation creates the target object - from the parts and subsequently deletes the parts from disk. This operation - is invoked on all storage nodes that will contain the target object when the - multipart upload is committed. + (using a PUT to electric-moray), and a 204 is returned to the client. At this + point, the object's data is recorded persistently on the requested number of + storage nodes, and the metadata is replicated on at least two index nodes. + +For read requests on objects, muskie instead contacts each of the storage nodes +hosting the data and streams data from whichever one responds first to the +client. ## Garbage Collection **Garbage collection** consists of several components in the `garbage-collector` and `storage` zones and is responsible for removing the storage used by objects -which have been removed from the metadata tier. It is also responsible for -removing metadata for finalized MPU uploads. +which have been removed from the metadata tier. -When an object is deleted from the metadata tier in either the Manta Directory -API or Manta Buckets API, the objects on disk are not immediately removed, nor are -all references in the metadata tier itself. The original record is moved into a -new deletion record which includes the information required to delete the -storage backing the now-deleted object. The garbage collection system is -responsible for actually performing the cleanup. +When an object is deleted from the metadata tier, the objects on disk are not +immediately removed, nor are all references in the metadata tier itself. The +original record is moved into a new deletion record which includes the +information required to delete the storage backing the now-deleted object. The +garbage collection system is responsible for actually performing the cleanup. Processes in the `garbage-collector` zone include: - * `garbage-buckets-consumer` -- consumes deletion records from `buckets-mdapi` - (created when an object is deleted by Manta Buckets API). The records found - are written to local `instructions` files in the `garbage-collector` zone. - * `garbage-dir-consumer` -- consumes deletion records from the `manta_fastdelete_queue` bucket (created when an object is deleted through the Manta Directory API). The records found are written to local @@ -450,11 +366,6 @@ Processes in the `garbage-collector` zone include: * `garbage-uploader` -- consumes the locally queued `instructions` and uploads them to the appropriate `storage` zone for processing. - * `garbage-mpu-cleaner` -- consumes "finalized" MPU uploads (Manta Directory - API only) and deletes the upload parts, upload directory and the finalize - record itself after the upload has been finalized for some period of time - (default: 5 minutes). - On the `storage` zones, there's an additional component of garbage collection: * `garbage-deleter` -- consumes `instructions` that were uploaded by diff --git a/docs/operator-guide/maintenance.md b/docs/operator-guide/maintenance.md index 4fcaffc..58e55f6 100644 --- a/docs/operator-guide/maintenance.md +++ b/docs/operator-guide/maintenance.md @@ -30,9 +30,6 @@ service logs; and some general inspection/debugging tasks. - [Real-time logs and log formats](#real-time-logs-and-log-formats) - [Request Throttling](#request-throttling) - [Throttle Parameter Trade-offs](#throttle-parameter-trade-offs) - - [Multipart uploads prefix length](#multipart-uploads-prefix-length) - - [Changing the prefix length](#changing-the-prefix-length) - - [Prefix length tradeoffs](#prefix-length-tradeoffs) - [Picker/Storinfo toggle](#pickerstorinfo-toggle) - [Debugging: general tasks](#debugging-general-tasks) - [Locating servers](#locating-servers) @@ -453,65 +450,6 @@ latency is not acceptable and the application is likely to retry on receipt of a 503. Low queue tolerance values are also desirable if the zone is under memory pressure. -## Multipart uploads prefix length - -The Manta multipart upload API (MPU) stores the part directories of an account's -ongoing multipart uploads under the directory tree `/$MANTA_USER/uploads`. -Within the top-level directory, part directories are stored in subdirectories -based on some number of the first characters of the multipart upload's UUID. -The number of characters used to split multipart uploads is referred to as the -"prefix length". - -For example, in a Manta deployment for which the prefix length is set to 3, -a multipart upload would have an upload directory that looks like this: - - /$MANTA_USER/uploads/f00/f00e51d2-7e47-4732-8edf-eb871296b343 - -Note that the parent directory of the parts directory, also referred to -as its "prefix directory", has 3 characters, the same as the prefix length. - -The following multipart upload would have been created in a Manta deployment -with a prefix length of 1: - - /$MANTA_USER/uploads/d/d77feb78-cd7f-481f-a6c7-f653c80c7331 - - -### Changing the prefix length - -The prefix length is configurable in SAPI, represented as the -`MUSKIE_MPU_PREFIX_DIR_LEN` SAPI variable under the "webapi" service. For -example, to change the prefix length of a deployment to 2, you could run: - - $ sapiadm update $(sdc-sapi /services?name=webapi | json -Ha uuid) \ - metadata."MUSKIE_MPU_PREFIX_DIR_LEN"=2 - -As with other configuration changes to the "webapi" service, you must restart -the "webapi" zones to see the configuration change. - -Multipart uploads created with a different prefix length within the same Manta -deployment will continue to work after the prefix length is changed. - -### Prefix length tradeoffs - -The prefix length dictates the number of subdirectories allowed in the top-level -`/$MANTA_USER/uploads` directory. Because the number of entries in a Manta -directory should be limited, this affects how many ongoing multipart uploads are -available for a given account. Increasing the prefix length also increases the -number of requests required to list all multipart uploads under a given account. -Consequently, a smaller prefix length allows for fewer ongoing multipart uploads -for a single account, but less work to list them all; larger prefix directories -allow more ongoing multipart uploads, but require more work to list them. - -For example, in a Manta deployment with a prefix length of 3, a given account -may have up to 4096 prefix directories, allowing for about 4 billion ongoing -multipart uploads for a given account. Listing all of the multipart uploads -ongoing requires a maximum of 4096 directory listings operations. Compare this -to a deployment with a prefix length of 1, which has a maximum of 256 prefix -directories and allows for about 256 million multipart uploads, but only up to -256 directory listings are required to list all multipart uploads under an -account. - - ## Picker/Storinfo toggle There are two options for webapi to obtain storage node information - "picker" @@ -520,9 +458,9 @@ node `statvfs` data, keep a local cache and periodically refresh it, and select storage nodes for object write requests. Storinfo is an optional service which is separate from webapi. If storinfo is -not deployed (because rebalancer and buckets API components are not in use), -you should configure webapi to use the local picker function by setting the -`WEBAPI_USE_PICKER` SAPI variable to `true` under the "webapi" service: +not deployed you should configure webapi to use the local picker function by +setting the `WEBAPI_USE_PICKER` SAPI variable to `true` under the "webapi" +service: $ sdc-sapi /services/$(sdc-sapi /services?name=webapi | json -Ha uuid) \ -X PUT -d '{"action": "update", "metadata": {"WEBAPI_USE_PICKER": true}}' diff --git a/docs/operator-guide/mantav2-migration.md b/docs/operator-guide/mantav2-migration.md index e8adb9d..accd148 100644 --- a/docs/operator-guide/mantav2-migration.md +++ b/docs/operator-guide/mantav2-migration.md @@ -50,7 +50,7 @@ Specific steps will be provided later in this document. sdcadm post-setup manta 2. **Snaplink cleanup**. Snaplinks must be cleaned from the system, otherwise - the new GC and rebalancer systems cannot guarantee data integrity. + the new GC system cannot guarantee data integrity. 3. Deploy the **new garbage collector** (GCv2) system. @@ -59,10 +59,7 @@ Specific steps will be provided later in this document. can be undeployed. As well, any or all remaining Manta services can be updated to their latest "mantav2-\*" images. -5. **Optional service updates.** The new rebalancer service and the services - that make up the new Buckets API can be deployed. - -6. **Additional clean up.** Some orphaned data (related to the removed jobs +5. **Additional clean up.** Some orphaned data (related to the removed jobs feature and to the earlier GC system) can be removed. Other than the usual possible brief downtimes for service upgrades, this @@ -234,8 +231,8 @@ Mantav1 supported a feature called "snaplinks" where a new object could be quickly created from an existing one by linking to it. These snaplinks must be "delinked" -- i.e. changed from being metadata-tier references to a shared storage-tier object, to being fully separate objects -- before the new -garbage-collector and rebalancer services in mantav2 can function. This section -walks through the process of removing snaplinks. +garbage-collector services in mantav2 can function. This section walks through +the process of removing snaplinks. Snaplink cleanup involves a few stages, some of which are manual. The `mantav2-migrate snaplink-cleanup` command is used to coordinate the process. @@ -1016,16 +1013,7 @@ A simplified procedure is as follows: the storinfo service.) -## Step 7: Optional service updates - -At this point the services for the new "Rebalancer" system and the "Buckets API" -can be deployed. - -(TODO: The operator guide and/or here should provide details for deploying -those.) - - -## Step 8: Additional clean up +## Step 7: Additional clean up There remain a few things that can be cleaned out of the system. They are: