DOC-4238, DOC-4510 Sideloader GA prep, Relaunch function, begin migration docs cleanup #194
Conversation
This comment was marked as resolved.
This comment was marked as resolved.
|
Build successful! ✅
Deploying draft. |
|
Build successful! ✅
Deploying draft. |
|
Build successful! ✅
Deploying draft. |
|
Build successful! ✅
Deploying draft. |
|
Build successful! ✅
Deploying draft. |
|
Build successful! ✅
Deploying draft. |
|
Build successful! ✅
Deploying draft. |
|
Build successful! ✅
Deploying draft. |
|
Build successful! ✅
Deploying draft. |
|
Build successful! ✅
Deploying draft. |
This comment was marked as resolved.
This comment was marked as resolved.
|
Build successful! ✅
Deploying draft. |
| -- | ||
| Because {sstable-sideloader} operations are typically short-term, resource-intensive events, you can create a flexible capacity PCU group exclusively to support your target database during the migration. | ||
|
|
||
| //After PCU PR merge: xref:astra-db-serverless:administration:create-pcu.adoc#flexible-capacity[create a flexible capacity PCU group] |
There was a problem hiding this comment.
Edit link before merge (1)
| -- | ||
| If you plan to keep your target database in a PCU group after the migration, you can create a committed capacity PCU group for your target database. | ||
|
|
||
| //After PCU PR merge: xref:astra-db-serverless:administration:create-pcu.adoc#committed-capacity[create a committed capacity PCU group] |
There was a problem hiding this comment.
Edit link before merge (2)
|
Build successful! ✅
Deploying draft. |
1 similar comment
|
Build successful! ✅
Deploying draft. |
|
Build successful! ✅
Deploying draft. |
| == Command line knowledge requirements | ||
|
|
||
| Due to the nature of the {sstable-sideloader} process and the tools involved, you need to be familiar with using the command line, including the following: | ||
|
|
||
| * Installing and using CLI tools | ||
| * Issuing curl commands | ||
| * Basic scripting | ||
| * Modifying example command to your environment | ||
| * Security best practices | ||
|
|
||
| [IMPORTANT] | ||
| ==== | ||
| The {sstable-sideloader} process uses authentication credentials to write to the migration directory and your database. | ||
|
|
||
| Make sure you understand how to securely store and use sensitive credentials when working on the command line. | ||
| ==== |
There was a problem hiding this comment.
Eric suggested moving this to the top of the page
|
|
||
| include::partial$cdm-prerequisites.adoc[] | ||
| It facilitates data transfer by creating multiple jobs that access the {cass-short} cluster concurrently, making it an ideal choice for migrating large datasets. |
There was a problem hiding this comment.
I generally favor repeating the noun instead of using a pronoun when starting new paragraphs. It might be better for SEO too, since I would expect that the para would be more likely to be chosen as a snippet in Google results. But who knows, really.
| It facilitates data transfer by creating multiple jobs that access the {cass-short} cluster concurrently, making it an ideal choice for migrating large datasets. | |
| {cass-migrator-short} facilitates data transfer by creating multiple jobs that access the {cass-short} cluster concurrently, making it an ideal choice for migrating large datasets. |
|
|
||
| [[cdm-build-jar-local]] | ||
| == Build {cass-migrator} JAR for local development (optional) | ||
| This container's `assets` directory includes all required migration tools: `cassandra-data-migrator`, `dsbulk`, and `cqlsh`. |
There was a problem hiding this comment.
| This container's `assets` directory includes all required migration tools: `cassandra-data-migrator`, `dsbulk`, and `cqlsh`. | |
| The container's `assets` directory includes all required migration tools: `cassandra-data-migrator`, `dsbulk`, and `cqlsh`. |
| [source,bash,subs="+quotes"] | ||
| ---- | ||
| # Replace **PATCH** with your Spark patch version | ||
| wget https://archive.apache.org/dist/spark/spark-3.5.**PATCH**/spark-3.5.**PATCH**-bin-hadoop3-scala2.13.tgz | ||
|
|
||
| include::partial$use-cdm-migrator.adoc[] | ||
| tar -xvzf spark-3.5.**PATCH**-bin-hadoop3-scala2.13.tgz | ||
| ---- |
There was a problem hiding this comment.
In the future, this should probably be broken into two separate code blocks for clarity and easy copy-ability. I initially didn't see the first command because my eyes tricked me into thinking it was part of the comment.
| == Migrate or validate specific partition ranges | ||
| . Add the `cassandra-data-migrator` dependency to `pom.xml`: | ||
| + | ||
| [source,xml] |
There was a problem hiding this comment.
| [source,xml] | |
| [source,xml,subs="+quotes"] |
| For example, you can do the following: | ||
|
|
||
| * Check for large field guardrail violations before migrating. | ||
| * Use the `partition.min` and `partition.max` parameters to migrate or validate specific token ranges |
There was a problem hiding this comment.
| * Use the `partition.min` and `partition.max` parameters to migrate or validate specific token ranges | |
| * Use the `partition.min` and `partition.max` parameters to migrate or validate specific token ranges. |
|
|
||
| * A log message with `level=debug` or `level=info` is very likely not an error, but something expected and normal. | ||
| There are cases where protocol errors are fatal, and they will kill an active connection that was being used to serve requests. | ||
| However, it is also possible to get normal protocol log message that contain wording that sounds like an error. |
There was a problem hiding this comment.
| However, it is also possible to get normal protocol log message that contain wording that sounds like an error. | |
| However, it is also possible to get normal protocol log messages that contain wording that sounds like an error. |
| ---- | ||
|
|
||
| You can also provide a `-version` command line parameter to the {product-proxy} and it will only print the version. | ||
| Example: | ||
| This message is immediately before the long `Parsed configuration` string. |
There was a problem hiding this comment.
| This message is immediately before the long `Parsed configuration` string. | |
| This message is logged immediately before the long `Parsed configuration` string. |
| If you encounter a problem during your migration, please contact us. | ||
| In the {product-proxy} GitHub repo, submit a https://github.com/datastax/zdm-proxy/issues[GitHub Issue]. | ||
| Only to the extent that the issue's description does not contain **your proprietary or private** information, please include the following: | ||
| If possible, include the following information in the issue description, and remove all proprietary and private information before submitting the issue: |
There was a problem hiding this comment.
Seems a bit redundant with the notice right above it.
| If possible, include the following information in the issue description, and remove all proprietary and private information before submitting the issue: | |
| If possible, include the following information in the issue description: |
|
|
||
| === Reporting a performance issue | ||
| * <<proxy-logs,{product-proxy} logs>>, ideally at `debug` level, if you can easily reproduce the issue and tolerate restarting the proxy instances to apply the log level configuration change. |
There was a problem hiding this comment.
Sorry if I'm making a mess of the log leveling. I'm assuming you're using CAPS to refer to the log level configuration setting, and smalls to refer to individual messages/entries.
| * <<proxy-logs,{product-proxy} logs>>, ideally at `debug` level, if you can easily reproduce the issue and tolerate restarting the proxy instances to apply the log level configuration change. | |
| * <<proxy-logs,{product-proxy} logs>>, ideally at `DEBUG` level, if you can easily reproduce the issue and tolerate restarting the proxy instances to apply the log level configuration change. |
|
|
||
| * The driver language and version that the client application is using. | ||
|
|
||
| For performance-related issues, provide following additional information: |
There was a problem hiding this comment.
| For performance-related issues, provide following additional information: | |
| For performance-related issues, provide the following additional information: |
Direct changes for Sideloader GA
Prepare for Sideloader GA (DOC-4238):
General migration doc cleanup
Begin general cleanup of the migration docs:
includesthe entire body of cassandra-data-migrator.adoc.ifdefand{imageprefix}The following pages will receive further edits in future PRs. They don't need a detailed review at this time:
This PR is dependent on: