Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ Crowdstrike can be downloaded from our repository at:
for all devices owned by Harvard staff/faculty and available for all operating
systems.

!!! note "Please note"
!!! note "Please Note"

To acess this repository you need to be in **Harvard Campus Network**.

Expand Down
15 changes: 15 additions & 0 deletions docs/migration-guides/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Migration Guides

This section provides guidance on migrating your workloads and data off the NERC
platform.

- [Migrate from NERC OpenStack](openstack.md) — Data migration and decommissioning
for NERC OpenStack resources, including VM data transfer, and object storage
export.

- [Migrate from NERC OpenShift (OCP) & OpenShift AI (RHOAI)](ocp-and-rhoai.md)
— Data migration, configuration export, image backup, and decommissioning for
NERC OpenShift and RHOAI resources, including PVC data transfer, MinIO
object storage export, and workbench notebook backup.

---
143 changes: 143 additions & 0 deletions docs/migration-guides/ocp-and-rhoai.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
# Migrate from NERC OpenShift and OpenShift AI (RHOAI)

This guide covers the process of migrating your workloads and data off the NERC
OpenShift (OCP) and Red Hat OpenShift AI (RHOAI) platforms.

## Overview

Migrating from NERC OpenShift and/or RHOAI involves four main phases:

1. **Bulk-Export All Project Configurations** — Export all application setups,
configurations, and secrets into a reusable YAML file.

2. **Data Migration** — Transfer your data from OpenShift persistent storage,
object storage (MinIO), and RHOAI workbenches/notebooks to your target platform.

3. **Backup Internal Container Images** — Pull images from the internal OpenShift
registry and push them to an external registry.

4. **Decommissioning** — Delete all resources and archive your ColdFront project.

## Bulk-Export All Project Configurations

Export all application setups, configurations, and secrets within a specific
project into a single reusable YAML file:

```sh
oc get all,pod,deployment,deploymentconfig,pvc,route,service,build,buildconfig,
statefulset,replicaset,replicationcontroller,job,cronjob,imagestream,revision,
configuration,notebook -o yaml > openshift_backup.yaml
```

## Data Migration

Before decommissioning your resources, ensure all critical data has been migrated
to your target environment. NERC provides several methods for transferring data
from OCP and RHOAI.

### Persistent Storage (PVCs)

Copy data from PVCs to a pod's filesystem and then transfer it
to your target system. Refer to the
[Use Persistent Storage section](../openshift/storage/data-transfer-from-to-openshift.md#use-persistent-storage)
of the Data Transfer guide for detailed instructions on the following methods:

- **`oc rsync`** — Recommended for transferring directories between your local
machine and a pod. Efficient for repeat transfers as it only copies changed
files. See [Using `oc rsync`](../openshift/storage/data-transfer-from-to-openshift.md#using-oc-rsync).

- **`oc cp`** — Convenient for copying individual files or small directories
to/from a container. See
[Copying a Single File](../openshift/storage/data-transfer-from-to-openshift.md#copying-a-single-file).

- **`tar` with `oc exec`** — Stream a tar archive through a pod for efficient
directory transfers. See
[Using `tar` with `oc exec`](../openshift/storage/data-transfer-from-to-openshift.md#using-tar-with-oc-exec).

- **Transfer data directly to a PVC** — Run a temporary pod that mounts the PVC
and transfer data using `oc rsync` or `oc cp`. See
[Transferring Data Directly to a PVC](../openshift/storage/data-transfer-from-to-openshift.md#transferring-data-directly-to-a-pvc).

- **Transfer between two PVCs** — Run a pod that mounts both PVCs and copy data
between them. See
[Transferring Between Two PVCs](../openshift/storage/data-transfer-from-to-openshift.md#transferring-between-two-pvcs).

For help choosing the right method, see the
[Choosing a Transfer Method](../openshift/storage/data-transfer-from-to-openshift.md#choosing-a-transfer-method)
table.

### Object Storage (MinIO)

If you use [MinIO](../openshift/storage/minio.md) for object storage on your
OpenShift project, export your data using the following approaches documented
in the
[For Object Storage Setup on NERC OCP](../openshift/storage/data-transfer-from-to-openshift.md#for-object-storage-setup-on-nerc-ocp)
section:

- **MinIO Web Console** — Upload and download data through the browser-based
interface. See [Using MinIO](../openshift/storage/data-transfer-from-to-openshift.md#using-minio).

- **Rclone** — Sync, copy, or mount object storage for transfer to any supported
destination. See [Using Rclone](../openshift/storage/data-transfer-from-to-openshift.md#using-rclone).

- **Rclone workbench (RHOAI)** — Deploy an Rclone-based workbench through RHOAI
to manage transfers via a web interface. See
[Using RHOAI Rclone Workbench](../openshift/storage/Rclone.md).

### RHOAI Workbenches, Notebooks, and Cluster Storage

For users of Red Hat OpenShift AI (RHOAI):

1. **Notebook data** — Connect to your JupyterLab environment (see
[Explore the JupyterLab Environment](../openshift-ai/data-science-project/explore-the-jupyterlab-environment.md))
and download any data, models, or notebooks from the workbench's filesystem.

2. **Data Science Projects** — Review your data science project resources (see
[Using Your Data Science Project (DSP)](../openshift-ai/data-science-project/using-projects-the-rhoai.md))
and export any stored artifacts, trained models, or pipelines.

3. **Cluster storage** — All RHOAI cluster storage is backed by PVCs in your
OpenShift project to store your Jupyter notebooks and associated data, ensuring
that your work remains persistent. You can download the data as described in
[Persistent Storage (PVCs)](#persistent-storage-pvcs) above.

!!! note "Important"

The PVC backing a workbench's cluster storage includes all files uploaded to
that workbench. When you download the PVC data, you will also get all
notebooks, applications, and data stored on that workbench.

## Backup Internal Container Images

If you have images stored in the internal OpenShift registry (ImageStreams),
pull them to your local machine and push them to an external registry:

```sh
# Log in to the OpenShift registry via docker or podman
podman login -u $(oc whoami) -p $(oc whoami -t) $(oc registry info)

# Pull the image to your local machine
podman pull $(oc registry info)/<project-name>/<image-name>:<tag>

# Tag and push to an external registry (e.g., Quay or Docker Hub)
podman tag $(oc registry info)/<project-name>/<image-name>:<tag> quay.io/<username>/<image-name>:<tag>
podman push quay.io/<username>/<image-name>:<tag>
```

## Decommissioning

Once your data has been safely migrated, follow the
[Decommission OpenShift Resources](../openshift/decommission/decommission-openshift-resources.md)
guide to:

1. Delete all resources (pods, deployments, PVCs, routes, services, builds, etc.)

2. Remove users from your ColdFront project (optional)

3. Reduce all resource quotas to zero via a ColdFront change request

4. Review project resource quotas

5. Archive your ColdFront project

---
120 changes: 120 additions & 0 deletions docs/migration-guides/openstack.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
# Migrate from NERC OpenStack

This guide covers migrating your workloads and data off the NERC
OpenStack platform, including data migration, resource cleanup, and project decommissioning.

## Overview

Migrating from NERC OpenStack involves two main phases:

1. **Data Migration** — Transfer your data from NERC OpenStack resources (volumes,
object storage, VM filesystems) to your target platform.

2. **Decommissioning** — Clean up all OpenStack resources and archive your ColdFront
project.

## Data Migration

Before decommissioning your resources, ensure all critical data has been migrated
to your target environment.

### Data Transfer from VMs

To transfer data from your virtual machine's filesystem to an external location,
use one of the following methods documented in the
[Data Transfer To/From NERC VM](../openstack/data-transfer/data-transfer-from-to-vm.md)
guide:

- **Globus** — Preferred method for large datasets; handles retries, error recovery,
and provides status updates. Set up a personal Globus endpoint on your VM.

- **SCP** — Secure copy for smaller files (<~10 GB) using SSH.

- **tar+ssh** — Stream a tar archive over SSH for efficient directory transfers.

- **rsync** — Fast, delta-transfer synchronization; ideal for repeat or incremental
transfers.

- **Rclone** — Sync, copy, or mount the VM's filesystem via SFTP for transfer to
any supported destination.

- **WinSCP** (Windows) — GUI-based SFTP/SCP client for drag-and-drop transfers.

- **Cyberduck** (macOS/Windows) — GUI-based SFTP/FTP client.

- **Filezilla** (cross-platform) — Open-source SFTP/FTP client.

!!! note "Data on Detached Volumes"

If you have a detached volume with data to migrate, attach it to a running VM
as described in the
[Attach a Volume to an Instance guide](../openstack/persistent-storage/attach-the-volume-to-an-instance.md),
then use any of the clients listed above to transfer the data from the VM to
your target location.

### Object Storage Data

Data stored in OpenStack Object Storage (Swift) containers can be migrated
using any of the clients documented in the
[Object Storage guide](../openstack/persistent-storage/object-storage.md):

- **Horizon Dashboard** — Download files through the web interface.

- **OpenStack CLI** — Use `openstack object store` commands to download objects.

- **Swift Interface** — Use the `swift` command-line client (`python-swiftclient`)
to download or sync data.

- **AWS CLI** — Use S3-compatible `aws s3` commands with the NERC endpoint.

- **s3cmd** — Sync or copy data from containers to your local system or another
S3 target.

- **Rclone** — Sync, copy, or mount object storage for transfer to any supported
destination.

- **Python libraries (Boto3, Minio)** — Programmatically download objects using
the S3 API.

- **GUI tools (WinSCP, Cyberduck)** — Connect via S3 protocol for a file-browser
experience.

!!! note "Mount Object Storage to a VM"

You can also mount your object storage container to a running instance as
described in the
[Mount The Object Storage to an Instance guide](../openstack/persistent-storage/mount-the-object-storage.md),
then use any of the clients listed in [Data Transfer from VMs](#data-transfer-from-vms)
to download the data from the VM, just like a mounted volume.

## Decommissioning

Once your data has been safely migrated, follow the
[Decommission Your NERC OpenStack Resources](../openstack/decommission/decommission-openstack-resources.md)
guide to:

1. Delete all VMs

2. Delete volumes and snapshots

3. Delete custom images

4. Delete private networks, routers, and interfaces

5. Release floating IPs

6. Clean up security groups

7. Delete key pairs

8. Delete buckets and objects

9. Remove users from your ColdFront project (optional)

10. Reduce storage quotas to zero

11. Review project resource quotas

12. Archive your ColdFront project

---
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ with your operational, security, and performance requirements.
- Once successfully initiated, click on the **minio** deployment and select
the "Resources" tab to review created *Pods*, *Services*, and *Routes*.

![MinIO Deployemnt Resources](images/minio-deployment-resources.png)
![MinIO Deployment Resources](images/minio-deployment-resources.png)

The **`minio-s3`** route URL (found under "Routes" -> `minio-s3` -> _Location_
path) is used to interact with the MinIO API **programmatically** and will
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -482,7 +482,7 @@ are created and grouped under "minio" application grouping on the **Workloads**
Once successfully initiated, click on the **minio** deployment and select the
"Resources" tab to review created *Pods*, *Services*, and *Routes*.

![MinIO Deployemnt Resources](images/minio-deployment-resources.png)
![MinIO Deployment Resources](images/minio-deployment-resources.png)

Please note the **minio-console** route URL by navigating to the "Routes" section
under the _Location_ path. When you click on the **minio-console** route URL, it
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -453,7 +453,7 @@ are created and grouped under "minio" application grouping on the **Workloads**
Once successfully initiated, click on the **minio** deployment and select the
"Resources" tab to review created *Pods*, *Services*, and *Routes*.

![MinIO Deployemnt Resources](images/minio-deployment-resources.png)
![MinIO Deployment Resources](images/minio-deployment-resources.png)

Please note the **minio-console** route URL by navigating to the "Routes" section
under the _Location_ path. When you click on the **minio-console** route URL, it
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ below.

- **Backup**: Back up any critical data or configurations stored on the resources
that going to be decommissioned. This ensures that important information is not
lost during the process.
lost during the process as [described here](../storage/data-transfer-from-to-openshift.md).

- **Kubernetes Objects (Resources)**: Please review all OpenShift Kubernetes Objects
(Resources) to ensure they are not actively used and ready to be decommissioned.
Expand Down
2 changes: 2 additions & 0 deletions docs/openshift/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,8 @@ the list below.

- [Rclone](storage/Rclone.md)

- [Data Transfer](storage/data-transfer-from-to-openshift.md)

## Deleting Applications

- [Deleting your applications](applications/deleting-applications.md)
Expand Down
Loading