cloud-images

Build commercial-grade custom cloud images from upstream Linux distribution images.

What this project does

This repository automates the image factory workflow for niche Linux distributions:

1. Download an official upstream cloud image.
2. Upload it to a cloud object store.
3. Import it as a custom image.
4. Boot a temporary instance from that image.
5. Provision cloud compatibility features such as cloud-init, Cloud Assistant, disk growth, and console settings.
6. Validate those capabilities before and after reboot.
7. Capture a final commercial custom image.

The current implementation is Alibaba Cloud first, with provider backends for AWS and Azure exposed through the same workflow and state model.

Current scope

• Implemented end-to-end workflow for Alibaba Cloud using OSS, ECS image import, Cloud Assistant, and CreateImage.
• Implemented provider backends for AWS (S3, VM Import, SSM, AMI capture) and Azure (Blob, managed image import, RunCommand, image capture).
• First distro family: Rocky Linux 9, AlmaLinux 9, Oracle Linux 9.
• Architectures: x86_64 and arm64.

Requirements

• Python 3.11+
• PyYAML
• Optional Alibaba Cloud dependencies for real cloud execution:

pip install -e '.[aliyun]'
pip install -e '.[aws]'
pip install -e '.[azure]'

• Alibaba Cloud credentials in environment variables:

export ALIBABA_CLOUD_ACCESS_KEY_ID=...
export ALIBABA_CLOUD_ACCESS_KEY_SECRET=...

Configuration

Data is split by ownership and lifecycle, and cloud-specific fields are kept out of the image catalog:

Path	Written by	Holds	Cloud-specific
configs/sources/*.yaml	hand	Upstream entry points (index URL, target versions/arches)	no
configs/distros/*.yaml	generated	Image catalog: version, release, arch, filename, url, sha256, format, boot_mode	no
configs/clouds/*.yaml	hand	Cloud targets: region, bucket, object_prefix, platform_map, instance types, roles	yes

cloudos distros-update <source> regenerates configs/distros/<source>.yaml from the source. Cloud destination details (for example the OSS object_prefix and the Alibaba platform_map) live in configs/clouds/ and are applied when the cloud migration list is produced.

When adding a new source or cloud target, start from configs/sources/_template.yaml or configs/clouds/_template.yaml instead of inventing the shape by hand.

CLI

Install the cloudos command once, then use it directly:

make cli          # installs cloudos onto PATH (/usr/local/bin)

cloudos --help
cloudos help
cloudos help distros-update
cloudos sources
cloudos distros-update oraclelinux
cloudos distros-update oraclelinux --select 1,3,5-7
cloudos distros-update oraclelinux --all
cloudos distros-update oraclelinux --check

• make cli symlinks bin/cloudos into /usr/local/bin (override with make cli BIN_DIR=...).
• sources lists every source config under configs/sources/.
• distros-update <source> resolves the upstream index and rewrites configs/distros/<source>.yaml.
• Without --select/--all, it prompts so you can pick which images to keep (multi-select).
• distros-update <source> --check exits non-zero when the file is out of date (for CI).

configs/sources/ and configs/distros/ are built-in seed data. They are not exposed as CLI options, so users only supply a source name.

The same CLI can also be run without installing, via ./bin/cloudos or python3 -m cloud_images.cloudos.

The older workflow command is still available:

python3 -m cloud_images.cli all \
  --cloud aliyun \
  --distro rocky \
  --version 9 \
  --arch x86_64 \
  --workdir .cloud-images

Individual stages are also available:

python3 -m cloud_images.cli fetch --cloud aliyun --distro rocky --version 9 --arch x86_64
python3 -m cloud_images.cli import --cloud aliyun --distro rocky --version 9 --arch x86_64
python3 -m cloud_images.cli launch --cloud aliyun --distro rocky --version 9 --arch x86_64
python3 -m cloud_images.cli provision --cloud aliyun --distro rocky --version 9 --arch x86_64
python3 -m cloud_images.cli validate --cloud aliyun --distro rocky --version 9 --arch x86_64
python3 -m cloud_images.cli capture --cloud aliyun --distro rocky --version 9 --arch x86_64

Testing

The repository uses the standard library test runner so it can run in minimal environments:

PYTHONPATH=src python3 -m unittest discover -s tests -v

Notes

• Alibaba Cloud image import requires the AliyunECSImageImportDefaultRole role and bucket access authorization.
• Pure online provisioning still depends on the imported image being able to boot on the target cloud.
• AWS and Azure provider code is present but was only verified with unit tests in this repository, not against live cloud accounts from this environment.