更新:
不要看下去了,这里有一个比本文更好的方法:
https://blog.dreamdusk.com/archives/571/
目录
[TOC]
说明
主要功能
将docker hub
中其他人的镜像备份到自己的仓库中,以防止其他人删除镜像或者仓库
如将{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}
中的镜像备份到{YOUR_DOCKERHUB_USERNAME}/{YOUR_DOCKERHUB_REPOSITORY}
中
比如其他人有一个镜像:{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}:其他人的标签
,你可以将其备份到自己的仓库中:{YOUR_DOCKERHUB_USERNAME}/{YOUR_DOCKERHUB_REPOSITORY}:{OTHER_DOCKERHUB_USERNAME}__{OTHER_DOCKERHUB_REPOSITORY}__其他人的标签
更具体一点的例子:
你想备份这个仓库bitnami/nginx
中的所有镜像到自己的仓库,运行脚本后会把bitnami/nginx:latest
下载、重命名并上传为你的名字/你的备份仓库:bitnami_nginx__latest
,不只是这一个镜像,仓库中所有的镜像都会备份到自己的仓库中。
并且支持备份一个或多个源仓库
到一个你的备份仓库
你要是问我给每个版本都更新了什么,说实话,我也不知道。
使用方法
新建一个.py
后缀的文件,将下面的代码复制进去,然后修改{}
内的内容(如backup_repo_prefix = "{YOUR_DOCKERHUB_USERNAME}/{YOUR_DOCKERHUB_REPOSITORY}"
改为backup_repo_prefix = "你的用户名/你的仓库名"
),然后运行即可。
用最新版本就行了:VERSION 17 (FINAL)
,旧的别用。醉了,FINAL一直从2增加到了15
其中有的使用的是环境变量来输入账号密码,需要在运行以前执行下列命令来设置环境变量
export DOCKERHUB_USERNAME={YOUR_DOCKERHUB_USERNAME}
export DOCKERHUB_PASSWORD={YOUR_DOCKERHUB_PASSWORD}
所有版本
主要
VERSION 17 (FINAL)
上一个版本中,每备份完一个tag都要删除对应的镜像,删除镜像时,也会删除相关的文件和依赖项。
而这个版本中,每备份完一个仓库中的所有tag,才会删除这个仓库中的所有镜像的相关的文件和依赖项。
要知道同一个仓库中的镜像有很多文件和依赖项是相同的,所以仓库中的其他tag在备份时,也需要这些被删除的文件和依赖项,检测到缺少文件和依赖项以后,又会重新下载这些文件和依赖项,造成了重复下载,资源浪费。
这次修改了删除镜像的逻辑,在备份完某个仓库中的所有镜像时,才开始删除文件和依赖项,避免了浪费。
但是不足的是,没有指定要删除哪个文件和依赖项,而是使用docker image prune -af
直接删除了所有的镜像。造成的后果就是,如果在这台机器上,有和本次备份无关的、没有被正在使用的镜像,也会被删除。这个问题还没有解决。
并且这次需要的硬盘空间也比上次多,上次需要的是空间至少要有所有仓库中的所有镜像中最大的那个镜像的大小,而这次需要的是所有仓库中所占空间最大的那个仓库所占空间的大小。(以前只要一个最大的镜像,现在要的是一个最大的仓库,仓库中可能包含有多个镜像)
吧
上一个版本中,是靠这块来删除指定镜像的:
# 检查 正在运行的容器中 是否有容器正在使用 准备删除的镜像
running_containers = client.containers.list()
for container in running_containers:
if old_image_name in container.image.tags:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"{old_image_name} 对应的容器 {container.name} 正在运行,无法删除该镜像")
break # 如果有一个容器正在使用该镜像,则跳出循环,不进行镜像删除操作
else: # 如果没有任何容器使用该镜像,则执行镜像删除操作
# 删除已备份的原始镜像改名后的新镜像(也即是备份到自己仓库中的镜像)
client.images.remove(backup_repo_prefix + ":" + new_tag, force=True) #注意,使用 force=True 参数可以强制删除镜像,即使该镜像仍然被其他容器所使用。
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"删除改名后的镜像 {backup_repo_prefix}:{new_tag} 成功")
# 删除已备份的原始镜像
client.images.remove(old_image_name, force=True) #注意,使用 force=True 参数可以强制删除镜像,即使该镜像仍然被其他容器所使用。
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"删除原始镜像 {old_image_name} 成功")
这次改成了,并且将代码的位置向前缩进了两层
# 清理不再使用的镜像
# 获取当前时间并格式化为需要的字符串形式
current_time = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3]
# 输出开始清理镜像的提示
print(f'{current_time} 开始清理不再使用的镜像')
# 定义清理镜像的命令
cmd = ['docker', 'image', 'prune', '-af']
# 启动进程并捕获标准输出和标准错误流
with subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) as proc:
# 遍历标准输出流中的每一行,添加时间戳,并输出
for line in iter(proc.stdout.readline, b''):
# 获取当前时间戳并格式化为需要的字符串形式
timestamp = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3]
# 检查当前行是否为空行,如果不是,则输出时间戳和内容
if line.strip() != b'':
print(f'{timestamp} {line.decode("utf-8").strip().replace("Deleted", "已删除").replace("Images:", "镜像:").replace("Untagged", "未标记").replace("Total reclaimed space", "总共回收空间")}')
# 遍历标准错误流中的每一行,添加时间戳,并输出
for line in iter(proc.stderr.readline, b''):
# 获取当前时间戳并格式化为需要的字符串形式
timestamp = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3]
# 检查当前行是否为空行,如果不是,则输出时间戳和内容
if line.strip() != b'':
print(f'{timestamp} 错误:{line.decode("utf-8").strip().replace("Error", "错误").replace("response from daemon:", "来自守护进程的响应:").replace("Conflict.", "冲突。").replace("The container name", "容器名称").replace("is already in use by container", "已被容器占用").replace("You have to remove (or rename) that container to be able to reuse that name.", "您必须删除(或重命名)该容器才能重新使用该名称。")}')
# 输出清理镜像完成的提示
current_time = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3]
print(f'{current_time} 无用镜像清理完成')
以下是这个版本的代码:
#!/usr/bin/env python3
import requests
from concurrent.futures import ThreadPoolExecutor, as_completed
import docker
import subprocess
import shutil
import datetime
# 定义需要备份的镜像列表和备份后的仓库前缀
repos_to_backup = [
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}",
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}"
]
# 自己的仓库的路径,也就是备份的目的地
backup_repo_prefix = "{YOUR_DOCKERHUB_USERNAME}/{YOUR_DOCKERHUB_REPOSITORY}"
# 定义dockerhub的用户名和密码
DOCKERHUB_USERNAME = '{YOUR_DOCKERHUB_USERNAME}'
DOCKERHUB_PASSWORD = "{YOUR_DOCKERHUB_PASSWORD}"
# 定义一个函数,用于检查需要的库是否已经安装,并自动安装未安装的库
def check_and_install_libraries(packages):
"""
检查需要的库是否已经安装,并自动安装未安装的库
:param packages: 需要检查和安装的库列表
:return: True表示所有库都已成功安装,False表示至少有一个库未能成功安装
"""
try:
import importlib.util
except ImportError:
subprocess.check_call([sys.executable, "-m", "pip", "install", "importlib"])
# 遍历库列表中的每个库
for package in packages:
# 尝试导入该库
spec = importlib.util.find_spec(package)
if spec is not None:
continue
# 使用subprocess调用pip工具安装该库
subprocess.check_call([sys.executable, "-m", "pip", "install", package])
# 再次遍历库列表中的每个库
for package in packages:
# 尝试导入该库
spec = importlib.util.find_spec(package)
# 如果导入失败,则输出相应提示信息到控制台
if spec is None:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" 未能成功安装 {package}")
# 返回False表示至少有一个库未能成功安装
return False
# 如果所有库都被成功安装,则返回True
return True
# 调用此函数以打印自适应屏幕宽度的分隔线
def print_separator(char="-"):
# 获取终端屏幕的宽度
terminal_width = shutil.get_terminal_size().columns
# 使用指定字符创建一个自适应的分隔符
separator = char * terminal_width
# 打印分隔符
print(separator)
# 分隔线
print_separator("*")
# 运行开始备份前的前置工作
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" 运行开始备份前的前置工作")
# 运行库检测
packages = ["requests", "concurrent.futures", "docker", "subprocess", "datetime", "shutil"]
# 调用check_and_install_libraries函数,传入需要检查和安装的库列表
success = check_and_install_libraries(packages)
# 根据返回结果打印相应的提示信息到控制台
if success:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + " 所有库都已成功安装。")
else:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + " 至少有一个库未能成功安装。")
# 获取指定仓库的所有标签
def get_all_tags(repo, session):
print_separator("*")
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" 开始备份仓库 {repo}:")
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" 获取{repo}仓库中的所有标签")
url = f"https://registry.hub.docker.com/v2/repositories/{repo}/tags/"
all_tags = []
next_page = url
while next_page is not None:
response = session.get(next_page)
response.raise_for_status()
data = response.json()
tags = [t["name"] for t in data["results"]]
all_tags += tags
next_page = data.get('next', None)
for tag in all_tags:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" 发现标签: {tag}")
yield tag
# 登录到 Docker Hub
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + " 登录 Docker Hub中")
client = docker.from_env()
client.login(username=DOCKERHUB_USERNAME, password=DOCKERHUB_PASSWORD)
# 获取我的仓库中已备份的所有镜像
# 构造 API URL
api_url = f'https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/'
# 存储所有标签
existing_images = []
# 初始请求链接为 API URL
next_page = api_url
# 当 next_page 不为 None 时继续循环
while next_page is not None:
# 发送 GET 请求获取响应
response = requests.get(next_page)
# 将 JSON 数据转换成 Python 对象(字典)
data = response.json()
# 获取本次请求返回的所有标签数据
tags = [result['name'] for result in data['results']]
# 将本次请求返回的标签数据存入 existing_images 列表
existing_images += tags
# 如果还有下一页,则更新 next_page 的值为下一页链接
next_page = data.get('next', None)
# 对现有镜像列表进行排序
existing_images_sorted = sorted(existing_images)
#以前备份过的镜像的数量,方便最后输出调用
backuped_lastname = len(existing_images_sorted)
# 打印需要备份的仓库的数量
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" 需备份的仓库数量: {len(repos_to_backup)}")
# 打印需要备份的仓库的名字
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + " 它们分别是: {}".format(repos_to_backup))
# 遍历 existing_images_sorted 列表并打印每个现有镜像
print("\n"+datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + ' 获取以前备份过的镜像列表:')
number = 0
for image in existing_images_sorted:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], image)
number+=1
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f' 共{number}个\n')
# 统计备份结果信息
backed_up_images = []
failed_images = []
total_new_images_to_backup = 0
total_failed_images = 0
#前置工作结束,进入正式备份流程
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f"前置工作结束,进入正式备份流程")
# 打印分隔符
print_separator("*")
# 并行处理每个镜像的备份任务
with ThreadPoolExecutor() as executor:
futures = []
with requests.Session() as session:
for repo in repos_to_backup:
# 获取给定镜像的所有标签
future = executor.submit(get_all_tags, repo, session)
futures.append(future)
for future in as_completed(futures):
repo = repos_to_backup[futures.index(future)]
tags_to_backup = list(future.result())
if len(tags_to_backup) == 0:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" {repo}没有标签,跳过。")
continue
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" {repo}的所有标签分(共{len(tags_to_backup)}个): {', '.join(tags_to_backup)}")
# 并行处理每个标签的备份任务
tag_futures = []
tag_number = 0 # 标签序号
for tag in tags_to_backup:
tag_number += 1 # 标签序号+1
# 构建原始镜像名称和标签
old_image_name = f"{repo}:{tag}"
# 构建新的镜像名称和标签
new_image_name = f"{repo.split('/')[0]}__{repo.split('/')[1]}__{tag}"
new_tag = new_image_name
# 检查是否已经备份过这个镜像
print("\n" + datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], "开始处理第 {} 个标签:{}".format(tag_number, tag))
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], "开始对比是否需要备份 {}".format(old_image_name))
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"以前备份过的镜像: (共{number}个)")
number = 0
for image in existing_images_sorted:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], image)
number+=1
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], '准备要备份的镜像:')
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], new_image_name)
if new_image_name in existing_images_sorted:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], '"准备要备份的镜像"已包含在"以前备份过的镜像"中')
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"{new_image_name} 镜像已备份,跳过。")
continue
# 打印源仓库和备份仓库的对比信息
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], '"以前备份过的镜像"中没有找到"准备要备份的镜像"')
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], '开始备份:')
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"{old_image_name} 将被备份为 {new_image_name} 到 {backup_repo_prefix}:{new_tag}")
# 备份 Docker 镜像
new_image = f"{backup_repo_prefix}:{new_tag}"
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"正在备份 {old_image_name} 到 {new_image_name}")
try:
arch_list = [
"linux/amd64",
"linux/arm64/v8",
"linux/arm/v6",
"linux/arm/v7",
"linux/ppc64le",
"linux/386",
"linux/s390x",
"windows/amd64",
"windows/386",
"darwin/amd64",
"darwin/386"
]
for arch in arch_list:
try:
image = client.images.pull(old_image_name, platform=arch)
break
except docker.errors.ImageNotFound:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"无法在默认 registry 中找到 {old_image_name} 的 {arch} 版本,尝试其他架构")
except docker.errors.APIError as e:
if 'no matching manifest' in str(e).lower():
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"{old_image_name} 不支持 {arch} 架构")
else:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"API 错误:{str(e)}")
if not image:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"无法从任何 registry 中拉取 {old_image_name}")
image.tag(new_image)
client.images.push(new_image)
backed_up_images.append(new_image_name)
total_new_images_to_backup += 1
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"备份镜像 {old_image_name} 到 {backup_repo_prefix}:{new_tag} 成功")
except Exception as e:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"备份镜像 {old_image_name} 失败")
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"失败原因: {str(e)}")
failed_images.append(new_image_name)
total_failed_images += 1
# 删除已备份的标签
tags_to_backup.remove(tag)
# 清理不再使用的镜像
# 获取当前时间并格式化为需要的字符串形式
current_time = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3]
# 输出开始清理镜像的提示
print(f'{current_time} 开始清理不再使用的镜像')
# 定义清理镜像的命令
cmd = ['docker', 'image', 'prune', '-af']
# 启动进程并捕获标准输出和标准错误流
with subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) as proc:
# 遍历标准输出流中的每一行,添加时间戳,并输出
for line in iter(proc.stdout.readline, b''):
# 获取当前时间戳并格式化为需要的字符串形式
timestamp = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3]
# 检查当前行是否为空行,如果不是,则输出时间戳和内容
if line.strip() != b'':
print(f'{timestamp} {line.decode("utf-8").strip().replace("Deleted", "已删除").replace("Images:", "镜像:").replace("Untagged", "未标记").replace("Total reclaimed space", "总共回收空间")}')
# 遍历标准错误流中的每一行,添加时间戳,并输出
for line in iter(proc.stderr.readline, b''):
# 获取当前时间戳并格式化为需要的字符串形式
timestamp = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3]
# 检查当前行是否为空行,如果不是,则输出时间戳和内容
if line.strip() != b'':
print(f'{timestamp} 错误:{line.decode("utf-8").strip().replace("Error", "错误").replace("response from daemon:", "来自守护进程的响应:").replace("Conflict.", "冲突。").replace("The container name", "容器名称").replace("is already in use by container", "已被容器占用").replace("You have to remove (or rename) that container to be able to reuse that name.", "您必须删除(或重命名)该容器才能重新使用该名称。")}')
# 输出清理镜像完成的提示
current_time = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3]
print(f'{current_time} 无用镜像清理完成')
#打印仓库备份成功信息
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"仓库 {repo} 中所有标签备份成功")
#打印分隔符
print_separator("*")
# 清空标签列表
del tags_to_backup[:]
# 再次获取我的仓库中已备份的所有镜像
# 构造 API URL
api_url = f'https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/'
# 存储所有标签
existing_images = []
# 初始请求链接为 API URL
next_page = api_url
# 当 next_page 不为 None 时继续循环
while next_page is not None:
# 发送 GET 请求获取响应
response = requests.get(next_page)
# 将 JSON 数据转换成 Python 对象(字典)
data = response.json()
# 获取本次请求返回的所有标签数据
tags = [result['name'] for result in data['results']]
# 将本次请求返回的标签数据存入 existing_images 列表
existing_images += tags
# 如果还有下一页,则更新 next_page 的值为下一页链接
next_page = data.get('next', None)
# 对现有镜像列表进行排序
existing_images_sorted = sorted(existing_images)
# 总结本次备份
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"完成所有备份流程,开始总结本次备份:")
# 遍历 existing_images_sorted 列表并打印每个现有镜像
number = 0 # 镜像数量计数器
# 第一次循环先获取目前备份过的镜像数量
for image in existing_images_sorted:
number+=1
print("\n" + datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + ' 目前备份过的镜像(共 {} 个):'.format(str(number)))
# 第二次循环打印镜像列表
for image in existing_images_sorted:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], image)
print("\n")
# 打印备份信息
total_backed_up_images = len(backed_up_images)
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"本次备份前总共备份过 {backuped_lastname} 个镜像。")
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"本次总共需要备份 {total_new_images_to_backup} 个新的镜像。")
if total_backed_up_images > 0:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"实际备份的所有镜像为 {', '.join(backed_up_images)},共备份了 {total_backed_up_images} 个镜像。")
else:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], "没有备份任何镜像。")
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"经过本次备份后,已备份的所有镜像数量由 {backuped_lastname} 个变为 {number} 个。")
# 打印备份失败的镜像信息
total_failed_images = len(failed_images)
if total_failed_images > 0:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"备份失败的镜像: {', '.join(failed_images)},共{total_failed_images}个。")
else:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], "所有镜像备份成功!")
次要
VERSION 16
#!/usr/bin/env python3
import requests
from concurrent.futures import ThreadPoolExecutor, as_completed
import docker
import subprocess
import datetime
import shutil
# 定义需要备份的镜像列表和备份后的仓库前缀
repos_to_backup = [
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}",
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}"
]
# 自己的仓库的路径,也就是备份的目的地
backup_repo_prefix = "{YOUR_DOCKERHUB_USERNAME}/{YOUR_DOCKERHUB_REPOSITORY}"
# 定义dockerhub的用户名和密码
DOCKERHUB_USERNAME = '{YOUR_DOCKERHUB_USERNAME}'
DOCKERHUB_PASSWORD = "{YOUR_DOCKERHUB_PASSWORD}"
# 定义一个函数,用于检查需要的库是否已经安装,并自动安装未安装的库
def check_and_install_libraries(packages):
"""
检查需要的库是否已经安装,并自动安装未安装的库
:param packages: 需要检查和安装的库列表
:return: True表示所有库都已成功安装,False表示至少有一个库未能成功安装
"""
try:
import importlib.util
except ImportError:
subprocess.check_call([sys.executable, "-m", "pip", "install", "importlib"])
# 遍历库列表中的每个库
for package in packages:
# 尝试导入该库
spec = importlib.util.find_spec(package)
if spec is not None:
continue
# 使用subprocess调用pip工具安装该库
subprocess.check_call([sys.executable, "-m", "pip", "install", package])
# 再次遍历库列表中的每个库
for package in packages:
# 尝试导入该库
spec = importlib.util.find_spec(package)
# 如果导入失败,则输出相应提示信息到控制台
if spec is None:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" 未能成功安装 {package}")
# 返回False表示至少有一个库未能成功安装
return False
# 如果所有库都被成功安装,则返回True
return True
# 调用此函数以打印自适应屏幕宽度的分隔线
def print_separator(char="-"):
# 获取终端屏幕的宽度
terminal_width = shutil.get_terminal_size().columns
# 使用指定字符创建一个自适应的分隔符
separator = char * terminal_width
# 打印分隔符
print(separator)
# 分隔线
print_separator("*")
# 运行开始备份前的前置工作
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" 运行开始备份前的前置工作")
# 运行库检测
packages = ["requests", "concurrent.futures", "docker", "subprocess", "datetime", "shutil"]
# 调用check_and_install_libraries函数,传入需要检查和安装的库列表
success = check_and_install_libraries(packages)
# 根据返回结果打印相应的提示信息到控制台
if success:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + " 所有库都已成功安装。")
else:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + " 至少有一个库未能成功安装。")
# 获取指定仓库的所有标签
def get_all_tags(repo, session):
print_separator("*")
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" 开始备份仓库 {repo}:")
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" 获取{repo}仓库中的所有标签")
url = f"https://registry.hub.docker.com/v2/repositories/{repo}/tags/"
all_tags = []
next_page = url
while next_page is not None:
response = session.get(next_page)
response.raise_for_status()
data = response.json()
tags = [t["name"] for t in data["results"]]
all_tags += tags
next_page = data.get('next', None)
for tag in all_tags:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" 发现标签: {tag}")
yield tag
# 登录到 Docker Hub
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + " 登录 Docker Hub中")
client = docker.from_env()
client.login(username=DOCKERHUB_USERNAME, password=DOCKERHUB_PASSWORD)
# 获取我的仓库中已备份的所有镜像
# 构造 API URL
api_url = f'https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/'
# 存储所有标签
existing_images = []
# 初始请求链接为 API URL
next_page = api_url
# 当 next_page 不为 None 时继续循环
while next_page is not None:
# 发送 GET 请求获取响应
response = requests.get(next_page)
# 将 JSON 数据转换成 Python 对象(字典)
data = response.json()
# 获取本次请求返回的所有标签数据
tags = [result['name'] for result in data['results']]
# 将本次请求返回的标签数据存入 existing_images 列表
existing_images += tags
# 如果还有下一页,则更新 next_page 的值为下一页链接
next_page = data.get('next', None)
# 对现有镜像列表进行排序
existing_images_sorted = sorted(existing_images)
#以前备份过的镜像的数量,方便最后输出调用
backuped_lastname = len(existing_images_sorted)
# 打印需要备份的仓库的数量
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" 需备份的仓库数量: {len(repos_to_backup)}")
# 打印需要备份的仓库的名字
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + " 它们分别是: {}".format(repos_to_backup))
# 遍历 existing_images_sorted 列表并打印每个现有镜像
print("\n"+datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + ' 获取以前备份过的镜像列表:')
number = 0
for image in existing_images_sorted:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], image)
number+=1
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f' 共{number}个\n')
# 统计备份结果信息
backed_up_images = []
failed_images = []
total_new_images_to_backup = 0
total_failed_images = 0
#前置工作结束,进入正式备份流程
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f"前置工作结束,进入正式备份流程")
# 打印分隔符
print_separator("*")
# 并行处理每个镜像的备份任务
with ThreadPoolExecutor() as executor:
futures = []
with requests.Session() as session:
for repo in repos_to_backup:
# 获取给定镜像的所有标签
future = executor.submit(get_all_tags, repo, session)
futures.append(future)
for future in as_completed(futures):
repo = repos_to_backup[futures.index(future)]
tags_to_backup = list(future.result())
if len(tags_to_backup) == 0:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" {repo}没有标签,跳过。")
continue
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" {repo}的所有标签分(共{len(tags_to_backup)}个): {', '.join(tags_to_backup)}")
# 并行处理每个标签的备份任务
tag_futures = []
tag_number = 0 # 标签序号
for tag in tags_to_backup:
tag_number += 1 # 标签序号+1
# 构建原始镜像名称和标签
old_image_name = f"{repo}:{tag}"
# 构建新的镜像名称和标签
new_image_name = f"{repo.split('/')[0]}__{repo.split('/')[1]}__{tag}"
new_tag = new_image_name
# 检查是否已经备份过这个镜像
print("\n" + datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], "开始处理第 {} 个标签:{}".format(tag_number, tag))
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], "开始对比是否需要备份 {}".format(old_image_name))
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"以前备份过的镜像: (共{number}个)")
number = 0
for image in existing_images_sorted:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], image)
number+=1
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], '准备要备份的镜像:')
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], new_image_name)
if new_image_name in existing_images_sorted:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], '"准备要备份的镜像"已包含在"以前备份过的镜像"中')
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"{new_image_name} 镜像已备份,跳过。")
continue
# 打印源仓库和备份仓库的对比信息
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], '"以前备份过的镜像"中没有找到"准备要备份的镜像"')
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], '开始备份:')
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"{old_image_name} 将被备份为 {new_image_name} 到 {backup_repo_prefix}:{new_tag}")
# 备份 Docker 镜像
new_image = f"{backup_repo_prefix}:{new_tag}"
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"正在备份 {old_image_name} 到 {new_image_name}")
try:
arch_list = [
"linux/amd64",
"linux/arm64/v8",
"linux/arm/v6",
"linux/arm/v7",
"linux/ppc64le",
"linux/386",
"linux/s390x",
"windows/amd64",
"windows/386",
"darwin/amd64",
"darwin/386"
]
for arch in arch_list:
try:
image = client.images.pull(old_image_name, platform=arch)
break
except docker.errors.ImageNotFound:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"无法在默认 registry 中找到 {old_image_name} 的 {arch} 版本,尝试其他架构")
except docker.errors.APIError as e:
if 'no matching manifest' in str(e).lower():
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"{old_image_name} 不支持 {arch} 架构")
else:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"API 错误:{str(e)}")
if not image:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"无法从任何 registry 中拉取 {old_image_name}")
image.tag(new_image)
client.images.push(new_image)
backed_up_images.append(new_image_name)
total_new_images_to_backup += 1
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"备份镜像 {old_image_name} 到 {backup_repo_prefix}:{new_tag} 成功")
# 检查 正在运行的容器中 是否有容器正在使用 准备删除的镜像
running_containers = client.containers.list()
for container in running_containers:
if old_image_name in container.image.tags:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"{old_image_name} 对应的容器 {container.name} 正在运行,无法删除该镜像")
break # 如果有一个容器正在使用该镜像,则跳出循环,不进行镜像删除操作
else: # 如果没有任何容器使用该镜像,则执行镜像删除操作
# 删除已备份的原始镜像改名后的新镜像(也即是备份到自己仓库中的镜像)
client.images.remove(backup_repo_prefix + ":" + new_tag, force=True) #注意,使用 force=True 参数可以强制删除镜像,即使该镜像仍然被其他容器所使用。
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"删除改名后的镜像 {backup_repo_prefix}:{new_tag} 成功")
# 删除已备份的原始镜像
client.images.remove(old_image_name, force=True) #注意,使用 force=True 参数可以强制删除镜像,即使该镜像仍然被其他容器所使用。
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"删除原始镜像 {old_image_name} 成功")
except Exception as e:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"备份镜像 {old_image_name} 失败")
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"失败原因: {str(e)}")
failed_images.append(new_image_name)
total_failed_images += 1
# 删除已备份的标签
tags_to_backup.remove(tag)
#打印仓库备份成功信息
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"仓库 {repo} 中所有标签备份成功")
#打印分隔符
print_separator("*")
# 清空标签列表
del tags_to_backup[:]
# 再次获取我的仓库中已备份的所有镜像
# 构造 API URL
api_url = f'https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/'
# 存储所有标签
existing_images = []
# 初始请求链接为 API URL
next_page = api_url
# 当 next_page 不为 None 时继续循环
while next_page is not None:
# 发送 GET 请求获取响应
response = requests.get(next_page)
# 将 JSON 数据转换成 Python 对象(字典)
data = response.json()
# 获取本次请求返回的所有标签数据
tags = [result['name'] for result in data['results']]
# 将本次请求返回的标签数据存入 existing_images 列表
existing_images += tags
# 如果还有下一页,则更新 next_page 的值为下一页链接
next_page = data.get('next', None)
# 对现有镜像列表进行排序
existing_images_sorted = sorted(existing_images)
# 总结本次备份
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"完成所有备份流程,开始总结本次备份:")
# 遍历 existing_images_sorted 列表并打印每个现有镜像
number = 0 # 镜像数量计数器
# 第一次循环先获取目前备份过的镜像数量
for image in existing_images_sorted:
number+=1
print("\n" + datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + ' 目前备份过的镜像(共 {} 个):'.format(str(number)))
# 第二次循环打印镜像列表
for image in existing_images_sorted:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], image)
print("\n")
# 打印备份信息
total_backed_up_images = len(backed_up_images)
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"本次备份前总共备份过 {backuped_lastname} 个镜像。")
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"本次总共需要备份 {total_new_images_to_backup} 个新的镜像。")
if total_backed_up_images > 0:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"实际备份的所有镜像为 {', '.join(backed_up_images)},共备份了 {total_backed_up_images} 个镜像。")
else:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], "没有备份任何镜像。")
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"经过本次备份后,已备份的所有镜像数量由 {backuped_lastname} 个变为 {number} 个。")
# 打印备份失败的镜像信息
total_failed_images = len(failed_images)
if total_failed_images > 0:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"备份失败的镜像: {', '.join(failed_images)},共{total_failed_images}个。")
else:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], "所有镜像备份成功!")
VERSION 15
#!/usr/bin/env python3
import requests
from concurrent.futures import ThreadPoolExecutor, as_completed
import docker
import subprocess
import datetime
import shutil
# 定义需要备份的镜像列表和备份后的仓库前缀
repos_to_backup = [
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}",
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}"
]
# 自己的仓库的路径,也就是备份的目的地
backup_repo_prefix = "{YOUR_DOCKERHUB_USERNAME}/{YOUR_DOCKERHUB_REPOSITORY}"
# 定义dockerhub的用户名和密码
DOCKERHUB_USERNAME = '{YOUR_DOCKERHUB_USERNAME}'
DOCKERHUB_PASSWORD = "{YOUR_DOCKERHUB_PASSWORD}"
# 检查需要的库是否已经安装,并自动安装未安装的库
try:
import importlib.util
except ImportError:
subprocess.check_call([sys.executable, "-m", "pip", "install", "importlib"])
def check_and_install_libraries(packages):
"""
检查需要的库是否已经安装,并自动安装未安装的库
:param packages: 需要检查和安装的库列表
:return: True表示所有库都已成功安装,False表示至少有一个库未能成功安装
"""
# 遍历库列表中的每个库
for package in packages:
# 尝试导入该库
spec = importlib.util.find_spec(package)
if spec is not None:
continue
# 使用subprocess调用pip工具安装该库
subprocess.check_call([sys.executable, "-m", "pip", "install", package])
# 再次遍历库列表中的每个库
for package in packages:
# 尝试导入该库
spec = importlib.util.find_spec(package)
# 如果导入失败,则输出相应提示信息到控制台
if spec is None:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" 未能成功安装 {package}")
# 返回False表示至少有一个库未能成功安装
return False
# 如果所有库都被成功安装,则返回True
return True
try:
import importlib.util
except ImportError:
subprocess.check_call([sys.executable, "-m", "pip", "install", "importlib"])
def check_and_install_libraries(packages):
"""
检查需要的库是否已经安装,并自动安装未安装的库
:param packages: 需要检查和安装的库列表
:return: True表示所有库都已成功安装,False表示至少有一个库未能成功安装
"""
# 遍历库列表中的每个库
for package in packages:
# 尝试导入该库
spec = importlib.util.find_spec(package)
if spec is not None:
continue
# 使用subprocess调用pip工具安装该库
subprocess.check_call(["pip", "install", package])
# 再次遍历库列表中的每个库
for package in packages:
# 尝试导入该库
try:
__import__(package)
# 如果导入失败,则输出相应提示信息到控制台
except ImportError:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" 未能成功安装 {package}")
# 返回False表示至少有一个库未能成功安装
return False
# 如果所有库都被成功安装,则返回True
return True
# 调用此函数以打印自适应屏幕宽度的分隔线
def print_separator(char="-"):
# 获取终端屏幕的宽度
terminal_width = shutil.get_terminal_size().columns
# 使用指定字符创建一个自适应的分隔符
separator = char * terminal_width
# 打印分隔符
print(separator)
# 分隔线
print_separator("*")
# 运行开始备份前的前置工作
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f"运行开始备份前的前置工作")
# 运行库检测
packages = ["requests", "concurrent.futures", "docker"]
# 调用check_and_install_libraries函数,传入需要检查和安装的库列表
success = check_and_install_libraries(packages)
# 根据返回结果打印相应的提示信息到控制台
if success:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + " 所有库都已成功安装。")
else:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + " 至少有一个库未能成功安装。")
# 获取指定仓库的所有标签
def get_all_tags(repo, session):
print_separator("*")
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" 开始备份仓库 {repo}:")
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" 获取{repo}仓库中的所有标签")
url = f"https://registry.hub.docker.com/v2/repositories/{repo}/tags/"
all_tags = []
next_page = url
while next_page is not None:
response = session.get(next_page)
response.raise_for_status()
data = response.json()
tags = [t["name"] for t in data["results"]]
all_tags += tags
next_page = data.get('next', None)
for tag in all_tags:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" 发现标签: {tag}")
yield tag
# 登录到 Docker Hub
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + " 登录 Docker Hub中")
client = docker.from_env()
client.login(username=DOCKERHUB_USERNAME, password=DOCKERHUB_PASSWORD)
# 获取我的仓库中已备份的所有镜像
# 构造 API URL
api_url = f'https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/'
# 存储所有标签
existing_images = []
# 初始请求链接为 API URL
next_page = api_url
# 当 next_page 不为 None 时继续循环
while next_page is not None:
# 发送 GET 请求获取响应
response = requests.get(next_page)
# 将 JSON 数据转换成 Python 对象(字典)
data = response.json()
# 获取本次请求返回的所有标签数据
tags = [result['name'] for result in data['results']]
# 将本次请求返回的标签数据存入 existing_images 列表
existing_images += tags
# 如果还有下一页,则更新 next_page 的值为下一页链接
next_page = data.get('next', None)
# 对现有镜像列表进行排序
existing_images_sorted = sorted(existing_images)
#以前备份过的镜像的数量,方便最后输出调用
backuped_lastname = len(existing_images_sorted)
# 打印需要备份的仓库的数量
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" 需备份的仓库数量: {len(repos_to_backup)}")
# 打印需要备份的仓库的名字
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + " 它们分别是: {}".format(repos_to_backup))
# 遍历 existing_images_sorted 列表并打印每个现有镜像
print("\n"+datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + ' 获取以前备份过的镜像列表:')
number = 0
for image in existing_images_sorted:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], image)
number+=1
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f' 共{number}个\n')
# 统计备份结果信息
backed_up_images = []
failed_images = []
total_new_images_to_backup = 0
total_failed_images = 0
#前置工作结束,进入正式备份流程
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f"前置工作结束,进入正式备份流程")
# 打印分隔符
print_separator("*")
# 并行处理每个镜像的备份任务
with ThreadPoolExecutor() as executor:
futures = []
with requests.Session() as session:
for repo in repos_to_backup:
# 获取给定镜像的所有标签
future = executor.submit(get_all_tags, repo, session)
futures.append(future)
for future in as_completed(futures):
repo = repos_to_backup[futures.index(future)]
tags_to_backup = list(future.result())
if len(tags_to_backup) == 0:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" {repo}没有标签,跳过。")
continue
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" {repo}的所有标签分(共{len(tags_to_backup)}个): {', '.join(tags_to_backup)}")
# 并行处理每个标签的备份任务
tag_futures = []
tag_number = 0 # 标签序号
for tag in tags_to_backup:
tag_number += 1 # 标签序号+1
# 构建原始镜像名称和标签
old_image_name = f"{repo}:{tag}"
# 构建新的镜像名称和标签
new_image_name = f"{repo.split('/')[0]}__{repo.split('/')[1]}__{tag}"
new_tag = new_image_name
# 检查是否已经备份过这个镜像
print("\n" + datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], "开始处理第 {} 个标签:{}".format(tag_number, tag))
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], "开始对比是否需要备份 {}".format(old_image_name))
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"以前备份过的镜像: (共{number}个)")
number = 0
for image in existing_images_sorted:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], image)
number+=1
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], '准备要备份的镜像:')
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], new_image_name)
if new_image_name in existing_images_sorted:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], '"准备要备份的镜像"已包含在"以前备份过的镜像"中')
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"{new_image_name} 镜像已备份,跳过。")
continue
# 打印源仓库和备份仓库的对比信息
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], '"以前备份过的镜像"中没有找到"准备要备份的镜像"')
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], '开始备份:')
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"{old_image_name} 将被备份为 {new_image_name} 到 {backup_repo_prefix}:{new_tag}")
# 备份 Docker 镜像
new_image = f"{backup_repo_prefix}:{new_tag}"
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"正在备份 {old_image_name} 到 {new_image_name}")
try:
arch_list = [
"linux/amd64",
"linux/arm64/v8",
"linux/arm/v6",
"linux/arm/v7",
"linux/ppc64le",
"linux/386",
"linux/s390x",
"windows/amd64",
"windows/386",
"darwin/amd64",
"darwin/386"
]
for arch in arch_list:
try:
image = client.images.pull(old_image_name, platform=arch)
break
except docker.errors.ImageNotFound:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"无法在默认 registry 中找到 {old_image_name} 的 {arch} 版本,尝试其他架构")
except docker.errors.APIError as e:
if 'no matching manifest' in str(e).lower():
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"{old_image_name} 不支持 {arch} 架构")
else:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"API 错误:{str(e)}")
if not image:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"无法从任何 registry 中拉取 {old_image_name}")
image.tag(new_image)
client.images.push(new_image)
backed_up_images.append(new_image_name)
total_new_images_to_backup += 1
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"备份镜像 {old_image_name} 到 {backup_repo_prefix}:{new_tag} 成功")
# 检查 正在运行的容器中 是否有容器正在使用 准备删除的镜像
running_containers = client.containers.list()
for container in running_containers:
if old_image_name in container.image.tags:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"{old_image_name} 对应的容器 {container.name} 正在运行,无法删除该镜像")
break # 如果有一个容器正在使用该镜像,则跳出循环,不进行镜像删除操作
else: # 如果没有任何容器使用该镜像,则执行镜像删除操作
# 删除已备份的原始镜像改名后的新镜像(也即是备份到自己仓库中的镜像)
client.images.remove(backup_repo_prefix + ":" + new_tag, force=True) #注意,使用 force=True 参数可以强制删除镜像,即使该镜像仍然被其他容器所使用。
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"删除改名后的镜像 {backup_repo_prefix}:{new_tag} 成功")
# 删除已备份的原始镜像
client.images.remove(old_image_name, force=True) #注意,使用 force=True 参数可以强制删除镜像,即使该镜像仍然被其他容器所使用。
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"删除原始镜像 {old_image_name} 成功")
except Exception as e:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"备份镜像 {old_image_name} 失败")
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"失败原因: {str(e)}")
failed_images.append(new_image_name)
total_failed_images += 1
# 删除已备份的标签
tags_to_backup.remove(tag)
#打印仓库备份成功信息
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"仓库 {repo} 中所有标签备份成功")
#打印分隔符
print_separator("*")
# 清空标签列表
del tags_to_backup[:]
# 再次获取我的仓库中已备份的所有镜像
# 构造 API URL
api_url = f'https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/'
# 存储所有标签
existing_images = []
# 初始请求链接为 API URL
next_page = api_url
# 当 next_page 不为 None 时继续循环
while next_page is not None:
# 发送 GET 请求获取响应
response = requests.get(next_page)
# 将 JSON 数据转换成 Python 对象(字典)
data = response.json()
# 获取本次请求返回的所有标签数据
tags = [result['name'] for result in data['results']]
# 将本次请求返回的标签数据存入 existing_images 列表
existing_images += tags
# 如果还有下一页,则更新 next_page 的值为下一页链接
next_page = data.get('next', None)
# 对现有镜像列表进行排序
existing_images_sorted = sorted(existing_images)
# 总结本次备份
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"完成所有备份流程,开始总结本次备份:")
# 遍历 existing_images_sorted 列表并打印每个现有镜像
number = 0 # 镜像数量计数器
# 第一次循环先获取镜像数量
for image in existing_images_sorted:
number+=1
print("\n" + datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + ' 目前备份过的镜像(共 {} 个):'.format(str(number)))
# 第二次循环打印镜像列表
for image in existing_images_sorted:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], image)
print("\n")
# 打印备份信息
total_backed_up_images = len(backed_up_images)
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"本次备份前总共备份过 {backuped_lastname} 个镜像。")
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"本次总共需要备份 {total_new_images_to_backup} 个新的镜像。")
if total_backed_up_images > 0:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"实际备份的所有镜像为 {', '.join(backed_up_images)},共备份了 {total_backed_up_images} 个镜像。")
else:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], "没有备份任何镜像。")
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"经过本次备份后,已备份的所有镜像数量由 {backuped_lastname} 个变为 {number} 个。")
# 打印备份失败的镜像信息
total_failed_images = len(failed_images)
if total_failed_images > 0:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"备份失败的镜像: {', '.join(failed_images)},共{total_failed_images}个。")
else:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], "所有镜像备份成功!")
VERSION 14
#!/usr/bin/env python3
import requests
from concurrent.futures import ThreadPoolExecutor, as_completed
import docker
import subprocess
import datetime
# 定义需要备份的镜像列表和备份后的仓库前缀
repos_to_backup = [
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}",
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}"
]
# 自己的仓库的路径,也就是备份的目的地
backup_repo_prefix = "{YOUR_DOCKERHUB_USERNAME}/{YOUR_DOCKERHUB_REPOSITORY}"
# 定义dockerhub的用户名和密码
DOCKERHUB_USERNAME = '{YOUR_DOCKERHUB_USERNAME}'
DOCKERHUB_PASSWORD = "{YOUR_DOCKERHUB_PASSWORD}"
def check_and_install_libraries(packages):
"""
检查需要的库是否已经安装,并自动安装未安装的库
:param packages: 需要检查和安装的库列表
:return: True表示所有库都已成功安装,False表示至少有一个库未能成功安装
"""
# 遍历库列表中的每个库
for package in packages:
# 尝试导入该库
try:
__import__(package)
# 如果导入失败,则执行以下操作
except ImportError:
# 使用subprocess调用pip工具安装该库
subprocess.check_call(["pip", "install", package])
# 再次遍历库列表中的每个库
for package in packages:
# 尝试导入该库
try:
__import__(package)
# 如果导入失败,则输出相应提示信息到控制台
except ImportError:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" 未能成功安装 {package}")
# 返回False表示至少有一个库未能成功安装
return False
# 如果所有库都被成功安装,则返回True
return True
# 运行库检测
packages = ["requests", "concurrent.futures", "docker"]
# 调用check_and_install_libraries函数,传入需要检查和安装的库列表
success = check_and_install_libraries(packages)
# 根据返回结果打印相应的提示信息到控制台
if success:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + " 所有库都已成功安装。")
else:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + " 至少有一个库未能成功安装。")
# 获取指定仓库的所有标签
def get_all_tags(repo, session):
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" 获取{repo}的所有标签中")
url = f"https://registry.hub.docker.com/v2/repositories/{repo}/tags/"
all_tags = []
next_page = url
while next_page is not None:
response = session.get(next_page)
response.raise_for_status()
data = response.json()
tags = [t["name"] for t in data["results"]]
all_tags += tags
next_page = data.get('next', None)
for tag in all_tags:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" 发现标签: {tag}")
yield tag
# 登录到 Docker Hub
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + " 登录 Docker Hub中")
client = docker.from_env()
client.login(username=DOCKERHUB_USERNAME, password=DOCKERHUB_PASSWORD)
# 获取我的仓库中已备份的所有镜像
# 构造 API URL
api_url = f'https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/'
# 存储所有标签
existing_images = []
# 初始请求链接为 API URL
next_page = api_url
# 当 next_page 不为 None 时继续循环
while next_page is not None:
# 发送 GET 请求获取响应
response = requests.get(next_page)
# 将 JSON 数据转换成 Python 对象(字典)
data = response.json()
# 获取本次请求返回的所有标签数据
tags = [result['name'] for result in data['results']]
# 将本次请求返回的标签数据存入 existing_images 列表
existing_images += tags
# 如果还有下一页,则更新 next_page 的值为下一页链接
next_page = data.get('next', None)
# 对现有镜像列表进行排序
existing_images_sorted = sorted(existing_images)
#以前备份过的镜像的数量,方便最后输出调用
backuped_lastname = len(existing_images_sorted)
# 遍历 existing_images_sorted 列表并打印每个现有镜像
print("\n"+datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + ' 以前备份过的镜像:')
number = 0
for image in existing_images_sorted:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], image)
number+=1
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f' 共{number}个\n')
# 统计备份结果信息
backed_up_images = []
failed_images = []
total_new_images_to_backup = 0
total_failed_images = 0
# 并行处理每个镜像的备份任务
with ThreadPoolExecutor() as executor:
futures = []
with requests.Session() as session:
for repo in repos_to_backup:
# 获取给定镜像的所有标签
future = executor.submit(get_all_tags, repo, session)
futures.append(future)
for future in as_completed(futures):
repo = repos_to_backup[futures.index(future)]
tags_to_backup = list(future.result())
if len(tags_to_backup) == 0:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" {repo}没有标签,跳过。")
continue
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + f" {repo}的所有标签分(共{len(tags_to_backup)}个): {', '.join(tags_to_backup)}")
# 并行处理每个标签的备份任务
tag_futures = []
for tag in tags_to_backup:
# 构建原始镜像名称和标签
old_image_name = f"{repo}:{tag}"
# 构建新的镜像名称和标签
new_image_name = f"{repo.split('/')[0]}__{repo.split('/')[1]}__{tag}"
new_tag = new_image_name
# 检查是否已经备份过这个镜像
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], "开始对比是否需要备份")
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"以前备份过的镜像: (共{number}个)")
number = 0
for image in existing_images_sorted:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], image)
number+=1
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], '准备要备份的镜像:')
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], new_image_name)
if new_image_name in existing_images_sorted:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], '"准备要备份的镜像"已包含在"以前备份过的镜像"中')
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"{new_image_name} 镜像已备份,跳过。")
continue
# 打印源仓库和备份仓库的对比信息
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], '"以前备份过的镜像"中没有找到"准备要备份的镜像"')
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], '开始备份:')
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"{old_image_name} 将被备份为 {new_image_name} 到 {backup_repo_prefix}:{new_tag}")
# 备份 Docker 镜像
new_image = f"{backup_repo_prefix}:{new_tag}"
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"正在备份 {old_image_name} 到 {new_image_name}")
try:
arch_list = [
"linux/amd64",
"linux/arm64/v8",
"linux/arm/v6",
"linux/arm/v7",
"linux/ppc64le",
"linux/386",
"linux/s390x",
"windows/amd64",
"windows/386",
"darwin/amd64",
"darwin/386"
]
for arch in arch_list:
try:
image = client.images.pull(old_image_name, platform=arch)
break
except docker.errors.ImageNotFound:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"无法在默认 registry 中找到 {old_image_name} 的 {arch} 版本,尝试其他架构")
except docker.errors.APIError as e:
if 'no matching manifest' in str(e).lower():
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"{old_image_name} 不支持 {arch} 架构")
else:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"API 错误:{str(e)}")
if not image:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"无法从任何 registry 中拉取 {old_image_name}")
image.tag(new_image)
client.images.push(new_image)
backed_up_images.append(new_image_name)
total_new_images_to_backup += 1
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"备份镜像 {old_image_name} 到 {backup_repo_prefix}:{new_tag} 成功")
# 检查 正在运行的容器中 是否有容器正在使用 准备删除的镜像
running_containers = client.containers.list()
for container in running_containers:
if old_image_name in container.image.tags:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"{old_image_name} 对应的容器 {container.name} 正在运行,无法删除该镜像")
break # 如果有一个容器正在使用该镜像,则跳出循环,不进行镜像删除操作
else: # 如果没有任何容器使用该镜像,则执行镜像删除操作
# 删除已备份的原始镜像改名后的新镜像(也即是备份到自己仓库中的镜像)
client.images.remove(backup_repo_prefix + ":" + new_tag, force=True) #注意,使用 force=True 参数可以强制删除镜像,即使该镜像仍然被其他容器所使用。
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"删除改名后的镜像 {backup_repo_prefix}:{new_tag} 成功")
# 删除已备份的原始镜像
client.images.remove(old_image_name, force=True) #注意,使用 force=True 参数可以强制删除镜像,即使该镜像仍然被其他容器所使用。
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"删除原始镜像 {old_image_name} 成功")
except Exception as e:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"备份镜像 {old_image_name} 失败")
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"失败原因: {str(e)}")
failed_images.append(new_image_name)
total_failed_images += 1
# 删除已备份的标签
tags_to_backup.remove(tag)
# 清空标签列表
del tags_to_backup[:]
# 再次获取我的仓库中已备份的所有镜像
# 构造 API URL
api_url = f'https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/'
# 存储所有标签
existing_images = []
# 初始请求链接为 API URL
next_page = api_url
# 当 next_page 不为 None 时继续循环
while next_page is not None:
# 发送 GET 请求获取响应
response = requests.get(next_page)
# 将 JSON 数据转换成 Python 对象(字典)
data = response.json()
# 获取本次请求返回的所有标签数据
tags = [result['name'] for result in data['results']]
# 将本次请求返回的标签数据存入 existing_images 列表
existing_images += tags
# 如果还有下一页,则更新 next_page 的值为下一页链接
next_page = data.get('next', None)
# 对现有镜像列表进行排序
existing_images_sorted = sorted(existing_images)
# 遍历 existing_images_sorted 列表并打印每个现有镜像
print("\n" + datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + ' 目前备份过的镜像:')
number = 0
for image in existing_images_sorted:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], image)
number+=1
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], '目前备份过的总镜像数量:', str(number) + "\n")
# 打印备份信息
total_backed_up_images = len(backed_up_images)
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"之前总共备份过 {backuped_lastname} 个镜像。")
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"本次总共需要备份 {total_new_images_to_backup} 个新的镜像。")
if total_backed_up_images > 0:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"实际备份的所有镜像为 {', '.join(backed_up_images)},共备份了 {total_backed_up_images} 个镜像。")
else:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], "没有备份任何镜像。")
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"经过本次备份后,已备份的所有镜像数量由 {backuped_lastname} 个变为 {number} 个。")
# 打印备份失败的镜像信息
total_failed_images = len(failed_images)
if total_failed_images > 0:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], f"备份失败的镜像: {', '.join(failed_images)},共{total_failed_images}个。")
else:
print(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3], "所有镜像备份成功!")
VERSION 13
#!/usr/bin/env python3
import requests
from concurrent.futures import ThreadPoolExecutor, as_completed
import docker
import subprocess
# 定义需要备份的镜像列表和备份后的仓库前缀
repos_to_backup = [
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}",
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}"
]
# 自己的仓库的路径,也就是备份的目的地
backup_repo_prefix = "{YOUR_DOCKERHUB_USERNAME}/{YOUR_DOCKERHUB_REPOSITORY}"
# 定义dockerhub的用户名和密码
DOCKERHUB_USERNAME = '{YOUR_DOCKERHUB_USERNAME}'
DOCKERHUB_PASSWORD = "{YOUR_DOCKERHUB_PASSWORD}"
def check_and_install_libraries(packages):
"""
检查需要的库是否已经安装,并自动安装未安装的库
:param packages: 需要检查和安装的库列表
:return: True表示所有库都已成功安装,False表示至少有一个库未能成功安装
"""
# 遍历库列表中的每个库
for package in packages:
# 尝试导入该库
try:
__import__(package)
# 如果导入失败,则执行以下操作
except ImportError:
# 使用subprocess调用pip工具安装该库
subprocess.check_call(["pip", "install", package])
# 再次遍历库列表中的每个库
for package in packages:
# 尝试导入该库
try:
__import__(package)
# 如果导入失败,则输出相应提示信息到控制台
except ImportError:
print(f"未能成功安装 {package}")
# 返回False表示至少有一个库未能成功安装
return False
# 如果所有库都被成功安装,则返回True
return True
# 运行库检测
packages = ["requests", "concurrent.futures", "docker"]
# 调用check_and_install_libraries函数,传入需要检查和安装的库列表
success = check_and_install_libraries(packages)
# 根据返回结果打印相应的提示信息到控制台
if success:
print("所有库都已成功安装。")
else:
print("至少有一个库未能成功安装。")
# 获取指定仓库的所有标签
def get_all_tags(repo, session):
print(f"获取{repo}的所有标签中")
url = f"https://registry.hub.docker.com/v2/repositories/{repo}/tags/"
all_tags = []
next_page = url
while next_page is not None:
response = session.get(next_page)
response.raise_for_status()
data = response.json()
tags = [t["name"] for t in data["results"]]
all_tags += tags
next_page = data.get('next', None)
for tag in all_tags:
print(f"发现标签: {tag}")
yield tag
# 登录到 Docker Hub
print("登录 Docker Hub中")
client = docker.from_env()
client.login(username=DOCKERHUB_USERNAME, password=DOCKERHUB_PASSWORD)
# 获取我的仓库中已备份的所有镜像
# 构造 API URL
api_url = f'https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/'
# 存储所有标签
existing_images = []
# 初始请求链接为 API URL
next_page = api_url
# 当 next_page 不为 None 时继续循环
while next_page is not None:
# 发送 GET 请求获取响应
response = requests.get(next_page)
# 将 JSON 数据转换成 Python 对象(字典)
data = response.json()
# 获取本次请求返回的所有标签数据
tags = [result['name'] for result in data['results']]
# 将本次请求返回的标签数据存入 existing_images 列表
existing_images += tags
# 如果还有下一页,则更新 next_page 的值为下一页链接
next_page = data.get('next', None)
# 对现有镜像列表进行排序
existing_images_sorted = sorted(existing_images)
#以前备份过的镜像的数量,方便最后输出调用
backuped_lastname = len(existing_images_sorted)
# 遍历 existing_images_sorted 列表并打印每个现有镜像
print('\n以前备份过的镜像:')
number = 0
for image in existing_images_sorted:
print(image)
number+=1
print(f'共{number}个\n')
# 统计备份结果信息
backed_up_images = []
failed_images = []
total_new_images_to_backup = 0
total_failed_images = 0
# 并行处理每个镜像的备份任务
with ThreadPoolExecutor() as executor:
futures = []
with requests.Session() as session:
for repo in repos_to_backup:
# 获取给定镜像的所有标签
future = executor.submit(get_all_tags, repo, session)
futures.append(future)
for future in as_completed(futures):
repo = repos_to_backup[futures.index(future)]
tags_to_backup = list(future.result())
if len(tags_to_backup) == 0:
print(f"{repo}没有标签,跳过。")
continue
print(f"{repo}的所有标签分(共{len(tags_to_backup)}个): {', '.join(tags_to_backup)}")
# 并行处理每个标签的备份任务
tag_futures = []
for tag in tags_to_backup:
# 构建原始镜像名称和标签
old_image_name = f"{repo}:{tag}"
# 构建新的镜像名称和标签
new_image_name = f"{repo.split('/')[0]}__{repo.split('/')[1]}__{tag}"
new_tag = new_image_name
# 检查是否已经备份过这个镜像
print('\n开始对比是否需要备份\n以前备份过的镜像: (共{}个)'.format(number))
number = 0
for image in existing_images_sorted:
print(image)
number+=1
print('准备要备份的镜像:\n' + new_image_name)
if new_image_name in existing_images_sorted:
print('"准备要备份的镜像"已包含在"以前备份过的镜像"中')
print(f"{new_image_name} 镜像已备份,跳过。")
continue
# 打印源仓库和备份仓库的对比信息
print('"以前备份过的镜像"中没有找到"准备要备份的镜像"\n开始备份:')
print(f"{old_image_name} 将被备份为 {new_image_name} 到 {backup_repo_prefix}:{new_tag}")
# 备份 Docker 镜像
new_image = f"{backup_repo_prefix}:{new_tag}"
print(f"正在备份 {old_image_name} 到 {new_image}")
try:
arch_list = [
"linux/amd64",
"linux/arm64/v8",
"linux/arm/v6",
"linux/arm/v7",
"linux/ppc64le",
"linux/386",
"linux/s390x",
"windows/amd64",
"windows/386",
"darwin/amd64",
"darwin/386"
]
for arch in arch_list:
try:
image = client.images.pull(old_image_name, platform=arch)
break
except docker.errors.ImageNotFound:
print(f"无法在默认 registry 中找到 {old_image_name} 的 {arch} 版本,尝试其他架构...")
except docker.errors.APIError as e:
if 'no matching manifest' in str(e).lower():
print(f"{old_image_name} 不支持 {arch} 架构")
else:
print(f"API 错误:{str(e)}")
if not image:
print(f"无法从任何 registry 中拉取 {old_image_name}")
image.tag(new_image)
client.images.push(new_image)
backed_up_images.append(new_image_name)
total_new_images_to_backup += 1
print(f"备份镜像 {old_image_name} 到 {backup_repo_prefix}:{new_tag} 成功")
# 检查 正在运行的容器中 是否有容器正在使用 准备删除的镜像
running_containers = client.containers.list()
for container in running_containers:
if old_image_name in container.image.tags:
print(f"{old_image_name} 对应的容器 {container.name} 正在运行,无法删除该镜像")
break # 如果有一个容器正在使用该镜像,则跳出循环,不进行镜像删除操作
else: # 如果没有任何容器使用该镜像,则执行镜像删除操作
# 删除已备份的原始镜像改名后的新镜像(也即是备份到自己仓库中的镜像)
client.images.remove(backup_repo_prefix + ":" + new_tag, force=True) #注意,使用 force=True 参数可以强制删除镜像,即使该镜像仍然被其他容器所使用。
print(f"删除改名后的镜像 {backup_repo_prefix}:{new_tag} 成功")
# 删除已备份的原始镜像
client.images.remove(old_image_name, force=True) #注意,使用 force=True 参数可以强制删除镜像,即使该镜像仍然被其他容器所使用。
print(f"删除原始镜像 {old_image_name} 成功")
except Exception as e:
print(f"备份镜像 {old_image_name} 失败\n失败原因: {str(e)} ")
failed_images.append(new_image_name)
total_failed_images += 1
# 删除已备份的标签
tags_to_backup.remove(tag)
# 清空标签列表
del tags_to_backup[:]
# 再次获取我的仓库中已备份的所有镜像
# 构造 API URL
api_url = f'https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/'
# 存储所有标签
existing_images = []
# 初始请求链接为 API URL
next_page = api_url
# 当 next_page 不为 None 时继续循环
while next_page is not None:
# 发送 GET 请求获取响应
response = requests.get(next_page)
# 将 JSON 数据转换成 Python 对象(字典)
data = response.json()
# 获取本次请求返回的所有标签数据
tags = [result['name'] for result in data['results']]
# 将本次请求返回的标签数据存入 existing_images 列表
existing_images += tags
# 如果还有下一页,则更新 next_page 的值为下一页链接
next_page = data.get('next', None)
# 对现有镜像列表进行排序
existing_images_sorted = sorted(existing_images)
# 遍历 existing_images_sorted 列表并打印每个现有镜像
print('\n目前备份过的镜像:')
number = 0
for image in existing_images_sorted:
print(image)
number+=1
print('目前备份过的总镜像数量:',number)
print('\n')
# 打印备份信息
total_backed_up_images = len(backed_up_images)
print(f"之前总共备份过 {backuped_lastname} 个镜像。")
print(f"本次总共需要备份 {total_new_images_to_backup} 个新的镜像。")
if total_backed_up_images > 0:
print(f"实际备份的所有镜像为 {', '.join(backed_up_images)},共备份了 {total_backed_up_images} 个镜像。")
else:
print("没有备份任何镜像。")
print(f"经过本次备份后,已备份的所有镜像数量由 {backuped_lastname} 个变为 {number} 个")
# 打印备份失败的镜像信息
total_failed_images = len(failed_images)
if total_failed_images > 0:
print(f"备份失败的镜像: {', '.join(failed_images)},共{total_failed_images}个。")
else:
print("所有镜像备份成功!")
VERSION 12
#!/usr/bin/env python3
import requests
from concurrent.futures import ThreadPoolExecutor, as_completed
import docker
import subprocess
# 定义需要备份的镜像列表和备份后的仓库前缀
repos_to_backup = [
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}",
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}"
]
backup_repo_prefix = "{YOUR_DOCKERHUB_USERNAME}/{YOUR_DOCKERHUB_REPOSITORY}"
# 定义dockerhub的用户名和密码
DOCKERHUB_USERNAME = 'DOCKERHUB_USERNAME'
DOCKERHUB_PASSWORD = "DOCKERHUB_PASSWORD"
def check_and_install_libraries(packages):
"""
检查需要的库是否已经安装,并自动安装未安装的库
:param packages: 需要检查和安装的库列表
:return: True表示所有库都已成功安装,False表示至少有一个库未能成功安装
"""
# 遍历库列表中的每个库
for package in packages:
# 尝试导入该库
try:
__import__(package)
# 如果导入失败,则执行以下操作
except ImportError:
# 使用subprocess调用pip工具安装该库
subprocess.check_call(["pip", "install", package])
# 再次遍历库列表中的每个库
for package in packages:
# 尝试导入该库
try:
__import__(package)
# 如果导入失败,则输出相应提示信息到控制台
except ImportError:
print(f"未能成功安装 {package}")
# 返回False表示至少有一个库未能成功安装
return False
# 如果所有库都被成功安装,则返回True
return True
# 运行库检测
packages = ["requests", "concurrent.futures", "docker"]
# 调用check_and_install_libraries函数,传入需要检查和安装的库列表
success = check_and_install_libraries(packages)
# 根据返回结果打印相应的提示信息到控制台
if success:
print("所有库都已成功安装。")
else:
print("至少有一个库未能成功安装。")
# 获取指定仓库的所有标签
def get_all_tags(repo, session):
print(f"获取{repo}的所有标签中")
url = f"https://registry.hub.docker.com/v2/repositories/{repo}/tags/"
all_tags = []
next_page = url
while next_page is not None:
response = session.get(next_page)
response.raise_for_status()
data = response.json()
tags = [t["name"] for t in data["results"]]
all_tags += tags
next_page = data.get('next', None)
for tag in all_tags:
print(f"发现标签: {tag}")
yield tag
# 登录到 Docker Hub
print("登录 Docker Hub中")
client = docker.from_env()
client.login(username=DOCKERHUB_USERNAME, password=DOCKERHUB_PASSWORD)
# 获取我的仓库中已备份的所有镜像
# 构造 API URL
api_url = f'https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/'
# 存储所有标签
existing_images = []
# 初始请求链接为 API URL
next_page = api_url
# 当 next_page 不为 None 时继续循环
while next_page is not None:
# 发送 GET 请求获取响应
response = requests.get(next_page)
# 将 JSON 数据转换成 Python 对象(字典)
data = response.json()
# 获取本次请求返回的所有标签数据
tags = [result['name'] for result in data['results']]
# 将本次请求返回的标签数据存入 existing_images 列表
existing_images += tags
# 如果还有下一页,则更新 next_page 的值为下一页链接
next_page = data.get('next', None)
# 对现有镜像列表进行排序
existing_images_sorted = sorted(existing_images)
#以前备份过的镜像的数量,方便最后输出调用
backuped_lastname = len(existing_images_sorted)
# 遍历 existing_images_sorted 列表并打印每个现有镜像
print('\n以前备份过的镜像:')
number = 0
for image in existing_images_sorted:
print(image)
number+=1
print(f'共{number}个\n')
# 统计备份结果信息
backed_up_images = []
failed_images = []
total_new_images_to_backup = 0
total_failed_images = 0
# 并行处理每个镜像的备份任务
with ThreadPoolExecutor() as executor:
futures = []
with requests.Session() as session:
for repo in repos_to_backup:
# 获取给定镜像的所有标签
future = executor.submit(get_all_tags, repo, session)
futures.append(future)
for future in as_completed(futures):
repo = repos_to_backup[futures.index(future)]
tags_to_backup = list(future.result())
if len(tags_to_backup) == 0:
print(f"{repo}没有标签,跳过。")
continue
print(f"{repo}的所有标签分(共{len(tags_to_backup)}个): {', '.join(tags_to_backup)}")
# 并行处理每个标签的备份任务
tag_futures = []
for tag in tags_to_backup:
# 构建原始镜像名称和标签
old_image_name = f"{repo}:{tag}"
# 构建新的镜像名称和标签
new_image_name = f"{repo.split('/')[0]}__{repo.split('/')[1]}__{tag}"
new_tag = new_image_name
# 检查是否已经备份过这个镜像
print('\n开始对比是否需要备份\n以前备份过的镜像: (共{}个)'.format(number))
number = 0
for image in existing_images_sorted:
print(image)
number+=1
print('准备要备份的镜像:\n' + new_image_name)
if new_image_name in existing_images_sorted:
print('"准备要备份的镜像"已包含在"以前备份过的镜像"中')
print(f"{new_image_name} 镜像已备份,跳过。")
continue
# 打印源仓库和备份仓库的对比信息
print('"以前备份过的镜像"中没有找到"准备要备份的镜像"\n开始备份:')
print(f"{old_image_name} 将被备份为 {new_image_name} 到 {backup_repo_prefix}:{new_tag}")
# 备份 Docker 镜像
new_image = f"{backup_repo_prefix}:{new_tag}"
print(f"备份 {old_image_name} 到 {new_image}中")
try:
image = client.images.pull(old_image_name)
image.tag(new_image)
client.images.push(new_image)
backed_up_images.append(new_image_name)
total_new_images_to_backup += 1
print(f"备份镜像 {old_image_name} 到 {backup_repo_prefix}:{new_tag} 成功")
except Exception as e:
print(f"备份镜像 {old_image_name} 失败: {str(e)}")
failed_images.append(new_image_name)
total_failed_images += 1
# 删除已备份的标签
tags_to_backup.remove(tag)
# 清空标签列表
del tags_to_backup[:]
# 清理本地系统上不再需要的镜像
client.images.prune()
# 再次获取我的仓库中已备份的所有镜像
# 构造 API URL
api_url = f'https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/'
# 存储所有标签
existing_images = []
# 初始请求链接为 API URL
next_page = api_url
# 当 next_page 不为 None 时继续循环
while next_page is not None:
# 发送 GET 请求获取响应
response = requests.get(next_page)
# 将 JSON 数据转换成 Python 对象(字典)
data = response.json()
# 获取本次请求返回的所有标签数据
tags = [result['name'] for result in data['results']]
# 将本次请求返回的标签数据存入 existing_images 列表
existing_images += tags
# 如果还有下一页,则更新 next_page 的值为下一页链接
next_page = data.get('next', None)
# 清理不再使用的镜像
print("开始清理不再使用的镜像")
subprocess.run(['docker', 'image', 'prune', '-af'])
# 对现有镜像列表进行排序
existing_images_sorted = sorted(existing_images)
# 遍历 existing_images_sorted 列表并打印每个现有镜像
print('\n目前备份过的镜像:')
number = 0
for image in existing_images_sorted:
print(image)
number+=1
print('目前备份过的总镜像数量:',number)
print('\n')
# 打印备份信息
total_backed_up_images = len(backed_up_images)
print(f"之前总共备份过 {backuped_lastname} 个镜像。")
print(f"本次总共需要备份 {total_new_images_to_backup} 个新的镜像。")
if total_backed_up_images > 0:
print(f"实际备份的所有镜像为 {', '.join(backed_up_images)},共备份了 {total_backed_up_images} 个镜像。")
else:
print("没有备份任何镜像。")
print(f"经过本次备份后,已备份的所有镜像数量由 {backuped_lastname} 个变为 {number} 个")
# 打印备份失败的镜像信息
total_failed_images = len(failed_images)
if total_failed_images > 0:
print(f"备份失败的镜像: {', '.join(failed_images)},共{total_failed_images}个。")
else:
print("所有镜像备份成功!")
VERSION 11
#!/usr/bin/env python3
import requests
from concurrent.futures import ThreadPoolExecutor, as_completed
import docker
import subprocess
# 定义需要备份的镜像列表和备份后的仓库前缀
repos_to_backup = [
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}",
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}"
]
backup_repo_prefix = "{YOUR_DOCKERHUB_USERNAME}/{YOUR_DOCKERHUB_REPOSITORY}"
# 定义dockerhub的用户名和密码
DOCKERHUB_USERNAME = "{YOUR_DOCKERHUB_USERNAME}"
DOCKERHUB_PASSWORD = "{YOUR_DOCKERHUB_REPOSITORY}"
# 获取指定仓库的所有标签
def get_all_tags(repo, session):
print(f"获取{repo}的所有标签中")
url = f"https://registry.hub.docker.com/v2/repositories/{repo}/tags/"
all_tags = []
next_page = url
while next_page is not None:
response = session.get(next_page)
response.raise_for_status()
data = response.json()
tags = [t["name"] for t in data["results"]]
all_tags += tags
next_page = data.get('next', None)
for tag in all_tags:
print(f"发现标签: {tag}")
yield tag
# 登录到 Docker Hub
print("登录 Docker Hub中")
client = docker.from_env()
client.login(username=DOCKERHUB_USERNAME, password=DOCKERHUB_PASSWORD)
# 获取我的仓库中已备份的所有镜像
# 构造 API URL
api_url = f'https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/'
# 存储所有标签
existing_images = []
# 初始请求链接为 API URL
next_page = api_url
# 当 next_page 不为 None 时继续循环
while next_page is not None:
# 发送 GET 请求获取响应
response = requests.get(next_page)
# 将 JSON 数据转换成 Python 对象(字典)
data = response.json()
# 获取本次请求返回的所有标签数据
tags = [result['name'] for result in data['results']]
# 将本次请求返回的标签数据存入 existing_images 列表
existing_images += tags
# 如果还有下一页,则更新 next_page 的值为下一页链接
next_page = data.get('next', None)
# 对现有镜像列表进行排序
existing_images_sorted = sorted(existing_images)
#以前备份过的镜像的数量,方便最后输出调用
backuped_lastname = len(existing_images_sorted)
# 遍历 existing_images_sorted 列表并打印每个现有镜像
print('\n以前备份过的镜像:')
number = 0
for image in existing_images_sorted:
print(image)
number+=1
print(f'共{number}个\n')
# 统计备份结果信息
backed_up_images = []
failed_images = []
total_new_images_to_backup = 0
total_failed_images = 0
# 并行处理每个镜像的备份任务
with ThreadPoolExecutor() as executor:
futures = []
with requests.Session() as session:
for repo in repos_to_backup:
# 获取给定镜像的所有标签
future = executor.submit(get_all_tags, repo, session)
futures.append(future)
for future in as_completed(futures):
repo = repos_to_backup[futures.index(future)]
tags_to_backup = list(future.result())
if len(tags_to_backup) == 0:
print(f"{repo}没有标签,跳过。")
continue
print(f"{repo}的所有标签分(共{len(tags_to_backup)}个): {', '.join(tags_to_backup)}")
# 并行处理每个标签的备份任务
tag_futures = []
for tag in tags_to_backup:
# 构建原始镜像名称和标签
old_image_name = f"{repo}:{tag}"
# 构建新的镜像名称和标签
new_image_name = f"{repo.split('/')[0]}__{repo.split('/')[1]}__{tag}"
new_tag = new_image_name
# 检查是否已经备份过这个镜像
print('\n开始对比是否需要备份\n以前备份过的镜像: (共{}个)'.format(number))
number = 0
for image in existing_images_sorted:
print(image)
number+=1
print('准备要备份的镜像:\n' + new_image_name)
if new_image_name in existing_images_sorted:
print('"准备要备份的镜像"已包含在"以前备份过的镜像"中')
print(f"{new_image_name} 镜像已备份,跳过。")
continue
# 打印源仓库和备份仓库的对比信息
print('"以前备份过的镜像"中没有找到"准备要备份的镜像"\n开始备份:')
print(f"{old_image_name} 将被备份为 {new_image_name} 到 {backup_repo_prefix}:{new_tag}")
# 备份 Docker 镜像
new_image = f"{backup_repo_prefix}:{new_tag}"
print(f"备份 {old_image_name} 到 {new_image}中")
try:
image = client.images.pull(old_image_name)
image.tag(new_image)
client.images.push(new_image)
backed_up_images.append(new_image_name)
total_new_images_to_backup += 1
print(f"备份镜像 {old_image_name} 到 {backup_repo_prefix}:{new_tag} 成功")
except Exception as e:
print(f"备份镜像 {old_image_name} 失败: {str(e)}")
failed_images.append(new_image_name)
total_failed_images += 1
# 删除已备份的标签
tags_to_backup.remove(tag)
# 清空标签列表
del tags_to_backup[:]
# 清理本地系统上不再需要的镜像
client.images.prune()
# 再次获取我的仓库中已备份的所有镜像
# 构造 API URL
api_url = f'https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/'
# 存储所有标签
existing_images = []
# 初始请求链接为 API URL
next_page = api_url
# 当 next_page 不为 None 时继续循环
while next_page is not None:
# 发送 GET 请求获取响应
response = requests.get(next_page)
# 将 JSON 数据转换成 Python 对象(字典)
data = response.json()
# 获取本次请求返回的所有标签数据
tags = [result['name'] for result in data['results']]
# 将本次请求返回的标签数据存入 existing_images 列表
existing_images += tags
# 如果还有下一页,则更新 next_page 的值为下一页链接
next_page = data.get('next', None)
# 清理不再使用的镜像
print("开始清理不再使用的镜像")
subprocess.run(['docker', 'image', 'prune', '-af'])
# 对现有镜像列表进行排序
existing_images_sorted = sorted(existing_images)
# 遍历 existing_images_sorted 列表并打印每个现有镜像
print('\n目前备份过的镜像:')
number = 0
for image in existing_images_sorted:
print(image)
number+=1
print('目前备份过的总镜像数量:',number)
print('\n')
# 打印备份信息
total_backed_up_images = len(backed_up_images)
print(f"之前总共备份过 {backuped_lastname} 个镜像。")
print(f"本次总共需要备份 {total_new_images_to_backup} 个新的镜像。")
if total_backed_up_images > 0:
print(f"实际备份的所有镜像为 {', '.join(backed_up_images)},共备份了 {total_backed_up_images} 个镜像。")
else:
print("没有备份任何镜像。")
print(f"经过本次备份后,已备份的所有镜像数量由 {backuped_lastname} 个变为 {number} 个")
# 打印备份失败的镜像信息
total_failed_images = len(failed_images)
if total_failed_images > 0:
print(f"备份失败的镜像: {', '.join(failed_images)},共{total_failed_images}个。")
else:
print("所有镜像备份成功!")
VERSION 10
#!/usr/bin/env python3
import requests
from concurrent.futures import ThreadPoolExecutor, as_completed
import docker
# 定义需要备份的镜像列表和备份后的仓库前缀
repos_to_backup = [
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}",
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}"
]
backup_repo_prefix = "{YOUR_DOCKERHUB_USERNAME}/{YOUR_DOCKERHUB_REPOSITORY}"
# 定义dockerhub的用户名和密码
DOCKERHUB_USERNAME = "{YOUR_DOCKERHUB_USERNAME}"
DOCKERHUB_PASSWORD = "{YOUR_DOCKERHUB_PASSWORD}"
# 获取指定仓库的所有标签
def get_all_tags(repo, session):
print(f"获取{repo}的所有标签中")
url = f"https://registry.hub.docker.com/v2/repositories/{repo}/tags/"
all_tags = []
next_page = url
while next_page is not None:
response = session.get(next_page)
response.raise_for_status()
data = response.json()
tags = [t["name"] for t in data["results"]]
all_tags += tags
next_page = data.get('next', None)
for tag in all_tags:
print(f"发现标签: {tag}")
yield tag
# 登录到 Docker Hub
print("登录 Docker Hub中")
client = docker.from_env()
client.login(username=DOCKERHUB_USERNAME, password=DOCKERHUB_PASSWORD)
# 获取我的仓库中已备份的所有镜像
# 构造 API URL
api_url = f'https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/'
# 存储所有标签
existing_images = []
# 初始请求链接为 API URL
next_page = api_url
# 当 next_page 不为 None 时继续循环
while next_page is not None:
# 发送 GET 请求获取响应
response = requests.get(next_page)
# 将 JSON 数据转换成 Python 对象(字典)
data = response.json()
# 获取本次请求返回的所有标签数据
tags = [result['name'] for result in data['results']]
# 将本次请求返回的标签数据存入 existing_images 列表
existing_images += tags
# 如果还有下一页,则更新 next_page 的值为下一页链接
next_page = data.get('next', None)
# 对现有镜像列表进行排序
existing_images_sorted = sorted(existing_images)
#以前备份过的镜像的数量,方便最后输出调用
backuped_lastname = len(existing_images_sorted)
# 遍历 existing_images_sorted 列表并打印每个现有镜像
print('\n以前备份过的镜像:')
number = 0
for image in existing_images_sorted:
print(image)
number+=1
print(f'共{number}个\n')
# 统计备份结果信息
backed_up_images = []
failed_images = []
total_new_images_to_backup = 0
total_failed_images = 0
# 并行处理每个镜像的备份任务
with ThreadPoolExecutor() as executor:
futures = []
with requests.Session() as session:
for repo in repos_to_backup:
# 获取给定镜像的所有标签
future = executor.submit(get_all_tags, repo, session)
futures.append(future)
for future in as_completed(futures):
repo = repos_to_backup[futures.index(future)]
tags_to_backup = list(future.result())
if len(tags_to_backup) == 0:
print(f"{repo}没有标签,跳过。")
continue
print(f"{repo}的所有标签分(共{len(tags_to_backup)}个): {', '.join(tags_to_backup)}")
# 并行处理每个标签的备份任务
tag_futures = []
for tag in tags_to_backup:
# 构建原始镜像名称和标签
old_image_name = f"{repo}:{tag}"
# 构建新的镜像名称和标签
new_image_name = f"{repo.split('/')[0]}__{repo.split('/')[1]}__{tag}"
new_tag = new_image_name
# 检查是否已经备份过这个镜像
print('\n开始对比是否需要备份\n以前备份过的镜像: (共{}个)'.format(number))
number = 0
for image in existing_images_sorted:
print(image)
number+=1
print('准备要备份的镜像:\n' + new_image_name)
if new_image_name in existing_images_sorted:
print('"准备要备份的镜像"已包含在"以前备份过的镜像"中')
print(f"{new_image_name} 镜像已备份,跳过。")
continue
# 打印源仓库和备份仓库的对比信息
print('"以前备份过的镜像"中没有找到"准备要备份的镜像"\n开始备份:')
print(f"{old_image_name} 将被备份为 {new_image_name} 到 {backup_repo_prefix}:{new_tag}")
# 备份 Docker 镜像
new_image = f"{backup_repo_prefix}:{new_tag}"
print(f"备份 {old_image_name} 到 {new_image}中")
try:
image = client.images.pull(old_image_name)
image.tag(new_image)
client.images.push(new_image)
backed_up_images.append(new_image_name)
total_new_images_to_backup += 1
print(f"备份镜像 {old_image_name} 到 {backup_repo_prefix}:{new_tag} 成功")
except Exception as e:
print(f"备份镜像 {old_image_name} 失败: {str(e)}")
failed_images.append(new_image_name)
total_failed_images += 1
# 删除已备份的标签
tags_to_backup.remove(tag)
# 清空标签列表
del tags_to_backup[:]
# 清理本地系统上不再需要的镜像
client.images.prune()
# 再次获取我的仓库中已备份的所有镜像
# 构造 API URL
api_url = f'https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/'
# 存储所有标签
existing_images = []
# 初始请求链接为 API URL
next_page = api_url
# 当 next_page 不为 None 时继续循环
while next_page is not None:
# 发送 GET 请求获取响应
response = requests.get(next_page)
# 将 JSON 数据转换成 Python 对象(字典)
data = response.json()
# 获取本次请求返回的所有标签数据
tags = [result['name'] for result in data['results']]
# 将本次请求返回的标签数据存入 existing_images 列表
existing_images += tags
# 如果还有下一页,则更新 next_page 的值为下一页链接
next_page = data.get('next', None)
# 对现有镜像列表进行排序
existing_images_sorted = sorted(existing_images)
# 遍历 existing_images_sorted 列表并打印每个现有镜像
print('\n目前备份过的镜像:')
number = 0
for image in existing_images_sorted:
print(image)
number+=1
print('目前备份过的总镜像数量:',number)
print('\n')
# 打印备份信息
total_backed_up_images = len(backed_up_images)
print(f"之前总共备份过 {backuped_lastname} 个镜像。")
print(f"本次总共需要备份 {total_new_images_to_backup} 个新的镜像。")
if total_backed_up_images > 0:
print(f"实际备份的所有镜像为 {', '.join(backed_up_images)},共备份了 {total_backed_up_images} 个镜像。")
else:
print("没有备份任何镜像。")
print(f"经过本次备份后,已备份的所有镜像数量由 {backuped_lastname} 个变为 {number} 个")
# 打印备份失败的镜像信息
total_failed_images = len(failed_images)
if total_failed_images > 0:
print(f"备份失败的镜像: {', '.join(failed_images)},共{total_failed_images}个。")
else:
print("所有镜像备份成功!")
# 清理不再使用的镜像
client.images.prune(filters={'dangling': True})
VERSION 9
#!/usr/bin/env python3
import os
import requests
from concurrent.futures import ThreadPoolExecutor, as_completed
import docker
# 获取指定仓库的所有标签
def get_all_tags(repo, session):
print(f"获取{repo}的所有标签中")
url = f"https://registry.hub.docker.com/v2/repositories/{repo}/tags/"
all_tags = []
next_page = url
while next_page is not None:
response = session.get(next_page)
response.raise_for_status()
data = response.json()
tags = [t["name"] for t in data["results"]]
all_tags += tags
next_page = data.get('next', None)
for tag in all_tags:
print(f"发现标签: {tag}")
yield tag
# 定义需要备份的镜像列表和备份后的仓库前缀
repos_to_backup = [
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}",
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}"
]
backup_repo_prefix = "{YOUR_DOCKERHUB_USERNAME}/{YOUR_DOCKERHUB_REPOSITORY}"
# 登录到 Docker Hub
print("登录 Docker Hub中")
client = docker.from_env()
client.login(username=os.environ['DOCKERHUB_USERNAME'], password=os.environ['DOCKERHUB_PASSWORD'])
# 获取我的仓库中已备份的所有镜像
# 构造 API URL
api_url = f'https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/'
# 存储所有标签
existing_images = []
# 初始请求链接为 API URL
next_page = api_url
# 当 next_page 不为 None 时继续循环
while next_page is not None:
# 发送 GET 请求获取响应
response = requests.get(next_page)
# 将 JSON 数据转换成 Python 对象(字典)
data = response.json()
# 获取本次请求返回的所有标签数据
tags = [result['name'] for result in data['results']]
# 将本次请求返回的标签数据存入 existing_images 列表
existing_images += tags
# 如果还有下一页,则更新 next_page 的值为下一页链接
next_page = data.get('next', None)
# 对现有镜像列表进行排序
existing_images_sorted = sorted(existing_images)
# 遍历 existing_images_sorted 列表并打印每个现有镜像
print('\n以前备份过的镜像:')
number = 0
for image in existing_images_sorted:
print(image)
number+=1
print(f'共{number}个\n')
# 统计备份结果信息
backed_up_images = []
failed_images = []
total_new_images_to_backup = 0
total_failed_images = 0
# 并行处理每个镜像的备份任务
with ThreadPoolExecutor() as executor:
futures = []
with requests.Session() as session:
for repo in repos_to_backup:
# 获取给定镜像的所有标签
future = executor.submit(get_all_tags, repo, session)
futures.append(future)
for future in as_completed(futures):
repo = repos_to_backup[futures.index(future)]
tags_to_backup = list(future.result())
if len(tags_to_backup) == 0:
print(f"{repo}没有标签,跳过。")
continue
print(f"{repo}的所有标签分(共{len(tags_to_backup)}个): {', '.join(tags_to_backup)}")
# 并行处理每个标签的备份任务
tag_futures = []
for tag in tags_to_backup:
# 构建原始镜像名称和标签
old_image_name = f"{repo}:{tag}"
# 构建新的镜像名称和标签
new_image_name = f"{repo.split('/')[0]}__{repo.split('/')[1]}__{tag}"
new_tag = new_image_name
# 检查是否已经备份过这个镜像
print('\n开始对比是否需要备份\n以前备份过的镜像: (共{}个)'.format(number))
number = 0
for image in existing_images_sorted:
print(image)
number+=1
print('准备要备份的镜像:\n' + new_image_name)
if new_image_name in existing_images_sorted:
print('"准备要备份的镜像"已包含在"以前备份过的镜像"中')
print(f"{new_image_name} 镜像已备份,跳过。")
continue
# 打印源仓库和备份仓库的对比信息
print('"以前备份过的镜像"中没有找到"准备要备份的镜像"\n开始备份:')
print(f"{old_image_name} 将被备份为 {new_image_name} 到 {backup_repo_prefix}:{new_tag}")
# 备份 Docker 镜像
new_image = f"{backup_repo_prefix}:{new_tag}"
print(f"备份 {old_image_name} 到 {new_image}中")
try:
image = client.images.pull(old_image_name)
image.tag(new_image)
client.images.push(new_image)
backed_up_images.append(new_image_name)
total_new_images_to_backup += 1
except Exception as e:
print(f"备份镜像 {old_image_name} 失败: {str(e)}")
failed_images.append(new_image_name)
total_failed_images += 1
# 删除已备份的标签
tags_to_backup.remove(tag)
# 清空标签列表
del tags_to_backup[:]
# 清理本地系统上不再需要的镜像
client.images.prune()
# 再次获取我的仓库中已备份的所有镜像
# 构造 API URL
api_url = f'https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/'
# 存储所有标签
existing_images = []
# 初始请求链接为 API URL
next_page = api_url
# 当 next_page 不为 None 时继续循环
while next_page is not None:
# 发送 GET 请求获取响应
response = requests.get(next_page)
# 将 JSON 数据转换成 Python 对象(字典)
data = response.json()
# 获取本次请求返回的所有标签数据
tags = [result['name'] for result in data['results']]
# 将本次请求返回的标签数据存入 existing_images 列表
existing_images += tags
# 如果还有下一页,则更新 next_page 的值为下一页链接
next_page = data.get('next', None)
# 对现有镜像列表进行排序
existing_images_sorted = sorted(existing_images)
# 遍历 existing_images_sorted 列表并打印每个现有镜像
print('\n目前备份过的镜像:')
number = 0
for image in existing_images_sorted:
print(image)
number+=1
print('目前备份过的总镜像数量:',number)
print('\n')
# 打印备份信息
total_backed_up_images = len(backed_up_images)
print(f"总共备份过 {len(existing_images_sorted)} 个镜像。")
print(f"本次总共需要备份 {total_new_images_to_backup} 个新的镜像。")
if total_backed_up_images > 0:
print(f"实际备份的所有镜像为 {', '.join(backed_up_images)},共备份了 {total_backed_up_images} 个镜像。")
else:
print("没有备份任何镜像。")
# 打印备份失败的镜像信息
total_failed_images = len(failed_images)
if total_failed_images > 0:
print(f"备份失败的镜像: {', '.join(failed_images)},共{total_failed_images}个。")
else:
print("所有镜像备份成功!")
VERSION 8
#!/usr/bin/env python3
import os
import requests
from concurrent.futures import ThreadPoolExecutor, as_completed
import docker
# 获取指定仓库的所有标签
def get_all_tags(repo, session):
print(f"获取{repo}的所有标签中")
url = f"https://registry.hub.docker.com/v2/repositories/{repo}/tags/"
all_tags = []
next_page = url
while next_page is not None:
response = session.get(next_page)
response.raise_for_status()
data = response.json()
tags = [t["name"] for t in data["results"]]
all_tags += tags
next_page = data.get('next', None)
for tag in all_tags:
print(f"发现标签: {tag}")
yield tag
# 定义需要备份的镜像列表和备份后的仓库前缀
repos_to_backup = [
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}",
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}"
]
backup_repo_prefix = "{YOUR_DOCKERHUB_USERNAME}/YOUR_DOCKERHUB_REPOSITORY"
# 登录到 Docker Hub
print("登录 Docker Hub中")
client = docker.from_env()
client.login(username=os.environ['DOCKERHUB_USERNAME'], password=os.environ['DOCKERHUB_PASSWORD'])
# 获取我的仓库中已备份的所有镜像
# existing_images = []
# url = f"https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/"
# with requests.Session() as session:
# response = session.get(url)
# if response.status_code == 200:
# existing_images = [t["name"] for t in response.json()["results"]]
# existing_images_sorted = sorted(existing_images)
# print('\n以前备份过的镜像:')
# number = 0
# for image in existing_images_sorted:
# print(image)
# number+=1
# print('以前备份过的总镜像数量:',number)
# print('\n')
# 构造 API URL
api_url = f'https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/'
# 存储所有标签
existing_images = []
# 初始请求链接为 API URL
next_page = api_url
# 当 next_page 不为 None 时继续循环
while next_page is not None:
# 发送 GET 请求获取响应
response = requests.get(next_page)
# 将 JSON 数据转换成 Python 对象(字典)
data = response.json()
# 获取本次请求返回的所有标签数据
tags = [result['name'] for result in data['results']]
# 将本次请求返回的标签数据存入 existing_images 列表
existing_images += tags
# 如果还有下一页,则更新 next_page 的值为下一页链接
next_page = data.get('next', None)
# 对现有镜像列表进行排序
existing_images_sorted = sorted(existing_images)
# 遍历 existing_images_sorted 列表并打印每个现有镜像
print('\n以前备份过的镜像:')
number = 0
for image in existing_images_sorted:
print(image)
number+=1
print('以前备份过的总镜像数量:',number)
print('\n')
# 统计备份结果信息
backed_up_images = []
failed_images = []
total_new_images_to_backup = 0
total_failed_images = 0
# 并行处理每个镜像的备份任务
with ThreadPoolExecutor() as executor:
futures = []
with requests.Session() as session:
for repo in repos_to_backup:
# 获取给定镜像的所有标签
future = executor.submit(get_all_tags, repo, session)
futures.append(future)
for future in as_completed(futures):
repo = repos_to_backup[futures.index(future)]
tags_to_backup = list(future.result())
if len(tags_to_backup) == 0:
print(f"{repo}没有标签,跳过。")
continue
print(f"{repo}的所有标签分别是: {', '.join(tags_to_backup)}, 共{len(tags_to_backup)}个")
# 并行处理每个标签的备份任务
tag_futures = []
for tag in tags_to_backup:
# 构建原始镜像名称和标签
old_image_name = f"{repo}:{tag}"
# 构建新的镜像名称和标签
new_image_name = f"{repo.split('/')[0]}__{repo.split('/')[1]}__{tag}"
new_tag = new_image_name
# 检查是否已经备份过这个镜像
print('\n以前备份过的镜像:')
number = 0
for image in existing_images_sorted:
print(image)
number+=1
print('以前备份过的总镜像数量:',number)
print('\n')
print('准备要备份的镜像:\n' + new_image_name)
if new_image_name in existing_images_sorted:
print('"准备要备份的镜像"已包含在"以前备份过的镜像"中')
print(f"{new_image_name} 镜像已备份,跳过。")
continue
# 打印源仓库和备份仓库的对比信息
print('"以前备份过的镜像"中没有找到"准备要备份的镜像"')
print(f"{old_image_name} 将被备份为 {new_image_name} 到 {backup_repo_prefix}:{new_tag}")
# 备份 Docker 镜像
new_image = f"{backup_repo_prefix}:{new_tag}"
print(f"备份 {old_image_name} 到 {new_image}中")
try:
image = client.images.pull(old_image_name)
image.tag(new_image)
client.images.push(new_image)
backed_up_images.append(new_image_name)
total_new_images_to_backup += 1
except Exception as e:
print(f"备份镜像 {old_image_name} 失败: {str(e)}")
failed_images.append(new_image_name)
total_failed_images += 1
# 删除已备份的标签
tags_to_backup.remove(tag)
# 清空标签列表
del tags_to_backup[:]
# 清理本地系统上不再需要的镜像
client.images.prune()
# 再次获取我的仓库中已备份的所有镜像
# existing_images = []
# url = f"https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/"
# with requests.Session() as session:
# response = session.get(url)
# if response.status_code == 200:
# existing_images = [t["name"] for t in response.json()["results"]]
# existing_images_sorted = sorted(existing_images)
# print('\n目前备份过的镜像:')
# number = 0
# for image in existing_images_sorted:
# print(image)
# number+=1
# print('目前备份过的总镜像数量:',number)
# print('\n')
# 构造 API URL
api_url = f'https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/'
# 存储所有标签
existing_images = []
# 初始请求链接为 API URL
next_page = api_url
# 当 next_page 不为 None 时继续循环
while next_page is not None:
# 发送 GET 请求获取响应
response = requests.get(next_page)
# 将 JSON 数据转换成 Python 对象(字典)
data = response.json()
# 获取本次请求返回的所有标签数据
tags = [result['name'] for result in data['results']]
# 将本次请求返回的标签数据存入 existing_images 列表
existing_images += tags
# 如果还有下一页,则更新 next_page 的值为下一页链接
next_page = data.get('next', None)
# 对现有镜像列表进行排序
existing_images_sorted = sorted(existing_images)
# 遍历 existing_images_sorted 列表并打印每个现有镜像
print('\n目前备份过的镜像:')
number = 0
for image in existing_images_sorted:
print(image)
number+=1
print('目前备份过的总镜像数量:',number)
print('\n')
# 打印备份信息
total_backed_up_images = len(backed_up_images)
print(f"总共备份过 {total_backed_up_images} 个镜像。")
print(f"本次总共需要备份 {total_new_images_to_backup} 个新的镜像。")
if total_backed_up_images > 0:
print(f"实际备份的所有镜像为 {', '.join(backed_up_images)},共备份了 {total_backed_up_images} 个镜像。")
else:
print("没有备份任何镜像。")
# 打印备份失败的镜像信息
total_failed_images = len(failed_images)
if total_failed_images > 0:
print(f"备份失败的镜像: {', '.join(failed_images)},共有{total_failed_images}个。")
else:
print("所有镜像备份成功!")
VERSION 7
#!/usr/bin/env python3
import os
import requests
from concurrent.futures import ThreadPoolExecutor, as_completed
import docker
# 获取指定仓库的所有标签
def get_all_tags(repo, session):
print(f"获取{repo}的所有标签中")
url = f"https://registry.hub.docker.com/v2/repositories/{repo}/tags/"
response = session.get(url)
response.raise_for_status()
tags = [t["name"] for t in response.json()["results"]]
for tag in tags:
print(f"发现标签: {tag}")
yield tag
# 定义需要备份的镜像列表和备份后的仓库前缀
repos_to_backup = [
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}",
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}"
]
backup_repo_prefix = "{YOUR_DOCKERHUB_USERNAME}/{YOUR_DOCKERHUB_REPOSITORY}"
# 登录到 Docker Hub
print("登录 Docker Hub中")
client = docker.from_env()
client.login(username=os.environ['DOCKERHUB_USERNAME'], password=os.environ['DOCKERHUB_PASSWORD'])
# 获取我的仓库中已备份的所有镜像
existing_images = []
url = f"https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/"
with requests.Session() as session:
response = session.get(url)
if response.status_code == 200:
existing_images = [t["name"] for t in response.json()["results"]]
existing_images_sorted = sorted(existing_images)
print('\n以前备份过的镜像:')
number = 0
for image in existing_images_sorted:
print(image)
number+=1
print('以前备份过的总镜像数量:',number)
print('\n')
# 统计备份结果信息
backed_up_images = []
failed_images = []
total_new_images_to_backup = 0
total_failed_images = 0
# 并行处理每个镜像的备份任务
with ThreadPoolExecutor() as executor:
futures = []
with requests.Session() as session:
for repo in repos_to_backup:
# 获取给定镜像的所有标签
future = executor.submit(get_all_tags, repo, session)
futures.append(future)
for future in as_completed(futures):
repo = repos_to_backup[futures.index(future)]
tags_to_backup = list(future.result())
if len(tags_to_backup) == 0:
print(f"{repo}没有标签,跳过。")
continue
print(f"{repo}的所有标签分别是: {', '.join(tags_to_backup)}, 共{len(tags_to_backup)}个")
# 并行处理每个标签的备份任务
tag_futures = []
for tag in tags_to_backup:
# 构建原始镜像名称和标签
old_image_name = f"{repo}:{tag}"
# 构建新的镜像名称和标签
new_image_name = f"{repo.split('/')[0]}__{repo.split('/')[1]}__{tag}"
new_tag = new_image_name
# 检查是否已经备份过这个镜像
print('\n以前备份过的镜像:')
for image in existing_images:
print(image)
print('准备要备份的镜像:\n' + new_image_name)
if new_image_name in existing_images:
print('"准备要备份的镜像"已包含在"以前备份过的镜像"中')
print(f"{new_image_name} 镜像已备份,跳过。")
continue
# 打印源仓库和备份仓库的对比信息
print('"以前备份过的镜像"中没有找到"准备要备份的镜像"')
print(f"{old_image_name} 将被备份为 {new_image_name} 到 {backup_repo_prefix}:{new_tag}")
# 备份 Docker 镜像
new_image = f"{backup_repo_prefix}:{new_tag}"
print(f"备份 {old_image_name} 到 {new_image}中")
try:
image = client.images.pull(old_image_name)
image.tag(new_image)
client.images.push(new_image)
backed_up_images.append(new_image_name)
total_new_images_to_backup += 1
except Exception as e:
print(f"备份镜像 {old_image_name} 失败: {str(e)}")
failed_images.append(new_image_name)
total_failed_images += 1
# 删除已备份的标签
tags_to_backup.remove(tag)
# 清空标签列表
del tags_to_backup[:]
# 清理本地系统上不再需要的镜像
client.images.prune()
# 再次获取我的仓库中已备份的所有镜像
existing_images = []
url = f"https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/"
with requests.Session() as session:
response = session.get(url)
if response.status_code == 200:
existing_images = [t["name"] for t in response.json()["results"]]
existing_images_sorted = sorted(existing_images)
print('\n目前备份过的镜像:')
number = 0
for image in existing_images_sorted:
print(image)
number+=1
print('目前备份过的总镜像数量:',number)
print('\n')
# 打印备份信息
total_backed_up_images = len(backed_up_images)
print(f"总共备份过 {total_backed_up_images} 个镜像。")
print(f"本次总共需要备份 {total_new_images_to_backup} 个新的镜像。")
if total_backed_up_images > 0:
print(f"实际备份的所有镜像为 {', '.join(backed_up_images)},共备份了 {total_backed_up_images} 个镜像。")
else:
print("没有备份任何镜像。")
# 打印备份失败的镜像信息
total_failed_images = len(failed_images)
if total_failed_images > 0:
print(f"备份失败的镜像: {', '.join(failed_images)},共有{total_failed_images}个。")
else:
print("所有镜像备份成功!")
VERSION 6
#!/usr/bin/env python3
import os
import requests
from concurrent.futures import ThreadPoolExecutor, as_completed
import docker
# 获取指定仓库的所有标签
def get_all_tags(repo, session):
print(f"获取{repo}的所有标签...")
url = f"https://registry.hub.docker.com/v2/repositories/{repo}/tags/"
response = session.get(url)
response.raise_for_status()
tags = [t["name"] for t in response.json()["results"]]
for tag in tags:
print(f"发现标签: {tag}")
yield tag
# 定义需要备份的镜像列表和备份后的仓库前缀
repos_to_backup = [
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}",
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}"
]
backup_repo_prefix = "{YOUR_DOCKERHUB_USERNAME}/{YOUR_DOCKERHUB_REPOSITORY}"
# 登录到 Docker Hub
print("登录 Docker Hub...")
client = docker.from_env()
client.login(username=os.environ['DOCKERHUB_USERNAME'], password=os.environ['DOCKERHUB_PASSWORD'])
# 统计备份结果信息
backed_up_images = []
failed_images = []
total_new_images_to_backup = 0
total_failed_images = 0
# 并行处理每个镜像的备份任务
with ThreadPoolExecutor() as executor:
futures = []
with requests.Session() as session:
for repo in repos_to_backup:
# 获取给定镜像的所有标签
future = executor.submit(get_all_tags, repo, session)
futures.append(future)
for future in as_completed(futures):
repo = repos_to_backup[futures.index(future)]
tags_to_backup = list(future.result())
if len(tags_to_backup) == 0:
print(f"{repo}没有标签,跳过。")
continue
print(f"找到的标签: {', '.join(tags_to_backup)}")
# 并行处理每个标签的备份任务
tag_futures = []
for tag in tags_to_backup:
# 构建原始镜像名称和标签
old_image_name = f"{repo}:{tag}"
# 构建新的镜像名称和标签
new_image_name = f"{repo.split('/')[0]}__{repo.split('/')[1]}__{tag}"
new_tag = new_image_name
# 获取我的仓库中已备份的所有镜像
existing_images = []
url = f"https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/"
response = session.get(url)
if response.status_code == 200:
existing_images = [t["name"] for t in response.json()["results"]]
# 检查是否已经备份过这个镜像
print('\n以前备份过的镜像:')
for image in existing_images:
print(image)
print('准备要备份的镜像:\n' + new_image_name)
if new_image_name in existing_images:
print('"准备要备份的镜像"已包含在"以前备份过的镜像"中')
print(f"{new_image_name} 镜像已备份,跳过。")
continue
# 打印源仓库和备份仓库的对比信息
print('以前没有备份过这个镜像')
print(f"{old_image_name} 将被备份为 {new_image_name} 到 {backup_repo_prefix}:{new_tag}")
# 备份 Docker 镜像
new_image = f"{backup_repo_prefix}:{new_tag}"
print(f"备份 {old_image_name} 到 {new_image}...")
try:
image = client.images.pull(old_image_name)
image.tag(new_image)
client.images.push(new_image)
backed_up_images.append(new_image_name)
total_new_images_to_backup += 1
except Exception as e:
print(f"备份镜像 {old_image_name} 失败: {str(e)}")
failed_images.append(new_image_name)
total_failed_images += 1
# 删除已备份的标签
tags_to_backup.remove(tag)
# 清空标签列表
del tags_to_backup[:]
# 清理本地系统上不再需要的镜像
client.images.prune()
# 打印备份信息
total_backed_up_images = len(backed_up_images)
print(f"总共备份过 {total_backed_up_images} 个镜像。")
print(f"本次总共需要备份 {total_new_images_to_backup} 个新的镜像。")
if total_backed_up_images > 0:
print(f"实际备份的所有镜像为 {', '.join(backed_up_images)},共备份了 {total_backed_up_images} 个镜像。")
else:
print("没有备份任何镜像。")
# 打印备份失败的镜像信息
total_failed_images = len(failed_images)
if total_failed_images > 0:
print(f"备份失败的镜像: {', '.join(failed_images)},共有{total_failed_images}个。")
else:
print("所有镜像备份成功!")
VERSION 5
#!/usr/bin/env python3
import os
import requests
from concurrent.futures import ThreadPoolExecutor, as_completed
import docker
def get_all_tags(repo, session):
"""
获取指定仓库的所有标签
"""
print(f"获取{repo}的所有标签...")
url = f"https://registry.hub.docker.com/v2/repositories/{repo}/tags/"
response = session.get(url)
response.raise_for_status()
tags = [t["name"] for t in response.json()["results"]]
for tag in tags:
print(f"发现标签: {tag}")
yield tag
repos_to_backup = [
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}",
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}"
]
backup_repo_prefix = "{YOUR_DOCKERHUB_USERNAME}/{YOUR_DOCKERHUB_REPOSITORY}"
print("登录 Docker Hub...")
client = docker.from_env()
client.login(username=os.environ['DOCKERHUB_USERNAME'], password=os.environ['DOCKERHUB_PASSWORD'])
backed_up_images = []
failed_images = []
with ThreadPoolExecutor() as executor:
total_new_images_to_backup = 0
total_failed_images = 0
with requests.Session() as session:
futures = []
for repo in repos_to_backup:
future = executor.submit(get_all_tags, repo, session)
futures.append(future)
for future in as_completed(futures):
tags_to_backup = list(future.result())
if len(tags_to_backup) == 0:
print(f"{repo}没有标签,跳过。")
continue
print(f"找到的标签: {', '.join(tags_to_backup)}")
total_new_images_to_backup += len(tags_to_backup)
tag_futures = []
for tag in tags_to_backup:
old_image_name = f"{repo}:{tag}"
new_image_name = f"{repo.split('/')[0]}__{repo.split('/')[1]}__{tag}"
new_tag = new_image_name
existing_images = []
url = f"https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/"
response = session.get(url)
if response.status_code == 200:
existing_images = [t["name"] for t in response.json()["results"]]
if new_image_name in existing_images:
print(f"{new_image_name} 镜像已备份,跳过。")
backed_up_images.append(new_image_name)
continue
new_image = f"{backup_repo_prefix}:{new_tag}"
print(f"备份 {old_image_name} 到 {new_image}...")
try:
image = client.images.pull(old_image_name)
image.tag(new_image)
client.images.push(new_image)
backed_up_images.append(new_image_name)
except Exception as e:
print(f"备份镜像 {old_image_name} 失败: {str(e)}")
failed_images.append(new_image_name)
total_failed_images += 1
# 清理本地系统上不再需要的镜像
client.images.prune()
# 打印备份信息
total_backed_up_images = len(backed_up_images)
print(f"总共备份过 {total_backed_up_images} 个镜像。")
print(f"本次总共需要备份 {total_new_images_to_backup} 个新的镜像。")
if total_backed_up_images > 0:
print(f"实际备份的所有镜像为 {', '.join(backed_up_images)},共备份了 {total_backed_up_images} 个镜像。")
else:
print("没有需要备份的镜像。")
if total_failed_images > 0:
print(f"备份失败的镜像有:")
for image in failed_images:
print(image)
print(f"共备份失败 {total_failed_images} 个镜像。")
else:
print("所有镜像备份成功!")
print(f"本次实际需要备份 {total_new_images_to_backup - total_backed_up_images} 个新的镜像。")
VERSION 4
import os
import requests
from concurrent.futures import ThreadPoolExecutor, as_completed
import docker
# 获取指定仓库的所有标签
def get_all_tags(repo, session):
print(f"获取 {repo} 的所有标签...")
url = f"https://registry.hub.docker.com/v2/repositories/{repo}/tags/"
response = session.get(url)
response.raise_for_status()
tags = [t["name"] for t in response.json()["results"]]
for tag in tags:
print(f"发现标签: {tag}")
yield tag
# 定义需要备份的镜像列表和备份后的仓库前缀
repos_to_backup = [
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}",
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}"
]
backup_repo_prefix = "{YOUR_DOCKERHUB_USERNAME}/{YOUR_DOCKERHUB_REPOSITORY}"
# 登录到 Docker Hub
print("登录 Docker Hub...")
client = docker.from_env()
client.login(username=os.environ['DOCKERHUB_USERNAME'], password=os.environ['DOCKERHUB_PASSWORD'])
# 统计信息变量
total_backed_up_images = 0
total_new_images_to_backup = 0
backed_up_images = []
failed_images = []
# 并行处理每个镜像的备份任务
with ThreadPoolExecutor() as executor, requests.Session() as session:
futures = [executor.submit(get_all_tags, repo, session) for repo in repos_to_backup]
for future in as_completed(futures):
repo = repos_to_backup[futures.index(future)]
tags_to_backup = list(future.result())
if not tags_to_backup: # 检查是否有要备份的标签
print(f"{repo} 没有标签,跳过。")
continue
print(f"{repo} 找到的标签: ")
for tag in tags_to_backup:
print(f"\t{tag}")
# 并行处理每个标签的备份任务
tag_futures = []
for tag in tags_to_backup:
total_backed_up_images += 1
# 构建原始镜像名称和标签
old_image_name = f"{repo}:{tag}"
# 构建新的镜像名称和标签
new_image_name = f"{repo.split('/')[0]}__{repo.split('/')[1]}__{tag}"
new_tag = new_image_name
# 获取我的仓库中已备份的所有镜像
url = f"https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/"
print(f"检查是否备份过 {new_image_name}...")
response = session.get(url)
if response.status_code == 200:
existing_images = [t["name"] for t in response.json()["results"]]
if len(existing_images) > 0:
print("在备份列表中找到的镜像:")
for existing_image in existing_images:
print(f"\t{existing_image}")
else:
print("在备份列表中未找到任何镜像")
# 将需要备份的镜像添加到现有镜像列表中
existing_images.append(new_image_name)
else:
existing_images = []
print(f"无法获取备份镜像列表,状态码为 {response.status_code}")
# 检查是否已经备份过这个镜像
if new_image_name in existing_images:
print(f"{new_image_name} 镜像已备份,跳过。")
continue
total_new_images_to_backup += 1
# 备份 Docker 镜像
new_image = f"{backup_repo_prefix}:{new_tag}"
print(f"备份 {old_image_name} 到 {new_image}...")
try:
image = client.images.pull(old_image_name)
image.tag(new_image)
client.images.push(new_image)
backed_up_images.append(new_image_name)
except Exception as e:
print(f"备份镜像 {old_image_name} 失败: {str(e)}")
failed_images.append(new_image_name)
# 清理本地系统上不再需要的镜像
client.images.prune()
# 打印统计信息
print(f"总共备份过 {total_backed_up_images} 个镜像。")
print(f"本次总共需要备份 {total_new_images_to_backup} 个新的")
print(f"需要备份的所有镜像为 {backed_up_images},共备份 {len(backed_up_images)} 个镜像。")
if failed_images:
print(f"备份失败 {failed_images},共 {len(failed_images)} 个镜像。")
# 登出 Docker Hub
print("登出 Docker Hub...")
client.close()
client = None
VERSION 3
#!/usr/bin/env python3
import os
import requests
from concurrent.futures import ThreadPoolExecutor, as_completed
import docker
# 获取指定仓库的所有标签
def get_all_tags(repo, session):
print(f"获取{repo}的所有标签...")
url = f"https://registry.hub.docker.com/v2/repositories/{repo}/tags/"
response = session.get(url)
response.raise_for_status()
tags = [t["name"] for t in response.json()["results"]]
for tag in tags:
print(f"发现标签: {tag}")
yield tag
# 定义需要备份的镜像列表和备份后的仓库前缀
repos_to_backup = [
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}",
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}"
]
backup_repo_prefix = "{YOUR_DOCKERHUB_USERNAME}/{YOUR_DOCKERHUB_REPOSITORY}"
# 登录到 Docker Hub
print("登录 Docker Hub...")
client = docker.from_env()
client.login(username=os.environ['DOCKERHUB_USERNAME'], password=os.environ['DOCKERHUB_PASSWORD'])
# 并行处理每个镜像的备份任务
with ThreadPoolExecutor() as executor:
futures = []
with requests.Session() as session:
for repo in repos_to_backup:
# 获取给定镜像的所有标签
future = executor.submit(get_all_tags, repo, session)
futures.append(future)
for future in as_completed(futures):
tags_to_backup = list(future.result())
if len(tags_to_backup) == 0:
print(f"{repo}没有标签,跳过。")
continue
print(f"找到的标签: {', '.join(tags_to_backup)}")
# 并行处理每个标签的备份任务
tag_futures = []
for tag in tags_to_backup:
# 构建原始镜像名称和标签
old_image_name = f"{repo}:{tag}"
# 构建新的镜像名称和标签
new_image_name = f"{repo.split('/')[0]}__{repo.split('/')[1]}__{tag}"
new_tag = new_image_name
# 获取我的仓库中已备份的所有镜像
existing_images = []
url = f"https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/"
response = session.get(url)
if response.status_code == 200:
existing_images = [t["name"] for t in response.json()["results"]]
# 检查是否已经备份过这个镜像
if new_image_name in existing_images:
print(f"{new_image_name} 镜像已备份,跳过。")
continue
# 备份 Docker 镜像
new_image = f"{backup_repo_prefix}:{new_tag}"
print(f"备份 {old_image_name} 到 {new_image}...")
try:
image = client.images.pull(old_image_name)
image.tag(new_image)
client.images.push(new_image)
except Exception as e:
print(f"备份镜像 {old_image_name} 失败: {str(e)}")
# 清理本地系统上不再需要的镜像
client.images.prune()
VERSION 2
使用了 docker 库来处理 Docker 相关操作,通过并发处理器 ThreadPoolExecutor 和 as_completed 函数以及 requests.Session 对象来提高效率。
#!/usr/bin/env python3
import requests
from concurrent.futures import ThreadPoolExecutor, as_completed
import docker
# 设置 Docker Hub 用户名和密码
dockerhub_username = "YOUR_DOCKERHUB_USERNAME"
dockerhub_password = "YOUR_DOCKERHUB_PASSWORD"
# 设置需要备份的镜像列表和备份后的仓库前缀
repos_to_backup = [
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}",
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}"
]
# 设置备份后的仓库前缀,所有备份的镜像将会被推送到这个仓库中
# 注意:请确保你拥有这个仓库的写权限,否则镜像推送将失败。
backup_repo_prefix = "{YOUR_DOCKERHUB_USERNAME}/{YOUR_DOCKERHUB_REPOSITORY}"
# 登录到 Docker Hub
print("登录 Docker Hub...")
client = docker.from_env()
client.login(username=dockerhub_username, password=dockerhub_password)
# 获取指定仓库的所有标签
def get_all_tags(repo, session):
print(f"获取{repo}的所有标签...")
url = f"https://registry.hub.docker.com/v2/repositories/{repo}/tags/"
response = session.get(url)
response.raise_for_status()
tags = [t["name"] for t in response.json()["results"]]
for tag in tags:
print(f"发现标签: {tag}")
yield tag
# 并行处理每个镜像的备份任务
with ThreadPoolExecutor() as executor:
futures = []
with requests.Session() as session:
for repo in repos_to_backup:
# 获取给定镜像的所有标签
future = executor.submit(get_all_tags, repo, session)
futures.append(future)
for future in as_completed(futures):
repo = repos_to_backup[futures.index(future)]
tags_to_backup = list(future.result())
if len(tags_to_backup) == 0:
print(f"{repo}没有标签,跳过。")
continue
print(f"找到的标签: {', '.join(tags_to_backup)}")
# 并行处理每个标签的备份任务
tag_futures = []
for tag in tags_to_backup:
# 构建原始镜像名称和标签
old_image_name = f"{repo}:{tag}"
# 构建新的镜像名称和标签
new_image_name = f"{repo.split('/')[0]}__{repo.split('/')[1]}__{tag}"
new_tag = new_image_name
# 获取我的仓库中已备份的所有镜像
existing_images = []
url = f"https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/"
response = session.get(url)
if response.status_code == 200:
existing_images = [t["name"] for t in response.json()["results"]]
# 检查是否已经备份过这个镜像
if new_image_name in existing_images:
print(f"{new_image_name} 镜像已备份,跳过。")
continue
# 备份 Docker 镜像
new_image = f"{backup_repo_prefix}:{new_tag}"
print(f"备份 {old_image_name} 到 {new_image}...")
try:
image = client.images.pull(old_image_name)
image.tag(new_image)
client.images.push(new_image)
except Exception as e:
print(f"备份镜像 {old_image_name} 失败: {str(e)}")
# 清理本地系统上不再需要的镜像
client.images.prune()
VERSION 1
使用 requests 库来发送 HTTP 请求获取镜像标签信息,使用 os.system 函数执行 shell 命令来备份和清理本地系统上的镜像。
#!/usr/binzh /env python3
import os
import requests
# 设置Docker Hub用户名和密码
DOCKERHUB_USERNAME = "{YOUR_DOCKERHUB_USERNAME}"
DOCKERHUB_PASSWORD = "YOUR_DOCKERHUB_PASSWORD"
# 获取指定仓库的所有标签
def get_all_tags(repo):
print(f"获取{repo}的所有标签...")
response = requests.get(f"https://registry.hub.docker.com/v2/repositories/{repo}/tags/")
response.raise_for_status()
tags = [t["name"] for t in response.json()["results"]]
for tag in tags:
print(f"发现标签: {tag}")
yield tag
# 定义需要备份的镜像列表和备份后的仓库前缀
repos_to_backup = [
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}",
"{OTHER_DOCKERHUB_USERNAME}/{OTHER_DOCKERHUB_REPOSITORY}"
]
# 设置备份后的仓库前缀,所有备份的镜像将会被推送到这个仓库中
# 注意:请确保你拥有这个仓库的写权限,否则镜像推送将失败。
backup_repo_prefix = "{YOUR_DOCKERHUB_USERNAME}/{YOUR_DOCKERHUB_REPOSITORY}"
# 登录到 Docker Hub
print("登录 Docker Hub...")
os.system(f"echo '{DOCKERHUB_PASSWORD}' | docker login --username='{DOCKERHUB_USERNAME}' --password-stdin")
# 遍历要备份的镜像
for repo in repos_to_backup:
# 获取给定镜像的所有标签
tags_to_backup = list(get_all_tags(repo))
if len(tags_to_backup) == 0:
print(f"{repo}没有标签,跳过。")
continue
print(f"找到的标签: {', '.join(tags_to_backup)}")
# 遍历要备份的标签
for tag in tags_to_backup:
# 构建原始镜像名称和标签
old_image_name = f"{repo}:{tag}"
# 构建新的镜像名称和标签
new_image_name = f"{repo.split('/')[0]}__{repo.split('/')[1]}__{tag}"
new_tag = new_image_name
# 获取我的仓库中已备份的所有镜像
existing_images = []
response = requests.get(f"https://registry.hub.docker.com/v2/repositories/{backup_repo_prefix}/tags/")
if response.status_code == 200:
existing_images = [t["name"] for t in response.json()["results"]]
# 检查是否已经备份过这个镜像
if new_image_name in existing_images:
print(f"{new_image_name} 镜像已备份,跳过。")
continue
# 备份 Docker 镜像
new_image = f"{backup_repo_prefix}:{new_tag}"
print(f"备份 {old_image_name} 到 {new_image}...")
if os.system(f"docker pull {old_image_name}") != 0:
print(f"拉取镜像 {old_image_name} 失败,跳过。")
continue
if os.system(f"docker tag {old_image_name} {new_image}") != 0:
print(f"将镜像 {old_image_name} 标记为 {new_image} 失败,跳过。")
continue
if os.system(f"docker push {new_image}") != 0:
print(f"推送镜像 {new_image} 失败,跳过。")
continue
# 清理本地系统上不再需要的镜像
os.system("docker image prune -af")