На днях, появилась задача — выполнить миграцию docker-registry в AWS ECR и сделать нормальную URL для AWS ECRб чтобы можно было работать и запомнить название домена легко. Сразу упала мысля о том, что можно сделать CNAME в AWS Route53 на уже созданный AWS ECR репозиторий, но не тут то было.
Создаю AWS Route53 CNAME, что-то типа такого:
docker-ecr.internal.vnatarov.io -> XXXXXXXXXX.dkr.ecr.us-east-1.amazonaws.comВыглядит это так:
Если вы выполните авторизацию по созданному CNAME, то получите ошибку:
$ CNAME_URL="docker-ecr.internal.vnatarov.io" &&\
aws ecr get-login-password --region us-east-1 --profile default | docker login --username AWS --password-stdin $CNAME_URL
Error response from daemon: Get "https://docker-ecr.internal.vnatarov.io/v2/": x509: certificate is valid for *.dkr.ecr.us-east-1.amazonaws.com, *.dkr.ecr.us-east-1.vpce.amazonaws.com, not docker-ecr.internal.vnatarov.io
Т.е нельзя использовать CNAME т.к Амазон подписал AWS ECR домен своим сертификатом. Окай…. Не логично, но ладно! Я думал все у Амазан будет интуитивно легко и просто — нет.
Подумав немного, придумал пару решений:
- Написать свою обретку при работе с AWS ECR.
- Использовать proxy.
- Попробовать использовать Amazon ECR interface VPC endpoints (AWS PrivateLink).
Приступим!
Pull/Push AWS ECR образов через AWS Route53 CNAME
Покопавшись в REST API Amazon (boto3), я увидел что можно подложить хеадеры и выполнить авторизацию и получить репозитории с AWS ECR. Сразу решил проверить, например вот так:
$ CNAME_URL="https://docker-ecr.internal.vnatarov.io" &&\
AWS_ACCOUNT_ID="XXXXXXXXXXXXXXX" &&\
AWS_ECR_REPO_NAME="repo-name" &&\
TOKEN=$(aws ecr get-authorization-token --region us-east-1 --profile default --output text --query 'authorizationData[].authorizationToken') && curl -k -i -H "Host: $AWS_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com" -H "Authorization: Basic $TOKEN" -H "X-Forwarded-Proto: https" -H "X-Forwarded-For 127.0.0.1" -H "X-Real-IP: 93.72.109.140" $CNAME_URL/v2/$AWS_ECR_REPO_NAME/tags/list
Вывод команды:
HTTP/1.1 200 OK
Docker-Distribution-Api-Version: registry/2.0
Link: <https://XXXXXXXXXXXXXXX.dkr.ecr.us-east-1.amazonaws.com/v2/repo-name/tags/list?last=ukD72mdD%2FmC8b5xV3susmJzzaTgp3hKwR9nRUW1yZZ7uIM42T5Qni9DYGx4CJpnhNeJ6HnjEzAukCdhHEgsIYR6ZPbIPWoMte%2Bkmss2BWER%2BOoldEmVa6n9tG88nfx8J3qz2X7nNmzKTvj9S75hqjlupun8iXyGm2Cef6EHKr6JqI7jXVAS0aBWToKUCsATn1R2LRKTxsdsk7HLTSmjieo3kdVioZ6%2F0%2BsiGagwFc6QZfrwH1%2Bl%2F%2Ba1ritf4IwZUXVC4kuID%2BzHKxz9rtgMBgDDBA1yjW8hZqO3K2tOa77h3i%2BPVqj6aHA096YMRh5BFiiLWPGgss0L4QQahaooOLRRg7kdr5k%2FZIqllcmGaLbioNLy3R5jOon7X61YbIGF7fUOkssj72o37fpPd%2FJG2g%3D%3D>; rel="next"
Date: Wed, 17 Nov 2021 11:06:47 GMT
Content-Type: text/plain; charset=utf-8
Transfer-Encoding: chunked
{"name":"repo-name","tags":["943bf60acceb327ee85717d2b52cfd97374f4340","ade31de1df53b7c96575f6f27bec17c9f7702cb8"]}С вывода команды можно понять что подложив хеадеры, можно получить репы. Пишем питон-скрипт…. Потребовалось часов 12 на написания данного скрипта и он довольно стабилен в работе.
Установкой Питон и pip я не буду. Но если нет в систему — ставим. А тем временем, нужно установить следующие пакеты:
# A Fast, Extensible Progress Bar for Python and CLI
tqdm==4.62.3
# AWS
boto3~=1.19.3
botocore~=1.22.3
# Others
setuptools~=57.4.0
urllib3~=1.22Python скрипт выглядит:
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import argparse
import json
import logging
import os
import shutil
import tarfile
import time
import urllib
from tempfile import mkdtemp
import boto3
import botocore
import botocore.session
import urllib3
from botocore.config import Config
# Initialize Logger
logger = logging.getLogger()
logger.setLevel(logging.INFO)
logging.captureWarnings(True)
CACHE_DIR_ROOT = os.path.expanduser("~")
assert os.path.isdir(CACHE_DIR_ROOT)
CACHE_DIR = CACHE_DIR_ROOT + '/.docker-pull-layers-cache/'
if not os.path.exists(CACHE_DIR):
print("Creating cache directory: " + CACHE_DIR)
os.makedirs(CACHE_DIR)
def ec2_connector(aws_settings):
if (aws_settings['client'] is not None) and (aws_settings['region'] is not None):
try:
session = botocore.session.get_session()
access_key = session.get_credentials().access_key
secret_key = session.get_credentials().secret_key
session_token = session.get_credentials().token
session = boto3.session.Session()
ec2 = session.client(aws_access_key_id=access_key,
aws_secret_access_key=secret_key,
aws_session_token=session_token,
service_name=aws_settings['client'],
region_name=aws_settings['region'],
config=Config(retries={'max_attempts': 3})
)
return ec2
except Exception as err:
print("Failed to create a boto3 client connection to ecr:\n", str(err))
logger.error('ERROR: Failed to create a boto3 client connection')
return False
else:
print('Please use/set [--bclient] and [--region]')
return False
def ecr_connector(aws_settings):
if (aws_settings['role_name'] is None or aws_settings['role_name'] == "None") \
and (aws_settings['role_session'] is None or aws_settings['role_session'] == "None"):
try:
session = boto3.session.Session(profile_name=aws_settings['profile_name'])
# Will retry any method call at most 3 time(s)
ecr = session.client(service_name=aws_settings['client'],
region_name=aws_settings['region'],
config=Config(retries={'max_attempts': 3})
)
return ecr
except Exception as err:
print("Failed to create a boto3 client connection to ecr:\n", str(err))
logger.error('ERROR: Failed to create a boto3 client connection to ecr')
return False
elif (aws_settings['profile_name'] is None or aws_settings['profile_name'] == "None") \
and (aws_settings['role_name'] is not None or aws_settings['role_name'] != "None") \
and (aws_settings['role_session'] is not None or aws_settings['role_session'] != "None"):
try:
session = boto3.session.Session()
sts = session.client(service_name='sts',
region_name=aws_settings['region'],
config=Config(retries={'max_attempts': 3})
)
assumed_role_object = sts.assume_role(
RoleArn="{0}".format(aws_settings['role_name']),
RoleSessionName='{0}'.format(aws_settings['role_session'])
)
# can be used ay name, but need to add restriction for the name!
ecr = session.client(aws_access_key_id=assumed_role_object['Credentials']['AccessKeyId'],
aws_secret_access_key=assumed_role_object['Credentials']['SecretAccessKey'],
aws_session_token=assumed_role_object['Credentials']['SessionToken'],
service_name=aws_settings['client'],
region_name=aws_settings['region'],
config=Config(retries={'max_attempts': 3})
)
return ecr
except Exception as err:
print("Failed to create a boto3 client connection to ecr:\n", str(err))
logger.error('ERROR: Failed to create a boto3 client connection to ecr')
return False
else:
print('Please use/set [--profile-name] or [--role-name] with [--role-session]')
return False
def get_ecr_repos(aws_settings):
ecr = ecr_connector(aws_settings)
if ecr:
try:
repos = ecr.describe_repositories()
print(repos)
print("The repos:\n {}!".format(repos))
except botocore.exceptions.ClientError as err:
error_code = str(err)
logger.error('ERROR: {0}. Forbidden Access!'.format(error_code))
else:
exit(-1)
return get_ecr_repos
def get_ecr_repo(aws_settings, ecr_repo):
ecr_repo_status = False
ecr = ecr_connector(aws_settings)
if ecr:
try:
repo = ecr.describe_repositories(repositoryNames=[ecr_repo])
# print("A repo {} is already exists!".format(ecr_repo))
ecr_repo_status = repo
return ecr_repo_status
except botocore.exceptions.ClientError as err:
error_code = str(err)
logger.error('ERROR: {0}. Forbidden Access!'.format(error_code))
ecr_repo_status = False
return ecr_repo_status
else:
exit(-1)
return ecr_repo_status
def get_authorization_token(aws_settings, ecr_repo):
global auth_token
ecr = ecr_connector(aws_settings)
ecr_repo = get_ecr_repo(aws_settings, ecr_repo)
ecr_repo_id = ecr_repo['repositories'][0]['registryId']
if ecr:
response = ecr.get_authorization_token(registryIds=[ecr_repo_id])
if response['ResponseMetadata']['HTTPStatusCode'] == 200:
auth_token = response['authorizationData'][0]['authorizationToken']
# print("Authorization token: ", auth_token)
elif response['ResponseMetadata']['HTTPStatusCode'] == 401:
print("You're not authorized")
exit(1)
else:
exit(-1)
return auth_token
def http_request(method='GET', url='', h=None, retries=False, timeout=30):
if h is None:
h = {}
http = urllib3.PoolManager()
response = http.request(method=method,
url=url,
headers=h,
retries=retries,
timeout=timeout)
if response.status == 200:
# print("response: ", response.data.decode('utf-8'))
logger.info('INFO: Successfully.....')
return response
elif response.status == 307:
# logger.error('HTTP 307 Temporary Redirect redirect status response')
return response
elif response.status == 401:
logger.error('ERROR: Please authorize, the issue: \n\t {}'.format(response.data.decode('utf-8')))
return response
elif response.status == 404:
logger.error('ERROR: 404 page not found')
return response
else:
logger.error('FAILURE: Got an error: \n\t {}'.format(response.data.decode('utf-8')))
return response
def urllib_request_urlopen(url):
response = urllib.request.urlopen(url).read()
return response
def downloading_layer(cache_dir=CACHE_DIR, layer_url=None, out_path='/', h=None):
"""
Get a layer in a compressed format, and saves it locally (unzipped).
The tar name is expected to contain a hash, thus to be cacheable.
"""
if h is None:
h = {}
cache_name = cache_dir + layer_url.split("/")[6].replace(':', '_')
repo_name = layer_url.split("/")[4]
response_headers = {}
if not os.path.exists(cache_name):
response = http_request(method='GET', url=layer_url, h=h, retries=False, timeout=30)
for key, val in response.headers.iteritems():
d = {key: val}
response_headers.update(d)
layer_location = response_headers['Location']
layer_location_data = urllib_request_urlopen(layer_location)
with open(cache_name, mode='wb') as localfile:
localfile.write(layer_location_data)
shutil.move(cache_name, cache_name)
os.makedirs(out_path[:out_path.rfind("/")], exist_ok=True)
shutil.copyfile(cache_name, out_path)
return downloading_layer
def ecr_pull(aws_settings, cname_url, ecr_repo, ecr_tag="latest", h=None):
if h is None:
h = {}
ecr = ecr_connector(aws_settings)
if ecr:
try:
manifests_url = cname_url + '/v2/{0}/manifests/{1}'.format(ecr_repo, ecr_tag)
response = http_request('GET', manifests_url, h)
web_manifest = json.loads(response.data.decode('utf-8'))
config_digest = web_manifest['config']['digest']
# ------------------------------------------------------------------------
config = cname_url + '/v2/{0}/blobs/{1}'.format(ecr_repo, config_digest)
response = http_request('GET', config, h)
response_headers = {}
for key, val in response.headers.iteritems():
d = {key: val}
response_headers.update(d)
config_location = response_headers['Location']
config_location_file = urllib_request_urlopen(config_location).decode('utf-8')
config_filename = config_digest.split(':')[1] + '.json'
with open(temp_dir + '/' + config_filename, 'w') as outfile:
json.dump(json.loads(config_location_file), outfile)
# ------------------------------------------------------------------------
layer_path_l = []
for layer in web_manifest['layers']:
layer_url = cname_url + '/v2/{0}/blobs/{1}'.format(ecr_repo, layer['digest'])
path = layer['digest'].split(':')[-1] + "/layer.tar"
out_path = temp_dir + '/' + path
downloading_layer(cache_dir=CACHE_DIR, layer_url=layer_url, out_path=out_path, h=h)
layer_path_l.append(path)
manifest = [{"Config": config_filename, "RepoTags": [], "Layers": layer_path_l}]
print("config_filename: ", config_filename)
print("manifest: ", manifest)
with open(temp_dir + '/' + 'manifest.json', 'w') as outfile:
json.dump(manifest, outfile)
with tarfile.open(ecr_tag, "w") as tar_out:
os.chdir(temp_dir)
tar_out.add(".")
except Exception as e:
logger.error('ERROR: {0}'.format(str(e)))
print(e)
exit(1)
else:
print("Could not get connect to AWS ECR or get needed repository")
exit(-1)
return ecr_pull
def ecr_push():
print("TBD")
return ecr_push
def ec2_handler(aws_settings, ecr_repo):
ecr = ec2_connector(aws_settings)
# TBD
return ec2_handler
if __name__ == '__main__':
start__time = time.time()
parser = argparse.ArgumentParser(prog='python3 script_name.py -h',
usage='python3 script_name.py {ARGS}',
add_help=True,
prefix_chars='--/',
epilog='''created by Vitalii Natarov'''
)
parser.add_argument('--version', action='version', version='v0.2.0')
parser.add_argument('--bclient', dest='boto3_client', help='Set boto3 client', default='ecr')
parser.add_argument('--region', dest='region', help='Set AWS region for boto3', default='us-east-1')
parser.add_argument('--pname', '--profile', dest='profile_name', help='Set profile name of AWS',
default=None)
parser.add_argument('--rname', '--role-name', dest='role_name', help='Set role ARN name',
default=None)
parser.add_argument('--rsession', '--role-session', dest='role_session', help='Set role session name',
default=None)
parser.add_argument('--registry-url', '--url', dest='registry_url', help='Set Registry URL',
default="https://docker-ecr.internal.logz.io")
parser.add_argument('--ecr-url', dest='ecr_url', help='Set URL from ECR registry',
default="406095609952.dkr.ecr.us-east-1.amazonaws.com")
parser.add_argument('--ecr-repo', '-repo', dest='ecr_repo', help='Set ECR repo name',
default="accounts")
parser.add_argument('--ecr-repo-tag', '-repo-tag', dest='ecr_repo_tag', help='Set ECR repo tag',
default="8f2b55cc45894247d880f7c4ac2042ed7c398dfa")
parser.add_argument('--ec2', dest='aws_ec2', help='Set ec2 usage', default='False')
results = parser.parse_args()
boto3_client = results.boto3_client
region = results.region
profile_name = results.profile_name
role_name = results.role_name
role_session = results.role_session
registry_url = results.registry_url
ecr_url_host = results.ecr_url
ecr_repo_name = results.ecr_repo
ecr_repo_tag = results.ecr_repo_tag
aws_ec2 = results.aws_ec2
try:
temp_dir = mkdtemp()
if aws_ec2 == 'True':
print("AWS EC2!")
aws_auth = {
"client": boto3_client,
"region": region,
"ec2_host": 'True'
}
ec2_handler(aws_auth)
else:
print("Local Usage!")
aws_auth = {
"client": boto3_client,
"region": region,
"profile_name": profile_name,
"role_name": role_name,
"role_session": role_session
}
authorization_token = get_authorization_token(aws_auth, ecr_repo_name)
headers = {
'Host': str(ecr_url_host),
'Accept': 'text/plain',
'X-Forwarded-Proto': 'https',
'X-Forwarded-For': '127.0.0.1',
'X-Real-IP': '66.66.66.66',
'Authorization': 'Basic {}'.format(str(authorization_token))
}
ecr_pull(aws_auth, registry_url, ecr_repo_name, ecr_repo_tag, headers)
finally:
shutil.rmtree(temp_dir)
end__time = round(time.time() - start__time, 2)
print("--- %s seconds ---" % end__time)
Использование — простое:
$ python3 ecr-containers.py --profile=default
config_filename: 2a5d611b6675bbcb15fea6050fdf5af2d974494511967e1ccd816efbe95fa348.json
manifest: [{'Config': '2a5d611b6675bbcb15fea6050fdf5af2d974494511967e1ccd816efbe95fa348.json', 'RepoTags': [], 'Layers': ['339de151aab4bc06eed8409daae147c408478cb538dacb90cc63f19ad4eba80b/layer.tar', 'c79c496f1543a6cf36a79305de7c709ea55c16b63ea86219c4b98c334b70488d/layer.tar', 'a5382bad4f98c8f742d943e2bd6930626b06af081c57062e06a501d449a4c472/layer.tar', '348e701aca2b0acab9fab3daaa10980e5507b8e55334671f503d1d5f5ef739a5/layer.tar', '965b2a30aa9c5ffea05d772b418c428cde9fa3f85faf37b2b66b719593f4c7b6/layer.tar', '9ee6edf857abe493670c386faa46127a9decea128da3fcafe25015afa8f6b9ef/layer.tar', '75f33f1b8076396b9a688c579ffb6ee4e7d7c83e94a9ba4d15b97a7efb118db7/layer.tar', 'a79e84406bd9b1f9c48e60589691a9ef807c7416cf1b2885eafc95fbbd3bbd8c/layer.tar', 'e5279c73940e27e2d0e32d8c288c32a795eeb12d4bf876e27067a7a49d032d6c/layer.tar', 'b088ed07f0856dbc5d7451536d3379a8c0dee79ddf98f7bdcef98e37549669fa/layer.tar', '09c3905aed6a2dd924c4a3fed0802d3cc191cdfc317c960ea9c43c9f8a4379c5/layer.tar', '7486d5d0bd29e293ed034751b9743c3f30405aa4746f353151ed44e2d827fc76/layer.tar']}]
--- 3.93 seconds ---Скрипт поддерживает простой хелп, вызвать его можно так:
$ python3 ecr-containers.py -h
usage: python3 script_name.py {ARGS}
optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit
--bclient BOTO3_CLIENT
Set boto3 client
--region REGION Set AWS region for boto3
--pname PROFILE_NAME, --profile PROFILE_NAME
Set profile name of AWS
--rname ROLE_NAME, --role-name ROLE_NAME
Set role ARN name
--rsession ROLE_SESSION, --role-session ROLE_SESSION
Set role session name
--registry-url REGISTRY_URL, --url REGISTRY_URL
Set Registry URL
--ecr-url ECR_URL Set URL from ECR registry
--ecr-repo ECR_REPO, -repo ECR_REPO
Set ECR repo name
--ecr-repo-tag ECR_REPO_TAG, -repo-tag ECR_REPO_TAG
Set ECR repo tag
--ec2 AWS_EC2 Set ec2 usage
created by Vitalii Natarov
После того как выполнили пуллинг образа, нужно его импортнуть в Докер:
$ docker load < 8f2b55cc45894247d880f7c4ac2042ed7c398dfa
32f366d666a5: Loading layer [==================================================>] 2.801MB/2.801MB
45449966e51a: Loading layer [==================================================>] 192.3MB/192.3MB
83ef90c94f12: Loading layer [==================================================>] 2.92MB/2.92MB
54037642f52c: Loading layer [==================================================>] 200kB/200kB
b9e5e90eb033: Loading layer [==================================================>] 42.09MB/42.09MB
539aca899eb8: Loading layer [==================================================>] 206B/206B
9d00a6a26f53: Loading layer [==================================================>] 4.668kB/4.668kB
bca0da0f1ea4: Loading layer [==================================================>] 1.631kB/1.631kB
cbb7b9b8cd31: Loading layer [==================================================>] 197.9MB/197.9MB
a0f1352db34e: Loading layer [==================================================>] 8.519kB/8.519kB
2543c13d39a0: Loading layer [==================================================>] 1.833kB/1.833kB
eb9876ca8f14: Loading layer [==================================================>] 41.2MB/41.2MB
Loaded image ID: sha256:ada32a4765be57eb1049808ebdbc7b8b6108847375383a21ffe004f3fffc3757
В плнах сделать push для работы с ECR, но это будет немного позже.
Репозиторий можно найти тут:
https://github.com/SebastianUA/ecr-pull-pushУ меня все, статья «Pull/Push AWS ECR образов через AWS Route53 CNAME» завершена!
Здравствуйте, могу ли я получить вашу контактную информацию для обсуждения делового сотрудничества?