Pull/Push AWS ECR образов через AWS Route53 CNAME

На днях, появилась задача — выполнить миграцию docker-registry в AWS ECR и сделать нормальную URL для AWS ECRб чтобы можно было работать и запомнить название домена легко. Сразу упала мысля о том, что можно сделать CNAME в AWS Route53 на уже созданный AWS ECR репозиторий, но не тут то было.

Создаю AWS Route53 CNAME, что-то типа такого:

docker-ecr.internal.vnatarov.io -> XXXXXXXXXX.dkr.ecr.us-east-1.amazonaws.com

Выглядит это так:

Если вы выполните авторизацию по созданному CNAME, то получите ошибку:

$ CNAME_URL="docker-ecr.internal.vnatarov.io" &&\
aws ecr get-login-password --region us-east-1 --profile default | docker login --username AWS --password-stdin $CNAME_URL
Error response from daemon: Get "https://docker-ecr.internal.vnatarov.io/v2/": x509: certificate is valid for *.dkr.ecr.us-east-1.amazonaws.com, *.dkr.ecr.us-east-1.vpce.amazonaws.com, not docker-ecr.internal.vnatarov.io

Т.е нельзя использовать CNAME т.к Амазон подписал AWS ECR домен своим сертификатом. Окай…. Не логично, но ладно! Я думал все у Амазан будет интуитивно легко и просто — нет.

Подумав немного, придумал пару решений:

  • Написать свою обретку при работе с AWS ECR.
  • Использовать proxy.
  • Попробовать использовать Amazon ECR interface VPC endpoints (AWS PrivateLink).

Приступим!

Pull/Push AWS ECR образов через AWS Route53 CNAME

Покопавшись в REST API Amazon (boto3), я увидел что можно подложить хеадеры и выполнить авторизацию и получить репозитории с AWS ECR. Сразу решил проверить, например вот так:

$ CNAME_URL="https://docker-ecr.internal.vnatarov.io" &&\
AWS_ACCOUNT_ID="XXXXXXXXXXXXXXX" &&\
AWS_ECR_REPO_NAME="repo-name" &&\
TOKEN=$(aws ecr get-authorization-token --region us-east-1 --profile default --output text --query 'authorizationData[].authorizationToken') && curl -k -i -H "Host: $AWS_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com" -H "Authorization: Basic $TOKEN" -H "X-Forwarded-Proto: https" -H "X-Forwarded-For 127.0.0.1" -H "X-Real-IP: 93.72.109.140" $CNAME_URL/v2/$AWS_ECR_REPO_NAME/tags/list

Вывод команды:

HTTP/1.1 200 OK
Docker-Distribution-Api-Version: registry/2.0
Link: <https://XXXXXXXXXXXXXXX.dkr.ecr.us-east-1.amazonaws.com/v2/repo-name/tags/list?last=ukD72mdD%2FmC8b5xV3susmJzzaTgp3hKwR9nRUW1yZZ7uIM42T5Qni9DYGx4CJpnhNeJ6HnjEzAukCdhHEgsIYR6ZPbIPWoMte%2Bkmss2BWER%2BOoldEmVa6n9tG88nfx8J3qz2X7nNmzKTvj9S75hqjlupun8iXyGm2Cef6EHKr6JqI7jXVAS0aBWToKUCsATn1R2LRKTxsdsk7HLTSmjieo3kdVioZ6%2F0%2BsiGagwFc6QZfrwH1%2Bl%2F%2Ba1ritf4IwZUXVC4kuID%2BzHKxz9rtgMBgDDBA1yjW8hZqO3K2tOa77h3i%2BPVqj6aHA096YMRh5BFiiLWPGgss0L4QQahaooOLRRg7kdr5k%2FZIqllcmGaLbioNLy3R5jOon7X61YbIGF7fUOkssj72o37fpPd%2FJG2g%3D%3D>; rel="next"
Date: Wed, 17 Nov 2021 11:06:47 GMT
Content-Type: text/plain; charset=utf-8
Transfer-Encoding: chunked

{"name":"repo-name","tags":["943bf60acceb327ee85717d2b52cfd97374f4340","ade31de1df53b7c96575f6f27bec17c9f7702cb8"]}

С вывода команды можно понять что подложив хеадеры, можно получить репы. Пишем питон-скрипт…. Потребовалось часов 12 на написания данного скрипта и он довольно стабилен в работе.

Установкой Питон и pip я не буду. Но если нет в систему — ставим. А тем временем, нужно установить следующие пакеты:

# A Fast, Extensible Progress Bar for Python and CLI
tqdm==4.62.3

# AWS
boto3~=1.19.3
botocore~=1.22.3

# Others
setuptools~=57.4.0
urllib3~=1.22

Python скрипт выглядит:

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

import argparse
import json
import logging
import os
import shutil
import tarfile
import time
import urllib
from tempfile import mkdtemp

import boto3
import botocore
import botocore.session
import urllib3
from botocore.config import Config

# Initialize Logger
logger = logging.getLogger()
logger.setLevel(logging.INFO)
logging.captureWarnings(True)

CACHE_DIR_ROOT = os.path.expanduser("~")
assert os.path.isdir(CACHE_DIR_ROOT)
CACHE_DIR = CACHE_DIR_ROOT + '/.docker-pull-layers-cache/'

if not os.path.exists(CACHE_DIR):
    print("Creating cache directory: " + CACHE_DIR)
    os.makedirs(CACHE_DIR)


def ec2_connector(aws_settings):
    if (aws_settings['client'] is not None) and (aws_settings['region'] is not None):
        try:
            session = botocore.session.get_session()
            access_key = session.get_credentials().access_key
            secret_key = session.get_credentials().secret_key
            session_token = session.get_credentials().token

            session = boto3.session.Session()
            ec2 = session.client(aws_access_key_id=access_key,
                                 aws_secret_access_key=secret_key,
                                 aws_session_token=session_token,
                                 service_name=aws_settings['client'],
                                 region_name=aws_settings['region'],
                                 config=Config(retries={'max_attempts': 3})
                                 )
            return ec2
        except Exception as err:
            print("Failed to create a boto3 client connection to ecr:\n", str(err))
            logger.error('ERROR: Failed to create a boto3 client connection')
            return False
    else:
        print('Please use/set [--bclient] and [--region]')
        return False


def ecr_connector(aws_settings):
    if (aws_settings['role_name'] is None or aws_settings['role_name'] == "None") \
            and (aws_settings['role_session'] is None or aws_settings['role_session'] == "None"):
        try:
            session = boto3.session.Session(profile_name=aws_settings['profile_name'])
            # Will retry any method call at most 3 time(s)
            ecr = session.client(service_name=aws_settings['client'],
                                 region_name=aws_settings['region'],
                                 config=Config(retries={'max_attempts': 3})
                                 )
            return ecr
        except Exception as err:
            print("Failed to create a boto3 client connection to ecr:\n", str(err))
            logger.error('ERROR: Failed to create a boto3 client connection to ecr')
            return False
    elif (aws_settings['profile_name'] is None or aws_settings['profile_name'] == "None") \
            and (aws_settings['role_name'] is not None or aws_settings['role_name'] != "None") \
            and (aws_settings['role_session'] is not None or aws_settings['role_session'] != "None"):
        try:
            session = boto3.session.Session()
            sts = session.client(service_name='sts',
                                 region_name=aws_settings['region'],
                                 config=Config(retries={'max_attempts': 3})
                                 )

            assumed_role_object = sts.assume_role(
                RoleArn="{0}".format(aws_settings['role_name']),
                RoleSessionName='{0}'.format(aws_settings['role_session'])
            )
            # can be used ay name, but need to add restriction for the name!
            ecr = session.client(aws_access_key_id=assumed_role_object['Credentials']['AccessKeyId'],
                                 aws_secret_access_key=assumed_role_object['Credentials']['SecretAccessKey'],
                                 aws_session_token=assumed_role_object['Credentials']['SessionToken'],
                                 service_name=aws_settings['client'],
                                 region_name=aws_settings['region'],
                                 config=Config(retries={'max_attempts': 3})
                                 )

            return ecr
        except Exception as err:
            print("Failed to create a boto3 client connection to ecr:\n", str(err))
            logger.error('ERROR: Failed to create a boto3 client connection to ecr')
            return False
    else:
        print('Please use/set [--profile-name] or [--role-name] with [--role-session]')
        return False


def get_ecr_repos(aws_settings):
    ecr = ecr_connector(aws_settings)

    if ecr:
        try:
            repos = ecr.describe_repositories()
            print(repos)

            print("The repos:\n {}!".format(repos))
        except botocore.exceptions.ClientError as err:
            error_code = str(err)
            logger.error('ERROR: {0}. Forbidden Access!'.format(error_code))
    else:
        exit(-1)

    return get_ecr_repos


def get_ecr_repo(aws_settings, ecr_repo):
    ecr_repo_status = False

    ecr = ecr_connector(aws_settings)

    if ecr:
        try:
            repo = ecr.describe_repositories(repositoryNames=[ecr_repo])
            # print("A repo {} is already exists!".format(ecr_repo))
            ecr_repo_status = repo
            return ecr_repo_status
        except botocore.exceptions.ClientError as err:
            error_code = str(err)
            logger.error('ERROR: {0}. Forbidden Access!'.format(error_code))
            ecr_repo_status = False
            return ecr_repo_status
    else:
        exit(-1)

    return ecr_repo_status


def get_authorization_token(aws_settings, ecr_repo):
    global auth_token

    ecr = ecr_connector(aws_settings)

    ecr_repo = get_ecr_repo(aws_settings, ecr_repo)
    ecr_repo_id = ecr_repo['repositories'][0]['registryId']
    if ecr:
        response = ecr.get_authorization_token(registryIds=[ecr_repo_id])
        if response['ResponseMetadata']['HTTPStatusCode'] == 200:
            auth_token = response['authorizationData'][0]['authorizationToken']
            # print("Authorization token: ", auth_token)
        elif response['ResponseMetadata']['HTTPStatusCode'] == 401:
            print("You're not authorized")
            exit(1)
    else:
        exit(-1)

    return auth_token


def http_request(method='GET', url='', h=None, retries=False, timeout=30):
    if h is None:
        h = {}
    http = urllib3.PoolManager()

    response = http.request(method=method,
                            url=url,
                            headers=h,
                            retries=retries,
                            timeout=timeout)
    if response.status == 200:
        # print("response: ", response.data.decode('utf-8'))
        logger.info('INFO: Successfully.....')
        return response
    elif response.status == 307:
        # logger.error('HTTP 307 Temporary Redirect redirect status response')
        return response
    elif response.status == 401:
        logger.error('ERROR: Please authorize, the issue: \n\t {}'.format(response.data.decode('utf-8')))
        return response
    elif response.status == 404:
        logger.error('ERROR: 404 page not found')
        return response
    else:
        logger.error('FAILURE: Got an error: \n\t {}'.format(response.data.decode('utf-8')))

        return response


def urllib_request_urlopen(url):
    response = urllib.request.urlopen(url).read()

    return response


def downloading_layer(cache_dir=CACHE_DIR, layer_url=None, out_path='/', h=None):
    """
    Get a layer in a compressed format, and saves it locally (unzipped).
    The tar name is expected to contain a hash, thus to be cacheable.
    """

    if h is None:
        h = {}

    cache_name = cache_dir + layer_url.split("/")[6].replace(':', '_')
    repo_name = layer_url.split("/")[4]

    response_headers = {}
    if not os.path.exists(cache_name):
        response = http_request(method='GET', url=layer_url, h=h, retries=False, timeout=30)
        for key, val in response.headers.iteritems():
            d = {key: val}
            response_headers.update(d)

        layer_location = response_headers['Location']
        layer_location_data = urllib_request_urlopen(layer_location)

        with open(cache_name, mode='wb') as localfile:
            localfile.write(layer_location_data)
            shutil.move(cache_name, cache_name)

    os.makedirs(out_path[:out_path.rfind("/")], exist_ok=True)
    shutil.copyfile(cache_name, out_path)

    return downloading_layer


def ecr_pull(aws_settings, cname_url, ecr_repo, ecr_tag="latest", h=None):
    if h is None:
        h = {}

    ecr = ecr_connector(aws_settings)

    if ecr:
        try:
            manifests_url = cname_url + '/v2/{0}/manifests/{1}'.format(ecr_repo, ecr_tag)
            response = http_request('GET', manifests_url, h)
            web_manifest = json.loads(response.data.decode('utf-8'))
            config_digest = web_manifest['config']['digest']

            # ------------------------------------------------------------------------
            config = cname_url + '/v2/{0}/blobs/{1}'.format(ecr_repo, config_digest)
            response = http_request('GET', config, h)

            response_headers = {}
            for key, val in response.headers.iteritems():
                d = {key: val}
                response_headers.update(d)

            config_location = response_headers['Location']
            config_location_file = urllib_request_urlopen(config_location).decode('utf-8')

            config_filename = config_digest.split(':')[1] + '.json'
            with open(temp_dir + '/' + config_filename, 'w') as outfile:
                json.dump(json.loads(config_location_file), outfile)

            # ------------------------------------------------------------------------
            layer_path_l = []
            for layer in web_manifest['layers']:
                layer_url = cname_url + '/v2/{0}/blobs/{1}'.format(ecr_repo, layer['digest'])
                path = layer['digest'].split(':')[-1] + "/layer.tar"
                out_path = temp_dir + '/' + path

                downloading_layer(cache_dir=CACHE_DIR, layer_url=layer_url, out_path=out_path, h=h)
                layer_path_l.append(path)

            manifest = [{"Config": config_filename, "RepoTags": [], "Layers": layer_path_l}]
            print("config_filename: ", config_filename)
            print("manifest: ", manifest)
            with open(temp_dir + '/' + 'manifest.json', 'w') as outfile:
                json.dump(manifest, outfile)

            with tarfile.open(ecr_tag, "w") as tar_out:
                os.chdir(temp_dir)
                tar_out.add(".")

        except Exception as e:
            logger.error('ERROR: {0}'.format(str(e)))
            print(e)
            exit(1)

    else:
        print("Could not get connect to AWS ECR or get needed repository")
        exit(-1)

    return ecr_pull


def ecr_push():
    print("TBD")

    return ecr_push


def ec2_handler(aws_settings, ecr_repo):
    ecr = ec2_connector(aws_settings)

    # TBD

    return ec2_handler


if __name__ == '__main__':
    start__time = time.time()
    parser = argparse.ArgumentParser(prog='python3 script_name.py -h',
                                     usage='python3 script_name.py {ARGS}',
                                     add_help=True,
                                     prefix_chars='--/',
                                     epilog='''created by Vitalii Natarov'''
                                     )
    parser.add_argument('--version', action='version', version='v0.2.0')
    parser.add_argument('--bclient', dest='boto3_client', help='Set boto3 client', default='ecr')
    parser.add_argument('--region', dest='region', help='Set AWS region for boto3', default='us-east-1')
    parser.add_argument('--pname', '--profile', dest='profile_name', help='Set profile name of AWS',
                        default=None)
    parser.add_argument('--rname', '--role-name', dest='role_name', help='Set role ARN name',
                        default=None)
    parser.add_argument('--rsession', '--role-session', dest='role_session', help='Set role session name',
                        default=None)
    parser.add_argument('--registry-url', '--url', dest='registry_url', help='Set Registry URL',
                        default="https://docker-ecr.internal.logz.io")
    parser.add_argument('--ecr-url', dest='ecr_url', help='Set URL from ECR registry',
                        default="406095609952.dkr.ecr.us-east-1.amazonaws.com")
    parser.add_argument('--ecr-repo', '-repo', dest='ecr_repo', help='Set ECR repo name',
                        default="accounts")
    parser.add_argument('--ecr-repo-tag', '-repo-tag', dest='ecr_repo_tag', help='Set ECR repo tag',
                        default="8f2b55cc45894247d880f7c4ac2042ed7c398dfa")
    parser.add_argument('--ec2', dest='aws_ec2', help='Set ec2 usage', default='False')

    results = parser.parse_args()

    boto3_client = results.boto3_client
    region = results.region
    profile_name = results.profile_name
    role_name = results.role_name
    role_session = results.role_session

    registry_url = results.registry_url
    ecr_url_host = results.ecr_url
    ecr_repo_name = results.ecr_repo
    ecr_repo_tag = results.ecr_repo_tag

    aws_ec2 = results.aws_ec2

    try:
        temp_dir = mkdtemp()

        if aws_ec2 == 'True':
            print("AWS EC2!")
            aws_auth = {
                "client": boto3_client,
                "region": region,
                "ec2_host": 'True'
            }

            ec2_handler(aws_auth)
        else:
            print("Local Usage!")
            aws_auth = {
                "client": boto3_client,
                "region": region,
                "profile_name": profile_name,
                "role_name": role_name,
                "role_session": role_session
            }

            authorization_token = get_authorization_token(aws_auth, ecr_repo_name)
            headers = {
                'Host': str(ecr_url_host),
                'Accept': 'text/plain',
                'X-Forwarded-Proto': 'https',
                'X-Forwarded-For': '127.0.0.1',
                'X-Real-IP': '66.66.66.66',
                'Authorization': 'Basic {}'.format(str(authorization_token))
            }
            ecr_pull(aws_auth, registry_url, ecr_repo_name, ecr_repo_tag, headers)
    finally:
        shutil.rmtree(temp_dir)

    end__time = round(time.time() - start__time, 2)
    print("--- %s seconds ---" % end__time)

Использование — простое:

$ python3 ecr-containers.py --profile=default
config_filename:  2a5d611b6675bbcb15fea6050fdf5af2d974494511967e1ccd816efbe95fa348.json
manifest:  [{'Config': '2a5d611b6675bbcb15fea6050fdf5af2d974494511967e1ccd816efbe95fa348.json', 'RepoTags': [], 'Layers': ['339de151aab4bc06eed8409daae147c408478cb538dacb90cc63f19ad4eba80b/layer.tar', 'c79c496f1543a6cf36a79305de7c709ea55c16b63ea86219c4b98c334b70488d/layer.tar', 'a5382bad4f98c8f742d943e2bd6930626b06af081c57062e06a501d449a4c472/layer.tar', '348e701aca2b0acab9fab3daaa10980e5507b8e55334671f503d1d5f5ef739a5/layer.tar', '965b2a30aa9c5ffea05d772b418c428cde9fa3f85faf37b2b66b719593f4c7b6/layer.tar', '9ee6edf857abe493670c386faa46127a9decea128da3fcafe25015afa8f6b9ef/layer.tar', '75f33f1b8076396b9a688c579ffb6ee4e7d7c83e94a9ba4d15b97a7efb118db7/layer.tar', 'a79e84406bd9b1f9c48e60589691a9ef807c7416cf1b2885eafc95fbbd3bbd8c/layer.tar', 'e5279c73940e27e2d0e32d8c288c32a795eeb12d4bf876e27067a7a49d032d6c/layer.tar', 'b088ed07f0856dbc5d7451536d3379a8c0dee79ddf98f7bdcef98e37549669fa/layer.tar', '09c3905aed6a2dd924c4a3fed0802d3cc191cdfc317c960ea9c43c9f8a4379c5/layer.tar', '7486d5d0bd29e293ed034751b9743c3f30405aa4746f353151ed44e2d827fc76/layer.tar']}]
--- 3.93 seconds ---

Скрипт поддерживает простой хелп, вызвать его можно так:

$ python3 ecr-containers.py -h
usage: python3 script_name.py {ARGS}

optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  --bclient BOTO3_CLIENT
                        Set boto3 client
  --region REGION       Set AWS region for boto3
  --pname PROFILE_NAME, --profile PROFILE_NAME
                        Set profile name of AWS
  --rname ROLE_NAME, --role-name ROLE_NAME
                        Set role ARN name
  --rsession ROLE_SESSION, --role-session ROLE_SESSION
                        Set role session name
  --registry-url REGISTRY_URL, --url REGISTRY_URL
                        Set Registry URL
  --ecr-url ECR_URL     Set URL from ECR registry
  --ecr-repo ECR_REPO, -repo ECR_REPO
                        Set ECR repo name
  --ecr-repo-tag ECR_REPO_TAG, -repo-tag ECR_REPO_TAG
                        Set ECR repo tag
  --ec2 AWS_EC2         Set ec2 usage

created by Vitalii Natarov

После того как выполнили пуллинг образа, нужно его импортнуть в Докер:

$ docker load < 8f2b55cc45894247d880f7c4ac2042ed7c398dfa
32f366d666a5: Loading layer [==================================================>]  2.801MB/2.801MB
45449966e51a: Loading layer [==================================================>]  192.3MB/192.3MB
83ef90c94f12: Loading layer [==================================================>]   2.92MB/2.92MB
54037642f52c: Loading layer [==================================================>]    200kB/200kB
b9e5e90eb033: Loading layer [==================================================>]  42.09MB/42.09MB
539aca899eb8: Loading layer [==================================================>]     206B/206B
9d00a6a26f53: Loading layer [==================================================>]  4.668kB/4.668kB
bca0da0f1ea4: Loading layer [==================================================>]  1.631kB/1.631kB
cbb7b9b8cd31: Loading layer [==================================================>]  197.9MB/197.9MB
a0f1352db34e: Loading layer [==================================================>]  8.519kB/8.519kB
2543c13d39a0: Loading layer [==================================================>]  1.833kB/1.833kB
eb9876ca8f14: Loading layer [==================================================>]   41.2MB/41.2MB
Loaded image ID: sha256:ada32a4765be57eb1049808ebdbc7b8b6108847375383a21ffe004f3fffc3757

В плнах сделать push для работы с ECR, но это будет немного позже.

Репозиторий можно найти тут:

https://github.com/SebastianUA/ecr-pull-push

У меня все, статья «Pull/Push AWS ECR образов через AWS Route53 CNAME» завершена!

Добавить комментарий

Ваш адрес email не будет опубликован. Обязательные поля помечены *

Этот сайт использует Akismet для борьбы со спамом. Узнайте, как обрабатываются ваши данные комментариев.