
Работа с Google Cloud Platform (google pubsub) и Terraform в Unix/Linux
Google Cloud Platrorm — это платформа вида «инфраструктура как сервис» (IaaS), позволяющая клиентам создавать, тестировать и развертывать собственные приложения на инфраструктуре Google, в высокопроизводительных виртуальных машинах.
Google Compute Engine предоставляет виртуальные машины, работающие в инновационных центрах обработки данных Google и всемирной сети.
Google pub/sub — сервис, который привносит в облако масштабируемость, гибкость и надежность и асинхронный обмен сообщениями.
Установка terraform в Unix/Linux
Установка крайне примитивная и я описал как это можно сделать тут:
Установка terraform в Unix/Linux
Вот еще полезные статьи по GCP + Terrafrom:
Работа с Google Cloud Platform (compute instance) и Terraform в Unix/Linux
Работа с Google Cloud Platform (compute health check) и Terraform в Unix/Linux
Работа с Google Cloud Platform (compute target pool) и Terraform в Unix/Linux
Работа с Google Cloud Platform (compute forwarding rule) и Terraform в Unix/Linux
Работа с Google Cloud Platform (compute firewall) и Terraform в Unix/Linux
Работа с Google Cloud Platform (compute disk) и Terraform в Unix/Linux
Работа с Google Cloud Platform (compute image) и Terraform в Unix/Linux
Работа с Google Cloud Platform (compute instance template) и Terraform в Unix/Linux
Работа с Google Cloud Platform (compute instance group manager) и Terraform в Unix/Linux
Работа с Google Cloud Platform (compute autoscaler) и Terraform в Unix/Linux
Работа с Google Cloud Platform (google kms) и Terraform в Unix/Linux
Работа с Google Cloud Platform (storage bucket) и Terraform в Unix/Linux
Генерация документации для Terraform с Python в Unix/Linux
Так же, в данной статье, я создал скрипт для автоматической установки данного ПО. Он был протестирован на CentOS 6/7, Debian 8 и на Mac OS X. Все работает должным образом!
Чтобы получить помощь по использованию команд, выполните:
$ terraform --help Usage: terraform [--version] [--help] <command> [args] The available commands for execution are listed below. The most common, useful commands are shown first, followed by less common or more advanced commands. If you're just getting started with Terraform, stick with the common commands. For the other commands, please read the help and docs before usage. Common commands: apply Builds or changes infrastructure console Interactive console for Terraform interpolations destroy Destroy Terraform-managed infrastructure env Workspace management fmt Rewrites config files to canonical format get Download and install modules for the configuration graph Create a visual graph of Terraform resources import Import existing infrastructure into Terraform init Initialize a Terraform working directory output Read an output from a state file plan Generate and show an execution plan providers Prints a tree of the providers used in the configuration push Upload this Terraform module to Atlas to run refresh Update local state file against real resources show Inspect Terraform state or plan taint Manually mark a resource for recreation untaint Manually unmark a resource as tainted validate Validates the Terraform files version Prints the Terraform version workspace Workspace management All other commands: debug Debug output management (experimental) force-unlock Manually unlock the terraform state state Advanced state management
Приступим к использованию!
Работа с Google Cloud Platform (google pubsub) и Terraform в Unix/Linux
Первое что нужно сделать — это настроить «Cloud Identity». С помощью сервиса Google Cloud Identity вы сможете предоставлять доменам, пользователям и аккаунтам в организации доступ к ресурсам Cloud, а также централизованно управлять пользователями и группами через консоль администратора Google.
Полезное чтиво:
Установка Google Cloud SDK/gcloud в Unix/Linux
У меня есть папка terraform, в ней у меня будут лежать провайдеры с которыми я буду работать. Т.к в этом примере я буду использовать google_cloud_platform, то создам данную папку и перейду в нее. Далее, в этой папке, стоит создать:
$ mkdir examples modules
В папке examples, я буду хранить так званые «плейбуки» для разварачивания различных служб, например — zabbix-server, grafana, web-серверы и так далее. В modules директории, я буду хранить все необходимые модули.
Начнем писать модуль, но для этой задачи, я создам папку:
$ mkdir modules/google_pubsub
Переходим в нее:
$ cd modules/google_pubsub
Открываем файл:
$ vim google_pubsub.tf
В данный файл, вставляем:
#--------------------------------------------------- # Create pubsub topic #--------------------------------------------------- resource "google_pubsub_topic" "pubsub_topic" { count = "${var.enable_pubsub_topic ? 1 : 0}" name = "${lower(var.name)}-ps-topic-${lower(var.environment)}" project = "${var.project}" lifecycle { ignore_changes = [] create_before_destroy = true } } #--------------------------------------------------- # Create pubsub topic iam policy #--------------------------------------------------- data "google_iam_policy" "iam_policy" { binding { role = "${var.role}" members = ["${var.members}"] } } resource "google_pubsub_topic_iam_policy" "pubsub_topic_iam_policy" { count = "${var.enable_pubsub_topic_iam_policy ? 1 : 0}" topic = "${var.topic}" policy_data = "${data.google_iam_policy.iam_policy.policy_data}" project = "${var.project}" depends_on = ["data.google_iam_policy.iam_policy"] lifecycle { ignore_changes = [] create_before_destroy = true } } #--------------------------------------------------- # Create pubsub topic iam binding #--------------------------------------------------- resource "google_pubsub_topic_iam_binding" "pubsub_topic_iam_binding" { count = "${var.enable_pubsub_topic_iam_binding ? 1 : 0}" topic = "${var.topic}" role = "${var.role}" members = ["${var.members}"] lifecycle { ignore_changes = [] create_before_destroy = true } } #--------------------------------------------------- # Create pubsub topic iam member #--------------------------------------------------- resource "google_pubsub_topic_iam_member" "pubsub_topic_iam_member" { count = "${var.enable_pubsub_topic_iam_member ? 1 : 0}" topic = "${var.topic}" role = "${var.role}" member = "${element(var.members, 0)}" lifecycle { ignore_changes = [] create_before_destroy = true } } #--------------------------------------------------- # Create pubsub subscription #--------------------------------------------------- resource "google_pubsub_subscription" "pubsub_subscription_default" { count = "${var.enable_pubsub_subscription_default ? 1 : 0}" name = "${lower(var.name)}-ps-subscription-${lower(var.environment)}" project = "${var.project}" topic = "${var.topic}" lifecycle { ignore_changes = [] create_before_destroy = true } } resource "google_pubsub_subscription" "pubsub_subscription" { count = "${!var.enable_pubsub_subscription_default ? 1 : 0}" name = "${lower(var.name)}-ps-subscription-${lower(var.environment)}" project = "${var.project}" topic = "${var.topic}" ack_deadline_seconds = "${var.ack_deadline_seconds}" push_config { push_endpoint = "${var.push_config_push_endpoint}" attributes { x-goog-version = "v1" } } lifecycle { ignore_changes = [] create_before_destroy = true } } #--------------------------------------------------- # Create pubsub subscription iam policy #--------------------------------------------------- data "google_iam_policy" "subscription_iam_policy" { binding { role = "${var.role}" members = ["${var.members}"] } } resource "google_pubsub_subscription_iam_policy" "pubsub_subscription_iam_policy" { count = "${var.enable_pubsub_subscription_iam_policy ? 1 : 0}" subscription = "${var.subscription}" policy_data = "${data.google_iam_policy.subscription_iam_policy.policy_data}" depends_on = ["data.google_iam_policy.subscription_iam_policy"] lifecycle { ignore_changes = [] create_before_destroy = true } } #--------------------------------------------------- # Create pubsub subscription iam binding #--------------------------------------------------- resource "google_pubsub_subscription_iam_binding" "pubsub_subscription_iam_binding" { count = "${var.enable_pubsub_subscription_iam_binding ? 1 : 0}" subscription = "${var.subscription}" role = "${var.role}" members = ["${var.members}"] lifecycle { ignore_changes = [] create_before_destroy = true } } #--------------------------------------------------- # Create pubsub subscription iam member #--------------------------------------------------- resource "google_pubsub_subscription_iam_member" "pubsub_subscription_iam_member" { count = "${var.enable_pubsub_subscription_iam_member ? 1 : 0}" subscription = "${var.subscription}" role = "${var.role}" member = "${element(var.members, 0)}" lifecycle { ignore_changes = [] create_before_destroy = true } }
Открываем файл:
$ vim variables.tf
И прописываем:
variable "name" { description = "A unique name for the resource, required by GCE. Changing this forces a new resource to be created." default = "TEST" } variable "environment" { description = "Environment for service" default = "STAGE" } variable "orchestration" { description = "Type of orchestration" default = "Terraform" } variable "project" { description = "(Optional) The ID of the project in which the resource belongs. If it is not provided, the provider project is used." default = "" } variable "enable_pubsub_topic" { description = "Enable pubsub topic" default = "true" } variable "enable_pubsub_topic_iam_policy" { description = "Enable pubsub topic iam policy" default = "false" } variable "role" { description = "(Required) The role that should be applied. Only one google_pubsub_topic_iam_binding can be used per role. Note that custom roles must be of the format [projects|organizations]/{parent-name}/roles/{role-name}." default = "roles/editor" } variable "members" { description = "(Required) Identities that will be granted the privilege in role." default = [] } variable "topic" { description = "(Required) The topic name or id to bind to attach IAM policy to." default = "" } variable "enable_pubsub_topic_iam_binding" { description = "Enable pubsub topic iam binding" default = "false" } variable "enable_pubsub_topic_iam_member" { description = "Enable pubsub topic iam member" default = "false" } variable "enable_pubsub_subscription_default" { description = "Enable pubsub subscription for default" default = "true" } variable "ack_deadline_seconds" { description = "(Optional) The maximum number of seconds a subscriber has to acknowledge a received message, otherwise the message is redelivered. Changing this forces a new resource to be created." default = "20" } variable "push_config_push_endpoint" { description = "(Required) The URL of the endpoint to which messages should be pushed. Changing this forces a new resource to be created." default = "" } variable "enable_pubsub_subscription_iam_policy" { description = "Enable pubsub subscription iam policy" default = "false" } variable "enable_pubsub_subscription_iam_binding" { description = "Enable pubsub subscription iam binding" default = "false" } variable "subscription" { description = "(Required) The subscription name or id to bind to attach IAM policy to." default = "" } variable "enable_pubsub_subscription_iam_member" { description = "Enable pubsub subscription iam member" default = "false" }
Собственно в этом файле храняться все переменные. Спасибо кэп!
Открываем последний файл:
$ vim outputs.tf
И в него вставить нужно следующие строки:
output "pubsub_topic_name" { description = "Name of pubsub topic" value = "${google_pubsub_topic.pubsub_topic.*.name}" } output "pubsub_topic_id" { description = "ID" value = "${google_pubsub_topic.pubsub_topic.*.id}" } output "google_pubsub_topic_iam_policy_etag" { description = "Etag" value = "${google_pubsub_topic_iam_policy.pubsub_topic_iam_policy.*.etag}" } output "pubsub_topic_iam_binding_etag" { description = "Etag" value = "${google_pubsub_topic_iam_binding.pubsub_topic_iam_binding.*.etag}" } output "pubsub_topic_iam_member_etag" { description = "Etag" value = "${google_pubsub_topic_iam_member.pubsub_topic_iam_member.*.etag}" } output "google_pubsub_subscription_default_path" { description = "Path of the subscription in the format projects/{project}/subscriptions/{sub}" value = "${google_pubsub_subscription.pubsub_subscription_default.*.path}" } output "google_pubsub_subscription_default_name" { description = "Name" value = "${google_pubsub_subscription.pubsub_subscription_default.*.name}" } output "google_pubsub_subscription_path" { description = "Path of the subscription in the format projects/{project}/subscriptions/{sub}" value = "${google_pubsub_subscription.pubsub_subscription.*.path}" } output "google_pubsub_subscription_name" { description = "Name" value = "${google_pubsub_subscription.pubsub_subscription.*.name}" } output "google_pubsub_subscription_iam_policy_etag" { description = "Etag" value = "${google_pubsub_subscription_iam_policy.pubsub_subscription_iam_policy.*.etag}" } output "google_pubsub_subscription_iam_binding_etag" { description = "Etag" value = "${google_pubsub_subscription_iam_binding.pubsub_subscription_iam_binding.*.etag}" } output "google_pubsub_subscription_iam_member_etag" { description = "Etag" value = "${google_pubsub_subscription_iam_member.pubsub_subscription_iam_member.*.etag}" }
Переходим теперь в папку google_cloud_platform/examples и создадим еще одну папку для проверки написанного чуда:
$ mkdir google_pubsub && cd $_
Внутри созданной папки открываем файл:
$ vim main.tf
Вставляем:
# # MAINTAINER Vitaliy Natarov "vitaliy.natarov@yahoo.com" # terraform { required_version = "> 0.9.0" } provider "google" { credentials = "${file("/Users/captain/.config/gcloud/creds/terraform_creds.json")}" project = "terraform-2018" region = "us-east1" } module "google_pubsub" { source = "../../modules/google_pubsub" name = "TEST" # enable_pubsub_topic = true # #enable_pubsub_topic_iam_policy = true #topic = "test-ps-topic-stage" # #enable_pubsub_topic_iam_binding = true #topic = "test-ps-topic-stage" #members = ["OWNER:solo.metal@bigmir.net"] # #enable_pubsub_topic_iam_member = true #members = ["OWNER:solo.metal@bigmir.net"] # #enable_pubsub_subscription_default = true # #enable_pubsub_subscription_iam_policy = true # #enable_pubsub_subscription_iam_binding = true #members = ["OWNER:solo.metal@bigmir.net"] # #enable_pubsub_subscription_iam_member = true #members = ["OWNER:solo.metal@bigmir.net"] }
Все уже написано и готово к использованию. Ну что, начнем тестирование. В папке с вашим плейбуком, выполняем:
$ terraform init
Этим действием я инициализирую проект. Затем, подтягиваю модуль:
$ terraform get
PS: Для обновление изменений в самом модуле, можно выполнять:
$ terraform get -update
Проверим валидацию:
$ terraform validate
Запускем прогон:
$ terraform plan
Мне вывело что все у меня хорошо и можно запускать деплой:
$ terraform apply
Как видно с вывода, — все прошло гладко! Чтобы удалить созданное творение, можно выполнить:
$ terraform destroy
Весь материал аплоаджу в github аккаунт для удобства использования:
$ git clone https://github.com/SebastianUA/terraform.git
Вот и все на этом. Данная статья «Работа с Google Cloud Platform (google pubsub) и Terraform в Unix/Linux» завершена.