ローカルで Mac 上で terraform init すると Mac の hash 値が追加され、 lock ファイルに差分が生じる
なので差分が出てしまった場合はコミットするで良いとは思いますが、そもそも CI で lock ファイルを更新する際に Mac の hash 値も追加してしまえばローカルで Mac 上で terraform init しても差分が出なくなります。ちなみに Windows 上で terraform init する人は自分の周りにはいなさそうなので、 Windows は対応しないことにしました。
例えば tfnotify plan がパースエラーになった場合、 cannot parse plan result というメッセージが標準エラー出力されます。
そこで標準エラー出力に cannot parse plan result が含まれていたら github-comment でコメントするようにします。
terraform plan | github-comment exec -k plan -- tfnotify plan
MySQL Provider の設定として MySQL に接続するための情報と、Terraform によって作成するデータベース foo の設定が定義されています。
password が平文で書かれているのが気になるかもしれませんが、一旦スルーしてください。
この状態で terraform plan を実行してみます。 terraform plan は terraform apply によるリソースの作成を DRY RUN するコマンドです。
$ terraform plan Error: Could not satisfy plugin requirements Plugin reinitialization required. Please run "terraform init". Plugins are external binaries that Terraform uses to access and manipulate resources. The configuration provided requires plugins which can't be located, don't satisfy the version constraints, or are otherwise incompatible. Terraform automatically discovers provider requirements from your configuration, including providers used in child modules. To see the requirements and constraints from each module, run "terraform providers". Error: provider.mysql: no suitable version installed version requirements: "(any version)" versions installed: none
失敗しました。
Plugin reinitialization required. Please run "terraform init".
とある通り、 terraform plan や apply などのコマンドを実行する前に terraform init を実行する必要があります。
$ terraform init Initializing the backend... Initializing provider plugins... - Checking for available provider plugins... - Downloading plugin for provider "mysql" (terraform-providers/mysql) 1.9.0... ^C The following providers do not have any version constraints in configuration, so the latest version was installed. To prevent automatic upgrades to new major versions that may contain breaking changes, it is recommended to add version = "..." constraints to the corresponding provider blocks in configuration, with the constraint strings suggested below. * provider.mysql: version = "~> 1.9" Terraform has been successfully initialized! You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure. All Terraform commands should now work. If you ever set or change modules or backend configuration for Terraform, rerun this command to reinitialize your working directory. If you forget, other commands will detect it and remind you to do so if necessary.
$ terraform plan Refreshing Terraform state in-memory prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage. ------------------------------------------------------------------------ An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # mysql_database.foo will be created + resource "mysql_database" "foo" { + default_character_set = "utf8" + default_collation = "utf8_general_ci" + id = (known after apply) + name = "foo" } Plan: 1 to add, 0 to change, 0 to destroy. ------------------------------------------------------------------------ Note: You didn't specify an "-out" parameter to save this plan, so Terraform can't guarantee that exactly these actions will be performed if "terraform apply" is subsequently run.
Plan: 1 to add, 0 to change, 0 to destroy.
とある通り、リソースが 1 つ作成されるようです。 DRY RUN なのでまだ作成されてません。
では実際に作成してみましょう。
$ terraform apply An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # mysql_database.foo will be created + resource "mysql_database" "foo" { + default_character_set = "utf8" + default_collation = "utf8_general_ci" + id = (known after apply) + name = "foo" } Plan: 1 to add, 0 to change, 0 to destroy. Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes mysql_database.foo: Creating... mysql_database.foo: Creation complete after 0s [id=foo] Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
途中確認があるので yes と入力すると実際に変更が適用されます。
本当に Database が作られているか確認します。
MySQL クエリを叩きます。
mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | foo | | mysql | | performance_schema | | sys | +--------------------+ 5 rows in set (0.01 sec) mysql> select * from INFORMATION_SCHEMA.SCHEMATA where SCHEMA_NAME='foo'; +--------------+-------------+----------------------------+------------------------+----------+ | CATALOG_NAME | SCHEMA_NAME | DEFAULT_CHARACTER_SET_NAME | DEFAULT_COLLATION_NAME | SQL_PATH | +--------------+-------------+----------------------------+------------------------+----------+ | def | foo | utf8 | utf8_general_ci | NULL | +--------------+-------------+----------------------------+------------------------+----------+ 1 row in set (0.00 sec)
ここであえて古いバージョンの Terraform を使って terraform plan を実行してみます。
$ echo 0.12.12 > .terraform-version $ tfenv install $ terraform version $ terraform plan Error: Error locking state: Error acquiring the state lock: state snapshot was created by Terraform v0.12.19, which is newer than current v0.12.12; upgrade to Terraform v0.12.19 or greater to work with this state Terraform acquires a state lock to protect the state from being written by multiple users at the same time. Please resolve the issue above and try again. For most commands, you can disable locking with the "-lock=false" flag, but this is not recommended.
同じ調子でもう一つ Database を作ってみましょう。
Database foo の設定をコピーして terraform plan を実行します。
resource "mysql_database" "foo" { name = "foo" } resource "mysql_database" "foo" { name = "foo" }
$ terraform plan Error: Duplicate resource "mysql_database" configuration on main.tf line 11: 11: resource "mysql_database" "foo" { A mysql_database resource named "foo" was already declared at main.tf:7,1-32. Resource names must be unique per type in each module.
$ terraform plan Terraform will perform the following actions: # mysql_database.bar will be created + resource "mysql_database" "bar" { + default_character_set = "utf8" + default_collation = "utf8_general_ci" + id = (known after apply) + name = "foo" } Plan: 1 to add, 0 to change, 0 to destroy.
作成されるようです。 apply してみましょう。
$ terraform apply mysql_database.bar: Creating... Error: Error 1007: Can't create database 'foo'; database exists on main.tf line 11, in resource "mysql_database" "bar": 11: resource "mysql_database" "bar" {
失敗しました。同じ名前のデータベースは作成できないので当然です。
このように plan に成功しても apply に失敗することはあります。
では name を修正しましょう。ついでに database foo の default character set を修正します。
resource "mysql_database" "foo" { name = "foo" default_character_set = "utf8mb4" # default character set を修正。デフォルトは utf8 } resource "mysql_database" "bar" { name = "bar" # 名前を foo から bar に変更 }
$ terraform plan Terraform will perform the following actions: # mysql_database.bar will be created + resource "mysql_database" "bar" { + default_character_set = "utf8" + default_collation = "utf8_general_ci" + id = (known after apply) + name = "bar" } # mysql_database.foo will be updated in-place ~ resource "mysql_database" "foo" { ~ default_character_set = "utf8" -> "utf8mb4" default_collation = "utf8_general_ci" id = "foo" name = "foo" } Plan: 1 to add, 1 to change, 0 to destroy.
$ terraform apply mysql_database.bar: Creating... mysql_database.foo: Modifying... [id=foo] mysql_database.bar: Creation complete after 0s [id=bar] Error: Error 1253: COLLATION 'utf8_general_ci' is not valid for CHARACTER SET 'utf8mb4' on main.tf line 7, in resource "mysql_database" "foo": 7: resource "mysql_database" "foo" {
$ terraform plan Terraform will perform the following actions: # mysql_database.foo must be replaced -/+ resource "mysql_database" "foo" { default_character_set = "utf8mb4" default_collation = "utf8mb4_general_ci" ~ id = "foo" -> (known after apply) ~ name = "foo" -> "foo2" # forces replacement } Plan: 1 to add, 0 to change, 1 to destroy.
$ terraform plan Error: Reference to undeclared input variable on main.tf line 9, in resource "mysql_database" "foo": 9: default_character_set = var.default_character_set An input variable with the name "default_character_set" has not been declared. This variable can be declared with a variable "default_character_set" {} block.
$ terraform plan var.default_character_set Enter a value: foo Terraform will perform the following actions: # mysql_database.foo will be updated in-place ~ resource "mysql_database" "foo" { ~ default_character_set = "utf8mb4" -> "foo" default_collation = "utf8mb4_general_ci" id = "foo2" name = "foo2" } Plan: 0 to add, 1 to change, 0 to destroy.
-input=false にすると入力を求められずにエラーを返します。
$ terraform plan -input=false Error: No value for required variable on variables.tf line 1: 1: variable "default_character_set" { The root module input variable "default_character_set" is not set, and has no default value. Use a -var or -var-file command line argument to provide a value for this variable.
terraform.tfvars というファイルを作成し、値を設定しましょう。
default_character_set = "utf8mb4"
$ terraform plan No changes. Infrastructure is up-to-date.
$ terraform plan Terraform will perform the following actions: # mysql_database.bar will be updated in-place ~ resource "mysql_database" "bar" { ~ default_character_set = "utf8" -> "utf8mb4" ~ default_collation = "utf8_general_ci" -> "utf8mb4_general_ci" id = "bar" name = "bar" } Plan: 0 to add, 1 to change, 0 to destroy.
Terraform は State によって設定ファイル中のリソースと実際のリソースをマッピングしています。
State からリソースを削除して terraform plan を実行してみると、 Terraform はそのリソースを新規で作成しようとします。
$ terraform plan No changes. Infrastructure is up-to-date. $ terraform state rm mysql_database.bar Removed mysql_database.bar Successfully removed 1 resource instance(s). $ terraform plan Terraform will perform the following actions: # mysql_database.bar will be created + resource "mysql_database" "bar" { + default_character_set = "utf8mb4" + default_collation = "utf8mb4_general_ci" + id = (known after apply) + name = "bar" } Plan: 1 to add, 0 to change, 0 to destroy.
実際には同じ名前のデータベースは作成できないので apply で失敗するはずです。
terraform state rm は元々 Terraform で管理していたものを Terraform で管理するのを止めたり、あるいは手動で削除してしまったリソースを State からも削除するのに使えます。
$ terraform import mysql_database.bar bar mysql_database.bar: Importing from ID "bar"... mysql_database.bar: Import prepared! Prepared mysql_database for import mysql_database.bar: Refreshing state... [id=bar] Import successful! The resources that were imported are shown above. These resources are now in your Terraform state and will henceforth be managed by Terraform.
Import 出来ました。 terraform plan を実行して差分がなくなっていることを確認します。
$ terraform plan No changes. Infrastructure is up-to-date.
$ terraform import mysql_database.zoo zoo Error: resource address "mysql_database.zoo" does not exist in the configuration. Before importing this resource, please create its configuration in the root module. For example: resource "mysql_database" "zoo" { # (resource arguments) }
$ terraform import mysql_database.zoo zoo mysql_database.zoo: Importing from ID "zoo"... mysql_database.zoo: Import prepared! Prepared mysql_database for import mysql_database.zoo: Refreshing state... [id=zoo] Import successful! The resources that were imported are shown above. These resources are now in your Terraform state and will henceforth be managed by Terraform.
Import 出来たので zoo を Terraform で管理できるようになりました。
terraform plan を実行してみます。
$ terraform plan Error: Missing required argument on main.tf line 19, in resource "mysql_database" "zoo": 19: resource "mysql_database" "zoo" { The argument "name" is required, but no definition was found.
zoo の設定が空で name が設定されていないので失敗しました。
修正します。
resource "mysql_database" "zoo" { name = "zoo" }
$ terraform plan Terraform will perform the following actions: # mysql_database.zoo will be updated in-place ~ resource "mysql_database" "zoo" { ~ default_character_set = "latin1" -> "utf8" ~ default_collation = "latin1_swedish_ci" -> "utf8_general_ci" id = "zoo" name = "zoo" } Plan: 0 to add, 1 to change, 0 to destroy.
まだ差分が出てしまいました。
Import は State にはリソースのデータを反映してくれますが、設定ファイルには反映してくれないので自分で反映させる必要があります。
一部の Provider では、設定ファイルに自動で反映させるためのサードパーティのツールもあります。
$ terraform plan Terraform will perform the following actions: # mysql_database.bar will be destroyed - resource "mysql_database" "bar" { - default_character_set = "utf8mb4" -> null - default_collation = "utf8mb4_general_ci" -> null - id = "bar" -> null - name = "bar" -> null } Plan: 0 to add, 0 to change, 1 to destroy.
$ terraform plan Terraform will perform the following actions: # mysql_database.foo will be destroyed - resource "mysql_database" "foo" { - default_character_set = "utf8mb4" -> null - default_collation = "utf8mb4_general_ci" -> null - id = "foo2" -> null - name = "foo2" -> null } # mysql_database.foo2 will be created + resource "mysql_database" "foo2" { + default_character_set = "utf8mb4" + default_collation = "utf8mb4_general_ci" + id = (known after apply) + name = "foo2" } Plan: 1 to add, 0 to change, 1 to destroy.
既存のデータベースが削除されて新しいデータベースが作成されてしまいそうです。
State に記録されているリソースのパスを修正する必要があります。
$ terraform state mv mysql_database.foo mysql_database.foo2 Move "mysql_database.foo" to "mysql_database.foo2" Successfully moved 1 object(s).
$ terraform plan No changes. Infrastructure is up-to-date.
差分がなくなりました。
このようにコマンドによって State を更新する必要があるので、特に複数人で作業する場合は安易にリソースパスを変更するべきではありません。
$ terraform plan -refresh=false Terraform will perform the following actions: # mysql_database.zoo will be updated in-place ~ resource "mysql_database" "zoo" { ~ default_character_set = "latin1" -> "utf8mb4" ~ default_collation = "latin1_swedish_ci" -> "utf8mb4_general_ci" id = "zoo" name = "zoo" } Plan: 0 to add, 1 to change, 0 to destroy. Warning: Resource targeting is in effect You are creating a plan with the -target option, which means that the result of this plan may not represent all of the changes requested by the current configuration. The -target option is not for routine use, and is provided only for exceptional situations such as recovering from errors or mistakes, or when Terraform specifically suggests to use it as part of an error message.
差分が出てしまいました。これは State が更新されていないからです。
terraform.tfstate を確認してみましょう。
$ terraform plan No changes. Infrastructure is up-to-date.
これは、実は terraform plan はデフォルトでは State の内容と設定ファイルを比較する前に実際のインフラの情報を取得し State の内容をインメモリで更新した上で設定ファイルと比較しているからです。
ただし、インメモリでの更新であり、実際の State が更新されているわけではないです。
今までスルーしてきましたが、そのことは terraform plan の結果でも説明されています。
$ terraform plan Refreshing Terraform state in-memory prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage.
ではなんで -refresh=false が必要になるかというと、理由の一つとして terraform plan が速くなることが挙げられます。
むしろ -refresh=false をつけないと、管理するリソースの数が増えれば増えるほど情報を取得してくるのに時間がかかるようになりますし、
場合によっては API の rate limit に引っかかるなんてこともあるかもしれません。
なので普段 -refresh=false をつけて terraform plan を実行するようにしていると Terraform を使わずに加えたインフラの修正を State に取り込むために terraform refresh を実行する必要が出てきたりします。
State は更新されていないので -refresh=false とすれば相変わらず差分は出ます。
$ terraform plan -refresh=false Terraform will perform the following actions: # mysql_database.zoo will be updated in-place ~ resource "mysql_database" "zoo" { ~ default_character_set = "utf8mb4" -> "latin1" ~ default_collation = "utf8mb4_general_ci" -> "latin1_swedish_ci" id = "zoo" name = "zoo" } Plan: 0 to add, 1 to change, 0 to destroy.
差分が出ました。 terraform refresh を実行すると State が更新され、差分がなくなるはずです。
$ terraform plan Error: Module not installed on main.tf line 25: 25: module "app" { This module is not yet installed. Run "terraform init" to install all modules required by this configuration.
失敗しました。 terraform init する必要があります。
$ terraform init
$ terraform plan Terraform will perform the following actions: # module.app.mysql_database.db will be created + resource "mysql_database" "db" { + default_character_set = "utf8mb4" + default_collation = "utf8mb4_general_ci" + id = (known after apply) + name = "app" } Plan: 1 to add, 0 to change, 0 to destroy.
$ terraform plan Error: Missing required argument on main.tf line 25, in module "app": 25: module "app" { The argument "name" is required, but no definition was found.
$ terraform plan No changes. Infrastructure is up-to-date.
パラメータを渡して name を変更しましょう。
module "app" { source = "./database" name = "app2" }
$ terraform plan Terraform will perform the following actions: # module.app.mysql_database.db must be replaced -/+ resource "mysql_database" "db" { default_character_set = "utf8mb4" default_collation = "utf8mb4_general_ci" ~ id = "app" -> (known after apply) ~ name = "app" -> "app2" # forces replacement }
パラメータを渡せました。もとに戻しておきます。
module "app" { source = "./database" name = "app" }
$ terraform destroy -target=module.app Terraform will perform the following actions: # mysql_database.zoo will be destroyed - resource "mysql_database" "zoo" { - default_character_set = "utf8mb4" -> null - default_collation = "utf8mb4_general_ci" -> null - id = "zoo" -> null - name = "zoo" -> null } # module.app.mysql_database.db will be destroyed - resource "mysql_database" "db" { - default_character_set = "utf8mb4" -> null - default_collation = "utf8mb4_general_ci" -> null - id = "app" -> null - name = "app" -> null } Plan: 0 to add, 0 to change, 2 to destroy. Do you really want to destroy all resources? Terraform will destroy all your managed infrastructure, as shown above. There is no undo. Only 'yes' will be accepted to confirm. Enter a value: yes
$ terraform destroy -target=mysql_database.zoo Terraform will perform the following actions: # mysql_database.zoo will be destroyed - resource "mysql_database" "zoo" { - default_character_set = "utf8mb4" -> null - default_collation = "utf8mb4_general_ci" -> null - id = "zoo" -> null - name = "zoo" -> null } Plan: 0 to add, 0 to change, 1 to destroy. Do you really want to destroy all resources? Terraform will destroy all your managed infrastructure, as shown above. There is no undo. Only 'yes' will be accepted to confirm. Enter a value:
確認されるので、 yes と入力して削除します。
Enter a value: yes mysql_database.zoo: Destroying... [id=zoo] mysql_database.zoo: Destruction complete after 0s Warning: Applied changes may be incomplete The plan was created with the -target option in effect, so some changes requested in the configuration may have been ignored and the output values may not be fully updated. Run the following command to verify that no other changes are pending: terraform plan Note that the -target option is not suitable for routine use, and is provided only for exceptional situations such as recovering from errors or mistakes, or when Terraform specifically suggests to use it as part of an error message. Destroy complete! Resources: 1 destroyed.
mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | app | | foo2 | | mysql | | performance_schema | | sys | +--------------------+ 6 rows in set (0.00 sec)
確かに消えています。
設定ファイルからは消えていないので terraform plan を実行すると Create しようとします。
$ terraform plan Terraform will perform the following actions: # mysql_database.zoo will be created + resource "mysql_database" "zoo" { + default_character_set = "utf8mb4" + default_collation = "utf8mb4_general_ci" + id = (known after apply) + name = "zoo" } Plan: 1 to add, 0 to change, 0 to destroy.
逆に、あまりないケースではありますが、あるリソースについて Terraform で管理するのをやめたい場合、
そのリソースを設定ファイルだけでなく State からも削除する必要があります。
単純に設定ファイルから削除するだけだと、 Terraform によって実際のインフラのリソースが削除されてしまいます。
State から削除するには terraform state rm というコマンドを使います。
plan, apply, refresh, state rm, state mv, state push には次のようなオプションがあります。
-lock=true Lock the state file when locking is supported. -lock-timeout=0s Duration to retry a state lock.
-lock はデフォルトで true なので State Locking のことを知らなくても実は State Locking 使ってたということもありえますが、 Backend type によっては State Locking のための設定をしていないと State Locking が無効になっている可能性があります。
例えば S3 backend で State Locking をするには DynamoDB が必要であり、 DynamoDB の設定 dynamodb_table が設定されていないと State Locking は無効になります。
lock が解放されていない状態で plan などを実行すると lock の取得に失敗し、次のようなエラーが起こります。
Acquiring state lock. This may take a few moments... Error: Error locking state: Error acquiring the state lock: ConditionalCheckFailedException: The conditional request failed status code: 400, request id: xxx Lock Info: ID: xxx Path: terraform.tfstate Operation: OperationTypePlan Who: xxx Version: 0.12.13 Created: 2020-01-09 09:30:37.41120929 +0000 UTC Info: Terraform acquires a state lock to protect the state from being written by multiple users at the same time. Please resolve the issue above and try again. For most commands, you can disable locking with the "-lock=false" flag, but this is not recommended.
ここで出力される ID を force-unlock の引数として指定します。
$ terraform force-unlock xxx Do you really want to force-unlock? Terraform will remove the lock on the remote state. This will allow local Terraform commands to modify this state, even though it may be still be in use. Only 'yes' will be accepted to confirm. Enter a value: yes Terraform state has been successfully unlocked! The state has been unlocked, and Terraform commands should now be able to obtain a new lock on the remote state.
dynamodb_table - (Optional) The name of a DynamoDB table to use for state locking and consistency. The table must have a primary key named LockID. If not present, locking will be disabled.
data "terraform_remote_state" "network" { backend = "s3" config = { bucket = "terraform-state-prod" key = "network/terraform.tfstate" region = "us-east-1" # state locking の設定 dynamodb_table = "???" } }
