Terraform으로 리소스를 관리하다 보면 자연히 관리하는 리소스가 많아지게 된다. 하나의 state 파일에서 너무 많은 리소스를 관리하지 않기를 권장하고 있지만, 너무 나누면 관리가 어려운 부분도 있고 성격상 어쩔 수 없이 많은 리소스가 생기게 되는 때도 있다. 이렇게 관리 리소스가 많았지만 plan이나 apply를 할 때 Terraform이 tfstate에 기록해 둔 내용과 AWS 같은 클라우드에 실제 구성을 비교해봐야 하므로 꽤 많은 시간이 걸리게 된다.

기본적으로 plan과 apply를 하면 tfstate 파일과 실제 상태를 비교하는 작업을 항상 진행하므로 아무리 간단한 수정을 하더라도 시간이 꽤 많이 걸리게 되고 항상 terraform plan을 한 뒤 terraform apply를 하게 되는데 양쪽 모두에서 이 작업을 하게 되므로 시간은 더욱 많이 걸리게 된다.

terraform plan -out=path

다음은 AWS Route53에 CNAME을 하나 추가하고 plan을 실행한 결과이다.

$ terraform plan
Acquiring state lock. This may take a few moments...
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

data.terraform_remote_state.vpc: Refreshing state...
data.terraform_remote_state.us_east_1: Refreshing state...
data.terraform_remote_state.ecs_services: Refreshing state...
aws_iam_user.apex-basic: Refreshing state... (ID: apex-basic)
aws_dynamodb_table.terraform-lock: Refreshing state... (ID: SideEffectTerraformStateLock)
aws_cloudfront_origin_access_identity.labs_sideeffect_kr: Refreshing state... (ID: E2WUVSG763JX3V)
data.aws_elb_service_account.main: Refreshing state...
data.aws_acm_certificate.sideeffect_kr: Refreshing state...
data.aws_iam_policy_document.ecs_service_role: Refreshing state...
data.aws_iam_policy_document.config_service: Refreshing state...
aws_s3_bucket.nodejs-ko: Refreshing state... (ID: nodejs-ko)
aws_route53_zone.sideeffect_kr: Refreshing state... (ID: Z12CBQ5AHVFFYA)
aws_iam_group.apex: Refreshing state... (ID: apex)
data.aws_iam_policy_document.ecs_instance_role: Refreshing state...
aws_cloudwatch_log_group.ap_northeast_1_ecs: Refreshing state... (ID: ap-northeast-1-ecs)
data.aws_iam_policy_document.nodejs-ko-twitter_lambda_logs: Refreshing state...
data.aws_iam_policy_document.apex-default: Refreshing state...
aws_iam_user.outsider: Refreshing state... (ID: outsider)
aws_cloudfront_origin_access_identity.nodejs_sideeffect_kr: Refreshing state... (ID: E25WJ3C3LDATWV)
data.aws_iam_policy_document.nodejs-ko-twitter_lambda_function: Refreshing state...
data.aws_iam_policy_document.vault_ecs_task: Refreshing state...
data.aws_iam_policy_document.aws_s3_bucket_logs: Refreshing state...
aws_iam_role.config_service: Refreshing state... (ID: config-service)
aws_iam_role.ecs_service_role: Refreshing state... (ID: ecsServiceRole)
aws_iam_policy.nodejs-ko-twitter_lambda_logs: Refreshing state... (ID: arn:aws:iam::410655858509:policy/nodejs-ko-twitter_lambda_logs)
aws_iam_role.ecs_instance_role: Refreshing state... (ID: ecsInstanceRole)
data.aws_iam_policy_document.labs_sideeffect_kr: Refreshing state...
aws_iam_policy.apex-default: Refreshing state... (ID: arn:aws:iam::410655858509:policy/apex-default)
aws_iam_group_membership.apex: Refreshing state... (ID: apex-group-membership)
aws_iam_role.nodejs-ko-twitter_lambda_function: Refreshing state... (ID: nodejs-ko-twitter_lambda_function)
aws_iam_role.vault_ecs_task: Refreshing state... (ID: vault-ecs-task)
aws_s3_bucket.logs: Refreshing state... (ID: kr.sideeffect.logs)
aws_config_configuration_recorder.ap_northeast_1: Refreshing state... (ID: aws-config-ap-northeast-1)
aws_iam_policy_attachment.AWSConfigRole-policy-attachment: Refreshing state... (ID: AWSConfigRole-policy-attachment)
aws_iam_policy_attachment.AdministratorAccess-policy-attachment: Refreshing state... (ID: AdministratorAccess-policy-attachment)
aws_iam_policy_attachment.IAMFullAccess-policy-attachment: Refreshing state... (ID: IAMFullAccess-policy-attachment)
aws_iam_policy_attachment.AWSLambdaFullAccess-policy-attachment: Refreshing state... (ID: AWSLambdaFullAccess-policy-attachment)
data.aws_iam_policy_document.nodejs_sideeffect_kr: Refreshing state...
aws_s3_bucket.labs_sideeffect_kr: Refreshing state... (ID: kr.sideeffect.labs)
aws_route53_record.vault_sideeffect_kr: Refreshing state... (ID: Z12CBQ5AHVFFYA_vault.sideeffect.kr_A)
aws_route53_record.www_sideeffect_kr: Refreshing state... (ID: Z12CBQ5AHVFFYA_www.sideeffect.kr_A)
aws_iam_policy_attachment.ecs_service_role: Refreshing state... (ID: AmazonEC2ContainerServiceforEC2Role)
aws_iam_policy_attachment.ecs_instance_role: Refreshing state... (ID: AmazonEC2ContainerServiceforEC2Role)
aws_iam_instance_profile.ecs_instance_role: Refreshing state... (ID: ecsInstanceRole)
aws_iam_policy_attachment.AmazonS3FullAccess-policy-attachment: Refreshing state... (ID: AmazonS3FullAccess-policy-attachment)
aws_config_config_rule.ap_northeast_1_cloudtrail_enabled: Refreshing state... (ID: cloudtrail-enabled)
aws_config_config_rule.ap_northeast_1_ec2_instances_in_vpc: Refreshing state... (ID: ec2-instances-in-vpc)
aws_s3_bucket.nodejs_sideeffect_kr: Refreshing state... (ID: kr.sideeffect.nodejs)
aws_iam_policy_attachment.apex-default-policy-attachment: Refreshing state... (ID: apex-default-policy-attachment)
aws_iam_policy_attachment.nodejs-ko-twitter_lambda_logs-policy-attachment: Refreshing state... (ID: nodejs-ko-twitter_lambda_logs-policy-attachment)
aws_route53_record.sideeffect_kr: Refreshing state... (ID: Z12CBQ5AHVFFYA_sideeffect.kr_A)
aws_s3_bucket.terraform-state: Refreshing state... (ID: kr.sideeffect.terraform.state)
aws_cloudtrail.ap_northeast_1: Refreshing state... (ID: ap-northeast-1)
aws_config_delivery_channel.ap_northeast_1: Refreshing state... (ID: aws-config-ap-northeast-1)
aws_s3_bucket.vault_sideeffect_kr: Refreshing state... (ID: kr.sideeffect.vault)
data.aws_iam_policy_document.config_service_delivery_permission: Refreshing state...
aws_iam_policy.config_service_delivery_permission: Refreshing state... (ID: arn:aws:iam::410655858509:policy/config-service-delivery-permission)
aws_config_configuration_recorder_status.ap_northeast_1: Refreshing state... (ID: aws-config-ap-northeast-1)
aws_cloudfront_distribution.labs_sideeffect_kr: Refreshing state... (ID: E3EP48YZYH5MDL)
aws_iam_policy_attachment.config_service_delivery_permission_attachment: Refreshing state... (ID: config-service-delivery-permission-attachment)
aws_cloudfront_distribution.nodejs_sideeffect_kr: Refreshing state... (ID: E3AG1R3KSY57S4)
aws_route53_record.labs_sideeffect_kr: Refreshing state... (ID: Z12CBQ5AHVFFYA_labs.sideeffect.kr_A)
aws_route53_record.nodejs_sideeffect_kr: Refreshing state... (ID: Z12CBQ5AHVFFYA_nodejs.sideeffect.kr_A)
data.aws_iam_policy_document.vault: Refreshing state...
aws_iam_policy.vault: Refreshing state... (ID: arn:aws:iam::410655858509:policy/vault-to-write-s3)
aws_iam_policy_attachment.vault-policy-attachment: Refreshing state... (ID: vault-policy-attachment)
The Terraform execution plan has been generated and is shown below.
Resources are shown in alphabetical order for quick scanning. Green resources
will be created (or destroyed and then created if an existing resource
exists), yellow resources are being changed in-place, and red resources
will be destroyed. Cyan entries are data sources to be read.

Note: You didn't specify an "-out" parameter to save this plan, so when
"apply" is called, Terraform can't guarantee this is what will execute.

+ aws_route53_record.demo_sideeffect_kr
    fqdn:               "<computed>"
    name:               "demo.sideeffect.kr"
    records.#:          "1"
    records.3313064457: "blog.outsider.ne.kr"
    type:               "CNAME"
    zone_id:            "Z12CBQ5AHVFFYA"


Plan: 1 to add, 0 to change, 0 to destroy.
Releasing state lock. This may take a few moments...

좀 길지만, 전체 출력을 모두 적었다. Route53에 CNAME을 생성한 것이지만 tfstate에 있는 모든 리소스를 다 검사한다. 상단에 나오는 출력 부분은 모두 이 부분의 상태를 검사하는 부분이고 최종적으로 CNAME을 1개 추가한다고 알려준다.

$ terraform apply
... 중략
aws_cloudfront_distribution.labs_sideeffect_kr: Refreshing state... (ID: E3EP48YZYH5MDL)
aws_config_configuration_recorder_status.ap_northeast_1: Refreshing state... (ID: aws-config-ap-northeast-1)
aws_cloudfront_distribution.nodejs_sideeffect_kr: Refreshing state... (ID: E3AG1R3KSY57S4)
aws_iam_policy_attachment.config_service_delivery_permission_attachment: Refreshing state... (ID: config-service-delivery-permission-attachment)
aws_route53_record.labs_sideeffect_kr: Refreshing state... (ID: Z12CBQ5AHVFFYA_labs.sideeffect.kr_A)
aws_route53_record.nodejs_sideeffect_kr: Refreshing state... (ID: Z12CBQ5AHVFFYA_nodejs.sideeffect.kr_A)
data.aws_iam_policy_document.vault: Refreshing state...
aws_iam_policy.vault: Refreshing state... (ID: arn:aws:iam::410655858509:policy/vault-to-write-s3)
aws_iam_policy_attachment.vault-policy-attachment: Refreshing state... (ID: vault-policy-attachment)
aws_route53_record.demo_sideeffect_kr: Creating...
  fqdn:               "" => "<computed>"
  name:               "" => "demo.sideeffect.kr"
  records.#:          "" => "1"
  records.3313064457: "" => "blog.outsider.ne.kr"
  ttl:                "" => "5"
  type:               "" => "CNAME"
  zone_id:            "" => "Z12CBQ5AHVFFYA"
aws_route53_record.demo_sideeffect_kr: Still creating... (10s elapsed)
aws_route53_record.demo_sideeffect_kr: Still creating... (20s elapsed)
aws_route53_record.demo_sideeffect_kr: Still creating... (30s elapsed)
aws_route53_record.demo_sideeffect_kr: Still creating... (40s elapsed)
aws_route53_record.demo_sideeffect_kr: Still creating... (40s elapsed)
aws_route53_record.demo_sideeffect_kr: Creation complete (ID: Z12CBQ5AHVFFYA_demo.sideeffect.kr_CNAME)

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

The state of your infrastructure has been saved to the path
below. This state is required to modify and destroy your
infrastructure, so keep it safe. To inspect the complete state
use the `terraform show` command.

State path:
Releasing state lock. This may take a few moments...

이를 적용하기 위해서 terraform apply를 실행하면 plan과 마찬가지로 다시 리소스를 모두 검사한다. 이는 plan과 apply를 실행하는 사이에 변경된 부분이 있을 수 있으므로 다시 검사를 다 실행하는 것인데 이 리소스가 많아질수록 점점 고통스러운 시간이 되고 그 결과 실패한다면 수정하고 다시 해야 하므로 귀찮아지게 된다.

앞에서 plan을 할 때 출력 로그를 보면 다음과 같은 문구가 있다.

Note: You didn't specify an "-out" parameter to save this plan, so when
"apply" is called, Terraform can't guarantee this is what will execute.

그냥 넘어가기 쉬운데 plan할 때 -out 파리미터로 플랜을 저장하지 않아서 apply 할 때 똑같이 실행된다고 보장할 수 없다는 부분이다. 이는 앞에서 apply 할 때 재검사하는 것과 같은 이유인데 plan 한 뒤에 다른 변경사항이 생긴다면 적용할 때 다른 변경사항(혹은 다른 사람이 만든 결과를 삭제하는..)이 적용될 수도 있다.

terraform plan -out=planfile처럼 -out 파라미터로 플랜파일을 지정하면(이름은 원하는 대로 지정할 수 있다.)

$ terraform plan -out=planfile
// 중략
Path: planfile

+ aws_route53_record.demo_sideeffect_kr
    fqdn:               "<computed>"
    name:               "demo.sideeffect.kr"
    records.#:          "1"
    records.3313064457: "blog.outsider.ne.kr"
    ttl:                "5"
    type:               "CNAME"
    zone_id:            "Z12CBQ5AHVFFYA"


Plan: 1 to add, 0 to change, 0 to destroy.
Releasing state lock. This may take a few moments...

Path: planfile에서 보듯이 저장한 플랜파일을 알려주고 이 결과가 이 파일에 저장이 된다. 이를 적용할 때는 terraform apply planfile로 앞에서 만든 파일을 지정해 주면 된다.

$ terraform apply planfile
Acquiring state lock. This may take a few moments...
Releasing state lock. This may take a few moments...
Acquiring state lock. This may take a few moments...
aws_route53_record.demo_sideeffect_kr: Creating...
  fqdn:               "" => "<computed>"
  name:               "" => "demo.sideeffect.kr"
  records.#:          "" => "1"
  records.3313064457: "" => "blog.outsider.ne.kr"
  ttl:                "" => "5"
  type:               "" => "CNAME"
  zone_id:            "" => "Z12CBQ5AHVFFYA"
aws_route53_record.demo_sideeffect_kr: Still creating... (10s elapsed)
aws_route53_record.demo_sideeffect_kr: Still creating... (20s elapsed)
aws_route53_record.demo_sideeffect_kr: Still creating... (30s elapsed)
aws_route53_record.demo_sideeffect_kr: Still creating... (40s elapsed)
aws_route53_record.demo_sideeffect_kr: Creation complete (ID: Z12CBQ5AHVFFYA_demo.sideeffect.kr_CNAME)

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

The state of your infrastructure has been saved to the path
below. This state is required to modify and destroy your
infrastructure, so keep it safe. To inspect the complete state
use the `terraform show` command.

State path:
Releasing state lock. This may take a few moments...

plan에서 검사한 결과를 저장했으므로 위처럼 planfile을 지정해서 apply 하면 다른 리소스를 재검사하지 않고 바로 plan 한 내용만 적용하게 되고 plan 한 부분을 적용함을 보장할 수 있다.