CDKTF执行destroy无资源可删，deploy却提示资源已存在的问题

现象 #

执行cdktf destroy DatasetCrawlerStack的结果 #

DatasetCrawlerStack  Initializing the backend...
DatasetCrawlerStack  Initializing provider plugins...
                     - Reusing previous version of hashicorp/aws from the dependency lock file
DatasetCrawlerStack  - Using previously-installed hashicorp/aws v5.49.0
DatasetCrawlerStack  Terraform has been successfully initialized!
                     
                     You may now begin working with Terraform. Try running "terraform plan" to see
                     any changes that are required for your infrastructure. All Terraform commands
                     should now work.

                     If you ever set or change modules or backend configuration for Terraform,
                     rerun this command to reinitialize your working directory. If you forget, other
                     commands will detect it and remind you to do so if necessary.
DatasetCrawlerStack  No changes. No objects need to be destroyed.

                     Either you have not created any objects yet or the existing objects were
                     already deleted outside of Terraform.
DatasetCrawlerStack  
                     Destroy complete! Resources: 0 destroyed.

执行cdktf deploy DatasetCrawlerStack的结果 #

DatasetCrawlerStack  Initializing the backend...
DatasetCrawlerStack  Initializing provider plugins...
                     - Reusing previous version of hashicorp/aws from the dependency lock file
DatasetCrawlerStack  - Using previously-installed hashicorp/aws v5.49.0
DatasetCrawlerStack  Terraform has been successfully initialized!
                     
                     You may now begin working with Terraform. Try running "terraform plan" to see
                     any changes that are required for your infrastructure. All Terraform commands
                     should now work.

                     If you ever set or change modules or backend configuration for Terraform,
                     rerun this command to reinitialize your working directory. If you forget, other
                     commands will detect it and remind you to do so if necessary.
DatasetCrawlerStack  Terraform used the selected providers to generate the following execution plan.
                     Resource actions are indicated with the following symbols:
                       + create

                     Terraform will perform the following actions:

                       # aws_glue_crawler.REDACTED-cdktf-crawler (REDACTED-cdktf-crawler) will be created
                       + resource "aws_glue_crawler" "REDACTED-cdktf-crawler" {
                           + arn           = (known after apply)
                           + database_name = "REDACTED"
                           + id            = (known after apply)
                           + name          = "REDACTED-cdktf-crawler"
                           + role          = "arn:aws:iam::822795565729:role/service-role/AWSGlueServiceRole-REDACTED-test-datasets"
                           + tags_all      = (known after apply)

                           + s3_target {
                               + path = "s3://REDACTED-lake/_datasets/_latest/"
                             }
                         }

                     Plan: 1 to add, 0 to change, 0 to destroy.
                     
                     Do you want to perform these actions?
DatasetCrawlerStack  Terraform will perform the actions described above.
                       Only 'yes' will be accepted to approve.
DatasetCrawlerStack  Enter a value: yes
DatasetCrawlerStack  aws_glue_crawler.REDACTED-cdktf-crawler: Creating...
DatasetCrawlerStack  ╷
                     │ Error: creating Glue Crawler (REDACTED-cdktf-crawler): AlreadyExistsException: 822795565729:REDACTED-cdktf-crawler already exists
                     │ 
                     │   with aws_glue_crawler.REDACTED-cdktf-crawler (REDACTED-cdktf-crawler),
                     │   on cdk.tf.json line 35, in resource.aws_glue_crawler.REDACTED-cdktf-crawler (REDACTED-cdktf-crawler):
                     │   35:       }
                     │ 
                     ╵


0 Stacks deploying     1 Stack done     0 Stacks waiting
Invoking Terraform CLI failed with exit code 1

代码实现 #

import boto3
from cdktf import S3Backend, TerraformStack
from cdktf_cdktf_provider_aws import provider,glue_crawler
# from imports.aws.glue_crawler import GlueCrawler
from pydantic_settings import BaseSettings


class StackConfig(BaseSettings):
    """Settings configured via environment variables."""

    aws_region: str = "us-east-2"
    crawler_role: str = "arn:aws:iam::822795565729:role/service-role/AWSGlueServiceRole-REDACTED-test-datasets"
    database_name: str = "REDACTED_datalake"
    s3_target_path: str = "s3://REDACTED-lake/_datasets/_latest/"


class DatasetMetadataCrawlerStack(TerraformStack):
    def __init__(self, scope, id, **kwargs):
        super().__init__(scope, id, **kwargs)

        config = StackConfig()

        provider.AwsProvider(self, f"{id}-AwsProvider", region=config.aws_region)
        boto3.client("sts").get_caller_identity()["Account"]
        account_alias = boto3.client("iam").list_account_aliases()["AccountAliases"][0]

        state_bucket = f"{account_alias}-tfstate-{config.aws_region}"
        state_key = "a2p-commons/dataset_metadata_crawler/terraform.tfstate"

        S3Backend(
            scope,
            bucket=state_bucket,
            key=state_key,
            encrypt=True,
            region=config.aws_region,
        )

        glue_crawler.GlueCrawler(
            self, 
            "REDACTED-cdktf-crawler",
            name="REDACTED-cdktf-crawler",
            role=config.crawler_role,
            database_name=config.database_name,
            s3_target=[
                glue_crawler.GlueCrawlerS3Target(
                    path=config.s3_target_path,
                )
            ],
        )

原因分析 #

• Terraform/CDKTF的核心是状态文件（state），它记录了当前栈所管理的资源。当前矛盾的本质是：状态文件中没有该Glue Crawler的记录，但AWS实际环境中已存在同名资源。
• 直接原因是S3Backend的作用域配置错误：代码中将S3Backend挂载到了根scope而非当前的DatasetMetadataCrawlerStack实例（self），导致当前栈没有正确关联到指定的S3状态存储，可能默认使用了本地状态文件，而本地状态中没有该资源的记录，所以destroy时提示无资源可销毁；但实际资源已经被创建过（可能是之前正确配置状态时部署的，或手动创建），因此deploy时触发资源已存在的错误。
• 栈初始化阶段调用boto3获取账号信息的操作，可能引发初始化顺序问题，但不是当前矛盾的直接原因。

解决方法 #

1. 修正S3Backend的作用域：将S3Backend的第一个参数从scope改为self，确保当前栈的状态正确存储到指定的S3桶中：

S3Backend(
    self,  # 修改此处
    bucket=state_bucket,
    key=state_key,
    encrypt=True,
    region=config.aws_region,
)

2. 将已存在的资源导入状态：让CDKTF识别并管理现有资源：

   • 先确保状态配置修正完成
   • 执行导入命令：

     cdktf import DatasetCrawlerStack.aws_glue_crawler.REDACTED-cdktf-crawler REDACTED-cdktf-crawler
     

   • 之后执行cdktf plan或cdktf deploy，Terraform会识别资源已存在，不会重复创建；执行cdktf destroy也能正确销毁该资源。

3. （可选）优化账号信息获取方式：移除栈初始化中的boto3调用，改用AWS Provider的数据源（如aws_caller_identity、aws_iam_account_alias）获取账号信息，避免初始化阶段的潜在问题。

内容的提问来源于stack exchange，提问作者rictuar