脱坑： Amazon EFS CSI 驱动程序

Posted on by

首先，通过 Provision an EKS Cluster (AWS) 这篇文章，可以快捷的利用 terraform 在AWS 上部署一套 EKS。

如果想使用 pvc 和 pv，我们还需要给 K8s 安装 CSI 驱动。

AWS 给了我们一偏很详尽的安装说明： https://docs.amazonaws.cn/eks/latest/userguide/efs-csi.html

但是，我真心不知道有多少朋友能一次性就安装成功的。

为了方便管理和部署，我把 efs csi 安装过程 翻译成了 terraform 语义, 作为 Provision an EKS Cluster (AWS) 的补充

1. 创建 IAM 策略和角色

resource "aws_iam_policy" "policy" {
  name        = "${local.cluster_name}_EKS_EFS_CSI_Driver_Policy"
  description = "A test policy"

  # Terraform's "jsonencode" function converts a
  # Terraform expression result to valid JSON syntax.
  policy = <<EOT
{
    "Version" : "2012-10-17",
    "Statement" : [
        {
        "Effect" : "Allow",
        "Action" : [
            "elasticfilesystem:DescribeAccessPoints",
            "elasticfilesystem:DescribeFileSystems"
        ],
        "Resource" : "*"
        },
        {
        "Effect" : "Allow",
        "Action" : [
            "elasticfilesystem:CreateAccessPoint"
        ],
        "Resource" : "*",
        "Condition" : {
            "StringLike" : {
            "aws:RequestTag/efs.csi.aws.com/cluster" : "true"
            }
        }
        },
        {
        "Effect" : "Allow",
        "Action" : "elasticfilesystem:DeleteAccessPoint",
        "Resource" : "*",
        "Condition" : {
            "StringEquals" : {
            "aws:ResourceTag/efs.csi.aws.com/cluster" : "true"
            }
        }
        }
    ]
}
EOT
}

data "tls_certificate" "tls" {
  url = module.eks.cluster_oidc_issuer_url
}

resource "aws_iam_openid_connect_provider" "cluster" {
  client_id_list  = ["sts.amazonaws.com"]
  thumbprint_list = [data.tls_certificate.tls.certificates[0].sha1_fingerprint]
  url             = module.eks.cluster_oidc_issuer_url
}

data "aws_iam_policy_document" "assume_role_policy" {
  statement {
    actions = ["sts:AssumeRoleWithWebIdentity"]
    effect  = "Allow"

    condition {
      test = "StringEquals"
      variable = "${replace(aws_iam_openid_connect_provider.cluster.url, "https://", "")}:sub"
      values   = ["system:serviceaccount:kube-system:efs-csi-controller-sa"]
    }

    principals {
      type        = "Federated"
      identifiers = [aws_iam_openid_connect_provider.cluster.arn]
    }
  }
}

# aws_iam_openid_connect_provider
# https://learn.hashicorp.com/tutorials/terraform/aws-iam-policy
resource "aws_iam_role" "role" {
  name = "${local.cluster_name}_EKS_EFS_CSI_DriverRole"

  assume_role_policy = data.aws_iam_policy_document.assume_role_policy.json

  depends_on = [aws_iam_openid_connect_provider.cluster]

  tags = {
    tag-key = "${local.cluster_name}_EKS_EFS_CSI_DriverRole"
  }
}

resource "aws_iam_role_policy_attachment" "attach" {
  role       = aws_iam_role.role.name
  policy_arn = aws_iam_policy.policy.arn
}

2. 安装亚马逊 EFS 驱动程序

这儿我直接使用了官方方法

北京区域使用一下代码安装

helm upgrade -i aws-efs-csi-driver aws-efs-csi-driver/aws-efs-csi-driver \
    --namespace kube-system \
    --set image.repository=918309763551.dkr.ecr.cn-north-1.amazonaws.com.cn/eks/aws-efs-csi-driver \
    --set controller.serviceAccount.create=false \
    --set controller.serviceAccount.name=efs-csi-controller-sa

查看结果:

kubectl get pod -n kube-system -l "app.kubernetes.io/name=aws-efs-csi-driver,app.kubernetes.io/instance=aws-efs-csi-driver"
# NAME                                  READY   STATUS    RESTARTS   AGE
# efs-csi-controller-84c6976997-qpc2h   3/3     Running   0          25s
# efs-csi-controller-84c6976997-vz297   3/3     Running   0          25s
# efs-csi-node-jrtqp                    3/3     Running   0          14m
# efs-csi-node-t7jcc                    3/3     Running   0          14m

但是，注意这儿的 image.repository地址，虽然 Amazon EKS 附加组件容器映像地址 这个页面列了两个中国区的地址，实际上 cn-north-1 也就是北京，是没有用的，至少说现在是，使用得用 cn-northwest-1 的镜像地址。这个区域好像没有什么关系，即使你是使用得北京区域，也可以使用宁夏的镜像地址。

3. 创建 Amazon EFS 文件系统

# 创建一个安全组，该安全组允许 Amazon EFS 装载点进入 NFS 流量。
resource "aws_security_group" "allow_efs" {
  name        = "MyEfsSecurityGroup"
  description = "My EFS security group"
  vpc_id      = module.vpc.vpc_id

  tags = {
    Name = "allow_efs"
  }
}

# 为集群的 VPC 创建允许来自 CIDR 的入站 NFS 流量的入站规则。
# cidr_range: 10.0.0.0/16
resource "aws_security_group_rule" "allow_efs" {
  type              = "ingress"
  from_port         = 2049
  to_port           = 2049
  protocol          = "tcp"
  cidr_blocks       = ["10.0.0.0/16"]
  security_group_id = aws_security_group.allow_efs.id
}

# 为您的 Amazon EKS 群集创建 Amazon EFS 文件系统。
# id: fs-6225acff
resource "aws_efs_file_system" "efs" {
  tags = {
    Name = "MyEKSEfs"
  }
}

# 确定 VPC 中子网的 ID 以及子网所在的可用区
# 为节点所在的子网添加装载目标

resource "aws_efs_mount_target" "alpha" {
  count           = length(module.vpc.private_subnets)
  file_system_id  = aws_efs_file_system.efs.id
  subnet_id       = element(module.vpc.private_subnets, count.index)
  security_groups = [aws_security_group.allow_efs.id]
}

4. （可选）部署示例应用程序

之前可能没有安装 efs-service-account.yaml，这儿可以补充一下

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: efs-csi-controller-sa
  namespace: kube-system
  labels:
    app.kubernetes.io/name: aws-efs-csi-driver
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::<你的AccountId>:role/AmazonEKS_EFS_CSI_DriverRole

应用:

kubectl apply -f efs-service-account.yaml

接下部分就没有什么幺蛾子，清放心测试。