在 Kubernetes 部署

架构概述

DataAgent 在 K8s 环境部署时需要注意以下有状态组件:

组件存储路径说明
DeepAgent 工作空间storage/uploads/deepagent-workspace/AI Agent 生成的临时文件
上传文件storage/uploads/用户上传的文件
日志文件storage/logs/应用日志(建议输出 stdout)
PM2 运行时.pm2/PM2 进程管理器数据

关键点:必须使用 PVC 持久化存储,否则 Pod 重启会导致数据丢失。

部署配置

1. 存储声明 (PVC)

创建支持多 Pod 共享读写的存储卷:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: yiask-storage
  namespace: default
spec:
  accessModes:
    - ReadWriteMany  # 多 Pod 共享读写
  resources:
    requests:
      storage: 100Gi  # 根据实际需求调整
  # storageClassName: nfs  # 使用支持 RWX 的 StorageClass

StorageClass 选择

  • NFS:适合小规模部署
  • Ceph Rook:适合生产环境,高可用
  • 云存储:AWS EFS、阿里云 NAS 等

2. 部署 (Deployment)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: yiask
  namespace: default
  labels:
    app: yiask
spec:
  replicas: 2  # 根据负载调整
  selector:
    matchLabels:
      app: yiask
  template:
    metadata:
      labels:
        app: yiask
    spec:
      volumes:
        - name: storage
          persistentVolumeClaim:
            claimName: yiask-storage
        - name: pm2-home
          emptyDir: {}  # PM2 临时数据使用 emptyDir
      containers:
        - name: yiask
          image: your-registry/yiask:latest
          ports:
            - containerPort: 13000
          env:
            # ===== 必需环境变量 =====
            - name: APP_ENV
              value: production
            - name: APP_KEY
              valueFrom:
                secretKeyRef:
                  name: yiask-secrets
                  key: app-key
            - name: DB_DIALECT
              value: postgres
            - name: DB_HOST
              value: postgres-service  # 替换为实际数据库地址
            - name: DB_PORT
              value: "5432"
            - name: DB_DATABASE
              valueFrom:
                secretKeyRef:
                  name: yiask-secrets
                  key: db-database
            - name: DB_USER
              valueFrom:
                secretKeyRef:
                  name: yiask-secrets
                  key: db-user
            - name: DB_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: yiask-secrets
                  key: db-password

            # ===== 集群模式配置(K8s 环境必需)=====
            - name: CLUSTER_MODE
              value: "1"
            # 说明:K8s 环境中多进程能力由 Pod 副本提供,因此 CLUSTER_MODE 建议设为 1

            # ===== 缓存配置(多 Pod 必需使用 Redis)=====
            - name: CACHE_DEFAULT_STORE
              value: redis
            - name: REDIS_URL
              value: redis://redis-service:6379/0

            # ===== 日志配置(输出到 stdout,配合 K8s 日志采集)=====
            - name: LOGGER_TRANSPORT
              value: console
            - name: LOGGER_LEVEL
              value: info

            # ===== 只读文件系统容器需要配置 =====
            - name: PM2_HOME
              value: /opt/pm2

          volumeMounts:
            - name: storage
              mountPath: /app/nocobase/storage  # 挂载存储目录
            - name: pm2-home
              mountPath: /opt/pm2      # PM2 工作目录
          resources:
            requests:
              memory: "1Gi"
              cpu: "500m"
            limits:
              memory: "4Gi"
              cpu: "2000m"
          livenessProbe:
            httpGet:
              path: /api/health
              port: 13000
            initialDelaySeconds: 60
            periodSeconds: 10
          readinessProbe:
            httpGet:
              path: /api/health
              port: 13000
            initialDelaySeconds: 30
            periodSeconds: 5

3. Service

apiVersion: v1
kind: Service
metadata:
  name: yiask-service
  namespace: default
spec:
  selector:
    app: yiask
  ports:
    - port: 80
      targetPort: 13000
  type: ClusterIP

4. Ingress

注意:DataAgent 使用 SSE (Server-Sent Events) 进行流式响应,需要增加超时时间。

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: yiask-ingress
  namespace: default
  annotations:
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-buffer-size: "16k"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - yiask.example.com
      secretName: yiask-tls
  rules:
    - host: yiask.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: yiask-service
                port:
                  number: 80

5. Secret

apiVersion: v1
kind: Secret
metadata:
  name: yiask-secrets
  namespace: default
type: Opaque
stringData:
  app-key: "your-random-app-key-min-32-chars"
  db-database: "yiask"
  db-user: "yiask"
  db-password: "your-db-password"

会话亲和性

为了优化性能,可以配置同一会话的请求路由到同一个 Pod:

# 在 Service 中添加
spec:
  sessionAffinity: ClientIP
  sessionAffinityConfig:
    clientIP:
      timeoutSeconds: 3600  # 1 小时

说明:这不是必需的,因为 PVC 已经实现了数据共享。但配置会话亲和性可以减少跨 Pod 访问文件的开销。

依赖服务

PostgreSQL (必需)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
        - name: postgres
          image: postgres:17
          env:
            - name: POSTGRES_DB
              value: yiask
            - name: POSTGRES_USER
              value: yiask
            - name: POSTGRES_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: yiask-secrets
                  key: db-password
          volumeMounts:
            - name: postgres-data
              mountPath: /var/lib/postgresql/data
      volumes:
        - name: postgres-data
          persistentVolumeClaim:
            claimName: postgres-storage
---
apiVersion: v1
kind: Service
metadata:
  name: postgres-service
spec:
  selector:
    app: postgres
  ports:
    - port: 5432

Redis (可选)

用于多 Pod 间缓存共享,非必需。

apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis
spec:
  replicas: 1
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
        - name: redis
          image: redis:7-alpine
          ports:
            - containerPort: 6379
---
apiVersion: v1
kind: Service
metadata:
  name: redis-service
spec:
  selector:
    app: redis
  ports:
    - port: 6379

只读文件系统的容器

部分客户的生产环境会将容器文件系统设置为只读。此时需要完成以下两项配置:

1. 映射 /app/nocobase 到可写 Volume

/app/nocobase 目录挂载到 PVC 可写存储卷:

volumeMounts:
  - name: storage
    mountPath: /app/nocobase  # 整个 nocobase 目录挂载到可写卷

2. 映射 /runtime/yiask 到 emptyDir

/runtime/yiask 目录挂载到 emptyDir,用于存储临时运行时数据:

volumes:
  - name: yiask-runtime
    emptyDir: {}  # 临时运行时数据
---
volumeMounts:
  - name: yiask-runtime
    mountPath: /runtime/yiask

完整配置示例

spec:
  volumes:
    - name: storage
      persistentVolumeClaim:
        claimName: yiask-storage
    - name: yiask-runtime
      emptyDir: {}
  containers:
    - name: yiask
      # ... 其他配置
      volumeMounts:
        - name: storage
          mountPath: /app/nocobase
        - name: yiask-runtime
          mountPath: /runtime/yiask

故障排查

Pod 无法启动

  1. 检查 PVC 是否已绑定:

    kubectl get pvc
    kubectl describe pvc yiask-storage
  2. 检查存储类是否支持 ReadWriteMany

文件丢失

  1. 确认 PVC 挂载路径正确:/app/nocobase/storage
  2. 检查 Pod 重启后文件是否存在

SSE 连接中断

  1. 检查 Ingress timeout 配置
  2. 检查负载均衡器(如 AWS ALB)的 idle timeout 设置