feat: implement config center phases 1-5
This commit is contained in:
7
.dockerignore
Normal file
7
.dockerignore
Normal file
@@ -0,0 +1,7 @@
|
||||
.git
|
||||
.env
|
||||
bin
|
||||
coverage.out
|
||||
web/dist
|
||||
web/node_modules
|
||||
*.log
|
||||
14
.env.example
Normal file
14
.env.example
Normal file
@@ -0,0 +1,14 @@
|
||||
HTTP_ADDR=:8080
|
||||
DATABASE_URL=postgres://configcenter:configcenter@localhost:5432/configcenter?sslmode=disable
|
||||
ETCD_ENDPOINTS=http://localhost:2379
|
||||
ETCD_DIAL_TIMEOUT=5s
|
||||
OUTBOX_INTERVAL=500ms
|
||||
OUTBOX_BATCH_SIZE=50
|
||||
OUTBOX_MAX_RETRY=12
|
||||
CORS_ALLOWED_ORIGINS=http://localhost:5173
|
||||
AUTH_ENABLED=false
|
||||
JWT_SECRET=replace-with-at-least-32-random-characters
|
||||
JWT_TTL=8h
|
||||
BOOTSTRAP_ADMIN_USERNAME=admin
|
||||
BOOTSTRAP_ADMIN_PASSWORD=replace-with-at-least-12-characters
|
||||
BOOTSTRAP_ADMIN_DISPLAY_NAME=Administrator
|
||||
10
.gitignore
vendored
Normal file
10
.gitignore
vendored
Normal file
@@ -0,0 +1,10 @@
|
||||
.env
|
||||
.idea/
|
||||
.vscode/
|
||||
bin/
|
||||
coverage.out
|
||||
web/dist/
|
||||
web/node_modules/
|
||||
*.log
|
||||
__pycache__/
|
||||
*.pyc
|
||||
1371
ConfigCenter.jsx
Normal file
1371
ConfigCenter.jsx
Normal file
File diff suppressed because it is too large
Load Diff
14
Dockerfile
Normal file
14
Dockerfile
Normal file
@@ -0,0 +1,14 @@
|
||||
FROM golang:1.24-alpine AS build
|
||||
WORKDIR /src
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
COPY cmd ./cmd
|
||||
COPY internal ./internal
|
||||
RUN CGO_ENABLED=0 go build -buildvcs=false -trimpath -ldflags="-s -w" -o /out/configcenter ./cmd/server
|
||||
|
||||
FROM alpine:3.21
|
||||
RUN apk add --no-cache ca-certificates && adduser -D -u 10001 configcenter
|
||||
USER configcenter
|
||||
COPY --from=build /out/configcenter /usr/local/bin/configcenter
|
||||
EXPOSE 8080
|
||||
ENTRYPOINT ["configcenter"]
|
||||
29
Makefile
Normal file
29
Makefile
Normal file
@@ -0,0 +1,29 @@
|
||||
.PHONY: dev test build web-install web-build compose-up compose-down integration-smoke loadtest
|
||||
|
||||
dev:
|
||||
go run ./cmd/server
|
||||
|
||||
test:
|
||||
GOCACHE=/tmp/configcenter-go-cache go test ./...
|
||||
|
||||
build:
|
||||
mkdir -p bin
|
||||
GOCACHE=/tmp/configcenter-go-cache go build -buildvcs=false -trimpath -o bin/configcenter ./cmd/server
|
||||
|
||||
web-install:
|
||||
npm --prefix web install
|
||||
|
||||
web-build:
|
||||
npm --prefix web run build
|
||||
|
||||
compose-up:
|
||||
docker compose up --build
|
||||
|
||||
compose-down:
|
||||
docker compose down
|
||||
|
||||
integration-smoke:
|
||||
sh scripts/integration-smoke.sh
|
||||
|
||||
loadtest:
|
||||
GOCACHE=/tmp/configcenter-go-cache go run ./cmd/loadtest -duration=30s -concurrency=32
|
||||
208
README.md
Normal file
208
README.md
Normal file
@@ -0,0 +1,208 @@
|
||||
# Config Center
|
||||
|
||||
一个按 `config-center-etcd-implementation.md` 落地的自建配置中心。PostgreSQL 保存编辑态、版本和审计记录;发布事务写入 release 与 outbox;worker 再将完整 namespace 快照原子下发到 etcd。Web Console 已接入真实 API,不再使用浏览器内存数据。
|
||||
|
||||
## 已实现
|
||||
|
||||
- 应用、环境、命名空间、配置项 CRUD;
|
||||
- 草稿/已发布值分离,新增、修改、待删除状态与发布 diff;
|
||||
- PostgreSQL 事务化发布,基于 advisory lock 的并发版本分配;
|
||||
- Outbox claim、租约恢复、指数退避、最大重试和 release 状态回写;
|
||||
- etcd 完整快照 + `__meta` 同事务写入;
|
||||
- 运行时配置读取与 SSE Watch,服务端同 key Watch 扇出;
|
||||
- 发布历史和“生成新版本”的可追溯回滚;
|
||||
- 审计日志查询;
|
||||
- 可选 JWT 登录、bcrypt 密码、全局管理员与应用级 `viewer` / `app-owner` RBAC;
|
||||
- IP/CIDR、实例 ID、稳定百分比三类灰度规则,支持优先级覆盖;
|
||||
- Go/Python SDK,支持自动重连、全量替换和本地文件缓存兜底;
|
||||
- React Web Console(含登录、灰度、用户授权页面)、Docker Compose 与本地内存开发模式;
|
||||
- Prometheus 指标与告警规则、三副本服务/三节点 etcd Kubernetes 清单、SLO 压测工具。
|
||||
|
||||
## 快速启动
|
||||
|
||||
不安装 PostgreSQL/etcd 也可以启动。未设置连接信息时,服务会使用带演示数据的内存控制面和内存运行时:
|
||||
|
||||
```bash
|
||||
make dev
|
||||
```
|
||||
|
||||
另开终端启动 Web Console:
|
||||
|
||||
```bash
|
||||
make web-install
|
||||
npm --prefix web run dev
|
||||
```
|
||||
|
||||
浏览器访问 `http://localhost:5173`。API 默认监听 `http://localhost:8080`。
|
||||
|
||||
完整依赖环境:
|
||||
|
||||
```bash
|
||||
docker compose up --build
|
||||
```
|
||||
|
||||
Compose 会启动 PostgreSQL、etcd、Redis、Config Server 和 Web Console。打开 `http://localhost:5173`。PostgreSQL migration 会在服务启动时幂等执行。
|
||||
|
||||
启用 Prometheus(`http://localhost:9090`):
|
||||
|
||||
```bash
|
||||
docker compose --profile monitoring up --build
|
||||
```
|
||||
|
||||
## 配置
|
||||
|
||||
| 环境变量 | 默认值 | 说明 |
|
||||
|---|---:|---|
|
||||
| `HTTP_ADDR` | `:8080` | HTTP 监听地址 |
|
||||
| `DATABASE_URL` | 空 | 为空时使用内存控制面 |
|
||||
| `ETCD_ENDPOINTS` | 空 | 逗号分隔;为空时使用内存运行时 |
|
||||
| `ETCD_DIAL_TIMEOUT` | `5s` | etcd 连接超时 |
|
||||
| `OUTBOX_INTERVAL` | `500ms` | outbox 扫描周期 |
|
||||
| `OUTBOX_BATCH_SIZE` | `50` | 单次 claim 数量 |
|
||||
| `OUTBOX_MAX_RETRY` | `12` | 下发失败最大重试次数 |
|
||||
| `CORS_ALLOWED_ORIGINS` | `http://localhost:5173` | 允许的 Web Origin,逗号分隔 |
|
||||
| `AUTH_ENABLED` | `false` | 是否启用 JWT 登录与 RBAC |
|
||||
| `JWT_SECRET` | 空 | HS256 密钥;启用认证时至少 32 个字符 |
|
||||
| `JWT_TTL` | `8h` | 登录令牌有效期 |
|
||||
| `BOOTSTRAP_ADMIN_USERNAME` | `admin` | 首次启动时幂等创建的管理员用户名 |
|
||||
| `BOOTSTRAP_ADMIN_PASSWORD` | 空 | 启用认证时必填,至少 12 个字符 |
|
||||
| `BOOTSTRAP_ADMIN_DISPLAY_NAME` | `Administrator` | 初始管理员显示名称 |
|
||||
|
||||
可复制 `.env.example` 后按部署环境调整。认证启用后,除健康检查、登录和 `/metrics` 外的接口都要求 `Authorization: Bearer <token>`。初始管理员只会在不存在时创建,不会在重启时覆盖已有密码。生产环境应通过 Secret 管理数据库密码、JWT 密钥和初始密码,并启用 PostgreSQL/etcd TLS。
|
||||
|
||||
## API
|
||||
|
||||
管理面响应统一为 `{ "data": ... }`,错误统一为 `{ "error": { "code", "message" } }`。
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
|---|---|---|
|
||||
| `GET/POST` | `/v1/applications` | 应用列表/创建 |
|
||||
| `PUT/DELETE` | `/v1/applications/{id}` | 应用更新/删除 |
|
||||
| `GET/POST` | `/v1/environments` | 环境列表/创建 |
|
||||
| `GET/POST` | `/v1/namespaces` | 命名空间列表/创建 |
|
||||
| `GET/POST` | `/v1/config-items` | 配置项列表/创建 |
|
||||
| `PUT/DELETE` | `/v1/config-items/{id}` | 修改/标记待删除 |
|
||||
| `POST` | `/v1/config-items/{id}/restore` | 撤销待删除 |
|
||||
| `POST` | `/v1/publish` | 创建发布版本,返回 `202` |
|
||||
| `GET` | `/v1/releases?appId=&nsId=&envId=` | 发布历史与下发状态 |
|
||||
| `POST` | `/v1/rollback` | 回滚并生成新版本 |
|
||||
| `GET` | `/v1/audit-logs` | 审计日志 |
|
||||
| `POST` | `/v1/auth/login` | 用户登录并签发 JWT |
|
||||
| `GET` | `/v1/me` | 当前用户与应用角色 |
|
||||
| `GET/POST` | `/v1/users` | 用户列表/创建(管理员) |
|
||||
| `GET/PUT/DELETE` | `/v1/users/{id}/roles[/{appId}]` | 应用角色管理(管理员) |
|
||||
| `GET/POST` | `/v1/gray-rules` | 当前范围灰度规则列表/创建 |
|
||||
| `PUT/DELETE` | `/v1/gray-rules/{id}` | 灰度规则更新/删除 |
|
||||
| `GET` | `/v1/config?env=&app=&namespace=&ip=&instance=` | 读取运行时快照并匹配灰度规则 |
|
||||
| `GET` | `/v1/watch?env=&app=&namespace=&ip=&instance=` | SSE 全量同步与更新事件 |
|
||||
| `GET` | `/metrics` | Prometheus 指标 |
|
||||
|
||||
认证关闭时服务以开发管理员身份运行,并可用 `X-User` 记录操作者;认证开启时操作者来自 JWT,客户端不能通过请求头伪造。`viewer` 可以读取应用配置,`app-owner` 还可以维护命名空间、配置、发布、回滚和灰度规则,全局管理员可管理应用、环境、审计和用户授权。
|
||||
|
||||
### 灰度匹配
|
||||
|
||||
`ip` 规则接受精确 IP 或 CIDR;`instance` 规则接受实例 ID 列表;`percentage` 使用 salt 与实例 ID(无实例时使用 IP)做稳定哈希,同一实例不会随机漂移。多条规则命中时按优先级从低到高覆盖,并在响应的 `grayRuleIds` 中返回命中规则。
|
||||
|
||||
### 发布示例
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8080/v1/publish \
|
||||
-H 'Content-Type: application/json' \
|
||||
-H 'X-User: alice' \
|
||||
-d '{"appId":1,"nsId":1,"envId":1,"comment":"enable feature"}'
|
||||
```
|
||||
|
||||
数据库事务提交后返回 `pending`;outbox 写入 etcd 后,`GET /v1/releases/{id}` 会变为 `applied` 并带上 `etcdRevision`。
|
||||
|
||||
## SDK
|
||||
|
||||
Go SDK:
|
||||
|
||||
```go
|
||||
client, err := configsdk.New(configsdk.Options{
|
||||
BaseURL: "http://localhost:8080",
|
||||
Env: "PROD",
|
||||
App: "order-service",
|
||||
Token: os.Getenv("CONFIGCENTER_TOKEN"),
|
||||
Instance: "order-3",
|
||||
CacheFile: "/var/lib/my-service/configcenter.json",
|
||||
})
|
||||
if err != nil { /* handle */ }
|
||||
|
||||
_ = client.Load(ctx, "application")
|
||||
go client.WatchAndSync(ctx, "application")
|
||||
port := client.GetInt("application", "server.port", 8080)
|
||||
```
|
||||
|
||||
Python SDK 位于 `sdk/python`:
|
||||
|
||||
```python
|
||||
from configcenter import ConfigClient
|
||||
|
||||
client = ConfigClient(
|
||||
"http://localhost:8080", "PROD", "order-service", "/tmp/order-config.json",
|
||||
token=os.environ.get("CONFIGCENTER_TOKEN"), instance="order-3",
|
||||
)
|
||||
client.load("application")
|
||||
client.start_background_watch("application")
|
||||
timeout = client.get("application", "order.timeout.minutes", "30")
|
||||
```
|
||||
|
||||
本地缓存采用完整快照和原子 rename;Config Server/etcd 暂时不可用时,进程可以读取上一次成功同步的缓存启动。
|
||||
|
||||
## 开发与验证
|
||||
|
||||
```bash
|
||||
make test
|
||||
make build
|
||||
make web-build
|
||||
docker compose config
|
||||
make integration-smoke
|
||||
make loadtest
|
||||
```
|
||||
|
||||
核心端到端测试覆盖:CRUD → 发布事务 → outbox worker → 运行时读取 → 灰度覆盖与指标;单元/HTTP 集成测试还覆盖 JWT 过期与篡改、应用级 RBAC、多版本回滚和 SDK 鉴权/灰度参数。
|
||||
|
||||
压测工具默认验证运行时读取的错误率不超过 1%、p95 不超过 200ms,可覆盖目标和阈值:
|
||||
|
||||
```bash
|
||||
go run ./cmd/loadtest \
|
||||
-base-url=http://127.0.0.1:8080 -env=DEV -app=demo-service -namespace=application \
|
||||
-token="$CONFIGCENTER_TOKEN" -concurrency=64 -duration=60s -max-p95=200ms
|
||||
```
|
||||
|
||||
## 生产部署
|
||||
|
||||
`deploy/kubernetes/configcenter.yaml` 提供三副本无状态 Config Server、三节点 etcd、跨节点调度、PDB、HPA、探针和资源限制。部署前替换镜像与 Origin,并创建密钥:
|
||||
|
||||
```bash
|
||||
kubectl create namespace configcenter
|
||||
kubectl -n configcenter create secret generic configcenter-secrets \
|
||||
--from-literal=DATABASE_URL='postgres://...' \
|
||||
--from-literal=JWT_SECRET='至少32位随机密钥' \
|
||||
--from-literal=BOOTSTRAP_ADMIN_PASSWORD='至少12位初始密码'
|
||||
kubectl apply -f deploy/kubernetes/configcenter.yaml
|
||||
```
|
||||
|
||||
生产 PostgreSQL 应使用托管高可用/主从方案;etcd 应跨故障域部署,定期 snapshot、compact 和 defrag。示例清单使用明文集群内 etcd 地址,生产落地需补充双向 TLS 和只允许 Config Server 访问的网络策略。
|
||||
|
||||
## 目录
|
||||
|
||||
```text
|
||||
cmd/server 服务入口
|
||||
internal/api/httpapi REST + SSE
|
||||
internal/store/postgres PostgreSQL repository 与 migration
|
||||
internal/store/memory 本地开发/测试实现
|
||||
internal/runtime/etcd etcd 快照与 Watch
|
||||
internal/outbox 异步下发 worker
|
||||
internal/watch 同 key Watch 扇出
|
||||
internal/auth JWT、密码与应用级 RBAC
|
||||
internal/gray 灰度规则校验与确定性匹配
|
||||
internal/metrics Prometheus 指标
|
||||
cmd/loadtest 运行时读取 SLO 压测
|
||||
deploy Kubernetes 与监控告警配置
|
||||
pkg/sdk/go Go SDK
|
||||
sdk/python Python SDK
|
||||
api/proto gRPC 契约
|
||||
web React Web Console
|
||||
```
|
||||
77
api/proto/configcenter/v1/config.proto
Normal file
77
api/proto/configcenter/v1/config.proto
Normal file
@@ -0,0 +1,77 @@
|
||||
syntax = "proto3";
|
||||
|
||||
package configcenter.v1;
|
||||
|
||||
option go_package = "github.com/longpeng/configcenter/pkg/proto/v1;configcenterv1";
|
||||
|
||||
message ConfigItem {
|
||||
string key = 1;
|
||||
string value = 2;
|
||||
}
|
||||
|
||||
message GetConfigRequest {
|
||||
string env = 1;
|
||||
string app = 2;
|
||||
string namespace = 3;
|
||||
}
|
||||
|
||||
message GetConfigResponse {
|
||||
repeated ConfigItem items = 1;
|
||||
int64 revision = 2;
|
||||
int64 release_version = 3;
|
||||
}
|
||||
|
||||
message WatchConfigRequest {
|
||||
string env = 1;
|
||||
string app = 2;
|
||||
string namespace = 3;
|
||||
int64 start_revision = 4;
|
||||
}
|
||||
|
||||
message ConfigEvent {
|
||||
enum EventType {
|
||||
FULL_SYNC = 0;
|
||||
UPDATED = 1;
|
||||
}
|
||||
EventType type = 1;
|
||||
repeated ConfigItem items = 2;
|
||||
int64 revision = 3;
|
||||
}
|
||||
|
||||
message PublishRequest {
|
||||
int64 env_id = 1;
|
||||
int64 app_id = 2;
|
||||
int64 namespace_id = 3;
|
||||
string comment = 4;
|
||||
string operator = 5;
|
||||
}
|
||||
|
||||
message PublishResponse {
|
||||
int64 release_id = 1;
|
||||
int64 release_version = 2;
|
||||
string status = 3;
|
||||
}
|
||||
|
||||
message RollbackRequest {
|
||||
int64 env_id = 1;
|
||||
int64 app_id = 2;
|
||||
int64 namespace_id = 3;
|
||||
int64 target_version = 4;
|
||||
string operator = 5;
|
||||
}
|
||||
|
||||
message RollbackResponse {
|
||||
int64 release_id = 1;
|
||||
int64 new_release_version = 2;
|
||||
string status = 3;
|
||||
}
|
||||
|
||||
service ConfigService {
|
||||
rpc GetConfig(GetConfigRequest) returns (GetConfigResponse);
|
||||
rpc WatchConfig(WatchConfigRequest) returns (stream ConfigEvent);
|
||||
}
|
||||
|
||||
service AdminService {
|
||||
rpc PublishConfig(PublishRequest) returns (PublishResponse);
|
||||
rpc RollbackConfig(RollbackRequest) returns (RollbackResponse);
|
||||
}
|
||||
107
cmd/loadtest/main.go
Normal file
107
cmd/loadtest/main.go
Normal file
@@ -0,0 +1,107 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"sort"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
type result struct {
|
||||
duration time.Duration
|
||||
ok bool
|
||||
}
|
||||
|
||||
func main() {
|
||||
baseURL := flag.String("base-url", "http://127.0.0.1:8080", "Config Center base URL")
|
||||
env := flag.String("env", "DEV", "environment code")
|
||||
app := flag.String("app", "demo-service", "application code")
|
||||
namespace := flag.String("namespace", "application", "namespace")
|
||||
token := flag.String("token", "", "optional bearer token")
|
||||
concurrency := flag.Int("concurrency", 32, "concurrent workers")
|
||||
duration := flag.Duration("duration", 30*time.Second, "test duration")
|
||||
maxErrorRate := flag.Float64("max-error-rate", 0.01, "failure threshold")
|
||||
maxP95 := flag.Duration("max-p95", 200*time.Millisecond, "p95 latency threshold")
|
||||
flag.Parse()
|
||||
if *concurrency <= 0 || *duration <= 0 {
|
||||
fmt.Fprintln(os.Stderr, "concurrency and duration must be positive")
|
||||
os.Exit(2)
|
||||
}
|
||||
|
||||
query := url.Values{"env": {*env}, "app": {*app}, "namespace": {*namespace}}
|
||||
endpoint := fmt.Sprintf("%s/v1/config?%s", *baseURL, query.Encode())
|
||||
client := &http.Client{Transport: &http.Transport{MaxIdleConns: *concurrency * 2, MaxIdleConnsPerHost: *concurrency, IdleConnTimeout: 30 * time.Second}, Timeout: 5 * time.Second}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), *duration)
|
||||
defer cancel()
|
||||
results := make(chan result, *concurrency*128)
|
||||
var workers sync.WaitGroup
|
||||
|
||||
for i := 0; i < *concurrency; i++ {
|
||||
workers.Add(1)
|
||||
go func() {
|
||||
defer workers.Done()
|
||||
for ctx.Err() == nil {
|
||||
request, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, nil)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
if *token != "" {
|
||||
request.Header.Set("Authorization", "Bearer "+*token)
|
||||
}
|
||||
begin := time.Now()
|
||||
response, err := client.Do(request)
|
||||
elapsed := time.Since(begin)
|
||||
if err != nil && ctx.Err() != nil {
|
||||
return
|
||||
}
|
||||
ok := err == nil && response.StatusCode == http.StatusOK
|
||||
if response != nil {
|
||||
_, _ = io.Copy(io.Discard, response.Body)
|
||||
response.Body.Close()
|
||||
}
|
||||
select {
|
||||
case results <- result{duration: elapsed, ok: ok}:
|
||||
case <-ctx.Done():
|
||||
return
|
||||
}
|
||||
}
|
||||
}()
|
||||
}
|
||||
go func() {
|
||||
workers.Wait()
|
||||
close(results)
|
||||
}()
|
||||
|
||||
latencies := make([]time.Duration, 0)
|
||||
failures := 0
|
||||
for item := range results {
|
||||
latencies = append(latencies, item.duration)
|
||||
if !item.ok {
|
||||
failures++
|
||||
}
|
||||
}
|
||||
if len(latencies) == 0 {
|
||||
fmt.Fprintln(os.Stderr, "no requests completed")
|
||||
os.Exit(1)
|
||||
}
|
||||
sort.Slice(latencies, func(i, j int) bool { return latencies[i] < latencies[j] })
|
||||
p50, p95, p99 := percentile(latencies, 0.50), percentile(latencies, 0.95), percentile(latencies, 0.99)
|
||||
errorRate := float64(failures) / float64(len(latencies))
|
||||
qps := float64(len(latencies)) / duration.Seconds()
|
||||
fmt.Printf("requests=%d failures=%d error_rate=%.4f qps=%.1f p50=%s p95=%s p99=%s\n", len(latencies), failures, errorRate, qps, p50, p95, p99)
|
||||
if errorRate > *maxErrorRate || p95 > *maxP95 {
|
||||
fmt.Fprintf(os.Stderr, "SLO failed: error_rate<=%.4f p95<=%s\n", *maxErrorRate, *maxP95)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
func percentile(values []time.Duration, quantile float64) time.Duration {
|
||||
index := int(float64(len(values)-1) * quantile)
|
||||
return values[index]
|
||||
}
|
||||
121
cmd/server/main.go
Normal file
121
cmd/server/main.go
Normal file
@@ -0,0 +1,121 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"flag"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/signal"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"github.com/longpeng/configcenter/internal/api/httpapi"
|
||||
"github.com/longpeng/configcenter/internal/auth"
|
||||
"github.com/longpeng/configcenter/internal/config"
|
||||
"github.com/longpeng/configcenter/internal/metrics"
|
||||
"github.com/longpeng/configcenter/internal/outbox"
|
||||
runtimepkg "github.com/longpeng/configcenter/internal/runtime"
|
||||
etcdstore "github.com/longpeng/configcenter/internal/runtime/etcd"
|
||||
memoryruntime "github.com/longpeng/configcenter/internal/runtime/memory"
|
||||
storepkg "github.com/longpeng/configcenter/internal/store"
|
||||
memory_store "github.com/longpeng/configcenter/internal/store/memory"
|
||||
postgres_store "github.com/longpeng/configcenter/internal/store/postgres"
|
||||
"github.com/longpeng/configcenter/internal/watch"
|
||||
)
|
||||
|
||||
func main() {
|
||||
migrateOnly := flag.Bool("migrate-only", false, "apply database migrations and exit")
|
||||
flag.Parse()
|
||||
|
||||
logger := slog.New(slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{Level: slog.LevelInfo}))
|
||||
cfg := config.Load()
|
||||
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
|
||||
defer stop()
|
||||
|
||||
repository, err := openStore(ctx, cfg, logger)
|
||||
if err != nil {
|
||||
logger.Error("open control-plane store", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
defer repository.Close()
|
||||
if *migrateOnly {
|
||||
logger.Info("database migrations applied")
|
||||
return
|
||||
}
|
||||
authorizer, err := auth.New(repository, cfg.AuthEnabled, cfg.JWTSecret, cfg.JWTTTL)
|
||||
if err != nil {
|
||||
logger.Error("initialize authentication", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
if err := authorizer.Bootstrap(ctx, cfg.BootstrapAdminUsername, cfg.BootstrapAdminPassword, cfg.BootstrapAdminDisplayName); err != nil {
|
||||
logger.Error("bootstrap administrator", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
runtimeStore, err := openRuntime(cfg, logger)
|
||||
if err != nil {
|
||||
logger.Error("open runtime store", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
defer runtimeStore.Close() //nolint:errcheck
|
||||
|
||||
hub := watch.New(ctx, runtimeStore)
|
||||
collector := metrics.New()
|
||||
worker := outbox.New(repository, runtimeStore, cfg.OutboxInterval, cfg.OutboxBatchSize, cfg.OutboxMaxRetry, collector, logger)
|
||||
go worker.Run(ctx)
|
||||
|
||||
api := httpapi.New(repository, runtimeStore, hub, authorizer, collector, cfg.AllowedOrigins, logger)
|
||||
server := &http.Server{
|
||||
Addr: cfg.HTTPAddr,
|
||||
Handler: api.Handler(),
|
||||
ReadHeaderTimeout: 5 * time.Second,
|
||||
ReadTimeout: 15 * time.Second,
|
||||
IdleTimeout: 60 * time.Second,
|
||||
}
|
||||
|
||||
serverErrors := make(chan error, 1)
|
||||
go func() {
|
||||
logger.Info("config center listening", "address", cfg.HTTPAddr)
|
||||
serverErrors <- server.ListenAndServe()
|
||||
}()
|
||||
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
logger.Info("shutting down config center")
|
||||
case err := <-serverErrors:
|
||||
if err != nil && err != http.ErrServerClosed {
|
||||
logger.Error("http server stopped", "error", err)
|
||||
}
|
||||
}
|
||||
|
||||
shutdownCtx, cancel := context.WithTimeout(context.Background(), cfg.ShutdownTimeout)
|
||||
defer cancel()
|
||||
if err := server.Shutdown(shutdownCtx); err != nil {
|
||||
logger.Error("graceful shutdown", "error", err)
|
||||
}
|
||||
}
|
||||
|
||||
func openStore(ctx context.Context, cfg config.Config, logger *slog.Logger) (storepkg.Store, error) {
|
||||
if cfg.DatabaseURL == "" {
|
||||
logger.Warn("DATABASE_URL is empty; using the in-memory control-plane store")
|
||||
return memory_store.New(true), nil
|
||||
}
|
||||
repository, err := postgres_store.Open(ctx, cfg.DatabaseURL)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := repository.Migrate(ctx); err != nil {
|
||||
repository.Close()
|
||||
return nil, err
|
||||
}
|
||||
return repository, nil
|
||||
}
|
||||
|
||||
func openRuntime(cfg config.Config, logger *slog.Logger) (runtimepkg.Store, error) {
|
||||
if len(cfg.EtcdEndpoints) == 0 {
|
||||
logger.Warn("ETCD_ENDPOINTS is empty; using the in-memory runtime store")
|
||||
return memoryruntime.New(), nil
|
||||
}
|
||||
return etcdstore.Open(cfg.EtcdEndpoints, cfg.EtcdDialTimeout)
|
||||
}
|
||||
97
compose.yaml
Normal file
97
compose.yaml
Normal file
@@ -0,0 +1,97 @@
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16-alpine
|
||||
environment:
|
||||
POSTGRES_DB: configcenter
|
||||
POSTGRES_USER: configcenter
|
||||
POSTGRES_PASSWORD: configcenter
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U configcenter -d configcenter"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 20
|
||||
volumes:
|
||||
- postgres-data:/var/lib/postgresql/data
|
||||
|
||||
etcd:
|
||||
image: quay.io/coreos/etcd:v3.5.17
|
||||
command:
|
||||
- /usr/local/bin/etcd
|
||||
- --name=etcd-1
|
||||
- --data-dir=/etcd-data
|
||||
- --listen-client-urls=http://0.0.0.0:2379
|
||||
- --advertise-client-urls=http://etcd:2379
|
||||
healthcheck:
|
||||
test: ["CMD", "etcdctl", "endpoint", "health"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 20
|
||||
volumes:
|
||||
- etcd-data:/etcd-data
|
||||
|
||||
redis:
|
||||
image: redis:7.4-alpine
|
||||
command: ["redis-server", "--appendonly", "yes"]
|
||||
healthcheck:
|
||||
test: ["CMD", "redis-cli", "ping"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 20
|
||||
volumes:
|
||||
- redis-data:/data
|
||||
|
||||
server:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
environment:
|
||||
HTTP_ADDR: :8080
|
||||
DATABASE_URL: postgres://configcenter:configcenter@postgres:5432/configcenter?sslmode=disable
|
||||
ETCD_ENDPOINTS: http://etcd:2379
|
||||
CORS_ALLOWED_ORIGINS: http://localhost:5173
|
||||
AUTH_ENABLED: ${AUTH_ENABLED:-false}
|
||||
JWT_SECRET: ${JWT_SECRET:-}
|
||||
JWT_TTL: ${JWT_TTL:-8h}
|
||||
BOOTSTRAP_ADMIN_USERNAME: ${BOOTSTRAP_ADMIN_USERNAME:-admin}
|
||||
BOOTSTRAP_ADMIN_PASSWORD: ${BOOTSTRAP_ADMIN_PASSWORD:-}
|
||||
BOOTSTRAP_ADMIN_DISPLAY_NAME: ${BOOTSTRAP_ADMIN_DISPLAY_NAME:-Administrator}
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
etcd:
|
||||
condition: service_healthy
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-q", "-O", "-", "http://127.0.0.1:8080/health/ready"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 20
|
||||
ports:
|
||||
- "8080:8080"
|
||||
|
||||
web:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: web/Dockerfile
|
||||
depends_on:
|
||||
server:
|
||||
condition: service_healthy
|
||||
ports:
|
||||
- "5173:80"
|
||||
|
||||
prometheus:
|
||||
image: prom/prometheus:v3.2.1
|
||||
profiles: ["monitoring"]
|
||||
command: ["--config.file=/etc/prometheus/prometheus.yml"]
|
||||
volumes:
|
||||
- ./deploy/monitoring/prometheus.yml:/etc/prometheus/prometheus.yml:ro
|
||||
- ./deploy/monitoring/alerts.yml:/etc/prometheus/alerts.yml:ro
|
||||
depends_on:
|
||||
server:
|
||||
condition: service_healthy
|
||||
ports:
|
||||
- "9090:9090"
|
||||
|
||||
volumes:
|
||||
postgres-data:
|
||||
etcd-data:
|
||||
redis-data:
|
||||
506
config-center-etcd-implementation.md
Normal file
506
config-center-etcd-implementation.md
Normal file
@@ -0,0 +1,506 @@
|
||||
# 自建配置中心实现方案(etcd + PostgreSQL + Redis + Go)
|
||||
|
||||
对应架构:
|
||||
|
||||
```
|
||||
Web Console
|
||||
│
|
||||
▼
|
||||
Config Server (Go)
|
||||
│
|
||||
┌──────────────┼──────────────┐
|
||||
│ │ │
|
||||
PostgreSQL etcd Redis
|
||||
配置/版本/RBAC 生效配置 缓存/事件
|
||||
发布记录/审计 Watch
|
||||
```
|
||||
|
||||
## 一、核心设计原则
|
||||
|
||||
**PostgreSQL 是控制面(source of truth),etcd 是数据面(运行时分发)。**
|
||||
|
||||
配置的"编辑态"(草稿)永远只存在 PostgreSQL 里;只有点了「发布」,才会把该 namespace 当前生效的完整配置写进 etcd。etcd 里任何时刻只保存"当前生效"的那一份,历史版本、审计、RBAC 全部留在 PG。
|
||||
|
||||
这样划分带来两个好处:
|
||||
- etcd 数据量可控(etcd 官方建议库大小控制在几个 GB 内,不适合塞历史/审计数据);
|
||||
- 复杂查询(按部门筛选应用、按时间查审计日志)交给关系型数据库,etcd 只做它最擅长的事:强一致 + Watch。
|
||||
|
||||
## 二、发布语义与一致性(Outbox 模式)
|
||||
|
||||
发布动作要跨两个存储写入,如果直接「先写 PG 再写 etcd」,中间进程崩溃会导致状态不一致(PG 说已发布,etcd 里其实是旧值)。
|
||||
|
||||
解决办法:**Outbox 模式**——发布在同一个 PG 事务里写入 `releases` 记录和 `release_outbox` 记录;一个独立的异步 worker 轮询 outbox,把内容幂等地 Put 进 etcd,成功后回写 `etcd_revision` 并标记 `applied`。这样即使 Config Server 中途重启,未完成的发布也能被 worker 续上,不会丢失,也不会重复产生副作用(Put 本身是幂等的)。
|
||||
|
||||
```
|
||||
发布请求
|
||||
│
|
||||
▼
|
||||
PG 事务:写 releases + release_outbox(DB 提交即成功返回给用户)
|
||||
│
|
||||
▼
|
||||
Outbox Worker(异步、可重试)
|
||||
│
|
||||
▼
|
||||
etcd.Put(key, snapshot) ──成功──▶ 回写 etcd_revision,release.status = applied
|
||||
└─失败──▶ retry_count++,下一轮重试
|
||||
```
|
||||
|
||||
## 三、etcd Key 设计
|
||||
|
||||
推荐按 **namespace 整体做一个 blob**,而不是每个 key 单独存一条,理由:一次发布往往同时改多个 key,整体 blob 能保证客户端拿到的永远是"某个版本的完整快照",不会出现 watch 到一半、配置项之间不一致的中间态。
|
||||
|
||||
```
|
||||
/config/{env}/{app}/{namespace} -> JSON: {"server.port":"8080", "log.level":"INFO", ...}
|
||||
/config/{env}/{app}/{namespace}/__meta -> JSON: {"version":12,"releaseId":88,"publishedAt":"...","publishedBy":"admin"}
|
||||
```
|
||||
|
||||
客户端只 watch `/config/{env}/{app}/{namespace}` 这一个 key,收到事件后解码出完整 map,本地直接替换。
|
||||
|
||||
## 四、PostgreSQL 数据模型
|
||||
|
||||
```sql
|
||||
CREATE TABLE applications (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
app_code VARCHAR(64) UNIQUE NOT NULL,
|
||||
name VARCHAR(128) NOT NULL,
|
||||
department VARCHAR(128),
|
||||
owner VARCHAR(64),
|
||||
description TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE environments (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
env_code VARCHAR(32) UNIQUE NOT NULL, -- DEV/TEST/STAGING/PROD
|
||||
name VARCHAR(64) NOT NULL,
|
||||
sort_order INT NOT NULL DEFAULT 0,
|
||||
description TEXT
|
||||
);
|
||||
|
||||
CREATE TABLE namespaces (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
app_id BIGINT NOT NULL REFERENCES applications(id) ON DELETE CASCADE,
|
||||
name VARCHAR(64) NOT NULL,
|
||||
format VARCHAR(16) NOT NULL DEFAULT 'properties', -- properties/yaml/json/xml
|
||||
description TEXT,
|
||||
UNIQUE(app_id, name)
|
||||
);
|
||||
|
||||
-- 草稿态配置项:value 是编辑中的值,released_value 是最近一次发布生效的值
|
||||
CREATE TABLE config_items (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
namespace_id BIGINT NOT NULL REFERENCES namespaces(id) ON DELETE CASCADE,
|
||||
env_id BIGINT NOT NULL REFERENCES environments(id) ON DELETE CASCADE,
|
||||
key VARCHAR(256) NOT NULL,
|
||||
value TEXT NOT NULL,
|
||||
released_value TEXT, -- NULL 表示从未发布过(待发布·新增)
|
||||
pending_delete BOOLEAN NOT NULL DEFAULT false,
|
||||
comment TEXT,
|
||||
updated_by VARCHAR(64),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
UNIQUE(namespace_id, env_id, key)
|
||||
);
|
||||
|
||||
CREATE TABLE releases (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
namespace_id BIGINT NOT NULL REFERENCES namespaces(id),
|
||||
env_id BIGINT NOT NULL REFERENCES environments(id),
|
||||
version INT NOT NULL,
|
||||
snapshot JSONB NOT NULL, -- 该版本完整的 key -> value
|
||||
diff_added INT DEFAULT 0,
|
||||
diff_modified INT DEFAULT 0,
|
||||
diff_removed INT DEFAULT 0,
|
||||
comment TEXT,
|
||||
operator VARCHAR(64) NOT NULL,
|
||||
etcd_revision BIGINT, -- 写入 etcd 成功后回填,用于对账
|
||||
status VARCHAR(16) NOT NULL DEFAULT 'pending', -- pending/applied/failed
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
UNIQUE(namespace_id, env_id, version)
|
||||
);
|
||||
|
||||
-- Outbox:保证 DB 事务与 etcd 写入之间的最终一致
|
||||
CREATE TABLE release_outbox (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
release_id BIGINT NOT NULL REFERENCES releases(id),
|
||||
etcd_key VARCHAR(512) NOT NULL,
|
||||
payload JSONB NOT NULL,
|
||||
status VARCHAR(16) NOT NULL DEFAULT 'pending', -- pending/done/failed
|
||||
retry_count INT NOT NULL DEFAULT 0,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE users (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
username VARCHAR(64) UNIQUE NOT NULL,
|
||||
password_hash VARCHAR(256) NOT NULL,
|
||||
display_name VARCHAR(64),
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE roles (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
name VARCHAR(32) UNIQUE NOT NULL -- admin / app-owner / viewer
|
||||
);
|
||||
|
||||
CREATE TABLE user_app_roles ( -- 用户对某个应用的角色(应用级 RBAC)
|
||||
user_id BIGINT NOT NULL REFERENCES users(id),
|
||||
app_id BIGINT NOT NULL REFERENCES applications(id),
|
||||
role_id BIGINT NOT NULL REFERENCES roles(id),
|
||||
PRIMARY KEY (user_id, app_id)
|
||||
);
|
||||
|
||||
CREATE TABLE audit_logs (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
actor VARCHAR(64) NOT NULL,
|
||||
action VARCHAR(32) NOT NULL, -- create/update/delete/publish/rollback
|
||||
target_type VARCHAR(32) NOT NULL, -- app/namespace/env/config/release
|
||||
target_id BIGINT,
|
||||
detail JSONB,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
-- 灰度发布规则(可选,Phase 4 再做)
|
||||
CREATE TABLE gray_rules (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
namespace_id BIGINT NOT NULL REFERENCES namespaces(id),
|
||||
env_id BIGINT NOT NULL REFERENCES environments(id),
|
||||
rule_type VARCHAR(16) NOT NULL, -- ip / instance / percentage
|
||||
rule_value JSONB NOT NULL, -- {"ips": [...]} 或 {"percentage": 20}
|
||||
overrides JSONB NOT NULL, -- 灰度期间覆盖的 key -> value
|
||||
enabled BOOLEAN NOT NULL DEFAULT true,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
```
|
||||
|
||||
## 五、Protobuf / gRPC 接口定义
|
||||
|
||||
建议用 **grpc-gateway** 从同一份 proto 同时生成 gRPC(给 Go/Python SDK 用)和 REST(给 Web Console 用),避免维护两套接口。
|
||||
|
||||
```protobuf
|
||||
syntax = "proto3";
|
||||
package configcenter.v1;
|
||||
option go_package = "github.com/yourorg/configcenter/pkg/proto/v1;configcenterv1";
|
||||
|
||||
message ConfigItem {
|
||||
string key = 1;
|
||||
string value = 2;
|
||||
}
|
||||
|
||||
// ---- 读取 / 监听 ----
|
||||
message GetConfigRequest {
|
||||
string env = 1;
|
||||
string app = 2;
|
||||
string namespace = 3;
|
||||
}
|
||||
message GetConfigResponse {
|
||||
repeated ConfigItem items = 1;
|
||||
int64 revision = 2;
|
||||
int64 release_version = 3;
|
||||
}
|
||||
|
||||
message WatchConfigRequest {
|
||||
string env = 1;
|
||||
string app = 2;
|
||||
string namespace = 3;
|
||||
int64 start_revision = 4; // 断线重连时从该 revision 继续,避免错过中间事件
|
||||
}
|
||||
message ConfigEvent {
|
||||
enum EventType { FULL_SYNC = 0; UPDATED = 1; }
|
||||
EventType type = 1;
|
||||
repeated ConfigItem items = 2;
|
||||
int64 revision = 3;
|
||||
}
|
||||
|
||||
service ConfigService {
|
||||
rpc GetConfig(GetConfigRequest) returns (GetConfigResponse);
|
||||
rpc WatchConfig(WatchConfigRequest) returns (stream ConfigEvent);
|
||||
}
|
||||
|
||||
// ---- 发布 / 回滚(管理面,Web Console 也走这里)----
|
||||
message PublishRequest {
|
||||
string env = 1;
|
||||
string app = 2;
|
||||
string namespace = 3;
|
||||
string comment = 4;
|
||||
string operator = 5;
|
||||
}
|
||||
message PublishResponse {
|
||||
int64 release_version = 1;
|
||||
int64 etcd_revision = 2;
|
||||
}
|
||||
|
||||
message RollbackRequest {
|
||||
string env = 1;
|
||||
string app = 2;
|
||||
string namespace = 3;
|
||||
int64 target_version = 4;
|
||||
string operator = 5;
|
||||
}
|
||||
message RollbackResponse {
|
||||
int64 new_release_version = 1;
|
||||
}
|
||||
|
||||
service AdminService {
|
||||
rpc PublishConfig(PublishRequest) returns (PublishResponse);
|
||||
rpc RollbackConfig(RollbackRequest) returns (RollbackResponse);
|
||||
// 应用/命名空间/环境/配置项的 CRUD 接口类似,此处略
|
||||
}
|
||||
```
|
||||
|
||||
## 六、Go Config Server 实现要点
|
||||
|
||||
### 目录结构
|
||||
|
||||
```
|
||||
configcenter/
|
||||
├── cmd/server/main.go
|
||||
├── internal/
|
||||
│ ├── api/ # gRPC handler + grpc-gateway REST
|
||||
│ ├── service/ # 发布 / 回滚 / watch 扇出等业务逻辑
|
||||
│ ├── store/
|
||||
│ │ ├── postgres/ # repository(推荐 sqlc 生成,类型安全)
|
||||
│ │ └── etcdstore/ # etcd client 封装
|
||||
│ ├── outbox/ # outbox worker
|
||||
│ ├── cache/ # redis 封装(读缓存 + 内部事件)
|
||||
│ └── auth/ # JWT + RBAC 中间件
|
||||
├── pkg/
|
||||
│ ├── proto/v1/ # protoc 生成代码
|
||||
│ └── sdk/go/ # Go SDK,独立 module,方便业务方单独引用
|
||||
└── web/ # Web Console 前端
|
||||
```
|
||||
|
||||
### 发布流程
|
||||
|
||||
```go
|
||||
func (s *ConfigService) Publish(ctx context.Context, req *PublishRequest) (*PublishResponse, error) {
|
||||
return s.db.WithTx(ctx, func(tx *sql.Tx) (*PublishResponse, error) {
|
||||
items, err := s.repo.ListConfigItems(tx, req.NamespaceID, req.EnvID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
pending := filterPending(items) // pendingDelete || releasedValue==nil || value!=releasedValue
|
||||
if len(pending) == 0 {
|
||||
return nil, ErrNoPendingChanges
|
||||
}
|
||||
|
||||
version := s.repo.NextReleaseVersion(tx, req.NamespaceID, req.EnvID)
|
||||
snapshot := buildSnapshot(items) // 去掉 pendingDelete 的 key
|
||||
releaseID, err := s.repo.InsertRelease(tx, version, snapshot, req.Comment, req.Operator)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// 关键:写 outbox 而不是直接写 etcd,和 DB 事务在同一个提交单元里
|
||||
etcdKey := buildEtcdKey(req.Env, req.App, req.Namespace)
|
||||
if err := s.repo.InsertOutbox(tx, releaseID, etcdKey, snapshot); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
s.repo.MarkConfigItemsReleased(tx, pending)
|
||||
|
||||
return &PublishResponse{ReleaseVersion: version}, nil
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
### Outbox Worker(异步写 etcd,失败可重试)
|
||||
|
||||
```go
|
||||
func (w *OutboxWorker) Run(ctx context.Context) {
|
||||
ticker := time.NewTicker(500 * time.Millisecond)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-ticker.C:
|
||||
rows, _ := w.repo.FetchPendingOutbox(ctx, 50)
|
||||
for _, row := range rows {
|
||||
payload, _ := json.Marshal(row.Payload)
|
||||
resp, err := w.etcd.Put(ctx, row.EtcdKey, string(payload))
|
||||
if err != nil {
|
||||
w.repo.MarkOutboxFailed(ctx, row.ID) // retry_count++
|
||||
continue
|
||||
}
|
||||
w.repo.MarkOutboxDone(ctx, row.ID)
|
||||
w.repo.UpdateReleaseRevision(ctx, row.ReleaseID, resp.Header.Revision, "applied")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Watch 扇出(避免 N 个 Server 实例 = N 倍 etcd watch 连接)
|
||||
|
||||
```go
|
||||
type WatchHub struct {
|
||||
mu sync.RWMutex
|
||||
subs map[string]map[chan *pb.ConfigEvent]struct{}
|
||||
etcd *clientv3.Client
|
||||
}
|
||||
|
||||
func (h *WatchHub) Subscribe(key string) (<-chan *pb.ConfigEvent, func()) {
|
||||
ch := make(chan *pb.ConfigEvent, 8)
|
||||
h.mu.Lock()
|
||||
if h.subs[key] == nil {
|
||||
h.subs[key] = map[chan *pb.ConfigEvent]struct{}{}
|
||||
go h.watchKey(key) // 同一个 key 只建一条 etcd watch
|
||||
}
|
||||
h.subs[key][ch] = struct{}{}
|
||||
h.mu.Unlock()
|
||||
return ch, func() { h.unsubscribe(key, ch) }
|
||||
}
|
||||
|
||||
func (h *WatchHub) watchKey(key string) {
|
||||
wc := h.etcd.Watch(context.Background(), key)
|
||||
for resp := range wc {
|
||||
for _, ev := range resp.Events {
|
||||
evt := &pb.ConfigEvent{Type: pb.ConfigEvent_UPDATED, Items: decode(ev.Kv.Value), Revision: ev.Kv.ModRevision}
|
||||
h.broadcast(key, evt)
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
gRPC 层只需要订阅 Hub 并转发给客户端流:
|
||||
|
||||
```go
|
||||
func (s *ConfigService) WatchConfig(req *pb.WatchConfigRequest, stream pb.ConfigService_WatchConfigServer) error {
|
||||
key := buildEtcdKey(req.Env, req.App, req.Namespace)
|
||||
initial, rev := s.loadCurrent(key)
|
||||
if err := stream.Send(&pb.ConfigEvent{Type: pb.ConfigEvent_FULL_SYNC, Items: initial, Revision: rev}); err != nil {
|
||||
return err
|
||||
}
|
||||
ch, cancel := s.hub.Subscribe(key)
|
||||
defer cancel()
|
||||
for {
|
||||
select {
|
||||
case evt := <-ch:
|
||||
if err := stream.Send(evt); err != nil {
|
||||
return err
|
||||
}
|
||||
case <-stream.Context().Done():
|
||||
return nil
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 回滚
|
||||
|
||||
回滚不是"覆盖旧记录",而是**生成一条新的 release**(内容等于目标历史版本的 snapshot),走和发布完全一样的 outbox 流程——这样回滚本身也留痕,历史可追溯。
|
||||
|
||||
## 七、Go SDK
|
||||
|
||||
```go
|
||||
package configsdk
|
||||
|
||||
type Client struct {
|
||||
stub pb.ConfigServiceClient
|
||||
cache sync.Map // namespace -> map[string]string
|
||||
}
|
||||
|
||||
func New(addr string) (*Client, error) {
|
||||
conn, err := grpc.Dial(addr, grpc.WithTransportCredentials(insecure.NewCredentials()))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &Client{stub: pb.NewConfigServiceClient(conn)}, nil
|
||||
}
|
||||
|
||||
func (c *Client) GetString(namespace, key, defaultVal string) string {
|
||||
if m, ok := c.cache.Load(namespace); ok {
|
||||
if v, ok := m.(map[string]string)[key]; ok {
|
||||
return v
|
||||
}
|
||||
}
|
||||
return defaultVal
|
||||
}
|
||||
|
||||
// 后台常驻 goroutine,断线自动重连
|
||||
func (c *Client) WatchAndSync(ctx context.Context, env, app, namespace string) {
|
||||
for {
|
||||
stream, err := c.stub.WatchConfig(ctx, &pb.WatchConfigRequest{Env: env, App: app, Namespace: namespace})
|
||||
if err == nil {
|
||||
for {
|
||||
evt, err := stream.Recv()
|
||||
if err != nil {
|
||||
break
|
||||
}
|
||||
c.applyEvent(namespace, evt)
|
||||
}
|
||||
}
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-time.After(2 * time.Second): // 重连退避
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 八、Python SDK
|
||||
|
||||
不直接连 etcd(etcd 官方不维护 Python client),而是通过 gRPC 连 Go Config Server:
|
||||
|
||||
```python
|
||||
import grpc
|
||||
import threading
|
||||
import time
|
||||
from configcenter.v1 import config_pb2, config_pb2_grpc
|
||||
|
||||
class ConfigClient:
|
||||
def __init__(self, addr: str):
|
||||
self._channel = grpc.insecure_channel(addr)
|
||||
self._stub = config_pb2_grpc.ConfigServiceStub(self._channel)
|
||||
self._cache = {}
|
||||
self._lock = threading.Lock()
|
||||
|
||||
def get(self, namespace: str, key: str, default=None):
|
||||
with self._lock:
|
||||
return self._cache.get(namespace, {}).get(key, default)
|
||||
|
||||
def start_background_watch(self, env: str, app: str, namespace: str):
|
||||
t = threading.Thread(target=self._watch_loop, args=(env, app, namespace), daemon=True)
|
||||
t.start()
|
||||
|
||||
def _watch_loop(self, env, app, namespace):
|
||||
req = config_pb2.WatchConfigRequest(env=env, app=app, namespace=namespace)
|
||||
while True:
|
||||
try:
|
||||
for event in self._stub.WatchConfig(req):
|
||||
with self._lock:
|
||||
self._cache[namespace] = {i.key: i.value for i in event.items}
|
||||
except grpc.RpcError:
|
||||
time.sleep(2) # 断线重连
|
||||
```
|
||||
|
||||
## 九、Web Console 对接方式
|
||||
|
||||
之前给你做的前端 Demo(应用/命名空间/环境/配置项 CRUD + 发布 diff 预览 + 发布历史回滚)里的数据结构和这里的 PG 表结构是对齐的,接入真实后端时只需要:
|
||||
|
||||
1. 把内存里的 `useState` 数据源换成对 `grpc-gateway` 生成的 REST 接口的 `fetch` 调用;
|
||||
2. 发布按钮调用 `POST /v1/publish`,历史列表调用 `GET /v1/releases`,回滚调用 `POST /v1/rollback`;
|
||||
3. 因为发布是异步落 etcd(outbox),Web Console 发布后可以轮询 release 的 `status` 字段(pending → applied),或者用 Redis pub/sub + SSE 做实时状态推送。
|
||||
|
||||
## 十、部署与运维要点
|
||||
|
||||
- **etcd 独立部署**:3 或 5 节点集群,不与业务系统共享,专门服务配置分发;
|
||||
- **控制 etcd 数据量**:只放"当前生效配置",历史/审计一律留在 PG,etcd 库大小尽量控制在几个 GB 以内;
|
||||
- **定期 compact + defrag**:etcd 的 MVCC 历史 revision 会持续占用磁盘,需要定期 `etcdctl compact` 和 `defrag`;
|
||||
- **监控**:etcd 自带 `/metrics`(Prometheus),重点关注 db 大小、leader 变更次数、watch 连接数;Config Server 侧关注发布成功率、outbox 积压量、gRPC 延迟;
|
||||
- **无状态水平扩展**:Config Server 本身无状态,可以随意加实例,配合前面的 WatchHub 设计,不会导致 etcd 连接数暴涨;
|
||||
- **灾备**:etcd 定期 `snapshot save`;PostgreSQL 走常规主从 + 定期备份;
|
||||
- **安全**:etcd 只对 Config Server 开放访问(不直接暴露给业务服务),Config Server 与 etcd 之间开 TLS。
|
||||
|
||||
## 十一、分阶段落地路线图
|
||||
|
||||
| 阶段 | 内容 | 周期建议 |
|
||||
|---|---|---|
|
||||
| Phase 1 | PG 建表 + CRUD REST API + Web Console 接入真实接口(发布先只落 PG,不接 etcd) | 2–3 周 |
|
||||
| Phase 2 | 接入 etcd:Outbox Worker、GetConfig 接口、Go/Python SDK(先只支持 Get,不支持 Watch) | 2 周 |
|
||||
| Phase 3 | WatchConfig 全链路:WatchHub 扇出、SDK 长连接自动重连、**SDK 本地文件缓存兜底**(Server/etcd 故障时业务进程仍能用最后一次缓存启动,这是必须做的,Apollo/Nacos 都有) | 2 周 |
|
||||
| Phase 4 | RBAC(应用级角色)、审计日志、灰度发布规则 | 2–3 周 |
|
||||
| Phase 5 | 高可用加固:etcd 多机房、监控告警、压测 | 持续 |
|
||||
|
||||
Phase 1 完全可以复用你现在这份前端 Demo 直接改造,不用重写界面。
|
||||
218
deploy/kubernetes/configcenter.yaml
Normal file
218
deploy/kubernetes/configcenter.yaml
Normal file
@@ -0,0 +1,218 @@
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: configcenter
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: configcenter
|
||||
namespace: configcenter
|
||||
data:
|
||||
HTTP_ADDR: ":8080"
|
||||
ETCD_ENDPOINTS: "http://etcd:2379"
|
||||
OUTBOX_INTERVAL: "500ms"
|
||||
OUTBOX_BATCH_SIZE: "50"
|
||||
OUTBOX_MAX_RETRY: "12"
|
||||
AUTH_ENABLED: "true"
|
||||
JWT_TTL: "8h"
|
||||
CORS_ALLOWED_ORIGINS: "https://config.example.com"
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: StatefulSet
|
||||
metadata:
|
||||
name: etcd
|
||||
namespace: configcenter
|
||||
spec:
|
||||
serviceName: etcd-peer
|
||||
replicas: 3
|
||||
selector:
|
||||
matchLabels:
|
||||
app: etcd
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: etcd
|
||||
annotations:
|
||||
prometheus.io/scrape: "true"
|
||||
prometheus.io/port: "2379"
|
||||
prometheus.io/path: /metrics
|
||||
spec:
|
||||
terminationGracePeriodSeconds: 30
|
||||
topologySpreadConstraints:
|
||||
- maxSkew: 1
|
||||
topologyKey: topology.kubernetes.io/zone
|
||||
whenUnsatisfiable: DoNotSchedule
|
||||
labelSelector:
|
||||
matchLabels: { app: etcd }
|
||||
affinity:
|
||||
podAntiAffinity:
|
||||
requiredDuringSchedulingIgnoredDuringExecution:
|
||||
- labelSelector:
|
||||
matchLabels:
|
||||
app: etcd
|
||||
topologyKey: kubernetes.io/hostname
|
||||
containers:
|
||||
- name: etcd
|
||||
image: quay.io/coreos/etcd:v3.5.17
|
||||
command: ["/bin/sh", "-ec"]
|
||||
args:
|
||||
- >-
|
||||
exec etcd
|
||||
--name="${HOSTNAME}"
|
||||
--data-dir=/var/lib/etcd
|
||||
--listen-client-urls=http://0.0.0.0:2379
|
||||
--advertise-client-urls="http://${HOSTNAME}.etcd-peer.configcenter.svc.cluster.local:2379"
|
||||
--listen-peer-urls=http://0.0.0.0:2380
|
||||
--initial-advertise-peer-urls="http://${HOSTNAME}.etcd-peer.configcenter.svc.cluster.local:2380"
|
||||
--initial-cluster="etcd-0=http://etcd-0.etcd-peer.configcenter.svc.cluster.local:2380,etcd-1=http://etcd-1.etcd-peer.configcenter.svc.cluster.local:2380,etcd-2=http://etcd-2.etcd-peer.configcenter.svc.cluster.local:2380"
|
||||
--initial-cluster-state=new
|
||||
--initial-cluster-token=configcenter-etcd
|
||||
--auto-compaction-retention=1
|
||||
--auto-compaction-mode=periodic
|
||||
ports:
|
||||
- { name: client, containerPort: 2379 }
|
||||
- { name: peer, containerPort: 2380 }
|
||||
readinessProbe:
|
||||
exec:
|
||||
command: ["etcdctl", "endpoint", "health"]
|
||||
initialDelaySeconds: 10
|
||||
periodSeconds: 5
|
||||
resources:
|
||||
requests: { cpu: "250m", memory: "512Mi" }
|
||||
limits: { cpu: "2", memory: "2Gi" }
|
||||
volumeMounts:
|
||||
- { name: data, mountPath: /var/lib/etcd }
|
||||
volumeClaimTemplates:
|
||||
- metadata:
|
||||
name: data
|
||||
spec:
|
||||
accessModes: ["ReadWriteOnce"]
|
||||
resources:
|
||||
requests:
|
||||
storage: 20Gi
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: etcd-peer
|
||||
namespace: configcenter
|
||||
spec:
|
||||
clusterIP: None
|
||||
publishNotReadyAddresses: true
|
||||
selector: { app: etcd }
|
||||
ports:
|
||||
- { name: peer, port: 2380, targetPort: peer }
|
||||
- { name: client, port: 2379, targetPort: client }
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: etcd
|
||||
namespace: configcenter
|
||||
spec:
|
||||
selector: { app: etcd }
|
||||
ports:
|
||||
- { name: client, port: 2379, targetPort: client }
|
||||
---
|
||||
apiVersion: policy/v1
|
||||
kind: PodDisruptionBudget
|
||||
metadata:
|
||||
name: etcd
|
||||
namespace: configcenter
|
||||
spec:
|
||||
maxUnavailable: 1
|
||||
selector:
|
||||
matchLabels: { app: etcd }
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: configcenter
|
||||
namespace: configcenter
|
||||
spec:
|
||||
replicas: 3
|
||||
selector:
|
||||
matchLabels: { app: configcenter }
|
||||
template:
|
||||
metadata:
|
||||
labels: { app: configcenter }
|
||||
annotations:
|
||||
prometheus.io/scrape: "true"
|
||||
prometheus.io/port: "8080"
|
||||
prometheus.io/path: /metrics
|
||||
spec:
|
||||
topologySpreadConstraints:
|
||||
- maxSkew: 1
|
||||
topologyKey: topology.kubernetes.io/zone
|
||||
whenUnsatisfiable: ScheduleAnyway
|
||||
labelSelector:
|
||||
matchLabels: { app: configcenter }
|
||||
affinity:
|
||||
podAntiAffinity:
|
||||
preferredDuringSchedulingIgnoredDuringExecution:
|
||||
- weight: 100
|
||||
podAffinityTerm:
|
||||
labelSelector:
|
||||
matchLabels: { app: configcenter }
|
||||
topologyKey: kubernetes.io/hostname
|
||||
containers:
|
||||
- name: server
|
||||
image: ghcr.io/yourorg/configcenter:latest
|
||||
envFrom:
|
||||
- configMapRef: { name: configcenter }
|
||||
- secretRef: { name: configcenter-secrets }
|
||||
ports:
|
||||
- { name: http, containerPort: 8080 }
|
||||
readinessProbe:
|
||||
httpGet: { path: /health/ready, port: http }
|
||||
periodSeconds: 5
|
||||
livenessProbe:
|
||||
httpGet: { path: /health/live, port: http }
|
||||
periodSeconds: 10
|
||||
resources:
|
||||
requests: { cpu: "200m", memory: "256Mi" }
|
||||
limits: { cpu: "1", memory: "1Gi" }
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: configcenter
|
||||
namespace: configcenter
|
||||
spec:
|
||||
selector: { app: configcenter }
|
||||
ports:
|
||||
- { name: http, port: 8080, targetPort: http }
|
||||
---
|
||||
apiVersion: policy/v1
|
||||
kind: PodDisruptionBudget
|
||||
metadata:
|
||||
name: configcenter
|
||||
namespace: configcenter
|
||||
spec:
|
||||
minAvailable: 2
|
||||
selector:
|
||||
matchLabels: { app: configcenter }
|
||||
---
|
||||
apiVersion: autoscaling/v2
|
||||
kind: HorizontalPodAutoscaler
|
||||
metadata:
|
||||
name: configcenter
|
||||
namespace: configcenter
|
||||
spec:
|
||||
scaleTargetRef:
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
name: configcenter
|
||||
minReplicas: 3
|
||||
maxReplicas: 10
|
||||
behavior:
|
||||
scaleDown:
|
||||
stabilizationWindowSeconds: 300
|
||||
metrics:
|
||||
- type: Resource
|
||||
resource:
|
||||
name: cpu
|
||||
target:
|
||||
type: Utilization
|
||||
averageUtilization: 65
|
||||
54
deploy/monitoring/alerts.yml
Normal file
54
deploy/monitoring/alerts.yml
Normal file
@@ -0,0 +1,54 @@
|
||||
groups:
|
||||
- name: configcenter
|
||||
rules:
|
||||
- alert: ConfigCenterDown
|
||||
expr: up{job="configcenter"} == 0
|
||||
for: 2m
|
||||
labels:
|
||||
severity: critical
|
||||
annotations:
|
||||
summary: Config Center is unavailable
|
||||
|
||||
- alert: ConfigCenterOutboxBacklog
|
||||
expr: configcenter_outbox_pending > 100
|
||||
for: 5m
|
||||
labels:
|
||||
severity: warning
|
||||
annotations:
|
||||
summary: Config Center outbox backlog is growing
|
||||
|
||||
- alert: ConfigCenterOutboxFailed
|
||||
expr: configcenter_outbox_failed > 0
|
||||
for: 1m
|
||||
labels:
|
||||
severity: critical
|
||||
annotations:
|
||||
summary: Config Center has terminally failed releases
|
||||
|
||||
- alert: ConfigCenterHighHTTPErrorRate
|
||||
expr: |
|
||||
sum(rate(configcenter_http_requests_total{status=~"5.."}[5m]))
|
||||
/
|
||||
clamp_min(sum(rate(configcenter_http_requests_total[5m])), 0.001)
|
||||
> 0.01
|
||||
for: 5m
|
||||
labels:
|
||||
severity: warning
|
||||
annotations:
|
||||
summary: Config Center 5xx rate exceeds 1%
|
||||
|
||||
- alert: EtcdDown
|
||||
expr: up{job="etcd"} == 0
|
||||
for: 2m
|
||||
labels:
|
||||
severity: critical
|
||||
annotations:
|
||||
summary: Config Center etcd is unavailable
|
||||
|
||||
- alert: EtcdNoLeader
|
||||
expr: etcd_server_has_leader{job="etcd"} == 0
|
||||
for: 1m
|
||||
labels:
|
||||
severity: critical
|
||||
annotations:
|
||||
summary: Config Center etcd member has no leader
|
||||
17
deploy/monitoring/prometheus.yml
Normal file
17
deploy/monitoring/prometheus.yml
Normal file
@@ -0,0 +1,17 @@
|
||||
global:
|
||||
scrape_interval: 15s
|
||||
evaluation_interval: 15s
|
||||
|
||||
rule_files:
|
||||
- /etc/prometheus/alerts.yml
|
||||
|
||||
scrape_configs:
|
||||
- job_name: configcenter
|
||||
metrics_path: /metrics
|
||||
static_configs:
|
||||
- targets: ["server:8080"]
|
||||
|
||||
- job_name: etcd
|
||||
metrics_path: /metrics
|
||||
static_configs:
|
||||
- targets: ["etcd:2379"]
|
||||
32
go.mod
Normal file
32
go.mod
Normal file
@@ -0,0 +1,32 @@
|
||||
module github.com/longpeng/configcenter
|
||||
|
||||
go 1.24.0
|
||||
|
||||
require (
|
||||
github.com/jackc/pgx/v5 v5.7.2
|
||||
go.etcd.io/etcd/client/v3 v3.6.7
|
||||
)
|
||||
|
||||
require (
|
||||
github.com/coreos/go-semver v0.3.1 // indirect
|
||||
github.com/coreos/go-systemd/v22 v22.5.0 // indirect
|
||||
github.com/gogo/protobuf v1.3.2 // indirect
|
||||
github.com/golang/protobuf v1.5.4 // indirect
|
||||
github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.3 // indirect
|
||||
github.com/jackc/pgpassfile v1.0.0 // indirect
|
||||
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
|
||||
github.com/jackc/puddle/v2 v2.2.2 // indirect
|
||||
go.etcd.io/etcd/api/v3 v3.6.7 // indirect
|
||||
go.etcd.io/etcd/client/pkg/v3 v3.6.7 // indirect
|
||||
go.uber.org/multierr v1.11.0 // indirect
|
||||
go.uber.org/zap v1.27.0 // indirect
|
||||
golang.org/x/crypto v0.42.0 // indirect
|
||||
golang.org/x/net v0.45.0 // indirect
|
||||
golang.org/x/sync v0.17.0 // indirect
|
||||
golang.org/x/sys v0.36.0 // indirect
|
||||
golang.org/x/text v0.29.0 // indirect
|
||||
google.golang.org/genproto/googleapis/api v0.0.0-20250303144028-a0af3efb3deb // indirect
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20250303144028-a0af3efb3deb // indirect
|
||||
google.golang.org/grpc v1.71.1 // indirect
|
||||
google.golang.org/protobuf v1.36.5 // indirect
|
||||
)
|
||||
112
go.sum
Normal file
112
go.sum
Normal file
@@ -0,0 +1,112 @@
|
||||
github.com/coreos/go-semver v0.3.1 h1:yi21YpKnrx1gt5R+la8n5WgS0kCrsPp33dmEyHReZr4=
|
||||
github.com/coreos/go-semver v0.3.1/go.mod h1:irMmmIw/7yzSRPWryHsK7EYSg09caPQL03VsM8rvUec=
|
||||
github.com/coreos/go-systemd/v22 v22.5.0 h1:RrqgGjYQKalulkV8NGVIfkXQf6YYmOyiJKk8iXXhfZs=
|
||||
github.com/coreos/go-systemd/v22 v22.5.0/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=
|
||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY=
|
||||
github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
|
||||
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
|
||||
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
|
||||
github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=
|
||||
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
|
||||
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
|
||||
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
|
||||
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
|
||||
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
|
||||
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
|
||||
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
|
||||
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
||||
github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.3 h1:5ZPtiqj0JL5oKWmcsq4VMaAW5ukBEgSGXEN89zeH1Jo=
|
||||
github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.3/go.mod h1:ndYquD05frm2vACXE1nsccT4oJzjhw2arTS2cpUD1PI=
|
||||
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
|
||||
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
|
||||
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7UlwOQYpKFhBabPMi4aNAfoODPEFNiAnClxo=
|
||||
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
|
||||
github.com/jackc/pgx/v5 v5.7.2 h1:mLoDLV6sonKlvjIEsV56SkWNCnuNv531l94GaIzO+XI=
|
||||
github.com/jackc/pgx/v5 v5.7.2/go.mod h1:ncY89UGWxg82EykZUwSpUKEfccBGGYq1xjrOpsbsfGQ=
|
||||
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
|
||||
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
|
||||
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
|
||||
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
|
||||
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
|
||||
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
|
||||
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
|
||||
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
|
||||
go.etcd.io/etcd/api/v3 v3.6.7 h1:7BNJ2gQmc3DNM+9cRkv7KkGQDayElg8x3X+tFDYS+E0=
|
||||
go.etcd.io/etcd/api/v3 v3.6.7/go.mod h1:xJ81TLj9hxrYYEDmXTeKURMeY3qEDN24hqe+q7KhbnI=
|
||||
go.etcd.io/etcd/client/pkg/v3 v3.6.7 h1:vvzgyozz46q+TyeGBuFzVuI53/yd133CHceNb/AhBVs=
|
||||
go.etcd.io/etcd/client/pkg/v3 v3.6.7/go.mod h1:2IVulJ3FZ/czIGl9T4lMF1uxzrhRahLqe+hSgy+Kh7Q=
|
||||
go.etcd.io/etcd/client/v3 v3.6.7 h1:9WqA5RpIBtdMxAy1ukXLAdtg2pAxNqW5NUoO2wQrE6U=
|
||||
go.etcd.io/etcd/client/v3 v3.6.7/go.mod h1:2XfROY56AXnUqGsvl+6k29wrwsSbEh1lAouQB1vHpeE=
|
||||
go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA=
|
||||
go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A=
|
||||
go.opentelemetry.io/otel v1.34.0 h1:zRLXxLCgL1WyKsPVrgbSdMN4c0FMkDAskSTQP+0hdUY=
|
||||
go.opentelemetry.io/otel v1.34.0/go.mod h1:OWFPOQ+h4G8xpyjgqo4SxJYdDQ/qmRH+wivy7zzx9oI=
|
||||
go.opentelemetry.io/otel/metric v1.34.0 h1:+eTR3U0MyfWjRDhmFMxe2SsW64QrZ84AOhvqS7Y+PoQ=
|
||||
go.opentelemetry.io/otel/metric v1.34.0/go.mod h1:CEDrp0fy2D0MvkXE+dPV7cMi8tWZwX3dmaIhwPOaqHE=
|
||||
go.opentelemetry.io/otel/sdk v1.34.0 h1:95zS4k/2GOy069d321O8jWgYsW3MzVV+KuSPKp7Wr1A=
|
||||
go.opentelemetry.io/otel/sdk v1.34.0/go.mod h1:0e/pNiaMAqaykJGKbi+tSjWfNNHMTxoC9qANsCzbyxU=
|
||||
go.opentelemetry.io/otel/sdk/metric v1.34.0 h1:5CeK9ujjbFVL5c1PhLuStg1wxA7vQv7ce1EK0Gyvahk=
|
||||
go.opentelemetry.io/otel/sdk/metric v1.34.0/go.mod h1:jQ/r8Ze28zRKoNRdkjCZxfs6YvBTG1+YIqyFVFYec5w=
|
||||
go.opentelemetry.io/otel/trace v1.34.0 h1:+ouXS2V8Rd4hp4580a8q23bg0azF2nI8cqLYnC8mh/k=
|
||||
go.opentelemetry.io/otel/trace v1.34.0/go.mod h1:Svm7lSjQD7kG7KJ/MUHPVXSDGz2OX4h0M2jHBhmSfRE=
|
||||
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
|
||||
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
|
||||
go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
|
||||
go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
|
||||
go.uber.org/zap v1.27.0 h1:aJMhYGrd5QSmlpLMr2MftRKl7t8J8PTZPA732ud/XR8=
|
||||
go.uber.org/zap v1.27.0/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=
|
||||
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
|
||||
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
|
||||
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
|
||||
golang.org/x/crypto v0.42.0 h1:chiH31gIWm57EkTXpwnqf8qeuMUi0yekh6mT2AvFlqI=
|
||||
golang.org/x/crypto v0.42.0/go.mod h1:4+rDnOTJhQCx2q7/j6rAN5XDw8kPjeaXEUR2eL94ix8=
|
||||
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
|
||||
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
|
||||
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
|
||||
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
||||
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
||||
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
|
||||
golang.org/x/net v0.45.0 h1:RLBg5JKixCy82FtLJpeNlVM0nrSqpCRYzVU1n8kj0tM=
|
||||
golang.org/x/net v0.45.0/go.mod h1:ECOoLqd5U3Lhyeyo/QDCEVQ4sNgYsqvCZ722XogGieY=
|
||||
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sync v0.17.0 h1:l60nONMj9l5drqw6jlhIELNv9I0A4OFgRsG9k2oT9Ug=
|
||||
golang.org/x/sync v0.17.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
|
||||
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
|
||||
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.36.0 h1:KVRy2GtZBrk1cBYA7MKu5bEZFxQk4NIDV6RLVcC8o0k=
|
||||
golang.org/x/sys v0.36.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
|
||||
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
|
||||
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
|
||||
golang.org/x/text v0.29.0 h1:1neNs90w9YzJ9BocxfsQNHKuAT4pkghyXc4nhZ6sJvk=
|
||||
golang.org/x/text v0.29.0/go.mod h1:7MhJOA9CD2qZyOKYazxdYMF85OwPdEr9jTtBpO7ydH4=
|
||||
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
|
||||
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
|
||||
golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
|
||||
golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
|
||||
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||
google.golang.org/genproto/googleapis/api v0.0.0-20250303144028-a0af3efb3deb h1:p31xT4yrYrSM/G4Sn2+TNUkVhFCbG9y8itM2S6Th950=
|
||||
google.golang.org/genproto/googleapis/api v0.0.0-20250303144028-a0af3efb3deb/go.mod h1:jbe3Bkdp+Dh2IrslsFCklNhweNTBgSYanP1UXhJDhKg=
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20250303144028-a0af3efb3deb h1:TLPQVbx1GJ8VKZxz52VAxl1EBgKXXbTiU9Fc5fZeLn4=
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20250303144028-a0af3efb3deb/go.mod h1:LuRYeWDFV6WOn90g357N17oMCaxpgCnbi/44qJvDn2I=
|
||||
google.golang.org/grpc v1.71.1 h1:ffsFWr7ygTUscGPI0KKK6TLrGz0476KUvvsbqWK0rPI=
|
||||
google.golang.org/grpc v1.71.1/go.mod h1:H0GRtasmQOh9LkFoCPDu3ZrwUtD1YGE+b2vYBYd/8Ec=
|
||||
google.golang.org/protobuf v1.36.5 h1:tPhr+woSbjfYvY6/GPufUoYizxw1cF/yFoxJ2fmpwlM=
|
||||
google.golang.org/protobuf v1.36.5/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
1023
internal/api/httpapi/server.go
Normal file
1023
internal/api/httpapi/server.go
Normal file
File diff suppressed because it is too large
Load Diff
232
internal/api/httpapi/server_test.go
Normal file
232
internal/api/httpapi/server_test.go
Normal file
@@ -0,0 +1,232 @@
|
||||
package httpapi_test
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/longpeng/configcenter/internal/api/httpapi"
|
||||
"github.com/longpeng/configcenter/internal/auth"
|
||||
"github.com/longpeng/configcenter/internal/domain"
|
||||
"github.com/longpeng/configcenter/internal/metrics"
|
||||
"github.com/longpeng/configcenter/internal/outbox"
|
||||
memoryruntime "github.com/longpeng/configcenter/internal/runtime/memory"
|
||||
memory_store "github.com/longpeng/configcenter/internal/store/memory"
|
||||
"github.com/longpeng/configcenter/internal/watch"
|
||||
)
|
||||
|
||||
func TestCRUDPublishAndRuntimeRead(t *testing.T) {
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
repository := memory_store.New(false)
|
||||
runtimeStore := memoryruntime.New()
|
||||
defer runtimeStore.Close() //nolint:errcheck
|
||||
logger := slog.New(slog.NewTextHandler(io.Discard, nil))
|
||||
hub := watch.New(ctx, runtimeStore)
|
||||
authorizer, err := auth.New(repository, false, "", 8*time.Hour)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
collector := metrics.New()
|
||||
worker := outbox.New(repository, runtimeStore, time.Millisecond, 10, 3, collector, logger)
|
||||
go worker.Run(ctx)
|
||||
handler := httpapi.New(repository, runtimeStore, hub, authorizer, collector, nil, logger).Handler()
|
||||
|
||||
var app domain.Application
|
||||
post(t, handler, "/v1/applications", map[string]any{"code": "orders", "name": "Orders"}, http.StatusCreated, &app)
|
||||
var env domain.Environment
|
||||
post(t, handler, "/v1/environments", map[string]any{"code": "PROD", "name": "Production", "order": 1}, http.StatusCreated, &env)
|
||||
var namespace domain.Namespace
|
||||
post(t, handler, "/v1/namespaces", map[string]any{"appId": app.ID, "name": "application", "type": "properties"}, http.StatusCreated, &namespace)
|
||||
var config domain.ConfigItem
|
||||
post(t, handler, "/v1/config-items", map[string]any{
|
||||
"appId": app.ID, "nsId": namespace.ID, "envId": env.ID, "key": "server.port", "value": "8080",
|
||||
}, http.StatusCreated, &config)
|
||||
|
||||
var release domain.Release
|
||||
post(t, handler, "/v1/publish", map[string]any{
|
||||
"appId": app.ID, "nsId": namespace.ID, "envId": env.ID, "comment": "first release",
|
||||
}, http.StatusAccepted, &release)
|
||||
if release.Status != "pending" || release.Version != 1 {
|
||||
t.Fatalf("unexpected accepted release: %#v", release)
|
||||
}
|
||||
|
||||
deadline := time.Now().Add(time.Second)
|
||||
for time.Now().Before(deadline) {
|
||||
get(t, handler, fmt.Sprintf("/v1/releases/%d", release.ID), http.StatusOK, &release)
|
||||
if release.Status == "applied" {
|
||||
break
|
||||
}
|
||||
time.Sleep(5 * time.Millisecond)
|
||||
}
|
||||
if release.Status != "applied" || release.EtcdRevision == nil {
|
||||
t.Fatalf("outbox was not applied: %#v", release)
|
||||
}
|
||||
|
||||
var current domain.RuntimeConfig
|
||||
get(t, handler, "/v1/config?env=PROD&app=orders&namespace=application", http.StatusOK, ¤t)
|
||||
if current.Items["server.port"] != "8080" || current.ReleaseVersion != 1 {
|
||||
t.Fatalf("unexpected runtime config: %#v", current)
|
||||
}
|
||||
|
||||
var grayRule domain.GrayRule
|
||||
post(t, handler, "/v1/gray-rules", map[string]any{
|
||||
"appId": app.ID, "nsId": namespace.ID, "envId": env.ID,
|
||||
"ruleType": "instance", "ruleValue": map[string]any{"instances": []string{"orders-7"}},
|
||||
"overrides": map[string]string{"server.port": "9080", "feature.gray": "on"}, "enabled": true, "priority": 10,
|
||||
}, http.StatusCreated, &grayRule)
|
||||
get(t, handler, "/v1/config?env=PROD&app=orders&namespace=application&instance=orders-7", http.StatusOK, ¤t)
|
||||
if current.Items["server.port"] != "9080" || current.Items["feature.gray"] != "on" || len(current.GrayRuleIDs) != 1 || current.GrayRuleIDs[0] != grayRule.ID {
|
||||
t.Fatalf("gray overrides were not applied: %#v", current)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAuthenticationRBACAndMetrics(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
repository := memory_store.New(false)
|
||||
runtimeStore := memoryruntime.New()
|
||||
defer runtimeStore.Close() //nolint:errcheck
|
||||
logger := slog.New(slog.NewTextHandler(io.Discard, nil))
|
||||
authorizer, err := auth.New(repository, true, "0123456789abcdef0123456789abcdef", 90*time.Minute)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := authorizer.Bootstrap(ctx, "root", "a-strong-password", "Root Admin"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
collector := metrics.New()
|
||||
handler := httpapi.New(repository, runtimeStore, watch.New(ctx, runtimeStore), authorizer, collector, nil, logger).Handler()
|
||||
|
||||
request := httptest.NewRequest(http.MethodGet, "/v1/applications", nil)
|
||||
doStatus(t, handler, request, http.StatusUnauthorized)
|
||||
|
||||
var adminLogin struct {
|
||||
Token string `json:"token"`
|
||||
ExpiresIn int64 `json:"expiresIn"`
|
||||
}
|
||||
requestJSON(t, handler, http.MethodPost, "/v1/auth/login", "", map[string]any{"username": "root", "password": "a-strong-password"}, http.StatusOK, &adminLogin)
|
||||
if adminLogin.Token == "" || adminLogin.ExpiresIn != 5400 {
|
||||
t.Fatalf("unexpected login response: %#v", adminLogin)
|
||||
}
|
||||
|
||||
var application domain.Application
|
||||
requestJSON(t, handler, http.MethodPost, "/v1/applications", adminLogin.Token, map[string]any{"code": "secure-app", "name": "Secure App"}, http.StatusCreated, &application)
|
||||
var viewer domain.User
|
||||
requestJSON(t, handler, http.MethodPost, "/v1/users", adminLogin.Token, map[string]any{
|
||||
"username": "reader", "displayName": "Read Only", "password": "reader-password-123",
|
||||
}, http.StatusCreated, &viewer)
|
||||
requestJSON(t, handler, http.MethodPut, fmt.Sprintf("/v1/users/%d/roles/%d", viewer.ID, application.ID), adminLogin.Token, map[string]any{"role": auth.RoleViewer}, http.StatusOK, &domain.UserAppRole{})
|
||||
var audits []domain.AuditLog
|
||||
requestJSON(t, handler, http.MethodGet, "/v1/audit-logs?limit=20", adminLogin.Token, nil, http.StatusOK, &audits)
|
||||
foundActor := false
|
||||
for _, audit := range audits {
|
||||
if audit.Actor == "root" && audit.TargetType == "app" && audit.Action == "create" {
|
||||
foundActor = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !foundActor {
|
||||
t.Fatalf("JWT actor was not propagated to audit log: %#v", audits)
|
||||
}
|
||||
|
||||
var viewerLogin struct {
|
||||
Token string `json:"token"`
|
||||
}
|
||||
requestJSON(t, handler, http.MethodPost, "/v1/auth/login", "", map[string]any{"username": "reader", "password": "reader-password-123"}, http.StatusOK, &viewerLogin)
|
||||
var applications []domain.Application
|
||||
requestJSON(t, handler, http.MethodGet, "/v1/applications", viewerLogin.Token, nil, http.StatusOK, &applications)
|
||||
if len(applications) != 1 || applications[0].ID != application.ID {
|
||||
t.Fatalf("viewer application filtering failed: %#v", applications)
|
||||
}
|
||||
requestJSON(t, handler, http.MethodPost, "/v1/namespaces", viewerLogin.Token, map[string]any{"appId": application.ID, "name": "application", "type": "properties"}, http.StatusForbidden, nil)
|
||||
|
||||
doStatus(t, handler, httptest.NewRequest(http.MethodGet, "/health/live", nil), http.StatusOK)
|
||||
recorder := httptest.NewRecorder()
|
||||
handler.ServeHTTP(recorder, httptest.NewRequest(http.MethodGet, "/metrics", nil))
|
||||
if recorder.Code != http.StatusOK || !strings.Contains(recorder.Body.String(), `configcenter_http_requests_total{method="GET",route="GET /health/live",status="200"}`) || !strings.Contains(recorder.Body.String(), "configcenter_outbox_pending 0") {
|
||||
t.Fatalf("unexpected metrics response: status=%d body=%s", recorder.Code, recorder.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
func post(t *testing.T, handler http.Handler, path string, input any, wantStatus int, output any) {
|
||||
t.Helper()
|
||||
payload, err := json.Marshal(input)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
request := httptest.NewRequest(http.MethodPost, path, bytes.NewReader(payload))
|
||||
request.Header.Set("Content-Type", "application/json")
|
||||
request.Header.Set("X-User", "tester")
|
||||
do(t, handler, request, wantStatus, output)
|
||||
}
|
||||
|
||||
func get(t *testing.T, handler http.Handler, path string, wantStatus int, output any) {
|
||||
t.Helper()
|
||||
request := httptest.NewRequest(http.MethodGet, path, nil)
|
||||
do(t, handler, request, wantStatus, output)
|
||||
}
|
||||
|
||||
func do(t *testing.T, handler http.Handler, request *http.Request, wantStatus int, output any) {
|
||||
t.Helper()
|
||||
recorder := httptest.NewRecorder()
|
||||
handler.ServeHTTP(recorder, request)
|
||||
response := recorder.Result()
|
||||
defer response.Body.Close()
|
||||
body, err := io.ReadAll(response.Body)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if response.StatusCode != wantStatus {
|
||||
t.Fatalf("%s %s: got status %d, want %d; body=%s", request.Method, request.URL, response.StatusCode, wantStatus, body)
|
||||
}
|
||||
var envelope struct {
|
||||
Data json.RawMessage `json:"data"`
|
||||
}
|
||||
if err := json.Unmarshal(body, &envelope); err != nil {
|
||||
t.Fatalf("decode response envelope: %v; body=%s", err, body)
|
||||
}
|
||||
if err := json.Unmarshal(envelope.Data, output); err != nil {
|
||||
t.Fatalf("decode response data: %v; body=%s", err, body)
|
||||
}
|
||||
}
|
||||
|
||||
func requestJSON(t *testing.T, handler http.Handler, method, path, token string, input any, wantStatus int, output any) {
|
||||
t.Helper()
|
||||
var body io.Reader
|
||||
if input != nil {
|
||||
payload, err := json.Marshal(input)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
body = bytes.NewReader(payload)
|
||||
}
|
||||
request := httptest.NewRequest(method, path, body)
|
||||
if input != nil {
|
||||
request.Header.Set("Content-Type", "application/json")
|
||||
}
|
||||
if token != "" {
|
||||
request.Header.Set("Authorization", "Bearer "+token)
|
||||
}
|
||||
if output == nil {
|
||||
doStatus(t, handler, request, wantStatus)
|
||||
return
|
||||
}
|
||||
do(t, handler, request, wantStatus, output)
|
||||
}
|
||||
|
||||
func doStatus(t *testing.T, handler http.Handler, request *http.Request, wantStatus int) {
|
||||
t.Helper()
|
||||
recorder := httptest.NewRecorder()
|
||||
handler.ServeHTTP(recorder, request)
|
||||
if recorder.Code != wantStatus {
|
||||
t.Fatalf("%s %s: got status %d, want %d; body=%s", request.Method, request.URL, recorder.Code, wantStatus, recorder.Body.String())
|
||||
}
|
||||
}
|
||||
224
internal/auth/auth.go
Normal file
224
internal/auth/auth.go
Normal file
@@ -0,0 +1,224 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/hmac"
|
||||
"crypto/sha256"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"golang.org/x/crypto/bcrypt"
|
||||
|
||||
"github.com/longpeng/configcenter/internal/domain"
|
||||
"github.com/longpeng/configcenter/internal/store"
|
||||
)
|
||||
|
||||
var (
|
||||
ErrUnauthorized = errors.New("authentication required")
|
||||
ErrForbidden = errors.New("permission denied")
|
||||
)
|
||||
|
||||
const (
|
||||
RoleViewer = "viewer"
|
||||
RoleAppOwner = "app-owner"
|
||||
RoleAdmin = "admin"
|
||||
)
|
||||
|
||||
type Manager struct {
|
||||
enabled bool
|
||||
secret []byte
|
||||
ttl time.Duration
|
||||
store store.Store
|
||||
now func() time.Time
|
||||
}
|
||||
|
||||
type tokenClaims struct {
|
||||
Issuer string `json:"iss"`
|
||||
Subject string `json:"sub"`
|
||||
UserID int64 `json:"uid"`
|
||||
DisplayName string `json:"name"`
|
||||
Admin bool `json:"admin"`
|
||||
IssuedAt int64 `json:"iat"`
|
||||
ExpiresAt int64 `json:"exp"`
|
||||
}
|
||||
|
||||
type principalKey struct{}
|
||||
|
||||
func New(repository store.Store, enabled bool, secret string, ttl time.Duration) (*Manager, error) {
|
||||
if enabled && len(secret) < 32 {
|
||||
return nil, errors.New("JWT_SECRET must contain at least 32 characters when authentication is enabled")
|
||||
}
|
||||
if ttl <= 0 {
|
||||
return nil, errors.New("JWT_TTL must be positive")
|
||||
}
|
||||
return &Manager{enabled: enabled, secret: []byte(secret), ttl: ttl, store: repository, now: time.Now}, nil
|
||||
}
|
||||
|
||||
func (m *Manager) Enabled() bool { return m.enabled }
|
||||
|
||||
func (m *Manager) TTLSeconds() int64 { return int64(m.ttl.Seconds()) }
|
||||
|
||||
func (m *Manager) Bootstrap(ctx context.Context, username, password, displayName string) error {
|
||||
if !m.enabled {
|
||||
return nil
|
||||
}
|
||||
if strings.TrimSpace(username) == "" || len(password) < 12 {
|
||||
return errors.New("bootstrap admin username is required and password must contain at least 12 characters")
|
||||
}
|
||||
hash, err := bcrypt.GenerateFromPassword([]byte(password), bcrypt.DefaultCost)
|
||||
if err != nil {
|
||||
return fmt.Errorf("hash bootstrap password: %w", err)
|
||||
}
|
||||
return m.store.EnsureBootstrapAdmin(ctx, normalizeUsername(username), string(hash), strings.TrimSpace(displayName))
|
||||
}
|
||||
|
||||
func (m *Manager) Login(ctx context.Context, username, password string) (string, domain.Principal, error) {
|
||||
if !m.enabled {
|
||||
principal := domain.Principal{Username: "admin", DisplayName: "Development Admin", IsAdmin: true}
|
||||
token, err := m.issue(principal)
|
||||
return token, principal, err
|
||||
}
|
||||
credential, err := m.store.FindUserByUsername(ctx, normalizeUsername(username))
|
||||
if err != nil || credential.Disabled || bcrypt.CompareHashAndPassword([]byte(credential.PasswordHash), []byte(password)) != nil {
|
||||
return "", domain.Principal{}, ErrUnauthorized
|
||||
}
|
||||
principal := domain.Principal{UserID: credential.ID, Username: credential.Username, DisplayName: credential.DisplayName, IsAdmin: credential.IsAdmin}
|
||||
token, err := m.issue(principal)
|
||||
return token, principal, err
|
||||
}
|
||||
|
||||
func (m *Manager) CreateUser(ctx context.Context, input domain.User, password string) (domain.User, error) {
|
||||
if len(password) < 12 {
|
||||
return domain.User{}, errors.New("password must contain at least 12 characters")
|
||||
}
|
||||
hash, err := bcrypt.GenerateFromPassword([]byte(password), bcrypt.DefaultCost)
|
||||
if err != nil {
|
||||
return domain.User{}, err
|
||||
}
|
||||
input.Username = normalizeUsername(input.Username)
|
||||
return m.store.CreateUser(ctx, input, string(hash))
|
||||
}
|
||||
|
||||
func (m *Manager) AuthenticateRequest(r *http.Request) (domain.Principal, error) {
|
||||
if !m.enabled {
|
||||
return domain.Principal{Username: "admin", DisplayName: "Development Admin", IsAdmin: true}, nil
|
||||
}
|
||||
header := strings.TrimSpace(r.Header.Get("Authorization"))
|
||||
if !strings.HasPrefix(header, "Bearer ") {
|
||||
return domain.Principal{}, ErrUnauthorized
|
||||
}
|
||||
return m.parse(strings.TrimSpace(strings.TrimPrefix(header, "Bearer ")))
|
||||
}
|
||||
|
||||
func WithPrincipal(ctx context.Context, principal domain.Principal) context.Context {
|
||||
return context.WithValue(ctx, principalKey{}, principal)
|
||||
}
|
||||
|
||||
func Principal(ctx context.Context) (domain.Principal, bool) {
|
||||
principal, ok := ctx.Value(principalKey{}).(domain.Principal)
|
||||
return principal, ok
|
||||
}
|
||||
|
||||
func (m *Manager) RequireAdmin(ctx context.Context) error {
|
||||
principal, ok := Principal(ctx)
|
||||
if !ok {
|
||||
return ErrUnauthorized
|
||||
}
|
||||
if !principal.IsAdmin {
|
||||
return ErrForbidden
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m *Manager) RequireAppRole(ctx context.Context, appID int64, minimum string) error {
|
||||
principal, ok := Principal(ctx)
|
||||
if !ok {
|
||||
return ErrUnauthorized
|
||||
}
|
||||
if principal.IsAdmin {
|
||||
return nil
|
||||
}
|
||||
role, err := m.store.GetUserAppRole(ctx, principal.UserID, appID)
|
||||
if err != nil {
|
||||
if errors.Is(err, store.ErrNotFound) {
|
||||
return ErrForbidden
|
||||
}
|
||||
return err
|
||||
}
|
||||
if roleRank(role) < roleRank(minimum) {
|
||||
return ErrForbidden
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m *Manager) CanAccessApp(ctx context.Context, appID int64, minimum string) bool {
|
||||
return m.RequireAppRole(ctx, appID, minimum) == nil
|
||||
}
|
||||
|
||||
func (m *Manager) issue(principal domain.Principal) (string, error) {
|
||||
now := m.now().UTC()
|
||||
claims := tokenClaims{Issuer: "configcenter", Subject: principal.Username, UserID: principal.UserID, DisplayName: principal.DisplayName, Admin: principal.IsAdmin, IssuedAt: now.Unix(), ExpiresAt: now.Add(m.ttl).Unix()}
|
||||
header, _ := json.Marshal(map[string]string{"alg": "HS256", "typ": "JWT"})
|
||||
payload, err := json.Marshal(claims)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
unsigned := encode(header) + "." + encode(payload)
|
||||
signature := m.sign(unsigned)
|
||||
return unsigned + "." + encode(signature), nil
|
||||
}
|
||||
|
||||
func (m *Manager) parse(token string) (domain.Principal, error) {
|
||||
parts := strings.Split(token, ".")
|
||||
if len(parts) != 3 {
|
||||
return domain.Principal{}, ErrUnauthorized
|
||||
}
|
||||
unsigned := parts[0] + "." + parts[1]
|
||||
signature, err := base64.RawURLEncoding.DecodeString(parts[2])
|
||||
if err != nil || !hmac.Equal(signature, m.sign(unsigned)) {
|
||||
return domain.Principal{}, ErrUnauthorized
|
||||
}
|
||||
payload, err := base64.RawURLEncoding.DecodeString(parts[1])
|
||||
if err != nil {
|
||||
return domain.Principal{}, ErrUnauthorized
|
||||
}
|
||||
var claims tokenClaims
|
||||
if err := json.Unmarshal(payload, &claims); err != nil || claims.Issuer != "configcenter" || claims.Subject == "" {
|
||||
return domain.Principal{}, ErrUnauthorized
|
||||
}
|
||||
now := m.now().Unix()
|
||||
if claims.ExpiresAt <= now || claims.IssuedAt > now+60 {
|
||||
return domain.Principal{}, ErrUnauthorized
|
||||
}
|
||||
return domain.Principal{UserID: claims.UserID, Username: claims.Subject, DisplayName: claims.DisplayName, IsAdmin: claims.Admin}, nil
|
||||
}
|
||||
|
||||
func (m *Manager) sign(input string) []byte {
|
||||
mac := hmac.New(sha256.New, m.secret)
|
||||
_, _ = mac.Write([]byte(input))
|
||||
return mac.Sum(nil)
|
||||
}
|
||||
|
||||
func encode(value []byte) string { return base64.RawURLEncoding.EncodeToString(value) }
|
||||
|
||||
func normalizeUsername(value string) string { return strings.ToLower(strings.TrimSpace(value)) }
|
||||
|
||||
func roleRank(role string) int {
|
||||
switch role {
|
||||
case RoleViewer:
|
||||
return 1
|
||||
case RoleAppOwner:
|
||||
return 2
|
||||
case RoleAdmin:
|
||||
return 3
|
||||
default:
|
||||
return 0
|
||||
}
|
||||
}
|
||||
|
||||
func ValidRole(role string) bool { return roleRank(role) > 0 }
|
||||
72
internal/auth/auth_test.go
Normal file
72
internal/auth/auth_test.go
Normal file
@@ -0,0 +1,72 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/longpeng/configcenter/internal/domain"
|
||||
memory_store "github.com/longpeng/configcenter/internal/store/memory"
|
||||
)
|
||||
|
||||
func TestLoginTokenExpiryAndApplicationRoles(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
repository := memory_store.New(false)
|
||||
manager, err := New(repository, true, "0123456789abcdef0123456789abcdef", time.Hour)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
clock := time.Date(2026, 8, 30, 2, 0, 0, 0, time.UTC)
|
||||
manager.now = func() time.Time { return clock }
|
||||
if err := manager.Bootstrap(ctx, "root", "a-strong-password", "Root"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
token, principal, err := manager.Login(ctx, "root", "a-strong-password")
|
||||
if err != nil || !principal.IsAdmin {
|
||||
t.Fatalf("bootstrap login failed: principal=%#v err=%v", principal, err)
|
||||
}
|
||||
request := httptest.NewRequest("GET", "/v1/applications", nil)
|
||||
request.Header.Set("Authorization", "Bearer "+token)
|
||||
parsed, err := manager.AuthenticateRequest(request)
|
||||
if err != nil || parsed.Username != "root" {
|
||||
t.Fatalf("token validation failed: principal=%#v err=%v", parsed, err)
|
||||
}
|
||||
|
||||
app, err := repository.CreateApplication(ctx, domain.Application{Code: "orders", Name: "Orders"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
viewer, err := manager.CreateUser(ctx, domain.User{Username: "Reader", DisplayName: "Reader"}, "another-strong-password")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if viewer.Username != "reader" {
|
||||
t.Fatalf("username was not normalized: %q", viewer.Username)
|
||||
}
|
||||
if err := repository.SetUserAppRole(ctx, domain.UserAppRole{UserID: viewer.ID, AppID: app.ID, Role: RoleViewer}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
viewerToken, viewerPrincipal, err := manager.Login(ctx, "READER", "another-strong-password")
|
||||
if err != nil || viewerToken == "" {
|
||||
t.Fatal(err)
|
||||
}
|
||||
viewerContext := WithPrincipal(ctx, viewerPrincipal)
|
||||
if err := manager.RequireAppRole(viewerContext, app.ID, RoleViewer); err != nil {
|
||||
t.Fatalf("viewer should read application: %v", err)
|
||||
}
|
||||
if err := manager.RequireAppRole(viewerContext, app.ID, RoleAppOwner); !errors.Is(err, ErrForbidden) {
|
||||
t.Fatalf("viewer must not edit application: %v", err)
|
||||
}
|
||||
|
||||
clock = clock.Add(2 * time.Hour)
|
||||
if _, err := manager.AuthenticateRequest(request); !errors.Is(err, ErrUnauthorized) {
|
||||
t.Fatalf("expired token must be rejected: %v", err)
|
||||
}
|
||||
tampered := httptest.NewRequest("GET", "/", nil)
|
||||
tampered.Header.Set("Authorization", "Bearer "+token+"x")
|
||||
if _, err := manager.AuthenticateRequest(tampered); !errors.Is(err, ErrUnauthorized) {
|
||||
t.Fatalf("tampered token must be rejected: %v", err)
|
||||
}
|
||||
}
|
||||
103
internal/config/config.go
Normal file
103
internal/config/config.go
Normal file
@@ -0,0 +1,103 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
type Config struct {
|
||||
HTTPAddr string
|
||||
DatabaseURL string
|
||||
EtcdEndpoints []string
|
||||
EtcdDialTimeout time.Duration
|
||||
OutboxInterval time.Duration
|
||||
OutboxBatchSize int
|
||||
OutboxMaxRetry int
|
||||
ShutdownTimeout time.Duration
|
||||
AllowedOrigins []string
|
||||
AuthEnabled bool
|
||||
JWTSecret string
|
||||
JWTTTL time.Duration
|
||||
BootstrapAdminUsername string
|
||||
BootstrapAdminPassword string
|
||||
BootstrapAdminDisplayName string
|
||||
}
|
||||
|
||||
func Load() Config {
|
||||
return Config{
|
||||
HTTPAddr: env("HTTP_ADDR", ":8080"),
|
||||
DatabaseURL: strings.TrimSpace(os.Getenv("DATABASE_URL")),
|
||||
EtcdEndpoints: csv(os.Getenv("ETCD_ENDPOINTS")),
|
||||
EtcdDialTimeout: duration("ETCD_DIAL_TIMEOUT", 5*time.Second),
|
||||
OutboxInterval: duration("OUTBOX_INTERVAL", 500*time.Millisecond),
|
||||
OutboxBatchSize: integer("OUTBOX_BATCH_SIZE", 50),
|
||||
OutboxMaxRetry: integer("OUTBOX_MAX_RETRY", 12),
|
||||
ShutdownTimeout: duration("SHUTDOWN_TIMEOUT", 10*time.Second),
|
||||
AllowedOrigins: csvDefault(os.Getenv("CORS_ALLOWED_ORIGINS"), []string{"http://localhost:5173"}),
|
||||
AuthEnabled: boolean("AUTH_ENABLED", false),
|
||||
JWTSecret: strings.TrimSpace(os.Getenv("JWT_SECRET")),
|
||||
JWTTTL: duration("JWT_TTL", 8*time.Hour),
|
||||
BootstrapAdminUsername: env("BOOTSTRAP_ADMIN_USERNAME", "admin"),
|
||||
BootstrapAdminPassword: os.Getenv("BOOTSTRAP_ADMIN_PASSWORD"),
|
||||
BootstrapAdminDisplayName: env("BOOTSTRAP_ADMIN_DISPLAY_NAME", "Administrator"),
|
||||
}
|
||||
}
|
||||
|
||||
func env(key, fallback string) string {
|
||||
if value := strings.TrimSpace(os.Getenv(key)); value != "" {
|
||||
return value
|
||||
}
|
||||
return fallback
|
||||
}
|
||||
|
||||
func duration(key string, fallback time.Duration) time.Duration {
|
||||
value := strings.TrimSpace(os.Getenv(key))
|
||||
if value == "" {
|
||||
return fallback
|
||||
}
|
||||
parsed, err := time.ParseDuration(value)
|
||||
if err != nil {
|
||||
return fallback
|
||||
}
|
||||
return parsed
|
||||
}
|
||||
|
||||
func integer(key string, fallback int) int {
|
||||
parsed, err := strconv.Atoi(strings.TrimSpace(os.Getenv(key)))
|
||||
if err != nil || parsed <= 0 {
|
||||
return fallback
|
||||
}
|
||||
return parsed
|
||||
}
|
||||
|
||||
func boolean(key string, fallback bool) bool {
|
||||
value := strings.TrimSpace(os.Getenv(key))
|
||||
if value == "" {
|
||||
return fallback
|
||||
}
|
||||
parsed, err := strconv.ParseBool(value)
|
||||
if err != nil {
|
||||
return fallback
|
||||
}
|
||||
return parsed
|
||||
}
|
||||
|
||||
func csv(value string) []string {
|
||||
parts := strings.Split(value, ",")
|
||||
result := make([]string, 0, len(parts))
|
||||
for _, part := range parts {
|
||||
if trimmed := strings.TrimSpace(part); trimmed != "" {
|
||||
result = append(result, trimmed)
|
||||
}
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
func csvDefault(value string, fallback []string) []string {
|
||||
if result := csv(value); len(result) > 0 {
|
||||
return result
|
||||
}
|
||||
return fallback
|
||||
}
|
||||
168
internal/domain/models.go
Normal file
168
internal/domain/models.go
Normal file
@@ -0,0 +1,168 @@
|
||||
package domain
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"time"
|
||||
)
|
||||
|
||||
type Application struct {
|
||||
ID int64 `json:"id"`
|
||||
Code string `json:"code"`
|
||||
Name string `json:"name"`
|
||||
Department string `json:"department"`
|
||||
Owner string `json:"owner"`
|
||||
Desc string `json:"desc"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
UpdatedAt time.Time `json:"updatedAt"`
|
||||
UpdatedBy string `json:"-"`
|
||||
}
|
||||
|
||||
type Environment struct {
|
||||
ID int64 `json:"id"`
|
||||
Code string `json:"code"`
|
||||
Name string `json:"name"`
|
||||
Order int `json:"order"`
|
||||
Desc string `json:"desc"`
|
||||
UpdatedBy string `json:"-"`
|
||||
}
|
||||
|
||||
type Namespace struct {
|
||||
ID int64 `json:"id"`
|
||||
AppID int64 `json:"appId"`
|
||||
Name string `json:"name"`
|
||||
Type string `json:"type"`
|
||||
Desc string `json:"desc"`
|
||||
UpdatedBy string `json:"-"`
|
||||
}
|
||||
|
||||
type ConfigItem struct {
|
||||
ID int64 `json:"id"`
|
||||
AppID int64 `json:"appId"`
|
||||
NamespaceID int64 `json:"nsId"`
|
||||
EnvironmentID int64 `json:"envId"`
|
||||
Key string `json:"key"`
|
||||
Value string `json:"value"`
|
||||
ReleasedValue *string `json:"releasedValue"`
|
||||
PendingDelete bool `json:"pendingDelete"`
|
||||
Comment string `json:"comment"`
|
||||
UpdatedBy string `json:"updatedBy"`
|
||||
UpdatedAt time.Time `json:"updatedAt"`
|
||||
}
|
||||
|
||||
func (item ConfigItem) Pending() bool {
|
||||
return item.PendingDelete || item.ReleasedValue == nil || *item.ReleasedValue != item.Value
|
||||
}
|
||||
|
||||
type Release struct {
|
||||
ID int64 `json:"id"`
|
||||
AppID int64 `json:"appId"`
|
||||
NamespaceID int64 `json:"nsId"`
|
||||
EnvironmentID int64 `json:"envId"`
|
||||
Version int `json:"version"`
|
||||
Snapshot map[string]string `json:"snapshot"`
|
||||
Added int `json:"added"`
|
||||
Modified int `json:"modified"`
|
||||
Removed int `json:"removed"`
|
||||
Comment string `json:"comment"`
|
||||
Operator string `json:"operator"`
|
||||
EtcdRevision *int64 `json:"etcdRevision"`
|
||||
Status string `json:"status"`
|
||||
Time time.Time `json:"time"`
|
||||
}
|
||||
|
||||
type PublishRequest struct {
|
||||
EnvironmentID int64 `json:"envId"`
|
||||
AppID int64 `json:"appId"`
|
||||
NamespaceID int64 `json:"nsId"`
|
||||
Comment string `json:"comment"`
|
||||
Operator string `json:"operator"`
|
||||
}
|
||||
|
||||
type RollbackRequest struct {
|
||||
EnvironmentID int64 `json:"envId"`
|
||||
AppID int64 `json:"appId"`
|
||||
NamespaceID int64 `json:"nsId"`
|
||||
TargetVersion int `json:"targetVersion"`
|
||||
Operator string `json:"operator"`
|
||||
}
|
||||
|
||||
type RuntimeConfig struct {
|
||||
Items map[string]string `json:"items"`
|
||||
Revision int64 `json:"revision"`
|
||||
ReleaseVersion int `json:"releaseVersion"`
|
||||
GrayRuleIDs []int64 `json:"grayRuleIds,omitempty"`
|
||||
}
|
||||
|
||||
type OutboxEntry struct {
|
||||
ID int64
|
||||
ReleaseID int64
|
||||
EtcdKey string
|
||||
Payload json.RawMessage
|
||||
RetryCount int
|
||||
}
|
||||
|
||||
type AuditLog struct {
|
||||
ID int64 `json:"id"`
|
||||
Actor string `json:"actor"`
|
||||
Action string `json:"action"`
|
||||
TargetType string `json:"targetType"`
|
||||
TargetID *int64 `json:"targetId"`
|
||||
Detail json.RawMessage `json:"detail"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
}
|
||||
|
||||
type ConfigEvent struct {
|
||||
Type string `json:"type"`
|
||||
Items map[string]string `json:"items"`
|
||||
Revision int64 `json:"revision"`
|
||||
GrayRuleIDs []int64 `json:"grayRuleIds,omitempty"`
|
||||
}
|
||||
|
||||
type User struct {
|
||||
ID int64 `json:"id"`
|
||||
Username string `json:"username"`
|
||||
DisplayName string `json:"displayName"`
|
||||
IsAdmin bool `json:"isAdmin"`
|
||||
Disabled bool `json:"disabled"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
UpdatedBy string `json:"-"`
|
||||
}
|
||||
|
||||
type UserCredential struct {
|
||||
User
|
||||
PasswordHash string `json:"-"`
|
||||
}
|
||||
|
||||
type UserAppRole struct {
|
||||
UserID int64 `json:"userId"`
|
||||
AppID int64 `json:"appId"`
|
||||
Role string `json:"role"`
|
||||
UpdatedBy string `json:"-"`
|
||||
}
|
||||
|
||||
type Principal struct {
|
||||
UserID int64 `json:"userId"`
|
||||
Username string `json:"username"`
|
||||
DisplayName string `json:"displayName"`
|
||||
IsAdmin bool `json:"isAdmin"`
|
||||
}
|
||||
|
||||
type GrayRule struct {
|
||||
ID int64 `json:"id"`
|
||||
AppID int64 `json:"appId"`
|
||||
NamespaceID int64 `json:"nsId"`
|
||||
EnvironmentID int64 `json:"envId"`
|
||||
RuleType string `json:"ruleType"`
|
||||
RuleValue json.RawMessage `json:"ruleValue"`
|
||||
Overrides map[string]string `json:"overrides"`
|
||||
Enabled bool `json:"enabled"`
|
||||
Priority int `json:"priority"`
|
||||
Desc string `json:"desc"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
}
|
||||
|
||||
type OutboxStats struct {
|
||||
Pending int64 `json:"pending"`
|
||||
Processing int64 `json:"processing"`
|
||||
Failed int64 `json:"failed"`
|
||||
}
|
||||
159
internal/gray/matcher.go
Normal file
159
internal/gray/matcher.go
Normal file
@@ -0,0 +1,159 @@
|
||||
package gray
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/binary"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"github.com/longpeng/configcenter/internal/domain"
|
||||
)
|
||||
|
||||
const (
|
||||
RuleIP = "ip"
|
||||
RuleInstance = "instance"
|
||||
RulePercentage = "percentage"
|
||||
)
|
||||
|
||||
type Target struct {
|
||||
IP string
|
||||
Instance string
|
||||
}
|
||||
|
||||
type ipRule struct {
|
||||
IPs []string `json:"ips"`
|
||||
}
|
||||
|
||||
type instanceRule struct {
|
||||
Instances []string `json:"instances"`
|
||||
}
|
||||
|
||||
type percentageRule struct {
|
||||
Percentage int `json:"percentage"`
|
||||
Salt string `json:"salt"`
|
||||
}
|
||||
|
||||
func Validate(rule domain.GrayRule) error {
|
||||
if rule.NamespaceID <= 0 || rule.EnvironmentID <= 0 || len(rule.Overrides) == 0 {
|
||||
return fmt.Errorf("scope and overrides are required")
|
||||
}
|
||||
for key := range rule.Overrides {
|
||||
if strings.TrimSpace(key) == "" {
|
||||
return fmt.Errorf("override key must not be empty")
|
||||
}
|
||||
}
|
||||
switch rule.RuleType {
|
||||
case RuleIP:
|
||||
var value ipRule
|
||||
if err := strictUnmarshal(rule.RuleValue, &value); err != nil || len(value.IPs) == 0 {
|
||||
return fmt.Errorf("ip rule requires a non-empty ips array")
|
||||
}
|
||||
for _, item := range value.IPs {
|
||||
if net.ParseIP(item) == nil {
|
||||
if _, _, err := net.ParseCIDR(item); err != nil {
|
||||
return fmt.Errorf("invalid IP or CIDR %q", item)
|
||||
}
|
||||
}
|
||||
}
|
||||
case RuleInstance:
|
||||
var value instanceRule
|
||||
if err := strictUnmarshal(rule.RuleValue, &value); err != nil || len(value.Instances) == 0 {
|
||||
return fmt.Errorf("instance rule requires a non-empty instances array")
|
||||
}
|
||||
case RulePercentage:
|
||||
var value percentageRule
|
||||
if err := strictUnmarshal(rule.RuleValue, &value); err != nil || value.Percentage < 0 || value.Percentage > 100 {
|
||||
return fmt.Errorf("percentage rule requires percentage between 0 and 100")
|
||||
}
|
||||
default:
|
||||
return fmt.Errorf("unsupported gray rule type %q", rule.RuleType)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func Apply(base map[string]string, rules []domain.GrayRule, target Target) (map[string]string, []int64) {
|
||||
result := clone(base)
|
||||
sorted := append([]domain.GrayRule(nil), rules...)
|
||||
sort.SliceStable(sorted, func(i, j int) bool {
|
||||
if sorted[i].Priority == sorted[j].Priority {
|
||||
return sorted[i].ID < sorted[j].ID
|
||||
}
|
||||
return sorted[i].Priority < sorted[j].Priority
|
||||
})
|
||||
matched := make([]int64, 0)
|
||||
for _, rule := range sorted {
|
||||
if !rule.Enabled || !matches(rule, target) {
|
||||
continue
|
||||
}
|
||||
for key, value := range rule.Overrides {
|
||||
result[key] = value
|
||||
}
|
||||
matched = append(matched, rule.ID)
|
||||
}
|
||||
return result, matched
|
||||
}
|
||||
|
||||
func matches(rule domain.GrayRule, target Target) bool {
|
||||
switch rule.RuleType {
|
||||
case RuleIP:
|
||||
parsed := net.ParseIP(target.IP)
|
||||
if parsed == nil {
|
||||
return false
|
||||
}
|
||||
var value ipRule
|
||||
if json.Unmarshal(rule.RuleValue, &value) != nil {
|
||||
return false
|
||||
}
|
||||
for _, item := range value.IPs {
|
||||
if ip := net.ParseIP(item); ip != nil && ip.Equal(parsed) {
|
||||
return true
|
||||
}
|
||||
if _, network, err := net.ParseCIDR(item); err == nil && network.Contains(parsed) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
case RuleInstance:
|
||||
var value instanceRule
|
||||
if target.Instance == "" || json.Unmarshal(rule.RuleValue, &value) != nil {
|
||||
return false
|
||||
}
|
||||
for _, instance := range value.Instances {
|
||||
if instance == target.Instance {
|
||||
return true
|
||||
}
|
||||
}
|
||||
case RulePercentage:
|
||||
identity := target.Instance
|
||||
if identity == "" {
|
||||
identity = target.IP
|
||||
}
|
||||
if identity == "" {
|
||||
return false
|
||||
}
|
||||
var value percentageRule
|
||||
if json.Unmarshal(rule.RuleValue, &value) != nil {
|
||||
return false
|
||||
}
|
||||
digest := sha256.Sum256([]byte(value.Salt + "\x00" + identity))
|
||||
bucket := binary.BigEndian.Uint64(digest[:8]) % 100
|
||||
return int(bucket) < value.Percentage
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func strictUnmarshal(payload []byte, target any) error {
|
||||
decoder := json.NewDecoder(strings.NewReader(string(payload)))
|
||||
decoder.DisallowUnknownFields()
|
||||
return decoder.Decode(target)
|
||||
}
|
||||
|
||||
func clone(input map[string]string) map[string]string {
|
||||
result := make(map[string]string, len(input))
|
||||
for key, value := range input {
|
||||
result[key] = value
|
||||
}
|
||||
return result
|
||||
}
|
||||
53
internal/gray/matcher_test.go
Normal file
53
internal/gray/matcher_test.go
Normal file
@@ -0,0 +1,53 @@
|
||||
package gray
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"testing"
|
||||
|
||||
"github.com/longpeng/configcenter/internal/domain"
|
||||
)
|
||||
|
||||
func TestApplyMatchesIPInstanceAndPriority(t *testing.T) {
|
||||
rules := []domain.GrayRule{
|
||||
{ID: 1, NamespaceID: 1, EnvironmentID: 1, RuleType: RuleIP, RuleValue: json.RawMessage(`{"ips":["10.0.0.0/24"]}`), Overrides: map[string]string{"color": "blue", "ip-only": "yes"}, Enabled: true, Priority: 10},
|
||||
{ID: 2, NamespaceID: 1, EnvironmentID: 1, RuleType: RuleInstance, RuleValue: json.RawMessage(`{"instances":["orders-7"]}`), Overrides: map[string]string{"color": "green"}, Enabled: true, Priority: 20},
|
||||
}
|
||||
for _, rule := range rules {
|
||||
if err := Validate(rule); err != nil {
|
||||
t.Fatalf("valid rule rejected: %v", err)
|
||||
}
|
||||
}
|
||||
items, matched := Apply(map[string]string{"color": "red", "stable": "value"}, rules, Target{IP: "10.0.0.42", Instance: "orders-7"})
|
||||
if items["color"] != "green" || items["ip-only"] != "yes" || items["stable"] != "value" {
|
||||
t.Fatalf("unexpected overrides: %#v", items)
|
||||
}
|
||||
if len(matched) != 2 || matched[0] != 1 || matched[1] != 2 {
|
||||
t.Fatalf("unexpected matched rules: %#v", matched)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPercentageRuleIsDeterministic(t *testing.T) {
|
||||
rule := domain.GrayRule{ID: 3, NamespaceID: 1, EnvironmentID: 1, RuleType: RulePercentage, RuleValue: json.RawMessage(`{"percentage":50,"salt":"release-a"}`), Overrides: map[string]string{"feature": "on"}, Enabled: true}
|
||||
if err := Validate(rule); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
firstItems, firstMatched := Apply(map[string]string{"feature": "off"}, []domain.GrayRule{rule}, Target{Instance: "orders-42"})
|
||||
for i := 0; i < 20; i++ {
|
||||
items, matched := Apply(map[string]string{"feature": "off"}, []domain.GrayRule{rule}, Target{Instance: "orders-42"})
|
||||
if items["feature"] != firstItems["feature"] || len(matched) != len(firstMatched) {
|
||||
t.Fatalf("percentage assignment changed between calls: first=%#v/%#v current=%#v/%#v", firstItems, firstMatched, items, matched)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateRejectsInvalidCIDRAndPercentage(t *testing.T) {
|
||||
cases := []domain.GrayRule{
|
||||
{NamespaceID: 1, EnvironmentID: 1, RuleType: RuleIP, RuleValue: json.RawMessage(`{"ips":["not-an-ip"]}`), Overrides: map[string]string{"x": "y"}},
|
||||
{NamespaceID: 1, EnvironmentID: 1, RuleType: RulePercentage, RuleValue: json.RawMessage(`{"percentage":101}`), Overrides: map[string]string{"x": "y"}},
|
||||
}
|
||||
for _, rule := range cases {
|
||||
if err := Validate(rule); err == nil {
|
||||
t.Fatalf("invalid rule accepted: %#v", rule)
|
||||
}
|
||||
}
|
||||
}
|
||||
130
internal/metrics/metrics.go
Normal file
130
internal/metrics/metrics.go
Normal file
@@ -0,0 +1,130 @@
|
||||
package metrics
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net/http"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"github.com/longpeng/configcenter/internal/store"
|
||||
)
|
||||
|
||||
type Collector struct {
|
||||
mu sync.Mutex
|
||||
http map[httpKey]httpValue
|
||||
|
||||
publishAccepted atomic.Uint64
|
||||
outboxApplied atomic.Uint64
|
||||
outboxFailed atomic.Uint64
|
||||
runtimeReads atomic.Uint64
|
||||
grayMatches atomic.Uint64
|
||||
watchers atomic.Int64
|
||||
}
|
||||
|
||||
type httpKey struct {
|
||||
method string
|
||||
route string
|
||||
status int
|
||||
}
|
||||
|
||||
type httpValue struct {
|
||||
count uint64
|
||||
durationSum float64
|
||||
}
|
||||
|
||||
func New() *Collector { return &Collector{http: make(map[httpKey]httpValue)} }
|
||||
|
||||
func (c *Collector) ObserveHTTP(method, route string, status int, duration time.Duration) {
|
||||
if route == "" {
|
||||
route = "unmatched"
|
||||
}
|
||||
key := httpKey{method: method, route: route, status: status}
|
||||
c.mu.Lock()
|
||||
value := c.http[key]
|
||||
value.count++
|
||||
value.durationSum += duration.Seconds()
|
||||
c.http[key] = value
|
||||
c.mu.Unlock()
|
||||
}
|
||||
|
||||
func (c *Collector) PublishAccepted() { c.publishAccepted.Add(1) }
|
||||
func (c *Collector) OutboxApplied() { c.outboxApplied.Add(1) }
|
||||
func (c *Collector) OutboxFailed() { c.outboxFailed.Add(1) }
|
||||
func (c *Collector) RuntimeRead() { c.runtimeReads.Add(1) }
|
||||
func (c *Collector) GrayMatched(count int) {
|
||||
if count > 0 {
|
||||
c.grayMatches.Add(uint64(count))
|
||||
}
|
||||
}
|
||||
func (c *Collector) WatcherAdded() { c.watchers.Add(1) }
|
||||
func (c *Collector) WatcherRemoved() { c.watchers.Add(-1) }
|
||||
|
||||
func (c *Collector) Handler(repository store.Store) http.HandlerFunc {
|
||||
return func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Type", "text/plain; version=0.0.4; charset=utf-8")
|
||||
stats, err := repository.OutboxStats(r.Context())
|
||||
if err != nil {
|
||||
http.Error(w, "collect outbox metrics", http.StatusServiceUnavailable)
|
||||
return
|
||||
}
|
||||
var output strings.Builder
|
||||
writeHelp(&output, "configcenter_http_requests_total", "HTTP requests by method, route and status", "counter")
|
||||
writeHelp(&output, "configcenter_http_request_duration_seconds_sum", "Cumulative HTTP request duration", "counter")
|
||||
c.mu.Lock()
|
||||
keys := make([]httpKey, 0, len(c.http))
|
||||
for key := range c.http {
|
||||
keys = append(keys, key)
|
||||
}
|
||||
sort.Slice(keys, func(i, j int) bool {
|
||||
if keys[i].route != keys[j].route {
|
||||
return keys[i].route < keys[j].route
|
||||
}
|
||||
if keys[i].method != keys[j].method {
|
||||
return keys[i].method < keys[j].method
|
||||
}
|
||||
return keys[i].status < keys[j].status
|
||||
})
|
||||
for _, key := range keys {
|
||||
value := c.http[key]
|
||||
labels := fmt.Sprintf(`method="%s",route="%s",status="%d"`, escape(key.method), escape(key.route), key.status)
|
||||
fmt.Fprintf(&output, "configcenter_http_requests_total{%s} %d\n", labels, value.count)
|
||||
fmt.Fprintf(&output, "configcenter_http_request_duration_seconds_sum{%s} %s\n", labels, strconv.FormatFloat(value.durationSum, 'f', 6, 64))
|
||||
}
|
||||
c.mu.Unlock()
|
||||
|
||||
writeCounter(&output, "configcenter_publish_accepted_total", "Accepted configuration releases", c.publishAccepted.Load())
|
||||
writeCounter(&output, "configcenter_outbox_applied_total", "Successfully applied outbox entries", c.outboxApplied.Load())
|
||||
writeCounter(&output, "configcenter_outbox_failed_attempts_total", "Failed outbox apply attempts", c.outboxFailed.Load())
|
||||
writeCounter(&output, "configcenter_runtime_reads_total", "Runtime configuration reads", c.runtimeReads.Load())
|
||||
writeCounter(&output, "configcenter_gray_rule_matches_total", "Gray rules matched during runtime reads and watches", c.grayMatches.Load())
|
||||
writeGauge(&output, "configcenter_watch_subscribers", "Current SSE watch subscribers", c.watchers.Load())
|
||||
writeGauge(&output, "configcenter_outbox_pending", "Pending outbox entries", stats.Pending)
|
||||
writeGauge(&output, "configcenter_outbox_processing", "Processing outbox entries", stats.Processing)
|
||||
writeGauge(&output, "configcenter_outbox_failed", "Terminally failed outbox entries", stats.Failed)
|
||||
_, _ = w.Write([]byte(output.String()))
|
||||
}
|
||||
}
|
||||
|
||||
func writeHelp(output *strings.Builder, name, help, metricType string) {
|
||||
fmt.Fprintf(output, "# HELP %s %s\n# TYPE %s %s\n", name, help, name, metricType)
|
||||
}
|
||||
|
||||
func writeCounter(output *strings.Builder, name, help string, value uint64) {
|
||||
writeHelp(output, name, help, "counter")
|
||||
fmt.Fprintf(output, "%s %d\n", name, value)
|
||||
}
|
||||
|
||||
func writeGauge(output *strings.Builder, name, help string, value int64) {
|
||||
writeHelp(output, name, help, "gauge")
|
||||
fmt.Fprintf(output, "%s %d\n", name, value)
|
||||
}
|
||||
|
||||
func escape(value string) string {
|
||||
value = strings.ReplaceAll(value, `\`, `\\`)
|
||||
value = strings.ReplaceAll(value, `"`, `\"`)
|
||||
return strings.ReplaceAll(value, "\n", `\n`)
|
||||
}
|
||||
67
internal/outbox/worker.go
Normal file
67
internal/outbox/worker.go
Normal file
@@ -0,0 +1,67 @@
|
||||
package outbox
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log/slog"
|
||||
"time"
|
||||
|
||||
metricspkg "github.com/longpeng/configcenter/internal/metrics"
|
||||
runtimepkg "github.com/longpeng/configcenter/internal/runtime"
|
||||
"github.com/longpeng/configcenter/internal/store"
|
||||
)
|
||||
|
||||
type Worker struct {
|
||||
store store.Store
|
||||
runtime runtimepkg.Store
|
||||
interval time.Duration
|
||||
batch int
|
||||
maxRetry int
|
||||
logger *slog.Logger
|
||||
metrics *metricspkg.Collector
|
||||
}
|
||||
|
||||
func New(repository store.Store, runtimeStore runtimepkg.Store, interval time.Duration, batch, maxRetry int, metrics *metricspkg.Collector, logger *slog.Logger) *Worker {
|
||||
return &Worker{store: repository, runtime: runtimeStore, interval: interval, batch: batch, maxRetry: maxRetry, metrics: metrics, logger: logger}
|
||||
}
|
||||
|
||||
func (w *Worker) Run(ctx context.Context) {
|
||||
ticker := time.NewTicker(w.interval)
|
||||
defer ticker.Stop()
|
||||
w.process(ctx)
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-ticker.C:
|
||||
w.process(ctx)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (w *Worker) process(ctx context.Context) {
|
||||
entries, err := w.store.ClaimOutbox(ctx, w.batch)
|
||||
if err != nil {
|
||||
w.logger.Error("claim outbox", "error", err)
|
||||
return
|
||||
}
|
||||
for _, entry := range entries {
|
||||
release, err := w.store.GetRelease(ctx, entry.ReleaseID)
|
||||
if err == nil {
|
||||
var revision int64
|
||||
revision, err = w.runtime.Put(ctx, entry.EtcdKey, entry.Payload, release)
|
||||
if err == nil {
|
||||
err = w.store.MarkOutboxDone(ctx, entry.ID, entry.ReleaseID, revision)
|
||||
if err == nil {
|
||||
w.metrics.OutboxApplied()
|
||||
}
|
||||
}
|
||||
}
|
||||
if err != nil {
|
||||
w.metrics.OutboxFailed()
|
||||
w.logger.Warn("apply release outbox", "outbox_id", entry.ID, "release_id", entry.ReleaseID, "retry", entry.RetryCount, "error", err)
|
||||
if markErr := w.store.MarkOutboxFailed(ctx, entry.ID, entry.ReleaseID, err.Error(), w.maxRetry); markErr != nil {
|
||||
w.logger.Error("mark outbox failed", "outbox_id", entry.ID, "error", markErr)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
122
internal/runtime/etcd/etcd.go
Normal file
122
internal/runtime/etcd/etcd.go
Normal file
@@ -0,0 +1,122 @@
|
||||
package etcd
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
clientv3 "go.etcd.io/etcd/client/v3"
|
||||
|
||||
"github.com/longpeng/configcenter/internal/domain"
|
||||
runtimepkg "github.com/longpeng/configcenter/internal/runtime"
|
||||
)
|
||||
|
||||
type Store struct {
|
||||
client *clientv3.Client
|
||||
}
|
||||
|
||||
type metadata struct {
|
||||
Version int `json:"version"`
|
||||
ReleaseID int64 `json:"releaseId"`
|
||||
PublishedAt time.Time `json:"publishedAt"`
|
||||
PublishedBy string `json:"publishedBy"`
|
||||
}
|
||||
|
||||
func Open(endpoints []string, dialTimeout time.Duration) (*Store, error) {
|
||||
client, err := clientv3.New(clientv3.Config{Endpoints: endpoints, DialTimeout: dialTimeout})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("create etcd client: %w", err)
|
||||
}
|
||||
return &Store{client: client}, nil
|
||||
}
|
||||
|
||||
func (s *Store) Close() error { return s.client.Close() }
|
||||
|
||||
func (s *Store) Ping(ctx context.Context) error {
|
||||
_, err := s.client.Get(ctx, "/configcenter/health", clientv3.WithLimit(1))
|
||||
return err
|
||||
}
|
||||
|
||||
func (s *Store) Put(ctx context.Context, key string, payload []byte, release domain.Release) (int64, error) {
|
||||
meta, err := json.Marshal(metadata{Version: release.Version, ReleaseID: release.ID, PublishedAt: release.Time, PublishedBy: release.Operator})
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
response, err := s.client.Txn(ctx).Then(
|
||||
clientv3.OpPut(key, string(payload)),
|
||||
clientv3.OpPut(key+"/__meta", string(meta)),
|
||||
).Commit()
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return response.Header.Revision, nil
|
||||
}
|
||||
|
||||
func (s *Store) Get(ctx context.Context, key string) (domain.RuntimeConfig, error) {
|
||||
response, err := s.client.Txn(ctx).Then(
|
||||
clientv3.OpGet(key),
|
||||
clientv3.OpGet(key+"/__meta"),
|
||||
).Commit()
|
||||
if err != nil {
|
||||
return domain.RuntimeConfig{}, err
|
||||
}
|
||||
result := domain.RuntimeConfig{Items: map[string]string{}, Revision: response.Header.Revision}
|
||||
if len(response.Responses) > 0 {
|
||||
values := response.Responses[0].GetResponseRange().Kvs
|
||||
if len(values) > 0 {
|
||||
if err := json.Unmarshal(values[0].Value, &result.Items); err != nil {
|
||||
return domain.RuntimeConfig{}, fmt.Errorf("decode runtime config: %w", err)
|
||||
}
|
||||
result.Revision = values[0].ModRevision
|
||||
}
|
||||
}
|
||||
if len(response.Responses) > 1 {
|
||||
values := response.Responses[1].GetResponseRange().Kvs
|
||||
if len(values) > 0 {
|
||||
var meta metadata
|
||||
if err := json.Unmarshal(values[0].Value, &meta); err != nil {
|
||||
return domain.RuntimeConfig{}, fmt.Errorf("decode runtime metadata: %w", err)
|
||||
}
|
||||
result.ReleaseVersion = meta.Version
|
||||
}
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (s *Store) Watch(ctx context.Context, key string, startRevision int64) <-chan runtimepkg.WatchResult {
|
||||
result := make(chan runtimepkg.WatchResult, 16)
|
||||
options := []clientv3.OpOption{}
|
||||
if startRevision > 0 {
|
||||
options = append(options, clientv3.WithRev(startRevision))
|
||||
}
|
||||
watch := s.client.Watch(ctx, key, options...)
|
||||
go func() {
|
||||
defer close(result)
|
||||
for response := range watch {
|
||||
if err := response.Err(); err != nil {
|
||||
select {
|
||||
case result <- runtimepkg.WatchResult{Err: err}:
|
||||
case <-ctx.Done():
|
||||
}
|
||||
return
|
||||
}
|
||||
for _, event := range response.Events {
|
||||
items := make(map[string]string)
|
||||
if err := json.Unmarshal(event.Kv.Value, &items); err != nil {
|
||||
select {
|
||||
case result <- runtimepkg.WatchResult{Err: err}:
|
||||
case <-ctx.Done():
|
||||
}
|
||||
continue
|
||||
}
|
||||
select {
|
||||
case result <- runtimepkg.WatchResult{Event: domain.ConfigEvent{Type: "UPDATED", Items: items, Revision: event.Kv.ModRevision}}:
|
||||
case <-ctx.Done():
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
}()
|
||||
return result
|
||||
}
|
||||
120
internal/runtime/memory/memory.go
Normal file
120
internal/runtime/memory/memory.go
Normal file
@@ -0,0 +1,120 @@
|
||||
package memory
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"sync"
|
||||
|
||||
"github.com/longpeng/configcenter/internal/domain"
|
||||
runtimepkg "github.com/longpeng/configcenter/internal/runtime"
|
||||
)
|
||||
|
||||
type value struct {
|
||||
items map[string]string
|
||||
revision int64
|
||||
version int
|
||||
}
|
||||
|
||||
type historyEvent struct {
|
||||
key string
|
||||
event domain.ConfigEvent
|
||||
}
|
||||
|
||||
type subscriber struct {
|
||||
key string
|
||||
ch chan runtimepkg.WatchResult
|
||||
}
|
||||
|
||||
type Store struct {
|
||||
mu sync.RWMutex
|
||||
revision int64
|
||||
values map[string]value
|
||||
subscribers map[int64]subscriber
|
||||
nextSubID int64
|
||||
history []historyEvent
|
||||
}
|
||||
|
||||
func New() *Store {
|
||||
return &Store{values: make(map[string]value), subscribers: make(map[int64]subscriber)}
|
||||
}
|
||||
|
||||
func (s *Store) Close() error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
for id, sub := range s.subscribers {
|
||||
close(sub.ch)
|
||||
delete(s.subscribers, id)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Store) Ping(context.Context) error { return nil }
|
||||
|
||||
func (s *Store) Put(_ context.Context, key string, payload []byte, release domain.Release) (int64, error) {
|
||||
items := make(map[string]string)
|
||||
if err := json.Unmarshal(payload, &items); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
s.mu.Lock()
|
||||
s.revision++
|
||||
revision := s.revision
|
||||
s.values[key] = value{items: clone(items), revision: revision, version: release.Version}
|
||||
event := domain.ConfigEvent{Type: "UPDATED", Items: clone(items), Revision: revision}
|
||||
s.history = append(s.history, historyEvent{key: key, event: event})
|
||||
if len(s.history) > 256 {
|
||||
s.history = append([]historyEvent(nil), s.history[len(s.history)-256:]...)
|
||||
}
|
||||
for _, sub := range s.subscribers {
|
||||
if sub.key != key {
|
||||
continue
|
||||
}
|
||||
select {
|
||||
case sub.ch <- runtimepkg.WatchResult{Event: event}:
|
||||
default:
|
||||
}
|
||||
}
|
||||
s.mu.Unlock()
|
||||
return revision, nil
|
||||
}
|
||||
|
||||
func (s *Store) Get(_ context.Context, key string) (domain.RuntimeConfig, error) {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
current, ok := s.values[key]
|
||||
if !ok {
|
||||
return domain.RuntimeConfig{Items: map[string]string{}, Revision: s.revision}, nil
|
||||
}
|
||||
return domain.RuntimeConfig{Items: clone(current.items), Revision: current.revision, ReleaseVersion: current.version}, nil
|
||||
}
|
||||
|
||||
func (s *Store) Watch(ctx context.Context, key string, startRevision int64) <-chan runtimepkg.WatchResult {
|
||||
ch := make(chan runtimepkg.WatchResult, 16)
|
||||
s.mu.Lock()
|
||||
s.nextSubID++
|
||||
id := s.nextSubID
|
||||
s.subscribers[id] = subscriber{key: key, ch: ch}
|
||||
for _, historical := range s.history {
|
||||
if historical.key == key && historical.event.Revision >= startRevision && startRevision > 0 {
|
||||
ch <- runtimepkg.WatchResult{Event: historical.event}
|
||||
}
|
||||
}
|
||||
s.mu.Unlock()
|
||||
go func() {
|
||||
<-ctx.Done()
|
||||
s.mu.Lock()
|
||||
if _, ok := s.subscribers[id]; ok {
|
||||
delete(s.subscribers, id)
|
||||
close(ch)
|
||||
}
|
||||
s.mu.Unlock()
|
||||
}()
|
||||
return ch
|
||||
}
|
||||
|
||||
func clone(input map[string]string) map[string]string {
|
||||
result := make(map[string]string, len(input))
|
||||
for key, item := range input {
|
||||
result[key] = item
|
||||
}
|
||||
return result
|
||||
}
|
||||
20
internal/runtime/runtime.go
Normal file
20
internal/runtime/runtime.go
Normal file
@@ -0,0 +1,20 @@
|
||||
package runtime
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/longpeng/configcenter/internal/domain"
|
||||
)
|
||||
|
||||
type WatchResult struct {
|
||||
Event domain.ConfigEvent
|
||||
Err error
|
||||
}
|
||||
|
||||
type Store interface {
|
||||
Close() error
|
||||
Ping(context.Context) error
|
||||
Put(context.Context, string, []byte, domain.Release) (int64, error)
|
||||
Get(context.Context, string) (domain.RuntimeConfig, error)
|
||||
Watch(context.Context, string, int64) <-chan WatchResult
|
||||
}
|
||||
884
internal/store/memory/memory.go
Normal file
884
internal/store/memory/memory.go
Normal file
@@ -0,0 +1,884 @@
|
||||
package memory
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/longpeng/configcenter/internal/domain"
|
||||
storepkg "github.com/longpeng/configcenter/internal/store"
|
||||
)
|
||||
|
||||
type Store struct {
|
||||
mu sync.RWMutex
|
||||
nextID int64
|
||||
applications map[int64]domain.Application
|
||||
environments map[int64]domain.Environment
|
||||
namespaces map[int64]domain.Namespace
|
||||
configs map[int64]domain.ConfigItem
|
||||
releases map[int64]domain.Release
|
||||
outbox map[int64]outboxRecord
|
||||
audits []domain.AuditLog
|
||||
users map[int64]domain.UserCredential
|
||||
userRoles map[[2]int64]string
|
||||
grayRules map[int64]domain.GrayRule
|
||||
}
|
||||
|
||||
type outboxRecord struct {
|
||||
domain.OutboxEntry
|
||||
Status string
|
||||
NextAttempt time.Time
|
||||
}
|
||||
|
||||
func New(withSeed bool) *Store {
|
||||
s := &Store{
|
||||
nextID: 100,
|
||||
applications: make(map[int64]domain.Application),
|
||||
environments: make(map[int64]domain.Environment),
|
||||
namespaces: make(map[int64]domain.Namespace),
|
||||
configs: make(map[int64]domain.ConfigItem),
|
||||
releases: make(map[int64]domain.Release),
|
||||
outbox: make(map[int64]outboxRecord),
|
||||
users: make(map[int64]domain.UserCredential),
|
||||
userRoles: make(map[[2]int64]string),
|
||||
grayRules: make(map[int64]domain.GrayRule),
|
||||
}
|
||||
if withSeed {
|
||||
s.seed()
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
func (s *Store) Close() {}
|
||||
|
||||
func (s *Store) Ping(context.Context) error { return nil }
|
||||
|
||||
func (s *Store) id() int64 {
|
||||
s.nextID++
|
||||
return s.nextID
|
||||
}
|
||||
|
||||
func (s *Store) seed() {
|
||||
now := time.Now().UTC()
|
||||
for _, env := range []domain.Environment{
|
||||
{ID: 1, Code: "DEV", Name: "开发环境", Order: 1, Desc: "开发联调使用"},
|
||||
{ID: 2, Code: "TEST", Name: "测试环境", Order: 2, Desc: "QA 测试验证"},
|
||||
{ID: 3, Code: "STAGING", Name: "预发环境", Order: 3, Desc: "上线前灰度验证"},
|
||||
{ID: 4, Code: "PROD", Name: "生产环境", Order: 4, Desc: "线上正式环境"},
|
||||
} {
|
||||
s.environments[env.ID] = env
|
||||
}
|
||||
app := domain.Application{ID: 10, Code: "demo-service", Name: "示例服务", Department: "平台架构部", Owner: "admin", Desc: "开箱即用的演示应用", CreatedAt: now, UpdatedAt: now}
|
||||
ns := domain.Namespace{ID: 20, AppID: app.ID, Name: "application", Type: "properties", Desc: "默认应用配置"}
|
||||
s.applications[app.ID] = app
|
||||
s.namespaces[ns.ID] = ns
|
||||
port := "8080"
|
||||
level := "INFO"
|
||||
s.configs[30] = domain.ConfigItem{ID: 30, AppID: app.ID, NamespaceID: ns.ID, EnvironmentID: 1, Key: "server.port", Value: port, ReleasedValue: &port, Comment: "服务监听端口", UpdatedBy: "system", UpdatedAt: now}
|
||||
s.configs[31] = domain.ConfigItem{ID: 31, AppID: app.ID, NamespaceID: ns.ID, EnvironmentID: 1, Key: "log.level", Value: level, ReleasedValue: &level, Comment: "日志级别", UpdatedBy: "system", UpdatedAt: now}
|
||||
release := domain.Release{ID: 40, AppID: app.ID, NamespaceID: ns.ID, EnvironmentID: 1, Version: 1, Snapshot: map[string]string{"server.port": port, "log.level": level}, Added: 2, Operator: "system", Comment: "初始化示例配置", Status: "pending", Time: now}
|
||||
s.releases[release.ID] = release
|
||||
s.enqueue(release)
|
||||
}
|
||||
|
||||
func (s *Store) ListApplications(context.Context) ([]domain.Application, error) {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
result := make([]domain.Application, 0, len(s.applications))
|
||||
for _, item := range s.applications {
|
||||
result = append(result, item)
|
||||
}
|
||||
sort.Slice(result, func(i, j int) bool { return result[i].Code < result[j].Code })
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (s *Store) CreateApplication(_ context.Context, input domain.Application) (domain.Application, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
for _, item := range s.applications {
|
||||
if strings.EqualFold(item.Code, input.Code) {
|
||||
return domain.Application{}, storepkg.ErrConflict
|
||||
}
|
||||
}
|
||||
now := time.Now().UTC()
|
||||
input.ID, input.CreatedAt, input.UpdatedAt = s.id(), now, now
|
||||
s.applications[input.ID] = input
|
||||
s.audit(defaultActor(input.UpdatedBy), "create", "app", input.ID, input)
|
||||
return input, nil
|
||||
}
|
||||
|
||||
func (s *Store) UpdateApplication(_ context.Context, id int64, input domain.Application) (domain.Application, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
current, ok := s.applications[id]
|
||||
if !ok {
|
||||
return domain.Application{}, storepkg.ErrNotFound
|
||||
}
|
||||
current.Name, current.Department, current.Owner, current.Desc = input.Name, input.Department, input.Owner, input.Desc
|
||||
current.UpdatedAt = time.Now().UTC()
|
||||
s.applications[id] = current
|
||||
s.audit(defaultActor(input.UpdatedBy), "update", "app", id, current)
|
||||
return current, nil
|
||||
}
|
||||
|
||||
func (s *Store) DeleteApplication(_ context.Context, id int64, actor string) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if _, ok := s.applications[id]; !ok {
|
||||
return storepkg.ErrNotFound
|
||||
}
|
||||
delete(s.applications, id)
|
||||
for nsID, ns := range s.namespaces {
|
||||
if ns.AppID == id {
|
||||
delete(s.namespaces, nsID)
|
||||
}
|
||||
}
|
||||
for configID, item := range s.configs {
|
||||
if item.AppID == id {
|
||||
delete(s.configs, configID)
|
||||
}
|
||||
}
|
||||
for releaseID, item := range s.releases {
|
||||
if item.AppID == id {
|
||||
delete(s.releases, releaseID)
|
||||
}
|
||||
}
|
||||
for ruleID, rule := range s.grayRules {
|
||||
if rule.AppID == id {
|
||||
delete(s.grayRules, ruleID)
|
||||
}
|
||||
}
|
||||
for key := range s.userRoles {
|
||||
if key[1] == id {
|
||||
delete(s.userRoles, key)
|
||||
}
|
||||
}
|
||||
s.audit(defaultActor(actor), "delete", "app", id, nil)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Store) ListEnvironments(context.Context) ([]domain.Environment, error) {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
result := make([]domain.Environment, 0, len(s.environments))
|
||||
for _, item := range s.environments {
|
||||
result = append(result, item)
|
||||
}
|
||||
sort.Slice(result, func(i, j int) bool {
|
||||
if result[i].Order == result[j].Order {
|
||||
return result[i].Code < result[j].Code
|
||||
}
|
||||
return result[i].Order < result[j].Order
|
||||
})
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (s *Store) CreateEnvironment(_ context.Context, input domain.Environment) (domain.Environment, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
for _, item := range s.environments {
|
||||
if strings.EqualFold(item.Code, input.Code) {
|
||||
return domain.Environment{}, storepkg.ErrConflict
|
||||
}
|
||||
}
|
||||
input.ID = s.id()
|
||||
input.Code = strings.ToUpper(input.Code)
|
||||
s.environments[input.ID] = input
|
||||
s.audit(defaultActor(input.UpdatedBy), "create", "env", input.ID, input)
|
||||
return input, nil
|
||||
}
|
||||
|
||||
func (s *Store) UpdateEnvironment(_ context.Context, id int64, input domain.Environment) (domain.Environment, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
current, ok := s.environments[id]
|
||||
if !ok {
|
||||
return domain.Environment{}, storepkg.ErrNotFound
|
||||
}
|
||||
current.Name, current.Order, current.Desc = input.Name, input.Order, input.Desc
|
||||
s.environments[id] = current
|
||||
s.audit(defaultActor(input.UpdatedBy), "update", "env", id, current)
|
||||
return current, nil
|
||||
}
|
||||
|
||||
func (s *Store) DeleteEnvironment(_ context.Context, id int64, actor string) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if _, ok := s.environments[id]; !ok {
|
||||
return storepkg.ErrNotFound
|
||||
}
|
||||
delete(s.environments, id)
|
||||
for configID, item := range s.configs {
|
||||
if item.EnvironmentID == id {
|
||||
delete(s.configs, configID)
|
||||
}
|
||||
}
|
||||
for releaseID, item := range s.releases {
|
||||
if item.EnvironmentID == id {
|
||||
delete(s.releases, releaseID)
|
||||
}
|
||||
}
|
||||
for ruleID, rule := range s.grayRules {
|
||||
if rule.EnvironmentID == id {
|
||||
delete(s.grayRules, ruleID)
|
||||
}
|
||||
}
|
||||
s.audit(defaultActor(actor), "delete", "env", id, nil)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Store) ListNamespaces(_ context.Context, appID *int64) ([]domain.Namespace, error) {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
result := make([]domain.Namespace, 0)
|
||||
for _, item := range s.namespaces {
|
||||
if appID == nil || item.AppID == *appID {
|
||||
result = append(result, item)
|
||||
}
|
||||
}
|
||||
sort.Slice(result, func(i, j int) bool {
|
||||
if result[i].AppID == result[j].AppID {
|
||||
return result[i].Name < result[j].Name
|
||||
}
|
||||
return result[i].AppID < result[j].AppID
|
||||
})
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (s *Store) GetNamespace(_ context.Context, id int64) (domain.Namespace, error) {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
item, ok := s.namespaces[id]
|
||||
if !ok {
|
||||
return domain.Namespace{}, storepkg.ErrNotFound
|
||||
}
|
||||
return item, nil
|
||||
}
|
||||
|
||||
func (s *Store) CreateNamespace(_ context.Context, input domain.Namespace) (domain.Namespace, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if _, ok := s.applications[input.AppID]; !ok {
|
||||
return domain.Namespace{}, storepkg.ErrNotFound
|
||||
}
|
||||
for _, item := range s.namespaces {
|
||||
if item.AppID == input.AppID && strings.EqualFold(item.Name, input.Name) {
|
||||
return domain.Namespace{}, storepkg.ErrConflict
|
||||
}
|
||||
}
|
||||
input.ID = s.id()
|
||||
s.namespaces[input.ID] = input
|
||||
s.audit(defaultActor(input.UpdatedBy), "create", "namespace", input.ID, input)
|
||||
return input, nil
|
||||
}
|
||||
|
||||
func (s *Store) UpdateNamespace(_ context.Context, id int64, input domain.Namespace) (domain.Namespace, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
current, ok := s.namespaces[id]
|
||||
if !ok {
|
||||
return domain.Namespace{}, storepkg.ErrNotFound
|
||||
}
|
||||
current.Type, current.Desc = input.Type, input.Desc
|
||||
s.namespaces[id] = current
|
||||
s.audit(defaultActor(input.UpdatedBy), "update", "namespace", id, current)
|
||||
return current, nil
|
||||
}
|
||||
|
||||
func (s *Store) DeleteNamespace(_ context.Context, id int64, actor string) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if _, ok := s.namespaces[id]; !ok {
|
||||
return storepkg.ErrNotFound
|
||||
}
|
||||
delete(s.namespaces, id)
|
||||
for configID, item := range s.configs {
|
||||
if item.NamespaceID == id {
|
||||
delete(s.configs, configID)
|
||||
}
|
||||
}
|
||||
for releaseID, item := range s.releases {
|
||||
if item.NamespaceID == id {
|
||||
delete(s.releases, releaseID)
|
||||
}
|
||||
}
|
||||
for ruleID, rule := range s.grayRules {
|
||||
if rule.NamespaceID == id {
|
||||
delete(s.grayRules, ruleID)
|
||||
}
|
||||
}
|
||||
s.audit(defaultActor(actor), "delete", "namespace", id, nil)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Store) ListConfigItems(_ context.Context, appID, namespaceID, envID int64) ([]domain.ConfigItem, error) {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
result := make([]domain.ConfigItem, 0)
|
||||
for _, item := range s.configs {
|
||||
if item.AppID == appID && item.NamespaceID == namespaceID && item.EnvironmentID == envID {
|
||||
result = append(result, item)
|
||||
}
|
||||
}
|
||||
sort.Slice(result, func(i, j int) bool { return result[i].Key < result[j].Key })
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (s *Store) GetConfigItem(_ context.Context, id int64) (domain.ConfigItem, error) {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
item, ok := s.configs[id]
|
||||
if !ok {
|
||||
return domain.ConfigItem{}, storepkg.ErrNotFound
|
||||
}
|
||||
return item, nil
|
||||
}
|
||||
|
||||
func (s *Store) CreateConfigItem(_ context.Context, input domain.ConfigItem) (domain.ConfigItem, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if !s.validScope(input.AppID, input.NamespaceID, input.EnvironmentID) {
|
||||
return domain.ConfigItem{}, storepkg.ErrNotFound
|
||||
}
|
||||
for _, item := range s.configs {
|
||||
if item.NamespaceID == input.NamespaceID && item.EnvironmentID == input.EnvironmentID && item.Key == input.Key {
|
||||
return domain.ConfigItem{}, storepkg.ErrConflict
|
||||
}
|
||||
}
|
||||
input.ID, input.UpdatedAt = s.id(), time.Now().UTC()
|
||||
input.UpdatedBy = defaultActor(input.UpdatedBy)
|
||||
input.ReleasedValue = nil
|
||||
s.configs[input.ID] = input
|
||||
s.audit(input.UpdatedBy, "create", "config", input.ID, input)
|
||||
return input, nil
|
||||
}
|
||||
|
||||
func (s *Store) UpdateConfigItem(_ context.Context, id int64, input domain.ConfigItem) (domain.ConfigItem, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
current, ok := s.configs[id]
|
||||
if !ok {
|
||||
return domain.ConfigItem{}, storepkg.ErrNotFound
|
||||
}
|
||||
current.Value, current.Comment = input.Value, input.Comment
|
||||
current.UpdatedBy, current.UpdatedAt = defaultActor(input.UpdatedBy), time.Now().UTC()
|
||||
current.PendingDelete = false
|
||||
s.configs[id] = current
|
||||
s.audit(current.UpdatedBy, "update", "config", id, current)
|
||||
return current, nil
|
||||
}
|
||||
|
||||
func (s *Store) SetConfigItemPendingDelete(_ context.Context, id int64, pending bool, actor string) (domain.ConfigItem, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
current, ok := s.configs[id]
|
||||
if !ok {
|
||||
return domain.ConfigItem{}, storepkg.ErrNotFound
|
||||
}
|
||||
if current.ReleasedValue == nil && pending {
|
||||
delete(s.configs, id)
|
||||
s.audit(defaultActor(actor), "delete", "config", id, map[string]bool{"draftOnly": true})
|
||||
return current, nil
|
||||
}
|
||||
current.PendingDelete, current.UpdatedBy, current.UpdatedAt = pending, defaultActor(actor), time.Now().UTC()
|
||||
s.configs[id] = current
|
||||
s.audit(current.UpdatedBy, "update", "config", id, map[string]bool{"pendingDelete": pending})
|
||||
return current, nil
|
||||
}
|
||||
|
||||
func (s *Store) Publish(_ context.Context, request domain.PublishRequest) (domain.Release, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if !s.validScope(request.AppID, request.NamespaceID, request.EnvironmentID) {
|
||||
return domain.Release{}, storepkg.ErrNotFound
|
||||
}
|
||||
items := s.scopeItems(request.AppID, request.NamespaceID, request.EnvironmentID)
|
||||
added, modified, removed := 0, 0, 0
|
||||
snapshot := make(map[string]string)
|
||||
for _, item := range items {
|
||||
if item.PendingDelete {
|
||||
removed++
|
||||
continue
|
||||
}
|
||||
snapshot[item.Key] = item.Value
|
||||
if item.ReleasedValue == nil {
|
||||
added++
|
||||
} else if *item.ReleasedValue != item.Value {
|
||||
modified++
|
||||
}
|
||||
}
|
||||
if added+modified+removed == 0 {
|
||||
return domain.Release{}, storepkg.ErrNoPendingChange
|
||||
}
|
||||
release := s.newRelease(request.AppID, request.NamespaceID, request.EnvironmentID, snapshot, added, modified, removed, request.Comment, defaultActor(request.Operator))
|
||||
for id, item := range s.configs {
|
||||
if item.AppID != request.AppID || item.NamespaceID != request.NamespaceID || item.EnvironmentID != request.EnvironmentID {
|
||||
continue
|
||||
}
|
||||
if item.PendingDelete {
|
||||
delete(s.configs, id)
|
||||
continue
|
||||
}
|
||||
value := item.Value
|
||||
item.ReleasedValue, item.PendingDelete = &value, false
|
||||
s.configs[id] = item
|
||||
}
|
||||
s.enqueue(release)
|
||||
s.audit(release.Operator, "publish", "release", release.ID, release)
|
||||
return release, nil
|
||||
}
|
||||
|
||||
func (s *Store) Rollback(_ context.Context, request domain.RollbackRequest) (domain.Release, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
var target *domain.Release
|
||||
var latest *domain.Release
|
||||
for _, release := range s.releases {
|
||||
if release.AppID != request.AppID || release.NamespaceID != request.NamespaceID || release.EnvironmentID != request.EnvironmentID {
|
||||
continue
|
||||
}
|
||||
copy := release
|
||||
if release.Version == request.TargetVersion {
|
||||
target = ©
|
||||
}
|
||||
if latest == nil || release.Version > latest.Version {
|
||||
latest = ©
|
||||
}
|
||||
}
|
||||
if target == nil {
|
||||
return domain.Release{}, storepkg.ErrNotFound
|
||||
}
|
||||
if latest != nil && target.Version == latest.Version {
|
||||
return domain.Release{}, storepkg.ErrInvalidRollback
|
||||
}
|
||||
current := map[string]string{}
|
||||
if latest != nil {
|
||||
current = latest.Snapshot
|
||||
}
|
||||
added, modified, removed := diff(current, target.Snapshot)
|
||||
for id, item := range s.configs {
|
||||
if item.AppID == request.AppID && item.NamespaceID == request.NamespaceID && item.EnvironmentID == request.EnvironmentID {
|
||||
delete(s.configs, id)
|
||||
}
|
||||
}
|
||||
for key, value := range target.Snapshot {
|
||||
copyValue := value
|
||||
item := domain.ConfigItem{ID: s.id(), AppID: request.AppID, NamespaceID: request.NamespaceID, EnvironmentID: request.EnvironmentID, Key: key, Value: value, ReleasedValue: ©Value, UpdatedBy: defaultActor(request.Operator), UpdatedAt: time.Now().UTC()}
|
||||
s.configs[item.ID] = item
|
||||
}
|
||||
release := s.newRelease(request.AppID, request.NamespaceID, request.EnvironmentID, cloneMap(target.Snapshot), added, modified, removed, fmt.Sprintf("回滚至版本 v%d", target.Version), defaultActor(request.Operator))
|
||||
s.enqueue(release)
|
||||
s.audit(release.Operator, "rollback", "release", release.ID, map[string]any{"targetVersion": target.Version, "newVersion": release.Version})
|
||||
return release, nil
|
||||
}
|
||||
|
||||
func (s *Store) ListReleases(_ context.Context, appID, namespaceID, envID int64) ([]domain.Release, error) {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
result := make([]domain.Release, 0)
|
||||
for _, release := range s.releases {
|
||||
if release.AppID == appID && release.NamespaceID == namespaceID && release.EnvironmentID == envID {
|
||||
release.Snapshot = cloneMap(release.Snapshot)
|
||||
result = append(result, release)
|
||||
}
|
||||
}
|
||||
sort.Slice(result, func(i, j int) bool { return result[i].Version > result[j].Version })
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (s *Store) GetRelease(_ context.Context, id int64) (domain.Release, error) {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
release, ok := s.releases[id]
|
||||
if !ok {
|
||||
return domain.Release{}, storepkg.ErrNotFound
|
||||
}
|
||||
release.Snapshot = cloneMap(release.Snapshot)
|
||||
return release, nil
|
||||
}
|
||||
|
||||
func (s *Store) ListAuditLogs(_ context.Context, limit int) ([]domain.AuditLog, error) {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
if limit <= 0 || limit > len(s.audits) {
|
||||
limit = len(s.audits)
|
||||
}
|
||||
result := make([]domain.AuditLog, 0, limit)
|
||||
for i := len(s.audits) - 1; i >= len(s.audits)-limit; i-- {
|
||||
result = append(result, s.audits[i])
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (s *Store) EnsureBootstrapAdmin(_ context.Context, username, passwordHash, displayName string) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
for _, credential := range s.users {
|
||||
if strings.EqualFold(credential.Username, username) {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
now := time.Now().UTC()
|
||||
id := s.id()
|
||||
s.users[id] = domain.UserCredential{User: domain.User{ID: id, Username: username, DisplayName: displayName, IsAdmin: true, CreatedAt: now}, PasswordHash: passwordHash}
|
||||
s.audit(username, "create", "user", id, map[string]any{"bootstrap": true, "isAdmin": true})
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Store) FindUserByUsername(_ context.Context, username string) (domain.UserCredential, error) {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
for _, credential := range s.users {
|
||||
if strings.EqualFold(credential.Username, username) {
|
||||
return credential, nil
|
||||
}
|
||||
}
|
||||
return domain.UserCredential{}, storepkg.ErrNotFound
|
||||
}
|
||||
|
||||
func (s *Store) ListUsers(context.Context) ([]domain.User, error) {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
result := make([]domain.User, 0, len(s.users))
|
||||
for _, credential := range s.users {
|
||||
result = append(result, credential.User)
|
||||
}
|
||||
sort.Slice(result, func(i, j int) bool { return result[i].Username < result[j].Username })
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (s *Store) CreateUser(_ context.Context, input domain.User, passwordHash string) (domain.User, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
for _, credential := range s.users {
|
||||
if strings.EqualFold(credential.Username, input.Username) {
|
||||
return domain.User{}, storepkg.ErrConflict
|
||||
}
|
||||
}
|
||||
input.ID, input.CreatedAt = s.id(), time.Now().UTC()
|
||||
s.users[input.ID] = domain.UserCredential{User: input, PasswordHash: passwordHash}
|
||||
s.audit(defaultActor(input.UpdatedBy), "create", "user", input.ID, input)
|
||||
return input, nil
|
||||
}
|
||||
|
||||
func (s *Store) SetUserAppRole(_ context.Context, input domain.UserAppRole) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if _, ok := s.users[input.UserID]; !ok {
|
||||
return storepkg.ErrNotFound
|
||||
}
|
||||
if _, ok := s.applications[input.AppID]; !ok {
|
||||
return storepkg.ErrNotFound
|
||||
}
|
||||
if input.Role != "admin" && input.Role != "app-owner" && input.Role != "viewer" {
|
||||
return errors.New("invalid role")
|
||||
}
|
||||
s.userRoles[[2]int64{input.UserID, input.AppID}] = input.Role
|
||||
s.audit(defaultActor(input.UpdatedBy), "update", "user_app_role", input.UserID, input)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Store) DeleteUserAppRole(_ context.Context, userID, appID int64, updatedBy string) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
key := [2]int64{userID, appID}
|
||||
if _, ok := s.userRoles[key]; !ok {
|
||||
return storepkg.ErrNotFound
|
||||
}
|
||||
delete(s.userRoles, key)
|
||||
s.audit(defaultActor(updatedBy), "delete", "user_app_role", userID, map[string]int64{"appId": appID})
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Store) GetUserAppRole(_ context.Context, userID, appID int64) (string, error) {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
role, ok := s.userRoles[[2]int64{userID, appID}]
|
||||
if !ok {
|
||||
return "", storepkg.ErrNotFound
|
||||
}
|
||||
return role, nil
|
||||
}
|
||||
|
||||
func (s *Store) ListUserAppRoles(_ context.Context, userID int64) ([]domain.UserAppRole, error) {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
if _, ok := s.users[userID]; !ok {
|
||||
return nil, storepkg.ErrNotFound
|
||||
}
|
||||
result := make([]domain.UserAppRole, 0)
|
||||
for key, role := range s.userRoles {
|
||||
if key[0] == userID {
|
||||
result = append(result, domain.UserAppRole{UserID: userID, AppID: key[1], Role: role})
|
||||
}
|
||||
}
|
||||
sort.Slice(result, func(i, j int) bool { return result[i].AppID < result[j].AppID })
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (s *Store) ResolveScope(_ context.Context, envCode, appCode, namespaceName string) (int64, int64, int64, error) {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
var appID, namespaceID, envID int64
|
||||
for id, item := range s.applications {
|
||||
if item.Code == appCode {
|
||||
appID = id
|
||||
break
|
||||
}
|
||||
}
|
||||
for id, item := range s.environments {
|
||||
if strings.EqualFold(item.Code, envCode) {
|
||||
envID = id
|
||||
break
|
||||
}
|
||||
}
|
||||
for id, item := range s.namespaces {
|
||||
if item.AppID == appID && item.Name == namespaceName {
|
||||
namespaceID = id
|
||||
break
|
||||
}
|
||||
}
|
||||
if appID == 0 || namespaceID == 0 || envID == 0 {
|
||||
return 0, 0, 0, storepkg.ErrNotFound
|
||||
}
|
||||
return appID, namespaceID, envID, nil
|
||||
}
|
||||
|
||||
func (s *Store) ListGrayRules(_ context.Context, appID, namespaceID, envID int64) ([]domain.GrayRule, error) {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
if !s.validScope(appID, namespaceID, envID) {
|
||||
return nil, storepkg.ErrNotFound
|
||||
}
|
||||
result := make([]domain.GrayRule, 0)
|
||||
for _, rule := range s.grayRules {
|
||||
if rule.AppID == appID && rule.NamespaceID == namespaceID && rule.EnvironmentID == envID {
|
||||
result = append(result, cloneGrayRule(rule))
|
||||
}
|
||||
}
|
||||
sort.Slice(result, func(i, j int) bool {
|
||||
if result[i].Priority == result[j].Priority {
|
||||
return result[i].ID < result[j].ID
|
||||
}
|
||||
return result[i].Priority < result[j].Priority
|
||||
})
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (s *Store) GetGrayRule(_ context.Context, id int64) (domain.GrayRule, error) {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
rule, ok := s.grayRules[id]
|
||||
if !ok {
|
||||
return domain.GrayRule{}, storepkg.ErrNotFound
|
||||
}
|
||||
return cloneGrayRule(rule), nil
|
||||
}
|
||||
|
||||
func (s *Store) CreateGrayRule(_ context.Context, input domain.GrayRule, updatedBy string) (domain.GrayRule, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if !s.validScope(input.AppID, input.NamespaceID, input.EnvironmentID) {
|
||||
return domain.GrayRule{}, storepkg.ErrNotFound
|
||||
}
|
||||
input.ID, input.CreatedAt = s.id(), time.Now().UTC()
|
||||
input = cloneGrayRule(input)
|
||||
s.grayRules[input.ID] = input
|
||||
s.audit(defaultActor(updatedBy), "create", "gray_rule", input.ID, input)
|
||||
return cloneGrayRule(input), nil
|
||||
}
|
||||
|
||||
func (s *Store) UpdateGrayRule(_ context.Context, id int64, input domain.GrayRule, updatedBy string) (domain.GrayRule, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
current, ok := s.grayRules[id]
|
||||
if !ok {
|
||||
return domain.GrayRule{}, storepkg.ErrNotFound
|
||||
}
|
||||
current.RuleType, current.RuleValue, current.Overrides = input.RuleType, append(json.RawMessage(nil), input.RuleValue...), cloneMap(input.Overrides)
|
||||
current.Enabled, current.Priority, current.Desc = input.Enabled, input.Priority, input.Desc
|
||||
s.grayRules[id] = current
|
||||
s.audit(defaultActor(updatedBy), "update", "gray_rule", id, current)
|
||||
return cloneGrayRule(current), nil
|
||||
}
|
||||
|
||||
func (s *Store) DeleteGrayRule(_ context.Context, id int64, updatedBy string) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if _, ok := s.grayRules[id]; !ok {
|
||||
return storepkg.ErrNotFound
|
||||
}
|
||||
delete(s.grayRules, id)
|
||||
s.audit(defaultActor(updatedBy), "delete", "gray_rule", id, nil)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Store) ClaimOutbox(_ context.Context, limit int) ([]domain.OutboxEntry, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
now := time.Now().UTC()
|
||||
ids := make([]int64, 0)
|
||||
for id, record := range s.outbox {
|
||||
if record.Status == "pending" && !record.NextAttempt.After(now) {
|
||||
ids = append(ids, id)
|
||||
}
|
||||
}
|
||||
sort.Slice(ids, func(i, j int) bool { return ids[i] < ids[j] })
|
||||
if len(ids) > limit {
|
||||
ids = ids[:limit]
|
||||
}
|
||||
result := make([]domain.OutboxEntry, 0, len(ids))
|
||||
for _, id := range ids {
|
||||
record := s.outbox[id]
|
||||
record.Status = "processing"
|
||||
s.outbox[id] = record
|
||||
result = append(result, record.OutboxEntry)
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (s *Store) MarkOutboxDone(_ context.Context, id, releaseID, revision int64) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
record, ok := s.outbox[id]
|
||||
if !ok {
|
||||
return storepkg.ErrNotFound
|
||||
}
|
||||
record.Status = "done"
|
||||
s.outbox[id] = record
|
||||
release, ok := s.releases[releaseID]
|
||||
if !ok {
|
||||
return storepkg.ErrNotFound
|
||||
}
|
||||
release.Status, release.EtcdRevision = "applied", &revision
|
||||
s.releases[releaseID] = release
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Store) MarkOutboxFailed(_ context.Context, id, releaseID int64, _ string, maxRetry int) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
record, ok := s.outbox[id]
|
||||
if !ok {
|
||||
return storepkg.ErrNotFound
|
||||
}
|
||||
record.RetryCount++
|
||||
if record.RetryCount >= maxRetry {
|
||||
record.Status = "failed"
|
||||
if release, exists := s.releases[releaseID]; exists {
|
||||
release.Status = "failed"
|
||||
s.releases[releaseID] = release
|
||||
}
|
||||
} else {
|
||||
record.Status = "pending"
|
||||
delay := time.Duration(1<<min(record.RetryCount, 6)) * time.Second
|
||||
record.NextAttempt = time.Now().UTC().Add(delay)
|
||||
}
|
||||
s.outbox[id] = record
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Store) OutboxStats(context.Context) (domain.OutboxStats, error) {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
var result domain.OutboxStats
|
||||
for _, record := range s.outbox {
|
||||
switch record.Status {
|
||||
case "pending":
|
||||
result.Pending++
|
||||
case "processing":
|
||||
result.Processing++
|
||||
case "failed":
|
||||
result.Failed++
|
||||
}
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (s *Store) validScope(appID, namespaceID, envID int64) bool {
|
||||
ns, ok := s.namespaces[namespaceID]
|
||||
_, envExists := s.environments[envID]
|
||||
_, appExists := s.applications[appID]
|
||||
return ok && ns.AppID == appID && envExists && appExists
|
||||
}
|
||||
|
||||
func (s *Store) scopeItems(appID, namespaceID, envID int64) []domain.ConfigItem {
|
||||
result := make([]domain.ConfigItem, 0)
|
||||
for _, item := range s.configs {
|
||||
if item.AppID == appID && item.NamespaceID == namespaceID && item.EnvironmentID == envID {
|
||||
result = append(result, item)
|
||||
}
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
func (s *Store) newRelease(appID, namespaceID, envID int64, snapshot map[string]string, added, modified, removed int, comment, operator string) domain.Release {
|
||||
version := 1
|
||||
for _, release := range s.releases {
|
||||
if release.NamespaceID == namespaceID && release.EnvironmentID == envID && release.Version >= version {
|
||||
version = release.Version + 1
|
||||
}
|
||||
}
|
||||
release := domain.Release{ID: s.id(), AppID: appID, NamespaceID: namespaceID, EnvironmentID: envID, Version: version, Snapshot: snapshot, Added: added, Modified: modified, Removed: removed, Comment: comment, Operator: operator, Status: "pending", Time: time.Now().UTC()}
|
||||
s.releases[release.ID] = release
|
||||
return release
|
||||
}
|
||||
|
||||
func (s *Store) enqueue(release domain.Release) {
|
||||
app := s.applications[release.AppID]
|
||||
ns := s.namespaces[release.NamespaceID]
|
||||
env := s.environments[release.EnvironmentID]
|
||||
payload, _ := json.Marshal(release.Snapshot)
|
||||
id := s.id()
|
||||
s.outbox[id] = outboxRecord{OutboxEntry: domain.OutboxEntry{ID: id, ReleaseID: release.ID, EtcdKey: fmt.Sprintf("/config/%s/%s/%s", strings.ToLower(env.Code), app.Code, ns.Name), Payload: payload}, Status: "pending"}
|
||||
}
|
||||
|
||||
func (s *Store) audit(actor, action, targetType string, targetID int64, detail any) {
|
||||
payload, _ := json.Marshal(detail)
|
||||
id := s.id()
|
||||
target := targetID
|
||||
s.audits = append(s.audits, domain.AuditLog{ID: id, Actor: defaultActor(actor), Action: action, TargetType: targetType, TargetID: &target, Detail: payload, CreatedAt: time.Now().UTC()})
|
||||
}
|
||||
|
||||
func diff(current, target map[string]string) (added, modified, removed int) {
|
||||
for key, value := range target {
|
||||
old, ok := current[key]
|
||||
if !ok {
|
||||
added++
|
||||
} else if old != value {
|
||||
modified++
|
||||
}
|
||||
}
|
||||
for key := range current {
|
||||
if _, ok := target[key]; !ok {
|
||||
removed++
|
||||
}
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
func cloneMap(input map[string]string) map[string]string {
|
||||
result := make(map[string]string, len(input))
|
||||
for key, value := range input {
|
||||
result[key] = value
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
func cloneGrayRule(input domain.GrayRule) domain.GrayRule {
|
||||
input.RuleValue = append(json.RawMessage(nil), input.RuleValue...)
|
||||
input.Overrides = cloneMap(input.Overrides)
|
||||
return input
|
||||
}
|
||||
|
||||
func defaultActor(actor string) string {
|
||||
if strings.TrimSpace(actor) == "" {
|
||||
return "admin"
|
||||
}
|
||||
return strings.TrimSpace(actor)
|
||||
}
|
||||
80
internal/store/memory/memory_test.go
Normal file
80
internal/store/memory/memory_test.go
Normal file
@@ -0,0 +1,80 @@
|
||||
package memory
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"testing"
|
||||
|
||||
"github.com/longpeng/configcenter/internal/domain"
|
||||
storepkg "github.com/longpeng/configcenter/internal/store"
|
||||
)
|
||||
|
||||
func TestPublishCreatesVersionSnapshotAndOutbox(t *testing.T) {
|
||||
repository := New(false)
|
||||
ctx := context.Background()
|
||||
app, err := repository.CreateApplication(ctx, domain.Application{Code: "orders", Name: "Orders"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
env, err := repository.CreateEnvironment(ctx, domain.Environment{Code: "PROD", Name: "Production", Order: 1})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ns, err := repository.CreateNamespace(ctx, domain.Namespace{AppID: app.ID, Name: "application", Type: "properties"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
item, err := repository.CreateConfigItem(ctx, domain.ConfigItem{AppID: app.ID, NamespaceID: ns.ID, EnvironmentID: env.ID, Key: "server.port", Value: "8080", UpdatedBy: "alice"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
release, err := repository.Publish(ctx, domain.PublishRequest{AppID: app.ID, NamespaceID: ns.ID, EnvironmentID: env.ID, Comment: "first", Operator: "alice"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if release.Version != 1 || release.Added != 1 || release.Snapshot["server.port"] != "8080" || release.Status != "pending" {
|
||||
t.Fatalf("unexpected release: %#v", release)
|
||||
}
|
||||
items, err := repository.ListConfigItems(ctx, app.ID, ns.ID, env.ID)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(items) != 1 || items[0].ReleasedValue == nil || *items[0].ReleasedValue != item.Value {
|
||||
t.Fatalf("published item was not marked released: %#v", items)
|
||||
}
|
||||
entries, err := repository.ClaimOutbox(ctx, 10)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(entries) != 1 || entries[0].ReleaseID != release.ID || entries[0].EtcdKey != "/config/prod/orders/application" {
|
||||
t.Fatalf("unexpected outbox: %#v", entries)
|
||||
}
|
||||
if _, err := repository.Publish(ctx, domain.PublishRequest{AppID: app.ID, NamespaceID: ns.ID, EnvironmentID: env.ID}); !errors.Is(err, storepkg.ErrNoPendingChange) {
|
||||
t.Fatalf("expected no pending changes, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRollbackCreatesANewRelease(t *testing.T) {
|
||||
repository := New(false)
|
||||
ctx := context.Background()
|
||||
app, _ := repository.CreateApplication(ctx, domain.Application{Code: "orders", Name: "Orders"})
|
||||
env, _ := repository.CreateEnvironment(ctx, domain.Environment{Code: "PROD", Name: "Production"})
|
||||
ns, _ := repository.CreateNamespace(ctx, domain.Namespace{AppID: app.ID, Name: "application", Type: "properties"})
|
||||
item, _ := repository.CreateConfigItem(ctx, domain.ConfigItem{AppID: app.ID, NamespaceID: ns.ID, EnvironmentID: env.ID, Key: "feature", Value: "off"})
|
||||
first, _ := repository.Publish(ctx, domain.PublishRequest{AppID: app.ID, NamespaceID: ns.ID, EnvironmentID: env.ID, Operator: "alice"})
|
||||
_, _ = repository.UpdateConfigItem(ctx, item.ID, domain.ConfigItem{Value: "on", UpdatedBy: "bob"})
|
||||
second, _ := repository.Publish(ctx, domain.PublishRequest{AppID: app.ID, NamespaceID: ns.ID, EnvironmentID: env.ID, Operator: "bob"})
|
||||
|
||||
rolledBack, err := repository.Rollback(ctx, domain.RollbackRequest{AppID: app.ID, NamespaceID: ns.ID, EnvironmentID: env.ID, TargetVersion: first.Version, Operator: "carol"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if rolledBack.Version != second.Version+1 || rolledBack.Snapshot["feature"] != "off" || rolledBack.Modified != 1 {
|
||||
t.Fatalf("unexpected rollback release: %#v", rolledBack)
|
||||
}
|
||||
releases, _ := repository.ListReleases(ctx, app.ID, ns.ID, env.ID)
|
||||
if len(releases) != 3 {
|
||||
t.Fatalf("rollback must retain history, got %d releases", len(releases))
|
||||
}
|
||||
}
|
||||
134
internal/store/postgres/migrations/001_init.sql
Normal file
134
internal/store/postgres/migrations/001_init.sql
Normal file
@@ -0,0 +1,134 @@
|
||||
CREATE TABLE IF NOT EXISTS applications (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
app_code VARCHAR(64) UNIQUE NOT NULL,
|
||||
name VARCHAR(128) NOT NULL,
|
||||
department VARCHAR(128) NOT NULL DEFAULT '',
|
||||
owner VARCHAR(64) NOT NULL DEFAULT '',
|
||||
description TEXT NOT NULL DEFAULT '',
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS environments (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
env_code VARCHAR(32) UNIQUE NOT NULL,
|
||||
name VARCHAR(64) NOT NULL,
|
||||
sort_order INT NOT NULL DEFAULT 0,
|
||||
description TEXT NOT NULL DEFAULT ''
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS namespaces (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
app_id BIGINT NOT NULL REFERENCES applications(id) ON DELETE CASCADE,
|
||||
name VARCHAR(64) NOT NULL,
|
||||
format VARCHAR(16) NOT NULL DEFAULT 'properties',
|
||||
description TEXT NOT NULL DEFAULT '',
|
||||
UNIQUE(app_id, name),
|
||||
CONSTRAINT namespaces_format_check CHECK (format IN ('properties', 'yaml', 'json', 'xml'))
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS config_items (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
namespace_id BIGINT NOT NULL REFERENCES namespaces(id) ON DELETE CASCADE,
|
||||
env_id BIGINT NOT NULL REFERENCES environments(id) ON DELETE CASCADE,
|
||||
key VARCHAR(256) NOT NULL,
|
||||
value TEXT NOT NULL,
|
||||
released_value TEXT,
|
||||
pending_delete BOOLEAN NOT NULL DEFAULT false,
|
||||
comment TEXT NOT NULL DEFAULT '',
|
||||
updated_by VARCHAR(64) NOT NULL DEFAULT 'admin',
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
UNIQUE(namespace_id, env_id, key)
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS releases (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
namespace_id BIGINT NOT NULL REFERENCES namespaces(id) ON DELETE CASCADE,
|
||||
env_id BIGINT NOT NULL REFERENCES environments(id) ON DELETE CASCADE,
|
||||
version INT NOT NULL,
|
||||
snapshot JSONB NOT NULL,
|
||||
diff_added INT NOT NULL DEFAULT 0,
|
||||
diff_modified INT NOT NULL DEFAULT 0,
|
||||
diff_removed INT NOT NULL DEFAULT 0,
|
||||
comment TEXT NOT NULL DEFAULT '',
|
||||
operator VARCHAR(64) NOT NULL,
|
||||
etcd_revision BIGINT,
|
||||
status VARCHAR(16) NOT NULL DEFAULT 'pending',
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
UNIQUE(namespace_id, env_id, version),
|
||||
CONSTRAINT releases_status_check CHECK (status IN ('pending', 'applied', 'failed'))
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS release_outbox (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
release_id BIGINT NOT NULL REFERENCES releases(id) ON DELETE CASCADE,
|
||||
etcd_key VARCHAR(512) NOT NULL,
|
||||
payload JSONB NOT NULL,
|
||||
status VARCHAR(16) NOT NULL DEFAULT 'pending',
|
||||
retry_count INT NOT NULL DEFAULT 0,
|
||||
last_error TEXT NOT NULL DEFAULT '',
|
||||
next_attempt_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
locked_at TIMESTAMPTZ,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
CONSTRAINT outbox_status_check CHECK (status IN ('pending', 'processing', 'done', 'failed'))
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS release_outbox_poll_idx
|
||||
ON release_outbox(status, next_attempt_at, id);
|
||||
CREATE INDEX IF NOT EXISTS releases_scope_idx
|
||||
ON releases(namespace_id, env_id, version DESC);
|
||||
CREATE INDEX IF NOT EXISTS config_items_scope_idx
|
||||
ON config_items(namespace_id, env_id, key);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS users (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
username VARCHAR(64) UNIQUE NOT NULL,
|
||||
password_hash VARCHAR(256) NOT NULL,
|
||||
display_name VARCHAR(64) NOT NULL DEFAULT '',
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS roles (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
name VARCHAR(32) UNIQUE NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS user_app_roles (
|
||||
user_id BIGINT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
app_id BIGINT NOT NULL REFERENCES applications(id) ON DELETE CASCADE,
|
||||
role_id BIGINT NOT NULL REFERENCES roles(id) ON DELETE CASCADE,
|
||||
PRIMARY KEY (user_id, app_id)
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS audit_logs (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
actor VARCHAR(64) NOT NULL,
|
||||
action VARCHAR(32) NOT NULL,
|
||||
target_type VARCHAR(32) NOT NULL,
|
||||
target_id BIGINT,
|
||||
detail JSONB,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS audit_logs_created_idx ON audit_logs(created_at DESC);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS gray_rules (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
namespace_id BIGINT NOT NULL REFERENCES namespaces(id) ON DELETE CASCADE,
|
||||
env_id BIGINT NOT NULL REFERENCES environments(id) ON DELETE CASCADE,
|
||||
rule_type VARCHAR(16) NOT NULL,
|
||||
rule_value JSONB NOT NULL,
|
||||
overrides JSONB NOT NULL,
|
||||
enabled BOOLEAN NOT NULL DEFAULT true,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
INSERT INTO environments(env_code, name, sort_order, description) VALUES
|
||||
('DEV', '开发环境', 1, '开发联调使用'),
|
||||
('TEST', '测试环境', 2, 'QA 测试验证'),
|
||||
('STAGING', '预发环境', 3, '上线前灰度验证'),
|
||||
('PROD', '生产环境', 4, '线上正式环境')
|
||||
ON CONFLICT (env_code) DO NOTHING;
|
||||
|
||||
INSERT INTO roles(name) VALUES ('admin'), ('app-owner'), ('viewer')
|
||||
ON CONFLICT (name) DO NOTHING;
|
||||
10
internal/store/postgres/migrations/002_auth_gray.sql
Normal file
10
internal/store/postgres/migrations/002_auth_gray.sql
Normal file
@@ -0,0 +1,10 @@
|
||||
ALTER TABLE users ADD COLUMN IF NOT EXISTS is_admin BOOLEAN NOT NULL DEFAULT false;
|
||||
ALTER TABLE users ADD COLUMN IF NOT EXISTS disabled BOOLEAN NOT NULL DEFAULT false;
|
||||
|
||||
CREATE INDEX IF NOT EXISTS user_app_roles_app_idx ON user_app_roles(app_id, user_id);
|
||||
|
||||
ALTER TABLE gray_rules ADD COLUMN IF NOT EXISTS priority INT NOT NULL DEFAULT 0;
|
||||
ALTER TABLE gray_rules ADD COLUMN IF NOT EXISTS description TEXT NOT NULL DEFAULT '';
|
||||
|
||||
CREATE INDEX IF NOT EXISTS gray_rules_scope_idx
|
||||
ON gray_rules(namespace_id, env_id, enabled, priority, id);
|
||||
930
internal/store/postgres/postgres.go
Normal file
930
internal/store/postgres/postgres.go
Normal file
@@ -0,0 +1,930 @@
|
||||
package postgres
|
||||
|
||||
import (
|
||||
"context"
|
||||
"embed"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/jackc/pgx/v5"
|
||||
"github.com/jackc/pgx/v5/pgconn"
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
"github.com/longpeng/configcenter/internal/domain"
|
||||
storepkg "github.com/longpeng/configcenter/internal/store"
|
||||
)
|
||||
|
||||
//go:embed migrations/*.sql
|
||||
var migrations embed.FS
|
||||
|
||||
type Store struct {
|
||||
pool *pgxpool.Pool
|
||||
}
|
||||
|
||||
func Open(ctx context.Context, databaseURL string) (*Store, error) {
|
||||
pool, err := pgxpool.New(ctx, databaseURL)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("create postgres pool: %w", err)
|
||||
}
|
||||
if err := pool.Ping(ctx); err != nil {
|
||||
pool.Close()
|
||||
return nil, fmt.Errorf("ping postgres: %w", err)
|
||||
}
|
||||
return &Store{pool: pool}, nil
|
||||
}
|
||||
|
||||
func (s *Store) Close() { s.pool.Close() }
|
||||
|
||||
func (s *Store) Ping(ctx context.Context) error { return s.pool.Ping(ctx) }
|
||||
|
||||
func (s *Store) Migrate(ctx context.Context) error {
|
||||
entries, err := migrations.ReadDir("migrations")
|
||||
if err != nil {
|
||||
return fmt.Errorf("list migrations: %w", err)
|
||||
}
|
||||
for _, entry := range entries {
|
||||
if entry.IsDir() || !strings.HasSuffix(entry.Name(), ".sql") {
|
||||
continue
|
||||
}
|
||||
content, err := migrations.ReadFile("migrations/" + entry.Name())
|
||||
if err != nil {
|
||||
return fmt.Errorf("read migration %s: %w", entry.Name(), err)
|
||||
}
|
||||
if _, err := s.pool.Exec(ctx, string(content)); err != nil {
|
||||
return fmt.Errorf("apply migration %s: %w", entry.Name(), err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Store) ListApplications(ctx context.Context) ([]domain.Application, error) {
|
||||
rows, err := s.pool.Query(ctx, `SELECT id, app_code, name, department, owner, description, created_at, updated_at FROM applications ORDER BY app_code`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
result := make([]domain.Application, 0)
|
||||
for rows.Next() {
|
||||
var item domain.Application
|
||||
if err := rows.Scan(&item.ID, &item.Code, &item.Name, &item.Department, &item.Owner, &item.Desc, &item.CreatedAt, &item.UpdatedAt); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result = append(result, item)
|
||||
}
|
||||
return result, rows.Err()
|
||||
}
|
||||
|
||||
func (s *Store) CreateApplication(ctx context.Context, input domain.Application) (domain.Application, error) {
|
||||
err := s.pool.QueryRow(ctx, `
|
||||
INSERT INTO applications(app_code, name, department, owner, description)
|
||||
VALUES($1, $2, $3, $4, $5)
|
||||
RETURNING id, app_code, name, department, owner, description, created_at, updated_at`,
|
||||
input.Code, input.Name, input.Department, input.Owner, input.Desc,
|
||||
).Scan(&input.ID, &input.Code, &input.Name, &input.Department, &input.Owner, &input.Desc, &input.CreatedAt, &input.UpdatedAt)
|
||||
if err != nil {
|
||||
return domain.Application{}, classify(err)
|
||||
}
|
||||
s.audit(ctx, actor(input.UpdatedBy), "create", "app", input.ID, input)
|
||||
return input, nil
|
||||
}
|
||||
|
||||
func (s *Store) UpdateApplication(ctx context.Context, id int64, input domain.Application) (domain.Application, error) {
|
||||
err := s.pool.QueryRow(ctx, `
|
||||
UPDATE applications SET name=$2, department=$3, owner=$4, description=$5, updated_at=now()
|
||||
WHERE id=$1
|
||||
RETURNING id, app_code, name, department, owner, description, created_at, updated_at`,
|
||||
id, input.Name, input.Department, input.Owner, input.Desc,
|
||||
).Scan(&input.ID, &input.Code, &input.Name, &input.Department, &input.Owner, &input.Desc, &input.CreatedAt, &input.UpdatedAt)
|
||||
if err != nil {
|
||||
return domain.Application{}, classify(err)
|
||||
}
|
||||
s.audit(ctx, actor(input.UpdatedBy), "update", "app", id, input)
|
||||
return input, nil
|
||||
}
|
||||
|
||||
func (s *Store) DeleteApplication(ctx context.Context, id int64, actor string) error {
|
||||
return s.deleteAndAudit(ctx, "applications", "app", id, actor)
|
||||
}
|
||||
|
||||
func (s *Store) ListEnvironments(ctx context.Context) ([]domain.Environment, error) {
|
||||
rows, err := s.pool.Query(ctx, `SELECT id, env_code, name, sort_order, description FROM environments ORDER BY sort_order, env_code`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
result := make([]domain.Environment, 0)
|
||||
for rows.Next() {
|
||||
var item domain.Environment
|
||||
if err := rows.Scan(&item.ID, &item.Code, &item.Name, &item.Order, &item.Desc); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result = append(result, item)
|
||||
}
|
||||
return result, rows.Err()
|
||||
}
|
||||
|
||||
func (s *Store) CreateEnvironment(ctx context.Context, input domain.Environment) (domain.Environment, error) {
|
||||
input.Code = strings.ToUpper(input.Code)
|
||||
err := s.pool.QueryRow(ctx, `
|
||||
INSERT INTO environments(env_code, name, sort_order, description) VALUES($1, $2, $3, $4)
|
||||
RETURNING id, env_code, name, sort_order, description`, input.Code, input.Name, input.Order, input.Desc,
|
||||
).Scan(&input.ID, &input.Code, &input.Name, &input.Order, &input.Desc)
|
||||
if err != nil {
|
||||
return domain.Environment{}, classify(err)
|
||||
}
|
||||
s.audit(ctx, actor(input.UpdatedBy), "create", "env", input.ID, input)
|
||||
return input, nil
|
||||
}
|
||||
|
||||
func (s *Store) UpdateEnvironment(ctx context.Context, id int64, input domain.Environment) (domain.Environment, error) {
|
||||
err := s.pool.QueryRow(ctx, `
|
||||
UPDATE environments SET name=$2, sort_order=$3, description=$4 WHERE id=$1
|
||||
RETURNING id, env_code, name, sort_order, description`, id, input.Name, input.Order, input.Desc,
|
||||
).Scan(&input.ID, &input.Code, &input.Name, &input.Order, &input.Desc)
|
||||
if err != nil {
|
||||
return domain.Environment{}, classify(err)
|
||||
}
|
||||
s.audit(ctx, actor(input.UpdatedBy), "update", "env", id, input)
|
||||
return input, nil
|
||||
}
|
||||
|
||||
func (s *Store) DeleteEnvironment(ctx context.Context, id int64, actor string) error {
|
||||
return s.deleteAndAudit(ctx, "environments", "env", id, actor)
|
||||
}
|
||||
|
||||
func (s *Store) ListNamespaces(ctx context.Context, appID *int64) ([]domain.Namespace, error) {
|
||||
query := `SELECT id, app_id, name, format, description FROM namespaces`
|
||||
args := []any{}
|
||||
if appID != nil {
|
||||
query += ` WHERE app_id=$1`
|
||||
args = append(args, *appID)
|
||||
}
|
||||
query += ` ORDER BY app_id, name`
|
||||
rows, err := s.pool.Query(ctx, query, args...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
result := make([]domain.Namespace, 0)
|
||||
for rows.Next() {
|
||||
var item domain.Namespace
|
||||
if err := rows.Scan(&item.ID, &item.AppID, &item.Name, &item.Type, &item.Desc); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result = append(result, item)
|
||||
}
|
||||
return result, rows.Err()
|
||||
}
|
||||
|
||||
func (s *Store) GetNamespace(ctx context.Context, id int64) (domain.Namespace, error) {
|
||||
var item domain.Namespace
|
||||
err := s.pool.QueryRow(ctx, `SELECT id, app_id, name, format, description FROM namespaces WHERE id=$1`, id).
|
||||
Scan(&item.ID, &item.AppID, &item.Name, &item.Type, &item.Desc)
|
||||
return item, classify(err)
|
||||
}
|
||||
|
||||
func (s *Store) CreateNamespace(ctx context.Context, input domain.Namespace) (domain.Namespace, error) {
|
||||
err := s.pool.QueryRow(ctx, `
|
||||
INSERT INTO namespaces(app_id, name, format, description) VALUES($1, $2, $3, $4)
|
||||
RETURNING id, app_id, name, format, description`, input.AppID, input.Name, input.Type, input.Desc,
|
||||
).Scan(&input.ID, &input.AppID, &input.Name, &input.Type, &input.Desc)
|
||||
if err != nil {
|
||||
return domain.Namespace{}, classify(err)
|
||||
}
|
||||
s.audit(ctx, actor(input.UpdatedBy), "create", "namespace", input.ID, input)
|
||||
return input, nil
|
||||
}
|
||||
|
||||
func (s *Store) UpdateNamespace(ctx context.Context, id int64, input domain.Namespace) (domain.Namespace, error) {
|
||||
err := s.pool.QueryRow(ctx, `
|
||||
UPDATE namespaces SET format=$2, description=$3 WHERE id=$1
|
||||
RETURNING id, app_id, name, format, description`, id, input.Type, input.Desc,
|
||||
).Scan(&input.ID, &input.AppID, &input.Name, &input.Type, &input.Desc)
|
||||
if err != nil {
|
||||
return domain.Namespace{}, classify(err)
|
||||
}
|
||||
s.audit(ctx, actor(input.UpdatedBy), "update", "namespace", id, input)
|
||||
return input, nil
|
||||
}
|
||||
|
||||
func (s *Store) DeleteNamespace(ctx context.Context, id int64, actor string) error {
|
||||
return s.deleteAndAudit(ctx, "namespaces", "namespace", id, actor)
|
||||
}
|
||||
|
||||
func (s *Store) ListConfigItems(ctx context.Context, appID, namespaceID, envID int64) ([]domain.ConfigItem, error) {
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT ci.id, n.app_id, ci.namespace_id, ci.env_id, ci.key, ci.value, ci.released_value,
|
||||
ci.pending_delete, ci.comment, ci.updated_by, ci.updated_at
|
||||
FROM config_items ci JOIN namespaces n ON n.id=ci.namespace_id
|
||||
WHERE n.app_id=$1 AND ci.namespace_id=$2 AND ci.env_id=$3 ORDER BY ci.key`, appID, namespaceID, envID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
result := make([]domain.ConfigItem, 0)
|
||||
for rows.Next() {
|
||||
item, err := scanConfig(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result = append(result, item)
|
||||
}
|
||||
return result, rows.Err()
|
||||
}
|
||||
|
||||
func (s *Store) GetConfigItem(ctx context.Context, id int64) (domain.ConfigItem, error) {
|
||||
row := s.pool.QueryRow(ctx, `
|
||||
SELECT ci.id, n.app_id, ci.namespace_id, ci.env_id, ci.key, ci.value, ci.released_value,
|
||||
ci.pending_delete, ci.comment, ci.updated_by, ci.updated_at
|
||||
FROM config_items ci JOIN namespaces n ON n.id=ci.namespace_id WHERE ci.id=$1`, id)
|
||||
item, err := scanConfig(row)
|
||||
return item, classify(err)
|
||||
}
|
||||
|
||||
func (s *Store) CreateConfigItem(ctx context.Context, input domain.ConfigItem) (domain.ConfigItem, error) {
|
||||
err := s.pool.QueryRow(ctx, `
|
||||
INSERT INTO config_items(namespace_id, env_id, key, value, comment, updated_by)
|
||||
SELECT n.id, $3, $4, $5, $6, $7 FROM namespaces n WHERE n.id=$2 AND n.app_id=$1
|
||||
RETURNING id, namespace_id, env_id, key, value, released_value, pending_delete, comment, updated_by, updated_at`,
|
||||
input.AppID, input.NamespaceID, input.EnvironmentID, input.Key, input.Value, input.Comment, actor(input.UpdatedBy),
|
||||
).Scan(&input.ID, &input.NamespaceID, &input.EnvironmentID, &input.Key, &input.Value, &input.ReleasedValue, &input.PendingDelete, &input.Comment, &input.UpdatedBy, &input.UpdatedAt)
|
||||
if err != nil {
|
||||
return domain.ConfigItem{}, classify(err)
|
||||
}
|
||||
s.audit(ctx, input.UpdatedBy, "create", "config", input.ID, input)
|
||||
return input, nil
|
||||
}
|
||||
|
||||
func (s *Store) UpdateConfigItem(ctx context.Context, id int64, input domain.ConfigItem) (domain.ConfigItem, error) {
|
||||
err := s.pool.QueryRow(ctx, `
|
||||
UPDATE config_items SET value=$2, comment=$3, updated_by=$4, updated_at=now(), pending_delete=false WHERE id=$1
|
||||
RETURNING id, namespace_id, env_id, key, value, released_value, pending_delete, comment, updated_by, updated_at`,
|
||||
id, input.Value, input.Comment, actor(input.UpdatedBy),
|
||||
).Scan(&input.ID, &input.NamespaceID, &input.EnvironmentID, &input.Key, &input.Value, &input.ReleasedValue, &input.PendingDelete, &input.Comment, &input.UpdatedBy, &input.UpdatedAt)
|
||||
if err != nil {
|
||||
return domain.ConfigItem{}, classify(err)
|
||||
}
|
||||
if err := s.pool.QueryRow(ctx, `SELECT app_id FROM namespaces WHERE id=$1`, input.NamespaceID).Scan(&input.AppID); err != nil {
|
||||
return domain.ConfigItem{}, err
|
||||
}
|
||||
s.audit(ctx, input.UpdatedBy, "update", "config", id, input)
|
||||
return input, nil
|
||||
}
|
||||
|
||||
func (s *Store) SetConfigItemPendingDelete(ctx context.Context, id int64, pending bool, updatedBy string) (domain.ConfigItem, error) {
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return domain.ConfigItem{}, err
|
||||
}
|
||||
defer tx.Rollback(ctx) //nolint:errcheck
|
||||
var released *string
|
||||
if err := tx.QueryRow(ctx, `SELECT released_value FROM config_items WHERE id=$1 FOR UPDATE`, id).Scan(&released); err != nil {
|
||||
return domain.ConfigItem{}, classify(err)
|
||||
}
|
||||
if released == nil && pending {
|
||||
if _, err := tx.Exec(ctx, `DELETE FROM config_items WHERE id=$1`, id); err != nil {
|
||||
return domain.ConfigItem{}, err
|
||||
}
|
||||
if err := insertAudit(ctx, tx, actor(updatedBy), "delete", "config", id, map[string]bool{"draftOnly": true}); err != nil {
|
||||
return domain.ConfigItem{}, err
|
||||
}
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return domain.ConfigItem{}, err
|
||||
}
|
||||
return domain.ConfigItem{ID: id}, nil
|
||||
}
|
||||
var item domain.ConfigItem
|
||||
err = tx.QueryRow(ctx, `
|
||||
UPDATE config_items SET pending_delete=$2, updated_by=$3, updated_at=now() WHERE id=$1
|
||||
RETURNING id, namespace_id, env_id, key, value, released_value, pending_delete, comment, updated_by, updated_at`,
|
||||
id, pending, actor(updatedBy),
|
||||
).Scan(&item.ID, &item.NamespaceID, &item.EnvironmentID, &item.Key, &item.Value, &item.ReleasedValue, &item.PendingDelete, &item.Comment, &item.UpdatedBy, &item.UpdatedAt)
|
||||
if err != nil {
|
||||
return domain.ConfigItem{}, classify(err)
|
||||
}
|
||||
if err := tx.QueryRow(ctx, `SELECT app_id FROM namespaces WHERE id=$1`, item.NamespaceID).Scan(&item.AppID); err != nil {
|
||||
return domain.ConfigItem{}, err
|
||||
}
|
||||
if err := insertAudit(ctx, tx, item.UpdatedBy, "update", "config", id, map[string]bool{"pendingDelete": pending}); err != nil {
|
||||
return domain.ConfigItem{}, err
|
||||
}
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return domain.ConfigItem{}, err
|
||||
}
|
||||
return item, nil
|
||||
}
|
||||
|
||||
func (s *Store) Publish(ctx context.Context, request domain.PublishRequest) (domain.Release, error) {
|
||||
tx, err := s.pool.BeginTx(ctx, pgx.TxOptions{IsoLevel: pgx.ReadCommitted})
|
||||
if err != nil {
|
||||
return domain.Release{}, err
|
||||
}
|
||||
defer tx.Rollback(ctx) //nolint:errcheck
|
||||
if _, err := tx.Exec(ctx, `SELECT pg_advisory_xact_lock($1)`, advisoryKey(request.NamespaceID, request.EnvironmentID)); err != nil {
|
||||
return domain.Release{}, err
|
||||
}
|
||||
appCode, namespaceName, envCode, err := loadScope(ctx, tx, request.AppID, request.NamespaceID, request.EnvironmentID)
|
||||
if err != nil {
|
||||
return domain.Release{}, classify(err)
|
||||
}
|
||||
items, err := listConfigTx(ctx, tx, request.AppID, request.NamespaceID, request.EnvironmentID, true)
|
||||
if err != nil {
|
||||
return domain.Release{}, err
|
||||
}
|
||||
snapshot := make(map[string]string)
|
||||
added, modified, removed := 0, 0, 0
|
||||
for _, item := range items {
|
||||
if item.PendingDelete {
|
||||
removed++
|
||||
continue
|
||||
}
|
||||
snapshot[item.Key] = item.Value
|
||||
if item.ReleasedValue == nil {
|
||||
added++
|
||||
} else if *item.ReleasedValue != item.Value {
|
||||
modified++
|
||||
}
|
||||
}
|
||||
if added+modified+removed == 0 {
|
||||
return domain.Release{}, storepkg.ErrNoPendingChange
|
||||
}
|
||||
version, err := nextVersion(ctx, tx, request.NamespaceID, request.EnvironmentID)
|
||||
if err != nil {
|
||||
return domain.Release{}, err
|
||||
}
|
||||
release, err := insertRelease(ctx, tx, request.AppID, request.NamespaceID, request.EnvironmentID, version, snapshot, added, modified, removed, request.Comment, actor(request.Operator))
|
||||
if err != nil {
|
||||
return domain.Release{}, err
|
||||
}
|
||||
key := fmt.Sprintf("/config/%s/%s/%s", strings.ToLower(envCode), appCode, namespaceName)
|
||||
if _, err := tx.Exec(ctx, `INSERT INTO release_outbox(release_id, etcd_key, payload) VALUES($1, $2, $3)`, release.ID, key, snapshot); err != nil {
|
||||
return domain.Release{}, err
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `DELETE FROM config_items WHERE namespace_id=$1 AND env_id=$2 AND pending_delete=true`, request.NamespaceID, request.EnvironmentID); err != nil {
|
||||
return domain.Release{}, err
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `UPDATE config_items SET released_value=value, pending_delete=false WHERE namespace_id=$1 AND env_id=$2`, request.NamespaceID, request.EnvironmentID); err != nil {
|
||||
return domain.Release{}, err
|
||||
}
|
||||
if err := insertAudit(ctx, tx, release.Operator, "publish", "release", release.ID, release); err != nil {
|
||||
return domain.Release{}, err
|
||||
}
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return domain.Release{}, err
|
||||
}
|
||||
return release, nil
|
||||
}
|
||||
|
||||
func (s *Store) Rollback(ctx context.Context, request domain.RollbackRequest) (domain.Release, error) {
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return domain.Release{}, err
|
||||
}
|
||||
defer tx.Rollback(ctx) //nolint:errcheck
|
||||
if _, err := tx.Exec(ctx, `SELECT pg_advisory_xact_lock($1)`, advisoryKey(request.NamespaceID, request.EnvironmentID)); err != nil {
|
||||
return domain.Release{}, err
|
||||
}
|
||||
appCode, namespaceName, envCode, err := loadScope(ctx, tx, request.AppID, request.NamespaceID, request.EnvironmentID)
|
||||
if err != nil {
|
||||
return domain.Release{}, classify(err)
|
||||
}
|
||||
var targetSnapshot, currentSnapshot map[string]string
|
||||
var targetVersion int
|
||||
err = tx.QueryRow(ctx, `SELECT version, snapshot FROM releases WHERE namespace_id=$1 AND env_id=$2 AND version=$3`, request.NamespaceID, request.EnvironmentID, request.TargetVersion).Scan(&targetVersion, &targetSnapshot)
|
||||
if err != nil {
|
||||
return domain.Release{}, classify(err)
|
||||
}
|
||||
var latestVersion int
|
||||
err = tx.QueryRow(ctx, `SELECT version, snapshot FROM releases WHERE namespace_id=$1 AND env_id=$2 ORDER BY version DESC LIMIT 1`, request.NamespaceID, request.EnvironmentID).Scan(&latestVersion, ¤tSnapshot)
|
||||
if err != nil {
|
||||
return domain.Release{}, classify(err)
|
||||
}
|
||||
if latestVersion == targetVersion {
|
||||
return domain.Release{}, storepkg.ErrInvalidRollback
|
||||
}
|
||||
added, modified, removed := diff(currentSnapshot, targetSnapshot)
|
||||
if _, err := tx.Exec(ctx, `DELETE FROM config_items WHERE namespace_id=$1 AND env_id=$2`, request.NamespaceID, request.EnvironmentID); err != nil {
|
||||
return domain.Release{}, err
|
||||
}
|
||||
for key, value := range targetSnapshot {
|
||||
if _, err := tx.Exec(ctx, `INSERT INTO config_items(namespace_id, env_id, key, value, released_value, updated_by) VALUES($1,$2,$3,$4,$4,$5)`, request.NamespaceID, request.EnvironmentID, key, value, actor(request.Operator)); err != nil {
|
||||
return domain.Release{}, err
|
||||
}
|
||||
}
|
||||
version := latestVersion + 1
|
||||
comment := fmt.Sprintf("回滚至版本 v%d", targetVersion)
|
||||
release, err := insertRelease(ctx, tx, request.AppID, request.NamespaceID, request.EnvironmentID, version, targetSnapshot, added, modified, removed, comment, actor(request.Operator))
|
||||
if err != nil {
|
||||
return domain.Release{}, err
|
||||
}
|
||||
key := fmt.Sprintf("/config/%s/%s/%s", strings.ToLower(envCode), appCode, namespaceName)
|
||||
if _, err := tx.Exec(ctx, `INSERT INTO release_outbox(release_id, etcd_key, payload) VALUES($1,$2,$3)`, release.ID, key, targetSnapshot); err != nil {
|
||||
return domain.Release{}, err
|
||||
}
|
||||
if err := insertAudit(ctx, tx, release.Operator, "rollback", "release", release.ID, map[string]int{"targetVersion": targetVersion, "newVersion": version}); err != nil {
|
||||
return domain.Release{}, err
|
||||
}
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return domain.Release{}, err
|
||||
}
|
||||
return release, nil
|
||||
}
|
||||
|
||||
func (s *Store) ListReleases(ctx context.Context, appID, namespaceID, envID int64) ([]domain.Release, error) {
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT r.id, n.app_id, r.namespace_id, r.env_id, r.version, r.snapshot, r.diff_added, r.diff_modified,
|
||||
r.diff_removed, r.comment, r.operator, r.etcd_revision, r.status, r.created_at
|
||||
FROM releases r JOIN namespaces n ON n.id=r.namespace_id
|
||||
WHERE n.app_id=$1 AND r.namespace_id=$2 AND r.env_id=$3 ORDER BY r.version DESC`, appID, namespaceID, envID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
result := make([]domain.Release, 0)
|
||||
for rows.Next() {
|
||||
item, err := scanRelease(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result = append(result, item)
|
||||
}
|
||||
return result, rows.Err()
|
||||
}
|
||||
|
||||
func (s *Store) GetRelease(ctx context.Context, id int64) (domain.Release, error) {
|
||||
row := s.pool.QueryRow(ctx, `
|
||||
SELECT r.id, n.app_id, r.namespace_id, r.env_id, r.version, r.snapshot, r.diff_added, r.diff_modified,
|
||||
r.diff_removed, r.comment, r.operator, r.etcd_revision, r.status, r.created_at
|
||||
FROM releases r JOIN namespaces n ON n.id=r.namespace_id WHERE r.id=$1`, id)
|
||||
item, err := scanRelease(row)
|
||||
return item, classify(err)
|
||||
}
|
||||
|
||||
func (s *Store) ListAuditLogs(ctx context.Context, limit int) ([]domain.AuditLog, error) {
|
||||
if limit <= 0 || limit > 500 {
|
||||
limit = 100
|
||||
}
|
||||
rows, err := s.pool.Query(ctx, `SELECT id, actor, action, target_type, target_id, detail, created_at FROM audit_logs ORDER BY created_at DESC LIMIT $1`, limit)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
result := make([]domain.AuditLog, 0)
|
||||
for rows.Next() {
|
||||
var item domain.AuditLog
|
||||
if err := rows.Scan(&item.ID, &item.Actor, &item.Action, &item.TargetType, &item.TargetID, &item.Detail, &item.CreatedAt); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result = append(result, item)
|
||||
}
|
||||
return result, rows.Err()
|
||||
}
|
||||
|
||||
func (s *Store) EnsureBootstrapAdmin(ctx context.Context, username, passwordHash, displayName string) error {
|
||||
_, err := s.pool.Exec(ctx, `
|
||||
INSERT INTO users(username, password_hash, display_name, is_admin)
|
||||
VALUES($1,$2,$3,true) ON CONFLICT(username) DO NOTHING`, username, passwordHash, displayName)
|
||||
return classify(err)
|
||||
}
|
||||
|
||||
func (s *Store) FindUserByUsername(ctx context.Context, username string) (domain.UserCredential, error) {
|
||||
var item domain.UserCredential
|
||||
err := s.pool.QueryRow(ctx, `
|
||||
SELECT id, username, password_hash, display_name, is_admin, disabled, created_at
|
||||
FROM users WHERE lower(username)=lower($1)`, username).
|
||||
Scan(&item.ID, &item.Username, &item.PasswordHash, &item.DisplayName, &item.IsAdmin, &item.Disabled, &item.CreatedAt)
|
||||
return item, classify(err)
|
||||
}
|
||||
|
||||
func (s *Store) ListUsers(ctx context.Context) ([]domain.User, error) {
|
||||
rows, err := s.pool.Query(ctx, `SELECT id, username, display_name, is_admin, disabled, created_at FROM users ORDER BY username`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
result := make([]domain.User, 0)
|
||||
for rows.Next() {
|
||||
var item domain.User
|
||||
if err := rows.Scan(&item.ID, &item.Username, &item.DisplayName, &item.IsAdmin, &item.Disabled, &item.CreatedAt); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result = append(result, item)
|
||||
}
|
||||
return result, rows.Err()
|
||||
}
|
||||
|
||||
func (s *Store) CreateUser(ctx context.Context, input domain.User, passwordHash string) (domain.User, error) {
|
||||
err := s.pool.QueryRow(ctx, `
|
||||
INSERT INTO users(username, password_hash, display_name, is_admin, disabled)
|
||||
VALUES($1,$2,$3,$4,$5)
|
||||
RETURNING id, username, display_name, is_admin, disabled, created_at`,
|
||||
input.Username, passwordHash, input.DisplayName, input.IsAdmin, input.Disabled,
|
||||
).Scan(&input.ID, &input.Username, &input.DisplayName, &input.IsAdmin, &input.Disabled, &input.CreatedAt)
|
||||
if err != nil {
|
||||
return domain.User{}, classify(err)
|
||||
}
|
||||
s.audit(ctx, actor(input.UpdatedBy), "create", "user", input.ID, input)
|
||||
return input, nil
|
||||
}
|
||||
|
||||
func (s *Store) SetUserAppRole(ctx context.Context, input domain.UserAppRole) error {
|
||||
command, err := s.pool.Exec(ctx, `
|
||||
INSERT INTO user_app_roles(user_id, app_id, role_id)
|
||||
SELECT $1, $2, r.id FROM roles r WHERE r.name=$3
|
||||
ON CONFLICT(user_id, app_id) DO UPDATE SET role_id=EXCLUDED.role_id`, input.UserID, input.AppID, input.Role)
|
||||
if err != nil {
|
||||
return classify(err)
|
||||
}
|
||||
if command.RowsAffected() == 0 {
|
||||
return storepkg.ErrNotFound
|
||||
}
|
||||
s.audit(ctx, actor(input.UpdatedBy), "update", "user_app_role", input.UserID, input)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Store) DeleteUserAppRole(ctx context.Context, userID, appID int64, updatedBy string) error {
|
||||
command, err := s.pool.Exec(ctx, `DELETE FROM user_app_roles WHERE user_id=$1 AND app_id=$2`, userID, appID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if command.RowsAffected() == 0 {
|
||||
return storepkg.ErrNotFound
|
||||
}
|
||||
s.audit(ctx, actor(updatedBy), "delete", "user_app_role", userID, map[string]int64{"appId": appID})
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Store) GetUserAppRole(ctx context.Context, userID, appID int64) (string, error) {
|
||||
var role string
|
||||
err := s.pool.QueryRow(ctx, `
|
||||
SELECT r.name FROM user_app_roles uar JOIN roles r ON r.id=uar.role_id
|
||||
WHERE uar.user_id=$1 AND uar.app_id=$2`, userID, appID).Scan(&role)
|
||||
return role, classify(err)
|
||||
}
|
||||
|
||||
func (s *Store) ListUserAppRoles(ctx context.Context, userID int64) ([]domain.UserAppRole, error) {
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT uar.user_id, uar.app_id, r.name
|
||||
FROM user_app_roles uar JOIN roles r ON r.id=uar.role_id
|
||||
WHERE uar.user_id=$1 ORDER BY uar.app_id`, userID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
result := make([]domain.UserAppRole, 0)
|
||||
for rows.Next() {
|
||||
var item domain.UserAppRole
|
||||
if err := rows.Scan(&item.UserID, &item.AppID, &item.Role); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result = append(result, item)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(result) == 0 {
|
||||
var exists bool
|
||||
if err := s.pool.QueryRow(ctx, `SELECT EXISTS(SELECT 1 FROM users WHERE id=$1)`, userID).Scan(&exists); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !exists {
|
||||
return nil, storepkg.ErrNotFound
|
||||
}
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (s *Store) ResolveScope(ctx context.Context, envCode, appCode, namespaceName string) (int64, int64, int64, error) {
|
||||
var appID, namespaceID, envID int64
|
||||
err := s.pool.QueryRow(ctx, `
|
||||
SELECT a.id, n.id, e.id
|
||||
FROM applications a JOIN namespaces n ON n.app_id=a.id CROSS JOIN environments e
|
||||
WHERE a.app_code=$1 AND n.name=$2 AND lower(e.env_code)=lower($3)`, appCode, namespaceName, envCode).
|
||||
Scan(&appID, &namespaceID, &envID)
|
||||
return appID, namespaceID, envID, classify(err)
|
||||
}
|
||||
|
||||
func (s *Store) ListGrayRules(ctx context.Context, appID, namespaceID, envID int64) ([]domain.GrayRule, error) {
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT g.id, n.app_id, g.namespace_id, g.env_id, g.rule_type, g.rule_value, g.overrides,
|
||||
g.enabled, g.priority, g.description, g.created_at
|
||||
FROM gray_rules g JOIN namespaces n ON n.id=g.namespace_id
|
||||
WHERE n.app_id=$1 AND g.namespace_id=$2 AND g.env_id=$3
|
||||
ORDER BY g.priority, g.id`, appID, namespaceID, envID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
result := make([]domain.GrayRule, 0)
|
||||
for rows.Next() {
|
||||
item, err := scanGrayRule(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result = append(result, item)
|
||||
}
|
||||
return result, rows.Err()
|
||||
}
|
||||
|
||||
func (s *Store) GetGrayRule(ctx context.Context, id int64) (domain.GrayRule, error) {
|
||||
row := s.pool.QueryRow(ctx, `
|
||||
SELECT g.id, n.app_id, g.namespace_id, g.env_id, g.rule_type, g.rule_value, g.overrides,
|
||||
g.enabled, g.priority, g.description, g.created_at
|
||||
FROM gray_rules g JOIN namespaces n ON n.id=g.namespace_id WHERE g.id=$1`, id)
|
||||
item, err := scanGrayRule(row)
|
||||
return item, classify(err)
|
||||
}
|
||||
|
||||
func (s *Store) CreateGrayRule(ctx context.Context, input domain.GrayRule, updatedBy string) (domain.GrayRule, error) {
|
||||
err := s.pool.QueryRow(ctx, `
|
||||
INSERT INTO gray_rules(namespace_id, env_id, rule_type, rule_value, overrides, enabled, priority, description)
|
||||
SELECT n.id, $3, $4, $5, $6, $7, $8, $9 FROM namespaces n
|
||||
WHERE n.id=$2 AND n.app_id=$1
|
||||
RETURNING id, namespace_id, env_id, rule_type, rule_value, overrides, enabled, priority, description, created_at`,
|
||||
input.AppID, input.NamespaceID, input.EnvironmentID, input.RuleType, input.RuleValue, input.Overrides, input.Enabled, input.Priority, input.Desc,
|
||||
).Scan(&input.ID, &input.NamespaceID, &input.EnvironmentID, &input.RuleType, &input.RuleValue, &input.Overrides, &input.Enabled, &input.Priority, &input.Desc, &input.CreatedAt)
|
||||
if err != nil {
|
||||
return domain.GrayRule{}, classify(err)
|
||||
}
|
||||
s.audit(ctx, actor(updatedBy), "create", "gray_rule", input.ID, input)
|
||||
return input, nil
|
||||
}
|
||||
|
||||
func (s *Store) UpdateGrayRule(ctx context.Context, id int64, input domain.GrayRule, updatedBy string) (domain.GrayRule, error) {
|
||||
err := s.pool.QueryRow(ctx, `
|
||||
UPDATE gray_rules SET rule_type=$2, rule_value=$3, overrides=$4, enabled=$5, priority=$6, description=$7
|
||||
WHERE id=$1
|
||||
RETURNING id, namespace_id, env_id, rule_type, rule_value, overrides, enabled, priority, description, created_at`,
|
||||
id, input.RuleType, input.RuleValue, input.Overrides, input.Enabled, input.Priority, input.Desc,
|
||||
).Scan(&input.ID, &input.NamespaceID, &input.EnvironmentID, &input.RuleType, &input.RuleValue, &input.Overrides, &input.Enabled, &input.Priority, &input.Desc, &input.CreatedAt)
|
||||
if err != nil {
|
||||
return domain.GrayRule{}, classify(err)
|
||||
}
|
||||
if err := s.pool.QueryRow(ctx, `SELECT app_id FROM namespaces WHERE id=$1`, input.NamespaceID).Scan(&input.AppID); err != nil {
|
||||
return domain.GrayRule{}, classify(err)
|
||||
}
|
||||
s.audit(ctx, actor(updatedBy), "update", "gray_rule", id, input)
|
||||
return input, nil
|
||||
}
|
||||
|
||||
func (s *Store) DeleteGrayRule(ctx context.Context, id int64, updatedBy string) error {
|
||||
command, err := s.pool.Exec(ctx, `DELETE FROM gray_rules WHERE id=$1`, id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if command.RowsAffected() == 0 {
|
||||
return storepkg.ErrNotFound
|
||||
}
|
||||
s.audit(ctx, actor(updatedBy), "delete", "gray_rule", id, nil)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Store) ClaimOutbox(ctx context.Context, limit int) ([]domain.OutboxEntry, error) {
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer tx.Rollback(ctx) //nolint:errcheck
|
||||
rows, err := tx.Query(ctx, `
|
||||
SELECT id, release_id, etcd_key, payload, retry_count
|
||||
FROM release_outbox
|
||||
WHERE (status='pending' AND next_attempt_at <= now())
|
||||
OR (status='processing' AND locked_at < now() - interval '60 seconds')
|
||||
ORDER BY id FOR UPDATE SKIP LOCKED LIMIT $1`, limit)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
entries := make([]domain.OutboxEntry, 0)
|
||||
ids := make([]int64, 0)
|
||||
for rows.Next() {
|
||||
var entry domain.OutboxEntry
|
||||
if err := rows.Scan(&entry.ID, &entry.ReleaseID, &entry.EtcdKey, &entry.Payload, &entry.RetryCount); err != nil {
|
||||
rows.Close()
|
||||
return nil, err
|
||||
}
|
||||
entries, ids = append(entries, entry), append(ids, entry.ID)
|
||||
}
|
||||
rows.Close()
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, id := range ids {
|
||||
if _, err := tx.Exec(ctx, `UPDATE release_outbox SET status='processing', locked_at=now() WHERE id=$1`, id); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return entries, nil
|
||||
}
|
||||
|
||||
func (s *Store) MarkOutboxDone(ctx context.Context, id, releaseID, revision int64) error {
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer tx.Rollback(ctx) //nolint:errcheck
|
||||
if _, err := tx.Exec(ctx, `UPDATE release_outbox SET status='done', locked_at=NULL WHERE id=$1`, id); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `UPDATE releases SET status='applied', etcd_revision=$2 WHERE id=$1`, releaseID, revision); err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.Commit(ctx)
|
||||
}
|
||||
|
||||
func (s *Store) MarkOutboxFailed(ctx context.Context, id, releaseID int64, message string, maxRetry int) error {
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer tx.Rollback(ctx) //nolint:errcheck
|
||||
var status string
|
||||
err = tx.QueryRow(ctx, `
|
||||
UPDATE release_outbox
|
||||
SET retry_count=retry_count+1,
|
||||
status=CASE WHEN retry_count+1 >= $3 THEN 'failed' ELSE 'pending' END,
|
||||
last_error=$2,
|
||||
next_attempt_at=now() + make_interval(secs => power(2, LEAST(retry_count+1, 6))::int),
|
||||
locked_at=NULL
|
||||
WHERE id=$1 RETURNING status`, id, message, maxRetry).Scan(&status)
|
||||
if err != nil {
|
||||
return classify(err)
|
||||
}
|
||||
if status == "failed" {
|
||||
if _, err := tx.Exec(ctx, `UPDATE releases SET status='failed' WHERE id=$1`, releaseID); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return tx.Commit(ctx)
|
||||
}
|
||||
|
||||
func (s *Store) OutboxStats(ctx context.Context) (domain.OutboxStats, error) {
|
||||
var result domain.OutboxStats
|
||||
err := s.pool.QueryRow(ctx, `
|
||||
SELECT COUNT(*) FILTER (WHERE status='pending'),
|
||||
COUNT(*) FILTER (WHERE status='processing'),
|
||||
COUNT(*) FILTER (WHERE status='failed')
|
||||
FROM release_outbox`).Scan(&result.Pending, &result.Processing, &result.Failed)
|
||||
return result, err
|
||||
}
|
||||
|
||||
type rowScanner interface {
|
||||
Scan(...any) error
|
||||
}
|
||||
|
||||
func scanConfig(row rowScanner) (domain.ConfigItem, error) {
|
||||
var item domain.ConfigItem
|
||||
err := row.Scan(&item.ID, &item.AppID, &item.NamespaceID, &item.EnvironmentID, &item.Key, &item.Value, &item.ReleasedValue, &item.PendingDelete, &item.Comment, &item.UpdatedBy, &item.UpdatedAt)
|
||||
return item, err
|
||||
}
|
||||
|
||||
func scanRelease(row rowScanner) (domain.Release, error) {
|
||||
var item domain.Release
|
||||
err := row.Scan(&item.ID, &item.AppID, &item.NamespaceID, &item.EnvironmentID, &item.Version, &item.Snapshot, &item.Added, &item.Modified, &item.Removed, &item.Comment, &item.Operator, &item.EtcdRevision, &item.Status, &item.Time)
|
||||
return item, err
|
||||
}
|
||||
|
||||
func scanGrayRule(row rowScanner) (domain.GrayRule, error) {
|
||||
var item domain.GrayRule
|
||||
err := row.Scan(&item.ID, &item.AppID, &item.NamespaceID, &item.EnvironmentID, &item.RuleType, &item.RuleValue, &item.Overrides, &item.Enabled, &item.Priority, &item.Desc, &item.CreatedAt)
|
||||
return item, err
|
||||
}
|
||||
|
||||
func listConfigTx(ctx context.Context, tx pgx.Tx, appID, namespaceID, envID int64, lock bool) ([]domain.ConfigItem, error) {
|
||||
query := `
|
||||
SELECT ci.id, n.app_id, ci.namespace_id, ci.env_id, ci.key, ci.value, ci.released_value,
|
||||
ci.pending_delete, ci.comment, ci.updated_by, ci.updated_at
|
||||
FROM config_items ci JOIN namespaces n ON n.id=ci.namespace_id
|
||||
WHERE n.app_id=$1 AND ci.namespace_id=$2 AND ci.env_id=$3 ORDER BY ci.key`
|
||||
if lock {
|
||||
query += ` FOR UPDATE OF ci`
|
||||
}
|
||||
rows, err := tx.Query(ctx, query, appID, namespaceID, envID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
items := make([]domain.ConfigItem, 0)
|
||||
for rows.Next() {
|
||||
item, err := scanConfig(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items = append(items, item)
|
||||
}
|
||||
return items, rows.Err()
|
||||
}
|
||||
|
||||
func loadScope(ctx context.Context, tx pgx.Tx, appID, namespaceID, envID int64) (string, string, string, error) {
|
||||
var appCode, namespaceName, envCode string
|
||||
err := tx.QueryRow(ctx, `
|
||||
SELECT a.app_code, n.name, e.env_code
|
||||
FROM applications a JOIN namespaces n ON n.app_id=a.id CROSS JOIN environments e
|
||||
WHERE a.id=$1 AND n.id=$2 AND e.id=$3`, appID, namespaceID, envID).Scan(&appCode, &namespaceName, &envCode)
|
||||
return appCode, namespaceName, envCode, err
|
||||
}
|
||||
|
||||
func nextVersion(ctx context.Context, tx pgx.Tx, namespaceID, envID int64) (int, error) {
|
||||
var version int
|
||||
err := tx.QueryRow(ctx, `SELECT COALESCE(MAX(version),0)+1 FROM releases WHERE namespace_id=$1 AND env_id=$2`, namespaceID, envID).Scan(&version)
|
||||
return version, err
|
||||
}
|
||||
|
||||
func insertRelease(ctx context.Context, tx pgx.Tx, appID, namespaceID, envID int64, version int, snapshot map[string]string, added, modified, removed int, comment, operator string) (domain.Release, error) {
|
||||
item := domain.Release{AppID: appID, NamespaceID: namespaceID, EnvironmentID: envID, Version: version, Snapshot: snapshot, Added: added, Modified: modified, Removed: removed, Comment: comment, Operator: operator}
|
||||
err := tx.QueryRow(ctx, `
|
||||
INSERT INTO releases(namespace_id, env_id, version, snapshot, diff_added, diff_modified, diff_removed, comment, operator)
|
||||
VALUES($1,$2,$3,$4,$5,$6,$7,$8,$9)
|
||||
RETURNING id, status, created_at`, namespaceID, envID, version, snapshot, added, modified, removed, comment, operator,
|
||||
).Scan(&item.ID, &item.Status, &item.Time)
|
||||
return item, err
|
||||
}
|
||||
|
||||
func (s *Store) deleteAndAudit(ctx context.Context, table, targetType string, id int64, updatedBy string) error {
|
||||
if table != "applications" && table != "environments" && table != "namespaces" {
|
||||
return errors.New("unsupported delete target")
|
||||
}
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer tx.Rollback(ctx) //nolint:errcheck
|
||||
command, err := tx.Exec(ctx, fmt.Sprintf("DELETE FROM %s WHERE id=$1", table), id) // #nosec G201 -- table is allow-listed above.
|
||||
if err != nil {
|
||||
return classify(err)
|
||||
}
|
||||
if command.RowsAffected() == 0 {
|
||||
return storepkg.ErrNotFound
|
||||
}
|
||||
if err := insertAudit(ctx, tx, actor(updatedBy), "delete", targetType, id, nil); err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.Commit(ctx)
|
||||
}
|
||||
|
||||
func (s *Store) audit(ctx context.Context, updatedBy, action, targetType string, targetID int64, detail any) {
|
||||
_, _ = s.pool.Exec(ctx, `INSERT INTO audit_logs(actor, action, target_type, target_id, detail) VALUES($1,$2,$3,$4,$5)`, actor(updatedBy), action, targetType, targetID, toJSON(detail))
|
||||
}
|
||||
|
||||
func insertAudit(ctx context.Context, tx pgx.Tx, updatedBy, action, targetType string, targetID int64, detail any) error {
|
||||
_, err := tx.Exec(ctx, `INSERT INTO audit_logs(actor, action, target_type, target_id, detail) VALUES($1,$2,$3,$4,$5)`, actor(updatedBy), action, targetType, targetID, toJSON(detail))
|
||||
return err
|
||||
}
|
||||
|
||||
func toJSON(value any) json.RawMessage {
|
||||
payload, _ := json.Marshal(value)
|
||||
return payload
|
||||
}
|
||||
|
||||
func classify(err error) error {
|
||||
if err == nil {
|
||||
return nil
|
||||
}
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return storepkg.ErrNotFound
|
||||
}
|
||||
var pgErr *pgconn.PgError
|
||||
if errors.As(err, &pgErr) {
|
||||
switch pgErr.Code {
|
||||
case "23505":
|
||||
return storepkg.ErrConflict
|
||||
case "23503", "23514", "23502":
|
||||
return fmt.Errorf("invalid data: %s", pgErr.Message)
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func actor(value string) string {
|
||||
if strings.TrimSpace(value) == "" {
|
||||
return "admin"
|
||||
}
|
||||
return strings.TrimSpace(value)
|
||||
}
|
||||
|
||||
func advisoryKey(namespaceID, envID int64) int64 {
|
||||
return namespaceID*1_000_000_000 + envID
|
||||
}
|
||||
|
||||
func diff(current, target map[string]string) (added, modified, removed int) {
|
||||
for key, value := range target {
|
||||
old, exists := current[key]
|
||||
if !exists {
|
||||
added++
|
||||
} else if old != value {
|
||||
modified++
|
||||
}
|
||||
}
|
||||
for key := range current {
|
||||
if _, exists := target[key]; !exists {
|
||||
removed++
|
||||
}
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
var _ = time.Second
|
||||
69
internal/store/store.go
Normal file
69
internal/store/store.go
Normal file
@@ -0,0 +1,69 @@
|
||||
package store
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
|
||||
"github.com/longpeng/configcenter/internal/domain"
|
||||
)
|
||||
|
||||
var (
|
||||
ErrNotFound = errors.New("resource not found")
|
||||
ErrConflict = errors.New("resource already exists")
|
||||
ErrNoPendingChange = errors.New("no pending configuration changes")
|
||||
ErrInvalidRollback = errors.New("invalid rollback target")
|
||||
)
|
||||
|
||||
type Store interface {
|
||||
Close()
|
||||
Ping(context.Context) error
|
||||
|
||||
ListApplications(context.Context) ([]domain.Application, error)
|
||||
CreateApplication(context.Context, domain.Application) (domain.Application, error)
|
||||
UpdateApplication(context.Context, int64, domain.Application) (domain.Application, error)
|
||||
DeleteApplication(context.Context, int64, string) error
|
||||
|
||||
ListEnvironments(context.Context) ([]domain.Environment, error)
|
||||
CreateEnvironment(context.Context, domain.Environment) (domain.Environment, error)
|
||||
UpdateEnvironment(context.Context, int64, domain.Environment) (domain.Environment, error)
|
||||
DeleteEnvironment(context.Context, int64, string) error
|
||||
|
||||
ListNamespaces(context.Context, *int64) ([]domain.Namespace, error)
|
||||
GetNamespace(context.Context, int64) (domain.Namespace, error)
|
||||
CreateNamespace(context.Context, domain.Namespace) (domain.Namespace, error)
|
||||
UpdateNamespace(context.Context, int64, domain.Namespace) (domain.Namespace, error)
|
||||
DeleteNamespace(context.Context, int64, string) error
|
||||
|
||||
ListConfigItems(context.Context, int64, int64, int64) ([]domain.ConfigItem, error)
|
||||
GetConfigItem(context.Context, int64) (domain.ConfigItem, error)
|
||||
CreateConfigItem(context.Context, domain.ConfigItem) (domain.ConfigItem, error)
|
||||
UpdateConfigItem(context.Context, int64, domain.ConfigItem) (domain.ConfigItem, error)
|
||||
SetConfigItemPendingDelete(context.Context, int64, bool, string) (domain.ConfigItem, error)
|
||||
|
||||
Publish(context.Context, domain.PublishRequest) (domain.Release, error)
|
||||
Rollback(context.Context, domain.RollbackRequest) (domain.Release, error)
|
||||
ListReleases(context.Context, int64, int64, int64) ([]domain.Release, error)
|
||||
GetRelease(context.Context, int64) (domain.Release, error)
|
||||
ListAuditLogs(context.Context, int) ([]domain.AuditLog, error)
|
||||
|
||||
EnsureBootstrapAdmin(context.Context, string, string, string) error
|
||||
FindUserByUsername(context.Context, string) (domain.UserCredential, error)
|
||||
ListUsers(context.Context) ([]domain.User, error)
|
||||
CreateUser(context.Context, domain.User, string) (domain.User, error)
|
||||
SetUserAppRole(context.Context, domain.UserAppRole) error
|
||||
DeleteUserAppRole(context.Context, int64, int64, string) error
|
||||
GetUserAppRole(context.Context, int64, int64) (string, error)
|
||||
ListUserAppRoles(context.Context, int64) ([]domain.UserAppRole, error)
|
||||
|
||||
ResolveScope(context.Context, string, string, string) (int64, int64, int64, error)
|
||||
ListGrayRules(context.Context, int64, int64, int64) ([]domain.GrayRule, error)
|
||||
GetGrayRule(context.Context, int64) (domain.GrayRule, error)
|
||||
CreateGrayRule(context.Context, domain.GrayRule, string) (domain.GrayRule, error)
|
||||
UpdateGrayRule(context.Context, int64, domain.GrayRule, string) (domain.GrayRule, error)
|
||||
DeleteGrayRule(context.Context, int64, string) error
|
||||
|
||||
ClaimOutbox(context.Context, int) ([]domain.OutboxEntry, error)
|
||||
MarkOutboxDone(context.Context, int64, int64, int64) error
|
||||
MarkOutboxFailed(context.Context, int64, int64, string, int) error
|
||||
OutboxStats(context.Context) (domain.OutboxStats, error)
|
||||
}
|
||||
82
internal/watch/hub.go
Normal file
82
internal/watch/hub.go
Normal file
@@ -0,0 +1,82 @@
|
||||
package watch
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sync"
|
||||
|
||||
"github.com/longpeng/configcenter/internal/domain"
|
||||
runtimepkg "github.com/longpeng/configcenter/internal/runtime"
|
||||
)
|
||||
|
||||
type Hub struct {
|
||||
ctx context.Context
|
||||
runtime runtimepkg.Store
|
||||
mu sync.Mutex
|
||||
keys map[string]*keyWatch
|
||||
}
|
||||
|
||||
type keyWatch struct {
|
||||
cancel context.CancelFunc
|
||||
nextID int64
|
||||
subs map[int64]chan domain.ConfigEvent
|
||||
}
|
||||
|
||||
func New(ctx context.Context, runtimeStore runtimepkg.Store) *Hub {
|
||||
return &Hub{ctx: ctx, runtime: runtimeStore, keys: make(map[string]*keyWatch)}
|
||||
}
|
||||
|
||||
func (h *Hub) Subscribe(key string) (<-chan domain.ConfigEvent, func()) {
|
||||
h.mu.Lock()
|
||||
watch, exists := h.keys[key]
|
||||
if !exists {
|
||||
ctx, cancel := context.WithCancel(h.ctx)
|
||||
watch = &keyWatch{cancel: cancel, subs: make(map[int64]chan domain.ConfigEvent)}
|
||||
h.keys[key] = watch
|
||||
go h.run(ctx, key, watch)
|
||||
}
|
||||
watch.nextID++
|
||||
id := watch.nextID
|
||||
channel := make(chan domain.ConfigEvent, 16)
|
||||
watch.subs[id] = channel
|
||||
h.mu.Unlock()
|
||||
|
||||
var once sync.Once
|
||||
return channel, func() {
|
||||
once.Do(func() {
|
||||
h.mu.Lock()
|
||||
current, ok := h.keys[key]
|
||||
if ok {
|
||||
if ch, found := current.subs[id]; found {
|
||||
delete(current.subs, id)
|
||||
close(ch)
|
||||
}
|
||||
if len(current.subs) == 0 {
|
||||
current.cancel()
|
||||
delete(h.keys, key)
|
||||
}
|
||||
}
|
||||
h.mu.Unlock()
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func (h *Hub) run(ctx context.Context, key string, target *keyWatch) {
|
||||
for result := range h.runtime.Watch(ctx, key, 0) {
|
||||
if result.Err != nil {
|
||||
return
|
||||
}
|
||||
h.mu.Lock()
|
||||
current, exists := h.keys[key]
|
||||
if !exists || current != target {
|
||||
h.mu.Unlock()
|
||||
return
|
||||
}
|
||||
for _, channel := range current.subs {
|
||||
select {
|
||||
case channel <- result.Event:
|
||||
default:
|
||||
}
|
||||
}
|
||||
h.mu.Unlock()
|
||||
}
|
||||
}
|
||||
324
pkg/sdk/go/client.go
Normal file
324
pkg/sdk/go/client.go
Normal file
@@ -0,0 +1,324 @@
|
||||
// Package configsdk provides a concurrency-safe Config Center client with
|
||||
// full-snapshot synchronization and an optional local-file fallback cache.
|
||||
package configsdk
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
type Options struct {
|
||||
BaseURL string
|
||||
Env string
|
||||
App string
|
||||
Token string
|
||||
IP string
|
||||
Instance string
|
||||
CacheFile string
|
||||
HTTPClient *http.Client
|
||||
}
|
||||
|
||||
type Client struct {
|
||||
baseURL string
|
||||
env string
|
||||
app string
|
||||
token string
|
||||
ip string
|
||||
instance string
|
||||
cacheFile string
|
||||
http *http.Client
|
||||
|
||||
mu sync.RWMutex
|
||||
cache map[string]map[string]string
|
||||
revisions map[string]int64
|
||||
}
|
||||
|
||||
type apiEnvelope struct {
|
||||
Data json.RawMessage `json:"data"`
|
||||
}
|
||||
|
||||
type runtimeConfig struct {
|
||||
Items map[string]string `json:"items"`
|
||||
Revision int64 `json:"revision"`
|
||||
ReleaseVersion int `json:"releaseVersion"`
|
||||
}
|
||||
|
||||
type configEvent struct {
|
||||
Type string `json:"type"`
|
||||
Items map[string]string `json:"items"`
|
||||
Revision int64 `json:"revision"`
|
||||
}
|
||||
|
||||
type diskCache struct {
|
||||
Namespaces map[string]map[string]string `json:"namespaces"`
|
||||
Revisions map[string]int64 `json:"revisions"`
|
||||
}
|
||||
|
||||
func New(options Options) (*Client, error) {
|
||||
if strings.TrimSpace(options.BaseURL) == "" || strings.TrimSpace(options.Env) == "" || strings.TrimSpace(options.App) == "" {
|
||||
return nil, errors.New("base URL, environment and application are required")
|
||||
}
|
||||
httpClient := options.HTTPClient
|
||||
if httpClient == nil {
|
||||
httpClient = &http.Client{Timeout: 15 * time.Second}
|
||||
}
|
||||
client := &Client{
|
||||
baseURL: strings.TrimRight(options.BaseURL, "/"),
|
||||
env: strings.TrimSpace(options.Env),
|
||||
app: strings.TrimSpace(options.App),
|
||||
token: strings.TrimSpace(options.Token),
|
||||
ip: strings.TrimSpace(options.IP),
|
||||
instance: strings.TrimSpace(options.Instance),
|
||||
cacheFile: options.CacheFile,
|
||||
http: httpClient,
|
||||
cache: make(map[string]map[string]string),
|
||||
revisions: make(map[string]int64),
|
||||
}
|
||||
if options.CacheFile != "" {
|
||||
if err := client.loadDiskCache(); err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||
return nil, fmt.Errorf("load config cache: %w", err)
|
||||
}
|
||||
}
|
||||
return client, nil
|
||||
}
|
||||
|
||||
func (c *Client) GetString(namespace, key, fallback string) string {
|
||||
c.mu.RLock()
|
||||
defer c.mu.RUnlock()
|
||||
if items, ok := c.cache[namespace]; ok {
|
||||
if value, exists := items[key]; exists {
|
||||
return value
|
||||
}
|
||||
}
|
||||
return fallback
|
||||
}
|
||||
|
||||
func (c *Client) GetInt(namespace, key string, fallback int) int {
|
||||
value := c.GetString(namespace, key, "")
|
||||
parsed, err := strconv.Atoi(value)
|
||||
if err != nil {
|
||||
return fallback
|
||||
}
|
||||
return parsed
|
||||
}
|
||||
|
||||
func (c *Client) GetBool(namespace, key string, fallback bool) bool {
|
||||
value := c.GetString(namespace, key, "")
|
||||
parsed, err := strconv.ParseBool(value)
|
||||
if err != nil {
|
||||
return fallback
|
||||
}
|
||||
return parsed
|
||||
}
|
||||
|
||||
func (c *Client) Snapshot(namespace string) map[string]string {
|
||||
c.mu.RLock()
|
||||
defer c.mu.RUnlock()
|
||||
return clone(c.cache[namespace])
|
||||
}
|
||||
|
||||
func (c *Client) Load(ctx context.Context, namespace string) error {
|
||||
endpoint := c.endpoint("/v1/config", namespace)
|
||||
request, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
c.authorize(request)
|
||||
response, err := c.http.Do(request)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer response.Body.Close()
|
||||
if response.StatusCode != http.StatusOK {
|
||||
return responseError(response)
|
||||
}
|
||||
var envelope apiEnvelope
|
||||
if err := json.NewDecoder(response.Body).Decode(&envelope); err != nil {
|
||||
return err
|
||||
}
|
||||
var current runtimeConfig
|
||||
if err := json.Unmarshal(envelope.Data, ¤t); err != nil {
|
||||
return err
|
||||
}
|
||||
return c.apply(namespace, current.Items, current.Revision)
|
||||
}
|
||||
|
||||
// WatchAndSync blocks until ctx is canceled. It reconnects with exponential
|
||||
// backoff and replaces the namespace cache only with complete snapshots.
|
||||
func (c *Client) WatchAndSync(ctx context.Context, namespace string) error {
|
||||
backoff := time.Second
|
||||
for {
|
||||
err := c.watchOnce(ctx, namespace)
|
||||
if ctx.Err() != nil {
|
||||
return nil
|
||||
}
|
||||
if err == nil {
|
||||
backoff = time.Second
|
||||
}
|
||||
timer := time.NewTimer(backoff)
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
timer.Stop()
|
||||
return nil
|
||||
case <-timer.C:
|
||||
}
|
||||
if backoff < 30*time.Second {
|
||||
backoff *= 2
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (c *Client) watchOnce(ctx context.Context, namespace string) error {
|
||||
request, err := http.NewRequestWithContext(ctx, http.MethodGet, c.endpoint("/v1/watch", namespace), nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
request.Header.Set("Accept", "text/event-stream")
|
||||
c.authorize(request)
|
||||
response, err := c.http.Do(request)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer response.Body.Close()
|
||||
if response.StatusCode != http.StatusOK {
|
||||
return responseError(response)
|
||||
}
|
||||
scanner := bufio.NewScanner(response.Body)
|
||||
scanner.Buffer(make([]byte, 64*1024), 10*1024*1024)
|
||||
var data strings.Builder
|
||||
for scanner.Scan() {
|
||||
line := scanner.Text()
|
||||
if line == "" {
|
||||
if data.Len() > 0 {
|
||||
var event configEvent
|
||||
if err := json.Unmarshal([]byte(data.String()), &event); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := c.apply(namespace, event.Items, event.Revision); err != nil {
|
||||
return err
|
||||
}
|
||||
data.Reset()
|
||||
}
|
||||
continue
|
||||
}
|
||||
if strings.HasPrefix(line, "data:") {
|
||||
if data.Len() > 0 {
|
||||
data.WriteByte('\n')
|
||||
}
|
||||
data.WriteString(strings.TrimSpace(strings.TrimPrefix(line, "data:")))
|
||||
}
|
||||
}
|
||||
if err := scanner.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
return io.EOF
|
||||
}
|
||||
|
||||
func (c *Client) endpoint(path, namespace string) string {
|
||||
query := url.Values{"env": {c.env}, "app": {c.app}, "namespace": {namespace}}
|
||||
if c.ip != "" {
|
||||
query.Set("ip", c.ip)
|
||||
}
|
||||
if c.instance != "" {
|
||||
query.Set("instance", c.instance)
|
||||
}
|
||||
return c.baseURL + path + "?" + query.Encode()
|
||||
}
|
||||
|
||||
func (c *Client) authorize(request *http.Request) {
|
||||
if c.token != "" {
|
||||
request.Header.Set("Authorization", "Bearer "+c.token)
|
||||
}
|
||||
}
|
||||
|
||||
func (c *Client) apply(namespace string, items map[string]string, revision int64) error {
|
||||
c.mu.Lock()
|
||||
if revision > 0 && revision < c.revisions[namespace] {
|
||||
c.mu.Unlock()
|
||||
return nil
|
||||
}
|
||||
c.cache[namespace] = clone(items)
|
||||
c.revisions[namespace] = revision
|
||||
err := c.saveDiskCacheLocked()
|
||||
c.mu.Unlock()
|
||||
return err
|
||||
}
|
||||
|
||||
func (c *Client) loadDiskCache() error {
|
||||
payload, err := os.ReadFile(c.cacheFile)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var cached diskCache
|
||||
if err := json.Unmarshal(payload, &cached); err != nil {
|
||||
return err
|
||||
}
|
||||
if cached.Namespaces != nil {
|
||||
c.cache = cached.Namespaces
|
||||
}
|
||||
if cached.Revisions != nil {
|
||||
c.revisions = cached.Revisions
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c *Client) saveDiskCacheLocked() error {
|
||||
if c.cacheFile == "" {
|
||||
return nil
|
||||
}
|
||||
payload, err := json.MarshalIndent(diskCache{Namespaces: c.cache, Revisions: c.revisions}, "", " ")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
directory := filepath.Dir(c.cacheFile)
|
||||
if err := os.MkdirAll(directory, 0o700); err != nil {
|
||||
return err
|
||||
}
|
||||
temporary, err := os.CreateTemp(directory, ".configcenter-cache-*")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
temporaryName := temporary.Name()
|
||||
defer os.Remove(temporaryName) //nolint:errcheck
|
||||
if err := temporary.Chmod(0o600); err != nil {
|
||||
temporary.Close()
|
||||
return err
|
||||
}
|
||||
if _, err := temporary.Write(payload); err != nil {
|
||||
temporary.Close()
|
||||
return err
|
||||
}
|
||||
if err := temporary.Sync(); err != nil {
|
||||
temporary.Close()
|
||||
return err
|
||||
}
|
||||
if err := temporary.Close(); err != nil {
|
||||
return err
|
||||
}
|
||||
return os.Rename(temporaryName, c.cacheFile)
|
||||
}
|
||||
|
||||
func responseError(response *http.Response) error {
|
||||
payload, _ := io.ReadAll(io.LimitReader(response.Body, 64*1024))
|
||||
return fmt.Errorf("config center returned %s: %s", response.Status, strings.TrimSpace(string(payload)))
|
||||
}
|
||||
|
||||
func clone(input map[string]string) map[string]string {
|
||||
result := make(map[string]string, len(input))
|
||||
for key, value := range input {
|
||||
result[key] = value
|
||||
}
|
||||
return result
|
||||
}
|
||||
42
pkg/sdk/go/client_test.go
Normal file
42
pkg/sdk/go/client_test.go
Normal file
@@ -0,0 +1,42 @@
|
||||
package configsdk
|
||||
|
||||
import (
|
||||
"context"
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
type roundTripFunc func(*http.Request) (*http.Response, error)
|
||||
|
||||
func (fn roundTripFunc) RoundTrip(request *http.Request) (*http.Response, error) { return fn(request) }
|
||||
|
||||
func TestLoadSendsAuthenticationAndGrayTarget(t *testing.T) {
|
||||
transport := roundTripFunc(func(r *http.Request) (*http.Response, error) {
|
||||
if got := r.Header.Get("Authorization"); got != "Bearer sdk-token" {
|
||||
t.Errorf("unexpected authorization header %q", got)
|
||||
}
|
||||
query := r.URL.Query()
|
||||
if query.Get("env") != "PROD" || query.Get("app") != "orders" || query.Get("namespace") != "application" || query.Get("ip") != "10.0.0.8" || query.Get("instance") != "orders-3" {
|
||||
t.Errorf("unexpected query: %s", r.URL.RawQuery)
|
||||
}
|
||||
return &http.Response{
|
||||
StatusCode: http.StatusOK,
|
||||
Header: http.Header{"Content-Type": []string{"application/json"}},
|
||||
Body: io.NopCloser(strings.NewReader(`{"data":{"items":{"feature.gray":"on"},"revision":12,"releaseVersion":3,"grayRuleIds":[7]}}`)),
|
||||
Request: r,
|
||||
}, nil
|
||||
})
|
||||
|
||||
client, err := New(Options{BaseURL: "https://config.test", Env: "PROD", App: "orders", Token: "sdk-token", IP: "10.0.0.8", Instance: "orders-3", HTTPClient: &http.Client{Transport: transport}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := client.Load(context.Background(), "application"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if value := client.GetString("application", "feature.gray", "off"); value != "on" {
|
||||
t.Fatalf("unexpected synchronized value %q", value)
|
||||
}
|
||||
}
|
||||
106
scripts/integration-smoke.sh
Normal file
106
scripts/integration-smoke.sh
Normal file
@@ -0,0 +1,106 @@
|
||||
#!/usr/bin/env sh
|
||||
set -eu
|
||||
|
||||
api_base="${CONFIGCENTER_API_BASE:-http://127.0.0.1:8080}"
|
||||
app_code="config-smoke-$(date +%s)-$$"
|
||||
app_id=""
|
||||
|
||||
json_data_field() {
|
||||
field="$1"
|
||||
python3 -c 'import json,sys; payload=json.load(sys.stdin)["data"]; print(payload[sys.argv[1]])' "$field"
|
||||
}
|
||||
|
||||
cleanup() {
|
||||
if [ -n "$app_id" ]; then
|
||||
curl -fsS -X DELETE -H 'X-User: integration-smoke' "$api_base/v1/applications/$app_id" >/dev/null || true
|
||||
fi
|
||||
}
|
||||
trap cleanup EXIT INT TERM
|
||||
|
||||
curl -fsS "$api_base/health/ready" >/dev/null
|
||||
|
||||
app_response="$(curl -fsS -X POST \
|
||||
-H 'Content-Type: application/json' \
|
||||
-H 'X-User: integration-smoke' \
|
||||
-d "{\"code\":\"$app_code\",\"name\":\"Integration Smoke\"}" \
|
||||
"$api_base/v1/applications")"
|
||||
app_id="$(printf '%s' "$app_response" | json_data_field id)"
|
||||
|
||||
env_id="$(curl -fsS "$api_base/v1/environments" | python3 -c '
|
||||
import json,sys
|
||||
for item in json.load(sys.stdin)["data"]:
|
||||
if item["code"] == "DEV":
|
||||
print(item["id"])
|
||||
break
|
||||
else:
|
||||
raise SystemExit("DEV environment not found")
|
||||
')"
|
||||
|
||||
namespace_response="$(curl -fsS -X POST \
|
||||
-H 'Content-Type: application/json' \
|
||||
-H 'X-User: integration-smoke' \
|
||||
-d "{\"appId\":$app_id,\"name\":\"application\",\"type\":\"properties\"}" \
|
||||
"$api_base/v1/namespaces")"
|
||||
namespace_id="$(printf '%s' "$namespace_response" | json_data_field id)"
|
||||
|
||||
curl -fsS -X POST \
|
||||
-H 'Content-Type: application/json' \
|
||||
-H 'X-User: integration-smoke' \
|
||||
-d "{\"appId\":$app_id,\"nsId\":$namespace_id,\"envId\":$env_id,\"key\":\"smoke.enabled\",\"value\":\"true\"}" \
|
||||
"$api_base/v1/config-items" >/dev/null
|
||||
|
||||
release_response="$(curl -fsS -X POST \
|
||||
-H 'Content-Type: application/json' \
|
||||
-H 'X-User: integration-smoke' \
|
||||
-d "{\"appId\":$app_id,\"nsId\":$namespace_id,\"envId\":$env_id,\"comment\":\"integration smoke\"}" \
|
||||
"$api_base/v1/publish")"
|
||||
release_id="$(printf '%s' "$release_response" | json_data_field id)"
|
||||
|
||||
attempt=0
|
||||
status="pending"
|
||||
while [ "$attempt" -lt 50 ]; do
|
||||
release_response="$(curl -fsS "$api_base/v1/releases/$release_id")"
|
||||
status="$(printf '%s' "$release_response" | json_data_field status)"
|
||||
if [ "$status" = "applied" ]; then
|
||||
break
|
||||
fi
|
||||
if [ "$status" = "failed" ]; then
|
||||
printf 'release failed: %s\n' "$release_response" >&2
|
||||
exit 1
|
||||
fi
|
||||
attempt=$((attempt + 1))
|
||||
sleep 0.2
|
||||
done
|
||||
|
||||
if [ "$status" != "applied" ]; then
|
||||
printf 'release did not become applied (last status: %s)\n' "$status" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
runtime_response="$(curl -fsS "$api_base/v1/config?env=DEV&app=$app_code&namespace=application")"
|
||||
printf '%s' "$runtime_response" | python3 -c '
|
||||
import json,sys
|
||||
data=json.load(sys.stdin)["data"]
|
||||
assert data["items"]["smoke.enabled"] == "true", data
|
||||
assert data["releaseVersion"] == 1, data
|
||||
assert data["revision"] > 0, data
|
||||
'
|
||||
|
||||
gray_response="$(curl -fsS -X POST \
|
||||
-H 'Content-Type: application/json' \
|
||||
-H 'X-User: integration-smoke' \
|
||||
-d "{\"appId\":$app_id,\"nsId\":$namespace_id,\"envId\":$env_id,\"ruleType\":\"instance\",\"ruleValue\":{\"instances\":[\"smoke-1\"]},\"overrides\":{\"smoke.enabled\":\"gray\"},\"enabled\":true,\"priority\":10,\"desc\":\"smoke rule\"}" \
|
||||
"$api_base/v1/gray-rules")"
|
||||
gray_id="$(printf '%s' "$gray_response" | json_data_field id)"
|
||||
|
||||
gray_runtime_response="$(curl -fsS "$api_base/v1/config?env=DEV&app=$app_code&namespace=application&instance=smoke-1")"
|
||||
printf '%s' "$gray_runtime_response" | python3 -c '
|
||||
import json,sys
|
||||
data=json.load(sys.stdin)["data"]
|
||||
assert data["items"]["smoke.enabled"] == "gray", data
|
||||
assert len(data["grayRuleIds"]) == 1, data
|
||||
'
|
||||
|
||||
curl -fsS "$api_base/metrics" | grep -q '^configcenter_gray_rule_matches_total 1$'
|
||||
|
||||
printf 'integration smoke passed: release=%s gray_rule=%s status=%s\n' "$release_id" "$gray_id" "$status"
|
||||
3
sdk/python/configcenter/__init__.py
Normal file
3
sdk/python/configcenter/__init__.py
Normal file
@@ -0,0 +1,3 @@
|
||||
from .client import ConfigClient
|
||||
|
||||
__all__ = ["ConfigClient"]
|
||||
129
sdk/python/configcenter/client.py
Normal file
129
sdk/python/configcenter/client.py
Normal file
@@ -0,0 +1,129 @@
|
||||
"""Thread-safe Config Center client using only Python's standard library."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
class ConfigClient:
|
||||
def __init__(
|
||||
self,
|
||||
base_url: str,
|
||||
env: str,
|
||||
app: str,
|
||||
cache_file: str | None = None,
|
||||
*,
|
||||
token: str | None = None,
|
||||
ip: str | None = None,
|
||||
instance: str | None = None,
|
||||
):
|
||||
if not base_url or not env or not app:
|
||||
raise ValueError("base_url, env and app are required")
|
||||
self._base_url = base_url.rstrip("/")
|
||||
self._env = env
|
||||
self._app = app
|
||||
self._token = token or ""
|
||||
self._ip = ip or ""
|
||||
self._instance = instance or ""
|
||||
self._cache_file = Path(cache_file) if cache_file else None
|
||||
self._cache: dict[str, dict[str, str]] = {}
|
||||
self._revisions: dict[str, int] = {}
|
||||
self._lock = threading.RLock()
|
||||
self._load_disk_cache()
|
||||
|
||||
def get(self, namespace: str, key: str, default: Any = None) -> Any:
|
||||
with self._lock:
|
||||
return self._cache.get(namespace, {}).get(key, default)
|
||||
|
||||
def snapshot(self, namespace: str) -> dict[str, str]:
|
||||
with self._lock:
|
||||
return dict(self._cache.get(namespace, {}))
|
||||
|
||||
def load(self, namespace: str, timeout: float = 10.0) -> None:
|
||||
with urllib.request.urlopen(self._request("/v1/config", namespace), timeout=timeout) as response:
|
||||
payload = json.load(response)["data"]
|
||||
self._apply(namespace, payload.get("items", {}), int(payload.get("revision", 0)))
|
||||
|
||||
def start_background_watch(self, namespace: str) -> threading.Thread:
|
||||
thread = threading.Thread(target=self.watch_and_sync, args=(namespace,), daemon=True)
|
||||
thread.start()
|
||||
return thread
|
||||
|
||||
def watch_and_sync(self, namespace: str) -> None:
|
||||
backoff = 1.0
|
||||
while True:
|
||||
try:
|
||||
request = self._request("/v1/watch", namespace, {"Accept": "text/event-stream"})
|
||||
with urllib.request.urlopen(request, timeout=None) as response:
|
||||
backoff = 1.0
|
||||
data_lines: list[str] = []
|
||||
for raw_line in response:
|
||||
line = raw_line.decode("utf-8").rstrip("\r\n")
|
||||
if not line:
|
||||
if data_lines:
|
||||
event = json.loads("\n".join(data_lines))
|
||||
self._apply(namespace, event.get("items", {}), int(event.get("revision", 0)))
|
||||
data_lines.clear()
|
||||
continue
|
||||
if line.startswith("data:"):
|
||||
data_lines.append(line[5:].strip())
|
||||
except Exception:
|
||||
time.sleep(backoff)
|
||||
backoff = min(backoff * 2, 30.0)
|
||||
|
||||
def _url(self, path: str, namespace: str) -> str:
|
||||
values = {"env": self._env, "app": self._app, "namespace": namespace}
|
||||
if self._ip:
|
||||
values["ip"] = self._ip
|
||||
if self._instance:
|
||||
values["instance"] = self._instance
|
||||
query = urllib.parse.urlencode(values)
|
||||
return f"{self._base_url}{path}?{query}"
|
||||
|
||||
def _request(
|
||||
self, path: str, namespace: str, headers: dict[str, str] | None = None
|
||||
) -> urllib.request.Request:
|
||||
request_headers = dict(headers or {})
|
||||
if self._token:
|
||||
request_headers["Authorization"] = f"Bearer {self._token}"
|
||||
return urllib.request.Request(self._url(path, namespace), headers=request_headers)
|
||||
|
||||
def _apply(self, namespace: str, items: dict[str, str], revision: int) -> None:
|
||||
with self._lock:
|
||||
if revision and revision < self._revisions.get(namespace, 0):
|
||||
return
|
||||
self._cache[namespace] = dict(items)
|
||||
self._revisions[namespace] = revision
|
||||
self._save_disk_cache()
|
||||
|
||||
def _load_disk_cache(self) -> None:
|
||||
if not self._cache_file or not self._cache_file.exists():
|
||||
return
|
||||
payload = json.loads(self._cache_file.read_text(encoding="utf-8"))
|
||||
self._cache = payload.get("namespaces", {})
|
||||
self._revisions = payload.get("revisions", {})
|
||||
|
||||
def _save_disk_cache(self) -> None:
|
||||
if not self._cache_file:
|
||||
return
|
||||
self._cache_file.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
|
||||
payload = {"namespaces": self._cache, "revisions": self._revisions}
|
||||
descriptor, temporary = tempfile.mkstemp(prefix=".configcenter-cache-", dir=self._cache_file.parent)
|
||||
try:
|
||||
with os.fdopen(descriptor, "w", encoding="utf-8") as handle:
|
||||
json.dump(payload, handle, ensure_ascii=False, indent=2)
|
||||
handle.flush()
|
||||
os.fsync(handle.fileno())
|
||||
os.chmod(temporary, 0o600)
|
||||
os.replace(temporary, self._cache_file)
|
||||
finally:
|
||||
if os.path.exists(temporary):
|
||||
os.unlink(temporary)
|
||||
13
sdk/python/pyproject.toml
Normal file
13
sdk/python/pyproject.toml
Normal file
@@ -0,0 +1,13 @@
|
||||
[build-system]
|
||||
requires = ["setuptools>=68"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "configcenter-client"
|
||||
version = "0.1.0"
|
||||
description = "Config Center Python client with SSE watch and local cache fallback"
|
||||
requires-python = ">=3.10"
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
where = ["."]
|
||||
include = ["configcenter*"]
|
||||
12
web/Dockerfile
Normal file
12
web/Dockerfile
Normal file
@@ -0,0 +1,12 @@
|
||||
FROM node:24-alpine AS build
|
||||
WORKDIR /src
|
||||
COPY web/package.json web/package-lock.json ./web/
|
||||
RUN cd web && npm ci
|
||||
COPY ConfigCenter.jsx ./ConfigCenter.jsx
|
||||
COPY web ./web
|
||||
RUN cd web && npm run build
|
||||
|
||||
FROM nginx:1.27-alpine
|
||||
COPY web/nginx.conf /etc/nginx/conf.d/default.conf
|
||||
COPY --from=build /src/web/dist /usr/share/nginx/html
|
||||
EXPOSE 80
|
||||
13
web/index.html
Normal file
13
web/index.html
Normal file
@@ -0,0 +1,13 @@
|
||||
<!doctype html>
|
||||
<html lang="zh-CN">
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<meta name="theme-color" content="#0f172a" />
|
||||
<title>Config Center</title>
|
||||
</head>
|
||||
<body>
|
||||
<div id="root"></div>
|
||||
<script type="module" src="/src/main.jsx"></script>
|
||||
</body>
|
||||
</html>
|
||||
24
web/nginx.conf
Normal file
24
web/nginx.conf
Normal file
@@ -0,0 +1,24 @@
|
||||
server {
|
||||
listen 80;
|
||||
server_name _;
|
||||
root /usr/share/nginx/html;
|
||||
index index.html;
|
||||
|
||||
location /v1/ {
|
||||
proxy_pass http://server:8080;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_buffering off;
|
||||
proxy_read_timeout 1h;
|
||||
}
|
||||
|
||||
location /health/ {
|
||||
proxy_pass http://server:8080;
|
||||
}
|
||||
|
||||
location / {
|
||||
try_files $uri $uri/ /index.html;
|
||||
}
|
||||
}
|
||||
1848
web/package-lock.json
generated
Normal file
1848
web/package-lock.json
generated
Normal file
File diff suppressed because it is too large
Load Diff
22
web/package.json
Normal file
22
web/package.json
Normal file
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"name": "configcenter-web",
|
||||
"private": true,
|
||||
"version": "0.1.0",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "vite",
|
||||
"build": "vite build",
|
||||
"preview": "vite preview"
|
||||
},
|
||||
"dependencies": {
|
||||
"@vitejs/plugin-react": "latest",
|
||||
"autoprefixer": "latest",
|
||||
"lucide-react": "latest",
|
||||
"postcss": "latest",
|
||||
"react": "latest",
|
||||
"react-dom": "latest",
|
||||
"tailwindcss": "^3.4.17",
|
||||
"vite": "latest"
|
||||
},
|
||||
"devDependencies": {}
|
||||
}
|
||||
6
web/postcss.config.js
Normal file
6
web/postcss.config.js
Normal file
@@ -0,0 +1,6 @@
|
||||
export default {
|
||||
plugins: {
|
||||
tailwindcss: {},
|
||||
autoprefixer: {},
|
||||
},
|
||||
};
|
||||
196
web/src/api.js
Normal file
196
web/src/api.js
Normal file
@@ -0,0 +1,196 @@
|
||||
const API_ROOT = (import.meta.env.VITE_API_BASE_URL || '').replace(/\/$/, '');
|
||||
const TOKEN_KEY = 'configcenter.accessToken';
|
||||
let unauthorizedHandler = null;
|
||||
|
||||
function accessToken() {
|
||||
return window.sessionStorage.getItem(TOKEN_KEY) || '';
|
||||
}
|
||||
|
||||
class ApiError extends Error {
|
||||
constructor(message, status, code) {
|
||||
super(message);
|
||||
this.name = 'ApiError';
|
||||
this.status = status;
|
||||
this.code = code;
|
||||
}
|
||||
}
|
||||
|
||||
function normalizeId(value) {
|
||||
return value == null ? value : String(value);
|
||||
}
|
||||
|
||||
function localTime(value, dateOnly = false) {
|
||||
if (!value) return '';
|
||||
const date = new Date(value);
|
||||
if (Number.isNaN(date.getTime())) return value;
|
||||
if (dateOnly) return date.toLocaleDateString('zh-CN');
|
||||
return date.toLocaleString('zh-CN', { hour12: false });
|
||||
}
|
||||
|
||||
function app(item) {
|
||||
return { ...item, id: normalizeId(item.id), createdAt: localTime(item.createdAt, true), updatedAt: localTime(item.updatedAt) };
|
||||
}
|
||||
|
||||
function env(item) {
|
||||
return { ...item, id: normalizeId(item.id) };
|
||||
}
|
||||
|
||||
function namespace(item) {
|
||||
return { ...item, id: normalizeId(item.id), appId: normalizeId(item.appId) };
|
||||
}
|
||||
|
||||
function config(item) {
|
||||
return {
|
||||
...item,
|
||||
id: normalizeId(item.id),
|
||||
appId: normalizeId(item.appId),
|
||||
nsId: normalizeId(item.nsId),
|
||||
envId: normalizeId(item.envId),
|
||||
updatedAt: localTime(item.updatedAt),
|
||||
};
|
||||
}
|
||||
|
||||
function release(item) {
|
||||
return {
|
||||
...item,
|
||||
id: normalizeId(item.id),
|
||||
appId: normalizeId(item.appId),
|
||||
nsId: normalizeId(item.nsId),
|
||||
envId: normalizeId(item.envId),
|
||||
time: localTime(item.time),
|
||||
};
|
||||
}
|
||||
|
||||
function grayRule(item) {
|
||||
return {
|
||||
...item,
|
||||
id: normalizeId(item.id),
|
||||
appId: normalizeId(item.appId),
|
||||
nsId: normalizeId(item.nsId),
|
||||
envId: normalizeId(item.envId),
|
||||
createdAt: localTime(item.createdAt),
|
||||
};
|
||||
}
|
||||
|
||||
async function request(path, options = {}) {
|
||||
const token = accessToken();
|
||||
const response = await fetch(`${API_ROOT}${path}`, {
|
||||
...options,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'X-User': 'admin',
|
||||
...(token ? { Authorization: `Bearer ${token}` } : {}),
|
||||
...options.headers,
|
||||
},
|
||||
});
|
||||
if (!response.ok) {
|
||||
let message = `请求失败 (${response.status})`;
|
||||
let code = 'http_error';
|
||||
try {
|
||||
const payload = await response.json();
|
||||
message = payload.error?.message || message;
|
||||
code = payload.error?.code || code;
|
||||
} catch {
|
||||
// Keep the HTTP status based fallback.
|
||||
}
|
||||
const error = new ApiError(message, response.status, code);
|
||||
if (response.status === 401 && path !== '/v1/auth/login') {
|
||||
window.sessionStorage.removeItem(TOKEN_KEY);
|
||||
unauthorizedHandler?.();
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
if (response.status === 204) return null;
|
||||
const payload = await response.json();
|
||||
return payload.data ?? payload;
|
||||
}
|
||||
|
||||
function json(method, body) {
|
||||
return { method, body: JSON.stringify(body) };
|
||||
}
|
||||
|
||||
function scopeQuery(appId, nsId, envId) {
|
||||
return new URLSearchParams({ appId, nsId, envId }).toString();
|
||||
}
|
||||
|
||||
export const api = {
|
||||
setToken(token) {
|
||||
if (token) window.sessionStorage.setItem(TOKEN_KEY, token);
|
||||
else window.sessionStorage.removeItem(TOKEN_KEY);
|
||||
},
|
||||
onUnauthorized(handler) {
|
||||
unauthorizedHandler = handler;
|
||||
return () => { if (unauthorizedHandler === handler) unauthorizedHandler = null; };
|
||||
},
|
||||
hasToken: () => Boolean(accessToken()),
|
||||
login: (username, password) => request('/v1/auth/login', json('POST', { username, password })),
|
||||
me: () => request('/v1/me'),
|
||||
async bootstrap() {
|
||||
const [apps, environments, namespaces, identity] = await Promise.all([
|
||||
request('/v1/applications'),
|
||||
request('/v1/environments'),
|
||||
request('/v1/namespaces'),
|
||||
request('/v1/me'),
|
||||
]);
|
||||
return {
|
||||
apps: apps.map(app),
|
||||
environments: environments.map(env),
|
||||
namespaces: namespaces.map(namespace),
|
||||
identity,
|
||||
};
|
||||
},
|
||||
|
||||
createApplication: (input) => request('/v1/applications', json('POST', input)).then(app),
|
||||
updateApplication: (id, input) => request(`/v1/applications/${id}`, json('PUT', input)).then(app),
|
||||
deleteApplication: (id) => request(`/v1/applications/${id}`, { method: 'DELETE' }),
|
||||
|
||||
createEnvironment: (input) => request('/v1/environments', json('POST', input)).then(env),
|
||||
updateEnvironment: (id, input) => request(`/v1/environments/${id}`, json('PUT', input)).then(env),
|
||||
deleteEnvironment: (id) => request(`/v1/environments/${id}`, { method: 'DELETE' }),
|
||||
|
||||
createNamespace: (input) => request('/v1/namespaces', json('POST', { ...input, appId: Number(input.appId) })).then(namespace),
|
||||
updateNamespace: (id, input) => request(`/v1/namespaces/${id}`, json('PUT', input)).then(namespace),
|
||||
deleteNamespace: (id) => request(`/v1/namespaces/${id}`, { method: 'DELETE' }),
|
||||
|
||||
async listConfigItems(appId, nsId, envId) {
|
||||
const data = await request(`/v1/config-items?${scopeQuery(appId, nsId, envId)}`);
|
||||
return data.map(config);
|
||||
},
|
||||
createConfigItem: (input) => request('/v1/config-items', json('POST', {
|
||||
...input,
|
||||
appId: Number(input.appId),
|
||||
nsId: Number(input.nsId),
|
||||
envId: Number(input.envId),
|
||||
})).then(config),
|
||||
updateConfigItem: (id, input) => request(`/v1/config-items/${id}`, json('PUT', input)).then(config),
|
||||
deleteConfigItem: (id) => request(`/v1/config-items/${id}`, { method: 'DELETE' }).then(config),
|
||||
restoreConfigItem: (id) => request(`/v1/config-items/${id}/restore`, { method: 'POST' }).then(config),
|
||||
|
||||
async listReleases(appId, nsId, envId) {
|
||||
const data = await request(`/v1/releases?${scopeQuery(appId, nsId, envId)}`);
|
||||
return data.map(release);
|
||||
},
|
||||
publish: (input) => request('/v1/publish', json('POST', {
|
||||
appId: Number(input.appId), nsId: Number(input.nsId), envId: Number(input.envId), comment: input.comment,
|
||||
})).then(release),
|
||||
rollback: (input) => request('/v1/rollback', json('POST', {
|
||||
appId: Number(input.appId), nsId: Number(input.nsId), envId: Number(input.envId), targetVersion: input.targetVersion,
|
||||
})).then(release),
|
||||
getRelease: (id) => request(`/v1/releases/${id}`).then(release),
|
||||
|
||||
async listGrayRules(appId, nsId, envId) {
|
||||
const data = await request(`/v1/gray-rules?${scopeQuery(appId, nsId, envId)}`);
|
||||
return data.map(grayRule);
|
||||
},
|
||||
createGrayRule: (input) => request('/v1/gray-rules', json('POST', {
|
||||
...input, appId: Number(input.appId), nsId: Number(input.nsId), envId: Number(input.envId),
|
||||
})).then(grayRule),
|
||||
updateGrayRule: (id, input) => request(`/v1/gray-rules/${id}`, json('PUT', input)).then(grayRule),
|
||||
deleteGrayRule: (id) => request(`/v1/gray-rules/${id}`, { method: 'DELETE' }),
|
||||
|
||||
listUsers: () => request('/v1/users'),
|
||||
createUser: (input) => request('/v1/users', json('POST', input)),
|
||||
listUserRoles: (id) => request(`/v1/users/${id}/roles`),
|
||||
setUserRole: (userId, appId, role) => request(`/v1/users/${userId}/roles/${appId}`, json('PUT', { role })),
|
||||
deleteUserRole: (userId, appId) => request(`/v1/users/${userId}/roles/${appId}`, { method: 'DELETE' }),
|
||||
};
|
||||
22
web/src/index.css
Normal file
22
web/src/index.css
Normal file
@@ -0,0 +1,22 @@
|
||||
@tailwind base;
|
||||
@tailwind components;
|
||||
@tailwind utilities;
|
||||
|
||||
html,
|
||||
body,
|
||||
#root {
|
||||
height: 100%;
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
body {
|
||||
min-width: 720px;
|
||||
background: #f8fafc;
|
||||
}
|
||||
|
||||
button,
|
||||
input,
|
||||
select,
|
||||
textarea {
|
||||
font: inherit;
|
||||
}
|
||||
10
web/src/main.jsx
Normal file
10
web/src/main.jsx
Normal file
@@ -0,0 +1,10 @@
|
||||
import React from 'react';
|
||||
import ReactDOM from 'react-dom/client';
|
||||
import ConfigCenter from '../../ConfigCenter.jsx';
|
||||
import './index.css';
|
||||
|
||||
ReactDOM.createRoot(document.getElementById('root')).render(
|
||||
<React.StrictMode>
|
||||
<ConfigCenter />
|
||||
</React.StrictMode>,
|
||||
);
|
||||
6
web/tailwind.config.js
Normal file
6
web/tailwind.config.js
Normal file
@@ -0,0 +1,6 @@
|
||||
/** @type {import('tailwindcss').Config} */
|
||||
export default {
|
||||
content: ['./index.html', './src/**/*.{js,jsx}', '../ConfigCenter.jsx'],
|
||||
theme: { extend: {} },
|
||||
plugins: [],
|
||||
};
|
||||
21
web/vite.config.js
Normal file
21
web/vite.config.js
Normal file
@@ -0,0 +1,21 @@
|
||||
import { defineConfig } from 'vite';
|
||||
import react from '@vitejs/plugin-react';
|
||||
import { fileURLToPath, URL } from 'node:url';
|
||||
|
||||
export default defineConfig({
|
||||
plugins: [react()],
|
||||
resolve: {
|
||||
alias: {
|
||||
react: fileURLToPath(new URL('./node_modules/react', import.meta.url)),
|
||||
'lucide-react': fileURLToPath(new URL('./node_modules/lucide-react', import.meta.url)),
|
||||
},
|
||||
},
|
||||
server: {
|
||||
port: 5173,
|
||||
proxy: {
|
||||
'/v1': 'http://localhost:8080',
|
||||
'/health': 'http://localhost:8080',
|
||||
},
|
||||
fs: { allow: ['..'] },
|
||||
},
|
||||
});
|
||||
Reference in New Issue
Block a user