Huawei Cloud CCE Pressure Test

Trigger keywords: pressure test, 压测, load test, 负载测试, stress test, 压力测试, performance test, 性能测试, k6 test, k6 压测, ELB traffic test, ELB 流量测试, end-to-end pressure test, 全链路压测, elasticity evaluation, 弹性评估, traffic generation, 流量生成

⚠️ Execution Method (Must Read): This skill executes queries via the local Python dispatcher script. Using hcloud, openstack, or other CLI tools or direct API calls is prohibited.

• The dispatcher script is located at scripts/huawei-cloud.py within the skill directory
• All scripts and environment check scripts are inside the skill package. You must use skill action=exec to execute them. Do not run them directly in a shell.
• Do not attempt hcloud, openstack, curl IAM, or any other CLI/API methods. This skill does not depend on those tools.
• All paths are relative to the skill directory, which is the directory where this SKILL.md is located.

Overview

This skill runs controlled workload pressure tests on Huawei Cloud CCE. It builds a complete traffic path from a k6 client through ELB to nginx-ingress to workload pods, supports short-connection, keepalive, and ramp traffic models, collects ELB metrics and AOM observability data, evaluates elasticity phases, and generates bilingual Markdown and HTML reports with performance curves.

Architecture: Python dispatcher (scripts/huawei-cloud.py) → Huawei Cloud Python SDK + Kubernetes client → ELB/CCE/AOM/APM APIs → Route preparation → Client generation → Traffic run → Observability collection → Report generation

Key Principle: Preview-first. Read-only checks may run immediately, but Service/Ingress changes, traffic generation, and workload scaling require explicit user approval with confirm=true before execution.

Related Skills:

Capabilities:

1. End-to-end traffic path: k6 client → ELB → nginx-ingress → workload Service → Pod
2. Short-connection (short), keepalive (keepalive), and staged ramp (ramp) traffic models
3. Isolated Java sample deployment for lab environments
4. ELB creation when no reusable ingress-controller ELB exists
5. Route preparation: Service and Ingress creation/patching with preview-first
6. k6 client generation with ConfigMap and Job
7. Traffic run with replica and Pod count monitoring
8. APM Java agent injection for distributed tracing during tests
9. Observability collection: ELB metrics, Pod CPU/memory, AOM logs
10. Bilingual Markdown and HTML report generation with SVG performance curves
11. Elasticity evaluation: baseline vs scaled phase comparison

Prerequisites

1. Python Requirements (MANDATORY)

• Python >= 3.6 installed
• Required packages: huaweicloudsdkcore, huaweicloudsdkcce, huaweicloudsdkvpc, huaweicloudsdkvpcep, huaweicloudsdkaom, huaweicloudsdkelb, kubernetes, matplotlib
• Verify: python3 --version
• Install packages: pip3 install huaweicloudsdkcore huaweicloudsdkcce huaweicloudsdkvpc huaweicloudsdkvpcep huaweicloudsdkaom huaweicloudsdkelb kubernetes matplotlib

2. Credential Configuration

• Valid Huawei Cloud credentials (AK/SK mode)
• Security Rules:
  • 🚫 Never expose AK/SK values in code, conversation, or commands
  • 🚫 Never use echo $HUAWEI_AK or echo $HUAWEI_SK to check credentials
  • 🚫 Never write credentials to files, logs, or responses
  • ✅ Use environment variables: HUAWEI_AK, HUAWEI_SK, HUAWEI_REGION
  • ✅ Credentials exist only in the current request call stack and are released after each invocation
  • ✅ Prefer IAM users over root account for cloud operations

Configuration Method (Environment Variables Only):

export HUAWEI_AK=\x3Cyour-ak>
export HUAWEI_SK=\x3Cyour-sk>
export HUAWEI_REGION=cn-north-4

Additional Variables:

3. IAM Permission Requirements

Permission Failure Handling:

1. When any action fails due to IAM permission errors, display the required permission list
2. Guide the user to create custom policies in the IAM console for Huawei Cloud permissions
3. Pause execution and wait for user confirmation that permissions have been granted
4. Retry the failed action

Core Commands

All actions are invoked via the dispatcher script:

python3 scripts/huawei-cloud.py \x3Caction> region=\x3Cregion> cluster_id=\x3Ccluster_id> [key=value ...]

1. Deploy Java Sample (Optional)

# Preview (no mutation)
python3 scripts/huawei-cloud.py huawei_deploy_cce_pressure_test_java_sample \
  region=cn-north-4 cluster_id=\x3Ccluster_id>

# Apply after user approval
python3 scripts/huawei-cloud.py huawei_deploy_cce_pressure_test_java_sample \
  region=cn-north-4 cluster_id=\x3Ccluster_id> confirm=true

Creates an isolated Namespace, ConfigMap (small Java HTTP server), and Deployment with readiness/liveness probes. Exposes /healthz, /api/hello, and /api/work on port 8080.

2. Create ELB (When No Reusable ELB Exists)

# Preview (no mutation)
python3 scripts/huawei-cloud.py huawei_create_elb \
  region=cn-north-4 name=\x3Celb-name> vip_subnet_cidr_id=\x3Csubnet-network-id>

# Apply after user approval
python3 scripts/huawei-cloud.py huawei_create_elb \
  region=cn-north-4 name=\x3Celb-name> vip_subnet_cidr_id=\x3Csubnet-network-id> confirm=true

Creates a chargeable ELB. Review billable fields before applying. Does not silently create an EIP.

3. Prepare Route

# Preview (no mutation)
python3 scripts/huawei-cloud.py huawei_prepare_cce_pressure_test_route \
  region=cn-north-4 cluster_id=\x3Ccluster_id> namespace=\x3Cnamespace> workload_name=\x3Cdeployment-name> target_port=8080

# Apply after user approval
python3 scripts/huawei-cloud.py huawei_prepare_cce_pressure_test_route \
  region=cn-north-4 cluster_id=\x3Ccluster_id> namespace=\x3Cnamespace> workload_name=\x3Cdeployment-name> target_port=8080 confirm=true

Creates or patches a ClusterIP Service and Ingress pointing to the workload. Discover an existing nginx ingress controller LoadBalancer address. Does not silently create a chargeable ELB.

4. Inject APM Java Agent (Optional)

python3 scripts/huawei-cloud.py huawei_inject_cce_apm_javaagent \
  region=cn-north-4 cluster_id=\x3Ccluster_id> namespace=\x3Cnamespace> \
  workload_name=\x3Cdeployment-name> app_name=\x3Capm-app-name> \
  business=\x3Capm-business> env_name=\x3Capm-env>

Injects the Huawei Cloud APM Java agent into the workload Deployment for distributed tracing during the pressure test.

5. Generate k6 Client

python3 scripts/huawei-cloud.py huawei_generate_cce_pressure_test_client \
  target_url=http://\x3Celb-address>/\x3Cpath> namespace=pressure-test \
  model=keepalive vus=10 duration_seconds=60

Generates the k6 ConfigMap and Job manifest. Use a regional SWR mirror for the k6 image when public image pulls are unavailable.

Supported Traffic Models:

6. Run Pressure Test

# Preview (no mutation)
python3 scripts/huawei-cloud.py huawei_run_cce_pressure_test \
  region=cn-north-4 cluster_id=\x3Ccluster_id> target_url=http://\x3Celb-address>/\x3Cpath>

# Apply after user approval
python3 scripts/huawei-cloud.py huawei_run_cce_pressure_test \
  region=cn-north-4 cluster_id=\x3Ccluster_id> target_url=http://\x3Celb-address>/\x3Cpath> confirm=true

Ensures the client namespace, creates the ConfigMap and Job, waits for completion by default, records replica/Pod count samples, and extracts the k6 summary from logs.

7. Collect Observability

python3 scripts/huawei-cloud.py huawei_collect_cce_pressure_test_observability \
  region=cn-north-4 cluster_id=\x3Ccluster_id> namespace=\x3Cnamespace> \
  workload_name=\x3Cdeployment-name> elb_id=\x3Celb-id>

Collects ELB QPS, active connections, layer-7 response time, 2xx ratio, Pod CPU/memory metrics, and AOM evidence. Pass label_selector to scope Pod metrics to the workload. Pod CPU and memory are collected from node metrics when aom_instance_id is not provided.

8. Generate Report

python3 scripts/huawei-cloud.py huawei_generate_cce_pressure_test_report \
  result_path=\x3Cpath-to-run-json>

Generates Markdown, HTML, and SVG curve artifacts from the run JSON and optional observability JSON.

9. Scale Workload (Elasticity Evaluation)

# Preview (no mutation)
python3 scripts/huawei-cloud.py huawei_scale_cce_workload \
  region=cn-north-4 cluster_id=\x3Ccluster_id> workload_type=Deployment \
  name=\x3Cdeployment-name> namespace=\x3Cnamespace> replicas=5

# Apply after user approval
python3 scripts/huawei-cloud.py huawei_scale_cce_workload \
  region=cn-north-4 cluster_id=\x3Ccluster_id> workload_type=Deployment \
  name=\x3Cdeployment-name> namespace=\x3Cnamespace> replicas=5 confirm=true

For elasticity evaluation: run a baseline phase, preview scaling, apply only after approval with confirm=true, then run a second phase and compare reports.

10. Supporting Actions (Read-Only)

Parameter Reference

Common Parameters

Deploy Java Sample Parameters

ELB Creation Parameters

Route Preparation Parameters

Client Generation Parameters

Traffic Run Parameters

Observability Collection Parameters

APM Injection Parameters

Report Generation Parameters

Common Region IDs

Output Format

All actions return JSON with the following common structure:

Run JSON (huawei_run_cce_pressure_test when output_dir is provided):

Observability JSON (huawei_collect_cce_pressure_test_observability):

Report Artifacts (huawei_generate_cce_pressure_test_report):

See Output Schema for the full JSON response schema.

Verification

Step-by-step Verification Checklist

1. Verify AK/SK credentials are configured via environment variables
2. Inspect existing Services, Ingresses, HPA, ingress controller, and ELB context
3. Confirm cluster, workload, namespace, target port, test window, traffic model, VUs, and output directory
4. Deploy Java sample (preview → confirm) if needed for an isolated lab
5. Create ELB (preview → confirm) if no reusable ingress-controller ELB exists
6. Prepare route (preview → confirm) — Service and Ingress
7. Inject APM Java agent (optional) for distributed tracing
8. Generate k6 client manifest — review ConfigMap and Job
9. Run pressure test (preview → confirm)
10. Collect observability with elb_id for full evidence
11. Generate report and review Markdown/HTML/SVG artifacts
12. For elasticity evaluation: run baseline → scale workload → run second phase → compare reports

Best Practices

1. Start with low traffic: Always begin with low VUs and short duration to validate the traffic path before ramping up
2. Preview before apply: Always run mutation actions without confirm=true first, review the plan, then re-run with confirm=true after explicit user approval
3. Use regional SWR images for k6: Docker Hub images may timeout; mirror k6 to a regional SWR namespace
4. Pass elb_id for observability: ELB metrics (QPS, connections, latency, success rate) are critical evidence; always pass elb_id when collecting observability
5. Run at least two phases for elasticity: Baseline + scaled phase comparison is the standard methodology for evaluating autoscaling behavior
6. Confirm test target before traffic: Never send traffic to production paths without an approved test window
7. Stop raising traffic when indicators degrade: Halt when success rate drops, latency rises sharply, or resource waterlines exceed agreed limits
8. Do not delete resources automatically: Never auto-delete ELBs, namespaces, workloads, Jobs, or ConfigMaps

Reference Documents

Notes

• Preview-first by design — Service/Ingress changes, traffic generation, and workload scaling return a preview without confirm=true; apply only after explicit user approval
• Standard traffic path — Pod → Service → nginx-ingress → ELB; the route preparation action discovers existing ingress controllers and does not silently create chargeable resources
• ELB is chargeable — creating a new ELB incurs hourly billing; always preview and confirm before creation
• No credential persistence — AK/SK exists only during API calls; never written to disk, logs, or reports
• Cross-skill escalation — If autoscaling does not respond during tests, hand off to huawei-cloud-cce-autoscaling-diagnoser; if Pods fail, hand off to huawei-cloud-cce-pod-failure-diagnoser

Common Pitfalls