Huawei Cloud NewAPI LLM Gateway

Overview

Deploy the "Building a NewAPI LLM Gateway" solution end-to-end on Huawei Cloud. The platform provides a NewAPI-based LLM API gateway for unified management and forwarding of multiple large model API requests, supporting load balancing, key rotation, and usage statistics.

Architecture: ECS (Ubuntu 22.04) and VPC and Subnet and Security Group (SSH port 22 and NewAPI Web port 3000) and EIP and EVS (system disk). Cloud-init installs Docker and launches the NewAPI gateway container.

Tool chain: Playwright CLI (solution info extraction) + Python 3.10+ (helper scripts) + Terraform 1.5+ (declarative deployment). No KooCLI — all resource operations through Terraform.

Prerequisites

• Python 3.10+, Playwright CLI, Terraform 1.5+ — see CLI Installation Guide
• Huawei Cloud AK/SK via environment variables (HW_ACCESS_KEY, HW_SECRET_KEY); if not set, prompt user to manually edit terraform.auto.tfvars.json to fill in AK/SK — ⛔ never read or display this file in conversation context
• IAM user with sufficient permissions — see IAM Policies

Security

• 🚫 Never expose AK/SK in conversation or output
• 🚫 Never ask user to type AK/SK in chat
• 🚫 Never read or display terraform.auto.tfvars.json in conversation context (contains AK/SK)
• ✅ Prefer IAM users over primary account
• ✅ Modification ops (apply, destroy) require explicit user confirmation

Core Commands

Placeholder values (see Parameters for per-OS resolution):

\x3Csolution_detail_page_url> = https://www.huaweicloud.com/solution/implementations/building-a-newapi-llm-gateway.html

\x3Ctf_template_url> = https://documentation-samples.obs.cn-north-4.myhuaweicloud.com/solution-as-code-publicbucket/solution-as-code-moudle/building-a-newapi-llm-gateway/building-a-newapi-llm-gateway.tf.json

# 1. Extract solution info
\x3Cpython> \x3Cscript_dir>/extract_sac_deploy_info.py \
  --url "\x3Csolution_detail_page_url>" \
  --out \x3Ctemp_dir>/sac_selected.json

# 2. Download and normalize template
\x3Cpython> \x3Cscript_dir>/download_tf_template_file.py \
  --url "\x3Ctf_template_url>" \
  --out-dir \x3Ctemp_dir>/\x3Cworkdir>

\x3Cpython> \x3Cscript_dir>/normalize_tf_providers.py \x3Ctemp_dir>/\x3Cworkdir> \
  --region "\x3Cregion>"

# 3. List variables for review
\x3Cpython> \x3Cscript_dir>/list_tf_variables.py \x3Ctemp_dir>/\x3Cworkdir>

# 4. Deploy
terraform -chdir=\x3Ctemp_dir>/\x3Cworkdir> init
terraform -chdir=\x3Ctemp_dir>/\x3Cworkdir> plan
# ⛔ STOP — Review the plan output above. Do NOT auto-apply.
# Confirm with the user (AskUserQuestion or equivalent) before proceeding.
# Only after explicit user confirmation:
terraform -chdir=\x3Ctemp_dir>/\x3Cworkdir> apply

# 5. Verify
terraform -chdir=\x3Ctemp_dir>/\x3Cworkdir> state list
terraform -chdir=\x3Ctemp_dir>/\x3Cworkdir> output -json

# 6. Cleanup
terraform -chdir=\x3Ctemp_dir>/\x3Cworkdir> destroy
python -c "import os; f='\x3Ctemp_dir>/\x3Cworkdir>/terraform.auto.tfvars.json'; os.path.exists(f) and os.remove(f)"

Workflow

1. Extract solution info

After running the Core Commands step 1, display the results to the user:

• Solution name: title field from output JSON
• Estimated price: estimated_price_text field
• Deploy links: list each text and url from deploy_links array
• If title or estimated_price_text is empty, warn the user and suggest manual verification on the solution page

2. Download and normalize template

normalize_tf_providers.py writes terraform.auto.tfvars.json (including region and other parameters). If environment variables HW_ACCESS_KEY/HW_SECRET_KEY are not set, AK/SK fields are left empty. Tell the user the file path and prompt them to manually edit it to fill in AK/SK. ⛔ Never read or display the file contents in conversation context. Then continue to the next step.

3. Confirm variables

Review with user. Block apply if sensitive variables are empty/weak.

4. Deploy

⛔ STOP — Before running terraform apply, review the terraform plan output and confirm with the user (AskUserQuestion or equivalent). Do NOT auto-apply. Only proceed after explicit user confirmation.

5. Verify

See Verification Method and Acceptance Criteria.

6. Cleanup

Parameters

Post-Deploy Output

• terraform output -json — includes access_instructions with NewAPI gateway URL
• NewAPI Web UI: http://\x3CEIP>:3000 (allow ~10 min for cloud-init)
• Verify: ssh root@\x3CEIP> "docker ps" shows NewAPI container running

Output Format

terraform output -json returns JSON with the following key fields:

{
  "access_instructions": { "value": "http://\x3CEIP>:3000" },
  "ecs_eip":             { "value": "\x3CElastic IP>" },
  "ecs_id":              { "value": "\x3CECS Instance ID>" },
  "vpc_id":              { "value": "\x3CVPC ID>" }
}

All script outputs are in JSON format: extract_sac_deploy_info.py outputs solution info JSON, list_tf_variables.py outputs variable list JSON.

Verification

Verify deployment results step by step:

1. Template extraction — Check \x3Ctemp_dir>/sac_selected.json contains solution_name, price fields
2. Template download — Confirm .tf files exist under \x3Ctemp_dir>/newapi-workdir and terraform validate passes
3. Variable confirmation — Sensitive variables (AK/SK, password) are not empty in list_tf_variables.py output; user confirmed overrides
4. Deployment — terraform plan shows no errors; user confirmed deployment; after apply, terraform state list shows all expected resources
5. Service reachability — Wait 10-15 min for cloud-init, then curl -s http://\x3CEIP>:3000 returns 200
6. Container — ssh root@\x3CEIP> "docker ps" shows NewAPI container running

See Verification Method and Acceptance Criteria for details.

Best Practices

• Always terraform plan before apply
• Start with charging_unit=month; switch to year after validation
• Allow 10-15 min post-deploy for cloud-init
• Monitor NewAPI dashboard for API usage and key rotation status

Reference Documents

Notes

• Only cn-north-4 region supported
• terraform.auto.tfvars.json is sensitive — never commit to VCS; never read or display in conversation context
• normalize_tf_providers.py writes region to tfvars; AK/SK left empty if env vars not set, user must fill manually
• Tool chain: Playwright CLI + Python + Terraform — no KooCLI