P8 deployment diff: code walkthrough

p8-api is a repository containing modules

p8-api/module_conf.yaml lists nine modules, including:

p8-api/p8-profile-management
p8-api/p8-metrics
p8-api/p8-statistics
p8-api/p8-audit-trail
p8-api/p8-announcement
p8-api/p8-maker-checker
p8-api/p8-notification
p8-api/p8-integrations
p8-api/p8-ui-log-management

Each module has a Terraform directory:

p8-api/<module>/deployments/

Each module’s deployments/provision.sh is a symlink to:

p8-deployments/scripts/provision.sh

p8-api/deployments/build_and_deploy.sh is also a symlink, pointing to:

scripts/deployments/build_and_deploy.sh

So there is one generic orchestrator and one shared Terraform provisioner. The old p8-api/scripts/provision.sh was an exception to that structure.

Important variables

• repo: repository name, calculated by the generic script. For this run it is p8-api.
• modules: module names selected from module_conf.yaml, or from --modules.
• mode: normally build_and_deploy, publish_only, deploy, or deploy_local.
• alias: environment name such as dev2.
• region: AWS region.
• TF_VAR_x: Terraform automatically reads an environment variable named TF_VAR_x as the value of Terraform variable x.

3.1 Generic deploy() before the deletion

At scripts/deployments/build_and_deploy.sh:334-344, UI repositories use a GUI builder. Other repositories enter the generic module loop. Before this diff, there was an additional branch immediately after that generic branch:

elif [[ $repo == "p8-api" ]]; then
  deploy_p8_api

When repo was p8-api, the generic per-module loop was bypassed and deploy_p8_api ran instead.

3.2 The deleted deploy_p8_api() caller

The deleted function was at old lines 407-430.

• function deploy_p8_api() { defined the exception path.
• if [[ $repo == "p8-api" ]]; then repeated the caller’s repository check. It was redundant but prevented the body from running for another repository.
• init_deploy_status created or preserved the overall status file.
• build_status=$(cat $status_file) copied that status into the local variable.
• pushd ../scripts changed from p8-api/deployments to p8-api/scripts.
• export P8_API_MODULES=$(printf '%s\n' "${modules[@]}" | tr '\n' ' ') converted the generic module selection into the environment variable expected by the old p8-api/scripts/provision.sh.
• export P8_PRODUCT_VARIATION="$P8_PRODUCT_VARIATION" re-exported the product variant. This is effectively a no-op if it was already exported, but made the child script’s input explicit.
• ./provision.sh -a ... -r ... -m ... -rt ... launched the old p8-api orchestrator. Notice that its -m means deployment mode, not module selection.
• || echo 'failed' >$status_file converted any non-zero result into an overall failed status.
• popd returned to p8-api/deployments.
• build_status=$(cat $status_file) refreshed the result.
• bash ./build_utils.sh bld_update 2 "overall_deployment" ... recorded one aggregate AWS-artifact deployment result.
• return stopped the generic deploy() function from continuing into any other deployment logic.
• The final two blank lines had no behavior.

Removing this function and its caller means p8-api now goes through the ordinary module loop.

Lines 1-3: shell setup

• Line 1 selected Bash.
• Line 3 enabled set -e, causing the script to stop when an unhandled command failed.
• The blank line had no behavior.

Lines 5-19: output helpers

The four functions only formatted terminal output:

• print_title: blue bold text.
• print_message: purple text.
• print_error: red ERROR: text.
• print_warning: yellow WARNING: text.

Deleting them removes the old script’s presentation layer. The generic orchestrator and shared provisioner have their own output helpers.

Lines 21-62: duplicate module deployability gate

is_deployable_module() was a copy of the function in scripts/deployments/build_and_deploy.sh.

• Lines 21-29 documented that it performed two checks: a feature condition from the repository config, then the module’s allowed variants.
• Line 30 defined the function.
• Line 31 read the module name.
• Lines 32-33 built paths to the repository-level and module-level module_conf.yaml files.
• Lines 34-36 selected a solution variant. It preferred P8_SOLUTION_VARIATION, then P8_SolutionVariant, then P8_PRODUCT_VARIATION.
• Lines 38-39 read .modules["<module>"].allowed_on_condition from the repository config.
• Lines 40-48 implemented the feature gate. If the config named a feature variable, the indirect expansion ${!condition} read that variable. Only the literal string true allowed deployment; otherwise the module was skipped.
• Lines 50-51 read the module’s .variants[] list.
• Lines 52-55 allowed a module whose variants contained all.
• Lines 56-59 allowed a module whose variants contained the selected solution variant.
• Lines 60-61 printed a skip message and returned failure when neither condition matched.
• Line 62 ended the function.

This gate was not unique in policy; it duplicated the generic script’s gate. The new flow keeps the generic version and deletes this copy.

Lines 64-81: old-script help text

help() described the deleted interface.

• Lines 65-66 printed the title and an empty line.
• Lines 67-76 documented -a, -r, -m, -rt, and --no-auto-approve.
• Line 69 said the script would run Terraform for every p8-api submodule.
• Line 70 specifically promised parallel execution.
• Lines 78-79 warned that p8-authorizer had to be applied first because modules read authorizer IDs from its Terraform remote state.
• Lines 80-81 ended the function.

This interface no longer exists. The supported interface is the generic build_and_deploy.sh interface, plus the shared module provision.sh interface.

Lines 83-89: old-script defaults

• Line 83 printed that the p8-api deployment script was starting.
• Lines 85-86 initialized alias and region to empty values.
• Line 87 defaulted the mode to build_and_deploy.
• Line 88 defaulted the release tag from P8_RELEASE_TAG.
• Line 89 defaulted to --auto-approve.

The generic orchestrator now obtains these values from --env-name, --aws-region, --mode, and --publish-tags, then passes them to each module provisioner.

Lines 91-123: old command-line parsing

The while loop consumed the old script’s arguments.

• -a stored the next argument as alias and shifted two arguments.
• -r stored the next argument as region and shifted two.
• -m stored the next argument as mode and shifted two.
• -rt stored the next argument as release_tag and shifted two.
• --no-auto-approve cleared the auto_approve string and shifted one.
• -h/--help printed help and exited successfully.
• Any other argument printed an error, printed help, and exited with status 1.
• The loop ended after all arguments were consumed.

The deleted script did not accept --modules; module selection came indirectly through P8_API_MODULES.

Lines 125-129: required-input validation

The script required both alias and region. If either was empty, it printed an error, printed help, and exited with status 1.

The generic entry point performs equivalent validation for env_name and aws_region.

Lines 131-170: old one-time API Gateway lookup

This was the most important deleted runtime block.

The intent was to look up four values once before starting parallel module deployments:

admin REST API ID
trader REST API ID
admin /api resource ID
trader /api resource ID

The comments at lines 131-137 explain the performance reason: every route-tree module otherwise performs API Gateway name/path lookups, and many concurrent lookups can be throttled.

• Line 138 skipped the entire block for publish_only because live API routes are not created in that mode.
• Line 139 printed that the fast lookup was starting.
• Lines 141-145 explained an AWS CLI pagination/output detail. The code intentionally avoided a query form that could produce None for nonmatching pages.
• Line 146 defined _first_id(), which split whitespace into tokens, removed empty values and None, and selected the first remaining ID.
• Lines 148-149 looked up the admin REST API by the exact name p8-<alias>-admin-api-<region>.
• Lines 150-151 looked up the trader REST API by the exact name p8-<alias>-api-<region>.
• Lines 153-154 initialized both parent resource IDs as empty.
• Lines 155-158 looked up /api in the admin REST API, but only if the admin API lookup succeeded.
• Lines 159-162 did the same for the trader API.
• Lines 164-167 exported all four values as TF_VAR_*, making them visible to child Terraform processes.
• Lines 168-169 printed the resolved values, displaying <lookup> when a value was empty.
• Line 170 ended the mode guard.

The new design still obtains these values, but through the shared export_config.sh cache and the shared populate_tf_vars() function rather than through this p8-api-only block.

Lines 172-195: old module selection and validation

• Lines 172-173 documented that P8_API_MODULES could override the default list.
• Lines 174-178 selected either the hard-coded nine-module list or split P8_API_MODULES into an array.
• Lines 180-182 explained why explicit validation was needed: a typo could otherwise look like a successful deployment that deployed nothing.
• Line 183 loaded the valid module names from ../module_conf.yaml.
• Lines 184-190 checked every selected name against that list. An unknown name caused an error, printed all valid names, and exited with status 1.
• Lines 192-195 printed the candidate module list.

The generic script now reads the same module_conf.yaml itself and supports --modules. The explicit p8-api-only validation is gone.

Lines 197-207: change to repository root and package detection

• Line 197 changed from p8-api/deployments to the p8-api repository root.
• Lines 199-204 documented a safety check: modules publishing lib.zip should have a freshly built lib/ directory, otherwise an old package or no package might be deployed.
• Lines 205-207 defined module_publishes_package(), using grep to inspect Terraform files for lib.zip.

This preflight check is lost. The shared package_lambda_code() now logs and continues when ../lib is absent.

Lines 209-229: old missing-build hard failure

This block ran except in deploy mode.

• Line 210 created an empty missing_builds array.
• Lines 211-216 iterated over selected deployable modules, ignored modules without deployments, ignored modules that did not publish a zip, and collected modules lacking lib/.
• Lines 218-228 failed the whole deployment if any module was missing build output.
• Lines 219-220 printed the missing modules and the requirement to build first.
• Lines 221-224 generated example scripts/build.sh commands, using the module’s npm package name when available.
• Line 225 suggested restricting deployment with --modules.
• Line 226 returned to the original directory.
• Line 227 exited with status 1.
• Lines 228-229 closed the condition.

After this diff, a missing lib/ does not stop all Terraform work before it starts. This is one of the concrete safeguards lost by deleting the orchestrator.

Lines 231-257: old provider-cache pre-warming

The old script tried to avoid provider-cache races before launching parallel modules.

• Lines 231-237 explained that concurrent terraform init operations could write or link the same cached provider and produce ETXTBSY (text file busy).
• Lines 238-239 set TF_PLUGIN_CACHE_DIR to $HOME/.terraform.d/plugin-cache and created it.
• Lines 241-242 defined warm_plugin_cache() and its loop variable.
• Lines 243-245 selected the first deployable module with a deployments directory.
• Line 246 logged the module used for pre-warming.
• Lines 247-250 entered that module’s deployment directory, removed local .terraform, ran terraform init -backend=false -input=false, and removed .terraform again if initialization succeeded.
• Lines 251-252 logged success and returned immediately after the first successful warm-up.
• Lines 253-255 tried the next module after failure and eventually printed a warning if none worked.
• Line 257 invoked the function.

This pre-warm pass is lost. The shared provisioner instead serializes each real terraform init with init_terraform_locked() at p8-deployments/scripts/provision.sh:101-113. That lock protects init-versus-init, not every possible init-versus-apply race.

Lines 259-265: parallel-deployment state

• Lines 259-260 created associative arrays for child process IDs and module statuses.
• Lines 262-263 created $HOME/p8_build_info/build_logs/p8-api/.
• Line 265 initialized the aggregate status to success.

The per-module log directory and PID/status arrays are lost. The generic flow records deployment status through build_utils.sh while processing each module.

Lines 267-272: old variant skip

• Lines 267-272 iterated through selected modules.
• Each module was checked with the duplicate is_deployable_module().
• A module failing the check was marked skipped in the status array and did not start a child process.

The generic loop still performs a deployability check, so this behavior is retained through a different implementation.

Lines 274-300: old parallel child deployment

This was the main deployment loop.

• Line 274 selected a per-module log path.
• Lines 276-297 started a background subshell for each module.
• Lines 277-296 redirected the whole child operation to that module’s log file.
• Lines 278-281 skipped a module with no deployments/ directory.
• Lines 283-286 explained and prepared Lambda packaging. The old code explicitly removed an existing lib.zip first, preventing a stale zip from surviving when lib/ disappeared.
• Lines 287-289 removed the old zip and zipped the contents of lib/ into lib.zip, excluding source maps.
• Lines 290-292 warned when lib/ was missing but continued.
• Lines 294-295 entered the module’s deployments directory and ran its shared provision.sh with init apply, passing alias, region, mode, product variant, release tag, module name, and auto-approval.
• Line 296 closed the log redirection.
• Line 297 backgrounded the subshell.
• Lines 299-300 saved the child PID under the module name.

The new generic loop still eventually runs each module’s provision.sh, and the shared provisioner still packages ../lib into ../lib.zip. The major differences are that packaging is now inside the shared provisioner and modules are not launched by this script in parallel.

Lines 302-310: waiting and result collection

• Lines 302-303 iterated over child module names and loaded each PID.
• Lines 304-305 marked a module successful when wait returned zero.
• Lines 306-309 marked it failed and changed the aggregate result when wait failed.
• Line 310 ended the wait loop.

This whole PID-based result collection is lost because the generic loop waits for each module’s provision call before starting the next one.

Lines 312-326: old summary and exit

• Lines 312-315 printed a module deployment summary.
• Lines 317-319 printed the overall status and returned to the original directory.
• Lines 321-324 printed an error and exited 1 when any child failed.
• Line 326 printed Success.

The generic script instead updates build status per module and later runs its normal end-of-build handling. The old p8-api-specific summary and log location disappear.

5.1 Module loop, line by line

• Lines 328-332 define the artifact-type-to-number map: Docker is 0, library is 1, AWS artifacts are 2.
• Lines 334-343 retain the special path only for p8-trader-ui and p8-ui. p8-api no longer appears here.
• Lines 344-346 enter the generic path and initialize the deployment status.
• Lines 346-351 iterate over modules and read each module’s build tool, relative path, artifact type, and API exposure flag from module_conf.yaml.
• Line 352 invokes the repository-level pre_deploy hook, if present.
• Lines 354-357 retain the deploy_local option to skip AWS artifacts.
• Lines 359-363 skip non-AWS modules that do not expose an API.
• Lines 366-369 apply the generic variant/feature gate.
• Lines 371-374 create a module-local status so a previous module failure does not incorrectly mark the current module’s own record.
• Lines 376-384 check for a module provisioner, enter its deployments directory, and call:

./provision.sh \
  -a "$env_name" \
  -r "$aws_region" \
  -m "$mode" \
  -v "$P8_PRODUCT_VARIATION" \
  -rt "$publish_tags" \
  --module "${module#p8-}" \
  --auto-approve init apply

${module#p8-} removes the p8- prefix. For example, p8-announcement becomes announcement, which is the Terraform module’s state/artifact name.

• Lines 386-388 make a failed module update the overall status file.
• Lines 390-399 calculate the build-info action type and call bld_update for the module deployment.
• Lines 401-403 end the loop and function.

The key result is that all p8-api modules use the same invocation shape as other Terraform modules.

Environment resolution

At p8-deployments/scripts/provision.sh:183-205:

1. resolve_from_envrc() does nothing in publish_only.
2. Otherwise it searches for an export_config.sh while walking upward from the module deployment directory.
3. For p8-api it finds p8-api/scripts/export_config.sh, which is a symlink to the shared p8-deployments/scripts/export_config.sh.
4. The export script creates or reuses /tmp/<alias>/.envrc.
5. load_envrc() sources that file with allexport, making values such as admin_api_id shell variables.
6. populate_tf_vars() copies selected values into TF_VAR_* variables.

The cache is shared by alias and normally lasts about two hours. The export script performs AWS lookups only when the cache is missing or expired.

Terraform initialization and apply

At p8-deployments/scripts/provision.sh:70-113:

• init_terraform() removes local .terraform, generated backend configuration, and generated terraform.tfvars.
• It calls module-specific export_vars to establish state-path and module variables.
• It generates the S3 backend configuration.
• It calls init_terraform_locked().
• init_terraform_locked() uses flock on .tf-init.lock in the shared plugin cache, then runs terraform init.
• The provisioner generates the module’s Terraform variable file and runs terraform apply -parallelism=20.

Within the new p8-api loop, modules are sequential, but Terraform itself still uses parallelism for resources inside a module.

p8-ai/deployments/variables.tf

Old comment:

REST API ids/resource ids; empty => resolved by the p8-api orchestrator and passed as TF_VAR_*.

New comment:

REST API ids/resource ids; populated from the shared environment config.

No Terraform variable, default, resource, or behavior changes in this file. The old explanation became false after removing the p8-api orchestrator. The actual source is now the shared .envrc/populate_tf_vars path.

p8-deployments/scripts/provision.sh:83

The phrase p8-api orchestrator becomes independent deployments.

This changes no shell behavior. It makes the locking comment apply to all callers, because the shared provisioner can be invoked concurrently by unrelated module or repository deployments.

p8-deployments/scripts/provision.sh:172

The parenthetical example naming the p8-api orchestrator is removed. The rule remains: pre-set TF_VAR_* values are respected.

p8-deployments/scripts/provision.sh:180-182

The old wording specifically mentioned REST API IDs exported by the p8-api orchestrator. It is replaced with generic wording saying caller-provided Terraform variables are respected.

Again, only comments changed in these locations.

scripts/deployments/build_and_deploy.sh:460-463

The packaging comment formerly gave p8-api: deploy_p8_api as an example of a repository-level orchestrator. That example is removed because deploy_p8_api no longer exists.

The code itself is unchanged: if a repository has a scripts/ directory, it is still copied into the Terraform release package. This remains important because p8-api/scripts/export_config.sh must be available when the release is deployed.