Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 19 additions & 15 deletions RELEASES.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,23 +164,27 @@ Use `source-tag = 3.11.0` and `release-tag = testnet-release` or
## Ops tooling

[`scripts/ops/menu.sh`](./scripts/ops/menu.sh) is the entry point for the
scripted parts of a release. It offers two things:
scripted parts of a release:

1. **release github code** — runs `prepare-github-release.sh` (step 1 above).
2. **migrate devnet cluster** — rolls a published release out to a NEAR One dev
cluster via [`scripts/ops/dev-cluster/dev-menu.sh`](./scripts/ops/dev-cluster/dev-menu.sh).

The dev-cluster flow asks for the network (testnet first, then mainnet), the
version, and the cluster's Nomad IP and credentials, then swaps each
`mpc-node-*` Nomad job to the release image (plan, confirm, run) and checks the
nodes report the new `release=` in their build info.

Every command is printed before it runs and every write is behind a
confirmation prompt, so a run can be stopped at any step. Nothing
cluster-specific is stored in this repo; addresses and credentials are typed in
per run, or supplied through the per-network `NOMAD_ADDR_DEV_*`,
`NOMAD_HTTP_AUTH_DEV_*`, and `MPC_NODE_ADDRS_DEV_*` environment variables to
skip the matching prompt.
2. **migrate devnet cluster** — rolls a published release out to a NEAR One
dev cluster ([`dev-menu.sh`](./scripts/ops/dev-cluster/dev-menu.sh)).

The dev-cluster flow asks for the network (testnet before mainnet), version,
and the cluster's Nomad IP and credentials, then follows the runbook: swap
each `mpc-node-*` Nomad job to the release image (plan, confirm, run), check
the nodes report the new `release=`, offer a test signature — a real on-chain
`sign` from the first member account, 1 NEAR (testnet) / 0.1 NEAR (mainnet)
deposit, behind its own confirmation — and finally, only for releases that
change `crates/contract`, propose and vote the contract update.

Every command is printed before it runs; every write sits behind a
confirmation prompt. Addresses and credentials are never stored in this
repo — they are typed in per run, or supplied via the per-network
`NOMAD_ADDR_DEV_*`, `NOMAD_HTTP_AUTH_DEV_*`, and `MPC_NODE_ADDRS_DEV_*`
variables. On-chain steps sign with near-cli (OS
keychain by default; `MPC_SIGN_WITH=sign-with-legacy-keychain` for
`~/.near-credentials` keys).

## Re-running after a failure

Expand Down
14 changes: 12 additions & 2 deletions scripts/ops/dev-cluster/dev-common.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@

SIGN_WITH="${MPC_SIGN_WITH:-sign-with-keychain}"

# Sets CONTRACT, NEAR_NET, MEMBER_ACCOUNTS, SIGN_DEPOSIT and re-points
# endpoint vars from per-cluster exports (NOMAD_ADDR_DEV_TESTNET, ...)
# Sets CONTRACT, NEAR_NET, MEMBER_ACCOUNTS, SIGN_DEPOSIT, PROPOSE_DEPOSIT and
# re-points endpoint vars from per-cluster exports (NOMAD_ADDR_DEV_TESTNET, ...)
# so the network choice drives every step; addresses stay out of this repo.
resolve_dev_cluster() {
local suffix var
Expand All @@ -25,6 +25,10 @@ resolve_dev_cluster() {
suffix="MAINNET" ;;
*) die "Unknown dev cluster '$1' (expected testnet|mainnet)." ;;
esac
# Over propose_update_required_deposit_yoctonear; excess is refunded.
# Read by upgrade-dev-contract.sh.
PROPOSE_DEPOSIT="16 NEAR"

var="NOMAD_ADDR_DEV_${suffix}"; [[ -z "${!var:-}" ]] || export NOMAD_ADDR="${!var}"
var="MPC_NODE_ADDRS_DEV_${suffix}"; [[ -z "${!var:-}" ]] || export MPC_NODE_ADDRS="${!var}"
# +set: an intentionally empty value still disables the prompt.
Expand Down Expand Up @@ -82,6 +86,12 @@ nomad_auth_state() {
else echo "(none)"; fi
}

# Read-only contract query against the resolved cluster.
near_view() {
run_cmd near contract call-function as-read-only "$CONTRACT" "$1" \
json-args '{}' network-config "$NEAR_NET" now
}

# Check every MPC_NODE_ADDRS node reports release="<version>". Retries per
# node — a node can still be warming up right after its allocation starts.
verify_nodes() {
Expand Down
21 changes: 18 additions & 3 deletions scripts/ops/dev-cluster/dev-menu.sh
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/usr/bin/env bash
#
# dev-menu.sh — entry point for dev-cluster work. Picks the network and
# version, then upgrades the cluster nodes and verifies them.
# version, then runs the upgrade in runbook order: nodes, verify, contract.
#
# Usage: ./scripts/ops/dev-cluster/dev-menu.sh [testnet|mainnet] [VERSION]
# Prompts for the Nomad IP, credentials, and node metrics addresses; exporting
Expand Down Expand Up @@ -62,7 +62,7 @@ confirm "Proceed?" || { echo "Aborted."; exit 0; }

step "### Step 1 — nodes"
run_cmd "${SCRIPT_DIR}/migrate-dev-nodes.sh" "$VERSION" \
|| die "Node upgrade did not complete."
|| die "Node upgrade did not complete — stopping before the contract step."

step "### Verify"
if [[ -n "${MPC_NODE_ADDRS:-}" ]]; then
Expand All @@ -72,5 +72,20 @@ else
fi
run_step test_sign "$NETWORK" || true

step "### Step 2 — contract"
echo "Only for releases that change crates/contract (diff it between the two tags)."
CONTRACT_RESULT="skipped — nodes only"
if confirm "Upgrade the contract too?"; then
if run_cmd "${SCRIPT_DIR}/upgrade-dev-contract.sh" "$VERSION" "$NETWORK"; then
CONTRACT_RESULT="upgraded"
else
CONTRACT_RESULT="FAILED — see the output above"
fi
fi

echo
ok "Done. Testnet first — upgrade the mainnet dev cluster only once this one is healthy."
if [[ "$CONTRACT_RESULT" == FAILED* ]]; then
warn "Nodes done; contract step ${CONTRACT_RESULT}. Do not upgrade the mainnet dev cluster yet."
else
ok "Done (contract: ${CONTRACT_RESULT}). Testnet first — upgrade the mainnet dev cluster only once this one is healthy."
fi
128 changes: 128 additions & 0 deletions scripts/ops/dev-cluster/upgrade-dev-contract.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
#!/usr/bin/env bash
#
# upgrade-dev-contract.sh — Step 2 of a dev-cluster upgrade: get the contract
# WASM (published release or a local build), borsh-serialize it, propose the
# update and vote it in with the cluster's member accounts. Run only after the
# nodes are on the new version.
#
# Usage: ./scripts/ops/dev-cluster/upgrade-dev-contract.sh <VERSION> <testnet|mainnet>
# Env: MPC_WASM_SOURCE=release|build skips the source prompt;
# MPC_SIGN_WITH overrides the signing method (default sign-with-keychain);
# MPC_OPS_CACHE (default ~/.cache/mpc-ops) holds the artifacts.
#

set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck source=../common.sh
source "${SCRIPT_DIR}/../common.sh"
# shellcheck source=dev-common.sh
source "${SCRIPT_DIR}/dev-common.sh"

write_u32_le() {
local n=$1 i
for i in 0 8 16 24; do
# shellcheck disable=SC2059
printf "\\x$(printf '%02x' $(( (n >> i) & 0xFF )))"
done
}

# Echoes the wasm path; progress goes to stderr so it stays capturable.
fetch_wasm() {
local version=$1 dir=$2 source=${MPC_WASM_SOURCE:-}
local wasm="${dir}/mpc-contract-v${version}.wasm"

if [[ -z "$source" ]]; then
local choice
read -rp "WASM source — (r)eleased ${version} or local (b)uild? [r] " choice >&2
case "${choice:-r}" in
r|R) source=release ;;
b|B) source=build ;;
*) die "Unknown source '${choice}'." ;;
esac
fi

if [[ "$source" == release ]]; then
if [[ -f "$wasm" ]]; then
step "==> Reusing ${wasm}" >&2
else
step "==> Downloading contract WASM from release ${version}..." >&2
require_cmds gh tar
run_cmd gh release download "$version" --repo near/mpc \
--pattern "mpc-contract-v${version}.tar.gz" --dir "$dir" --clobber >&2
run_cmd tar xzf "${dir}/mpc-contract-v${version}.tar.gz" -C "$dir" >&2
[[ -f "$wasm" ]] || die "Expected ${wasm} after extracting the tarball."
fi
else
require_cmds cargo git
local root built
root=$(git -C "$SCRIPT_DIR" rev-parse --show-toplevel)
step "==> Building the contract from ${root} (local build — not a released artifact)..." >&2
( cd "$root" && run_cmd cargo near build non-reproducible-wasm --features abi \
--profile=release-contract --manifest-path crates/contract/Cargo.toml --locked >&2 )
# Named, not globbed: target/near also holds tee_verifier and
# test_parallel_contract.
built="${root}/target/near/mpc_contract/mpc_contract.wasm"
[[ -f "$built" ]] || die "Expected ${built} after the cargo-near build."
cp "$built" "$wasm"
fi
echo "$wasm"
}

[[ $# -eq 2 ]] || die "Usage: $0 <VERSION> <testnet|mainnet>"
VERSION=$1
check_version "$VERSION"
resolve_dev_cluster "$2"
require_cmds near

CACHE="${MPC_OPS_CACHE:-$HOME/.cache/mpc-ops}/${VERSION}"
mkdir -p "$CACHE"

WASM=$(fetch_wasm "$VERSION" "$CACHE")
echo " wasm sha256: $(sha256_of "$WASM")"

SERIALIZED="${CACHE}/serialized.bin"
WASM_SIZE=$(wc -c < "$WASM")
# borsh ProposeUpdateArgs { code: Some(wasm), config: None }
{
printf '\x01'
write_u32_le "$WASM_SIZE"
cat "$WASM"
printf '\x00'
} > "$SERIALIZED"
[[ "$(wc -c < "$SERIALIZED")" -eq $((WASM_SIZE + 6)) ]] \
|| die "serialized.bin has an unexpected length."
step "==> ${SERIALIZED} ready ($(wc -c < "$SERIALIZED") bytes)"

PROPOSER=${MEMBER_ACCOUNTS%% *}
PROPOSE_CMD=(near contract call-function as-transaction "$CONTRACT" propose_update
file-args "$SERIALIZED" prepaid-gas '100.0 Tgas' attached-deposit "$PROPOSE_DEPOSIT"
sign-as "$PROPOSER" network-config "$NEAR_NET" "$SIGN_WITH" send)

step "About to propose the ${VERSION} contract on ${CONTRACT} (${NEAR_NET})"
echo " proposer: ${PROPOSER}, deposit ${PROPOSE_DEPOSIT}"
show_cmd "${PROPOSE_CMD[@]}"
confirm "Send propose_update?" || { echo "Aborted before proposing."; exit 0; }

"${PROPOSE_CMD[@]}" \
|| die "propose_update failed (an account low on NEAR is the usual cause — top it up)."

step "==> Pending proposals:"
near_view proposed_updates || true

# near-cli's result format is too unstable to parse an id out of.
read -rp "UpdateId to vote on: " UPDATE_ID
[[ "$UPDATE_ID" =~ ^[0-9]+$ ]] || die "'${UPDATE_ID}' is not a numeric UpdateId."

# The deciding vote deploys + migrates inline, hence 300 Tgas.
for account in $MEMBER_ACCOUNTS; do
vote_cmd=(near contract call-function as-transaction "$CONTRACT" vote_update
json-args "{\"id\": ${UPDATE_ID}}" prepaid-gas '300.0 Tgas' attached-deposit '0 NEAR'
sign-as "$account" network-config "$NEAR_NET" "$SIGN_WITH" send)
show_cmd "${vote_cmd[@]}"
confirm "Vote for update ${UPDATE_ID} as ${account}?" || { echo " skipped."; continue; }
"${vote_cmd[@]}" || echo " vote failed for ${account}."
done

step "==> Contract version (expect ${VERSION} once threshold was reached):"
near_view version || true