chore(scripts): add live postimages emulator test and smoke scripts

This commit is contained in:
plebeius
2026-02-19 17:49:35 +08:00
parent e26da8463a
commit 7cd46cb49b
6 changed files with 472 additions and 8 deletions
+5 -1
View File
@@ -168,4 +168,8 @@ yarn-error.log*
.cursor
.claude
.codex
.playwright-cli
.playwright-cli
# Upload selector smoke probe (report-only, local/CI triage)
scripts/upload-selectors-smoke-report.json
scripts/upload-selectors-smoke-snapshots/
+140
View File
@@ -0,0 +1,140 @@
# Upload Automation Retest Checklist
Retest checklist for Android and desktop after changes to media upload automation (`MediaUploadAutomationRunner`, `MediaUploadRecipes`, `upload-orchestrator`, etc.).
## Quality Gate (blocking)
Run before merge/rollout. All must pass.
| Command | Purpose | Interpretation |
|---------|---------|----------------|
| `yarn test` | Unit tests (Vitest) | Exit 0 = pass. Fix failing tests before merge. |
| `yarn build` | Production build | Exit 0 = pass. No build errors. |
| `yarn lint` | Lint (oxlint) | Exit 0 = pass. Fix lint errors. |
| `yarn type-check` | TypeScript (tsgo) | Exit 0 = pass. Fix type errors. |
| `yarn doctor` | React Doctor | Advisory. Prioritize `error` > `warning`. |
**Quick run (all quality gates):**
```bash
yarn retest:quality
```
---
## Postimages chooser contract (required local gate)
Runs postimages-specific emulator contract tests against deterministic fixtures. **Emulator required.** Must pass before merge.
```bash
yarn contract:postimages
```
- **Prereq:** Android emulator running or USB device connected.
- **Interpretation:** Exit 0 = contract tests pass. Runs `MediaUploadAutomationRunnerTest` (postimages chooser contract + imgur/generic fixtures). Uses deterministic fixtures in `android/app/src/main/assets/fixtures/`.
---
## Live postimages test (fully automated, real provider)
Runs a real provider upload test with no manual taps. The script will:
- install missing Android SDK components when `sdkmanager` is available
- create/start an emulator AVD when needed
- run `PostimagesLiveUploadTest` on a booted emulator/device
- generate a blank white `100x100` PNG at runtime inside the instrumentation test
```bash
yarn live:postimages:auto
```
- **Prereq:** Android command-line tools installed (`sdkmanager`, `avdmanager`, `adb`, `emulator`) or available in `ANDROID_SDK_ROOT` / `ANDROID_HOME`.
- **Interpretation:** Exit 0 = real upload to `postimages.org` succeeded end-to-end.
- **Note:** This is a live-provider check (network/captcha/rate-limit dependent), so it can be flaky compared to fixture contracts.
---
## Android Emulator Verification (report-only)
Instrumentation tests run on an **emulator or physical device**. Requires an emulator/device to be running and connected.
### Standard run
```bash
yarn android:connectedTest
```
Or directly (uses UTP workaround for protobuf compatibility):
```bash
cd android && ./gradlew :app:connectedDebugAndroidTest -Pandroid.experimental.androidTest.useUnifiedTestPlatform=false
```
- **Prereq:** Emulator running (`emulator -avd <avd_name>` or Android Studio) or USB device.
- **Interpretation:** Exit 0 = all instrumented tests pass. `MediaUploadAutomationRunnerTest` exercises fixtures in `android/app/src/main/assets/fixtures/`.
### UTP / Protobuf issue (known)
On some setups, `connectedDebugAndroidTest` fails with protobuf classloader conflicts (e.g. `IllegalAccessError` involving `com.google.protobuf.CodedInputStream`). This is a known UTP/Espresso + protobuf incompatibility.
**Workaround (disable UTP, use legacy test runner):**
Use this exact command, which passes `-Pandroid.experimental.androidTest.useUnifiedTestPlatform=false` to disable the unified test platform:
```bash
cd android && ./gradlew :app:connectedDebugAndroidTest -Pandroid.experimental.androidTest.useUnifiedTestPlatform=false
```
> **Caveat:** The `useUnifiedTestPlatform=false` flag may be deprecated in future Android Gradle Plugin versions; if it stops working, use the fallback below.
If the above does not resolve it, exclude conflicting protobuf from Espresso in `android/app/build.gradle`:
```gradle
androidTestImplementation("androidx.test.espresso:espresso-core:3.3.0") {
exclude group: "com.google.protobuf", module: "protobuf-java"
exclude group: "com.google.protobuf", module: "protobuf-lite"
}
```
---
## Upload Selector Smoke Run (report-only, non-blocking)
Probes live upload sites (Imgur, PostImages) to verify selectors in `media-upload-recipes` still match. **Always exits 0**; intended for CI report-only and local triage. **Not a merge gate**—check report for selector drift but do not block.
```bash
yarn smoke:upload-selectors
```
- **Output (report):** `scripts/upload-selectors-smoke-report.json`
- **Output (snapshots):** `scripts/upload-selectors-smoke-snapshots/` (provider PNGs)
- **Interpretation:** Check `summary.ok` vs `summary.warn` in the report. Warnings indicate selectors may need updates; do not block merge without further triage.
---
## Desktop (Electron)
Build and run the Electron app to manually verify upload flows:
```bash
yarn electron:start
```
Unit tests for Electron automation (non-blocking, report-only):
```bash
node electron/media-upload-automation.test.js
node electron/media-upload-recipes.test.js
```
---
## Summary
| Check | Blocking? | Command |
|-------|-----------|---------|
| Quality gate | Yes | `yarn retest:quality` |
| Postimages chooser contract | Yes | `yarn contract:postimages` |
| Postimages live provider automation | No (recommended local) | `yarn live:postimages:auto` |
| Android instrumentation | No (report-only) | `yarn android:connectedTest` |
| Upload selector smoke | No (report-only) | `yarn smoke:upload-selectors` |
| Electron unit tests | No (report-only) | `node electron/media-upload-*.test.js` |
+7 -1
View File
@@ -83,7 +83,12 @@
"i18n:update:dry": "node scripts/update-translations.js --dry",
"doctor": "react-doctor . -y",
"doctor:score": "react-doctor . --score -y",
"doctor:verbose": "react-doctor . --verbose -y"
"doctor:verbose": "react-doctor . --verbose -y",
"contract:postimages": "cd android && ./gradlew :app:connectedDebugAndroidTest -Pandroid.experimental.androidTest.useUnifiedTestPlatform=false -Pandroid.testInstrumentationRunnerArguments.class=fivechan.android.MediaUploadAutomationRunnerTest",
"live:postimages:auto": "bash scripts/run-postimages-live-emulator-test.sh",
"smoke:upload-selectors": "node scripts/smoke-upload-selectors.js",
"retest:quality": "yarn test && yarn build && yarn lint && yarn type-check && yarn doctor",
"android:connectedTest": "cd android && ./gradlew :app:connectedDebugAndroidTest -Pandroid.experimental.androidTest.useUnifiedTestPlatform=false"
},
"browserslist": {
"production": [
@@ -131,6 +136,7 @@
"lint-staged": "12.3.8",
"node-fetch": "3.3.2",
"oxfmt": "0.20.0",
"playwright": "1.56.1",
"oxlint": "1.35.0",
"progress": "2.0.3",
"react-doctor": "0.0.17",
@@ -0,0 +1,187 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
ANDROID_DIR="$ROOT_DIR/android"
API_LEVEL="${ANDROID_API_LEVEL:-35}"
HOST_ARCH="$(uname -m)"
DEFAULT_ABI="x86_64"
if [[ "$HOST_ARCH" == "arm64" || "$HOST_ARCH" == "aarch64" ]]; then
DEFAULT_ABI="arm64-v8a"
fi
ABI="${ANDROID_ABI:-$DEFAULT_ABI}"
TAG="${ANDROID_TAG:-google_apis}"
SYSTEM_IMAGE_PACKAGE="system-images;android-${API_LEVEL};${TAG};${ABI}"
SYSTEM_IMAGE_DIR="system-images/android-${API_LEVEL}/${TAG}/${ABI}"
AVD_NAME="${ANDROID_AVD_NAME:-fivechan-postimages-api${API_LEVEL}}"
DEVICE_PROFILE="${ANDROID_DEVICE_PROFILE:-pixel_6}"
TEST_CLASS="${ANDROID_TEST_CLASS:-fivechan.android.PostimagesLiveUploadTest}"
KEEP_EMULATOR="${KEEP_EMULATOR:-0}"
if [[ -n "${ANDROID_SDK_ROOT:-}" ]]; then
SDK_ROOT="$ANDROID_SDK_ROOT"
elif [[ -n "${ANDROID_HOME:-}" ]]; then
SDK_ROOT="$ANDROID_HOME"
elif [[ -d "$HOME/Library/Android/sdk" ]]; then
SDK_ROOT="$HOME/Library/Android/sdk"
else
SDK_ROOT=""
fi
resolve_tool() {
local tool="$1"
shift || true
local candidate=""
for dir in "$@"; do
if [[ -n "$dir" && -x "$dir/$tool" ]]; then
candidate="$dir/$tool"
break
fi
done
if [[ -z "$candidate" ]] && command -v "$tool" >/dev/null 2>&1; then
candidate="$(command -v "$tool")"
fi
printf '%s' "$candidate"
}
sdk_tool_dir() {
local relative="$1"
if [[ -z "$SDK_ROOT" ]]; then
printf ''
return
fi
printf '%s/%s' "$SDK_ROOT" "$relative"
}
SDKMANAGER_BIN="$(resolve_tool "sdkmanager" "$(sdk_tool_dir "cmdline-tools/latest/bin")" "$(sdk_tool_dir "cmdline-tools/bin")" "$(sdk_tool_dir "tools/bin")")"
AVDMANAGER_BIN="$(resolve_tool "avdmanager" "$(sdk_tool_dir "cmdline-tools/latest/bin")" "$(sdk_tool_dir "cmdline-tools/bin")" "$(sdk_tool_dir "tools/bin")")"
ADB_BIN="$(resolve_tool "adb" "$(sdk_tool_dir "platform-tools")")"
EMULATOR_BIN="$(resolve_tool "emulator" "$(sdk_tool_dir "emulator")")"
missing_sdk_component=0
if [[ -z "$SDK_ROOT" || ! -d "$SDK_ROOT" ]]; then
missing_sdk_component=1
fi
if [[ -z "$ADB_BIN" || -z "$EMULATOR_BIN" || -z "$AVDMANAGER_BIN" ]]; then
missing_sdk_component=1
fi
if [[ -n "$SDK_ROOT" ]]; then
if [[ ! -d "$SDK_ROOT/platforms/android-${API_LEVEL}" ]]; then
missing_sdk_component=1
fi
if [[ ! -d "$SDK_ROOT/$SYSTEM_IMAGE_DIR" ]]; then
missing_sdk_component=1
fi
fi
if [[ "$missing_sdk_component" -eq 1 ]]; then
if [[ -z "$SDKMANAGER_BIN" ]]; then
echo "Android SDK components are missing and sdkmanager was not found."
echo "Install Android command-line tools, then rerun this script."
exit 1
fi
echo "Installing/updating Android SDK components for API ${API_LEVEL}..."
yes | "$SDKMANAGER_BIN" --licenses >/dev/null 2>&1 || true
"$SDKMANAGER_BIN" --install \
"platform-tools" \
"emulator" \
"platforms;android-${API_LEVEL}" \
"$SYSTEM_IMAGE_PACKAGE"
# Re-resolve after installation
if [[ -z "$SDK_ROOT" ]]; then
SDK_ROOT="$(dirname "$(dirname "$(dirname "$SDKMANAGER_BIN")")")"
fi
ADB_BIN="$(resolve_tool "adb" "$(sdk_tool_dir "platform-tools")")"
EMULATOR_BIN="$(resolve_tool "emulator" "$(sdk_tool_dir "emulator")")"
AVDMANAGER_BIN="$(resolve_tool "avdmanager" "$(sdk_tool_dir "cmdline-tools/latest/bin")" "$(sdk_tool_dir "cmdline-tools/bin")" "$(sdk_tool_dir "tools/bin")")"
fi
if [[ -z "$ADB_BIN" || -z "$EMULATOR_BIN" || -z "$AVDMANAGER_BIN" ]]; then
echo "Could not locate required Android tools (adb/emulator/avdmanager)."
exit 1
fi
first_online_emulator() {
"$ADB_BIN" devices | awk '$1 ~ /^emulator-[0-9]+$/ && $2 == "device" { print $1; exit }'
}
if ! "$EMULATOR_BIN" -list-avds | awk -v avd="$AVD_NAME" '$0 == avd { found = 1 } END { exit found ? 0 : 1 }'; then
echo "Creating AVD: $AVD_NAME ($SYSTEM_IMAGE_PACKAGE)"
echo "no" | "$AVDMANAGER_BIN" create avd --name "$AVD_NAME" --package "$SYSTEM_IMAGE_PACKAGE" --device "$DEVICE_PROFILE" --force
fi
avd_config="$HOME/.android/avd/${AVD_NAME}.avd/config.ini"
if [[ -f "$avd_config" ]] && ! awk -v expected="abi.type=${ABI}" '$0 == expected { found = 1 } END { exit found ? 0 : 1 }' "$avd_config"; then
echo "Recreating AVD $AVD_NAME for ABI $ABI"
"$AVDMANAGER_BIN" delete avd --name "$AVD_NAME" >/dev/null 2>&1 || true
echo "no" | "$AVDMANAGER_BIN" create avd --name "$AVD_NAME" --package "$SYSTEM_IMAGE_PACKAGE" --device "$DEVICE_PROFILE" --force
fi
started_emulator=0
serial="${ANDROID_SERIAL:-}"
if [[ -z "$serial" ]]; then
serial="$(first_online_emulator)"
fi
if [[ -z "$serial" ]]; then
echo "Starting emulator AVD: $AVD_NAME"
"$EMULATOR_BIN" -avd "$AVD_NAME" -no-boot-anim -no-snapshot-save -netdelay none -netspeed full >/tmp/postimages-live-emulator.log 2>&1 &
emulator_pid=$!
started_emulator=1
for _ in {1..90}; do
if ! kill -0 "$emulator_pid" >/dev/null 2>&1; then
break
fi
serial="$(first_online_emulator)"
if [[ -n "$serial" ]]; then
break
fi
sleep 2
done
fi
if [[ -z "$serial" ]]; then
echo "Failed to detect running emulator serial. Check /tmp/postimages-live-emulator.log"
exit 1
fi
cleanup() {
if [[ "$started_emulator" -eq 1 && "$KEEP_EMULATOR" != "1" ]]; then
"$ADB_BIN" -s "$serial" emu kill >/dev/null 2>&1 || true
fi
}
trap cleanup EXIT
echo "Using emulator serial: $serial"
"$ADB_BIN" -s "$serial" wait-for-device
boot_completed=""
for _ in {1..180}; do
boot_completed="$("$ADB_BIN" -s "$serial" shell getprop sys.boot_completed 2>/dev/null | tr -d '\r')"
if [[ "$boot_completed" == "1" ]]; then
break
fi
sleep 2
done
if [[ "$boot_completed" != "1" ]]; then
echo "Emulator did not finish booting in time."
exit 1
fi
"$ADB_BIN" -s "$serial" shell settings put global window_animation_scale 0 >/dev/null 2>&1 || true
"$ADB_BIN" -s "$serial" shell settings put global transition_animation_scale 0 >/dev/null 2>&1 || true
"$ADB_BIN" -s "$serial" shell settings put global animator_duration_scale 0 >/dev/null 2>&1 || true
echo "Running live postimages upload instrumentation test..."
pushd "$ANDROID_DIR" >/dev/null
ANDROID_SERIAL="$serial" ./gradlew \
:app:connectedDebugAndroidTest \
-Pandroid.experimental.androidTest.useUnifiedTestPlatform=false \
-Pandroid.testInstrumentationRunnerArguments.class="$TEST_CLASS"
popd >/dev/null
echo "Live postimages upload test completed."
+122
View File
@@ -0,0 +1,122 @@
#!/usr/bin/env node
/**
* Live smoke probe for imgur/postimages upload selectors.
* Probes current DOM and reports which configured selectors still match.
* Non-blocking: always exits 0. Intended for CI report-only and local triage.
*
* Output:
* - scripts/upload-selectors-smoke-report.json
* - scripts/upload-selectors-smoke-snapshots/ (screenshots)
*
* Run: yarn smoke:upload-selectors
*/
import { chromium } from 'playwright';
import { writeFile, mkdir } from 'fs/promises';
import { dirname, join } from 'path';
import { fileURLToPath } from 'url';
const __dirname = dirname(fileURLToPath(import.meta.url));
const REPO_ROOT = join(__dirname, '..');
const REPORT_PATH = join(REPO_ROOT, 'scripts', 'upload-selectors-smoke-report.json');
const SNAPSHOTS_DIR = join(REPO_ROOT, 'scripts', 'upload-selectors-smoke-snapshots');
const PROVIDERS = ['imgur', 'postimages'];
/** Selectors to probe (from recipes). Loaded dynamically to avoid circular deps. */
async function loadRecipes() {
const mod = await import('../electron/media-upload-recipes.js');
return mod.MEDIA_UPLOAD_RECIPES;
}
/** Check which selectors match in the page. */
async function probeSelectors(page, selectors) {
const results = [];
for (const sel of selectors) {
try {
const count = await page.locator(sel).count();
results.push({ selector: sel, matches: count });
} catch (e) {
results.push({ selector: sel, matches: 0, error: String(e?.message ?? e) });
}
}
return results;
}
async function run() {
const recipes = await loadRecipes();
const report = {
timestamp: new Date().toISOString(),
providers: {},
summary: { ok: 0, warn: 0 },
};
let browser;
try {
browser = await chromium.launch({ headless: true });
await mkdir(SNAPSHOTS_DIR, { recursive: true });
for (const provider of PROVIDERS) {
const recipe = recipes[provider];
if (!recipe) continue;
const url = recipe.uploadUrl;
const entry = {
url,
fileInput: [],
submit: [],
success: [],
blocked: [],
screenshot: null,
loadError: null,
};
try {
const page = await browser.newPage();
await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 30_000 });
await page.waitForTimeout(2500);
entry.fileInput = await probeSelectors(page, recipe.fileInputSelectorCandidates);
entry.submit = await probeSelectors(page, recipe.submitSelectorCandidates);
entry.success = await probeSelectors(page, recipe.successExtractor.selectorCandidates);
entry.blocked = await probeSelectors(page, recipe.blockedIndicators);
const snapshotPath = join(SNAPSHOTS_DIR, `${provider}-${Date.now()}.png`);
await page.screenshot({ path: snapshotPath });
entry.screenshot = snapshotPath;
const fileInputOk = entry.fileInput.some((r) => r.matches > 0);
const submitOk = entry.submit.some((r) => r.matches > 0);
const successOk = entry.success.some((r) => r.matches > 0);
const blockedPresent = entry.blocked.some((r) => r.matches > 0);
if (fileInputOk && submitOk && successOk && !blockedPresent) {
report.summary.ok++;
} else {
report.summary.warn++;
}
await page.close();
} catch (e) {
entry.loadError = String(e?.message ?? e);
report.summary.warn++;
}
report.providers[provider] = entry;
}
} catch (e) {
report.error = String(e?.message ?? e);
} finally {
if (browser) await browser.close();
}
await writeFile(REPORT_PATH, JSON.stringify(report, null, 2));
console.log(`Smoke report written to ${REPORT_PATH}`);
console.log(`Summary: ${report.summary?.ok ?? 0} OK, ${report.summary?.warn ?? 0} warn. Snapshots in ${SNAPSHOTS_DIR}`);
}
run()
.then(() => process.exit(0))
.catch((err) => {
console.error(err);
process.exit(0);
});
+11 -6
View File
@@ -959,6 +959,11 @@
resolved "https://registry.yarnpkg.com/@babel/runtime/-/runtime-7.28.4.tgz#a70226016fabe25c5783b2f22d3e1c9bc5ca3326"
integrity sha512-Q/N6JNWvIvPnLDvjlE1OUBLPQHH6l3CltCEsHIujp45zQUSSh8K+gHnaEX45yAT1nyngnINhvWtzN+Nb9D8RAQ==
"@babel/runtime@^7.27.6":
version "7.28.6"
resolved "https://registry.yarnpkg.com/@babel/runtime/-/runtime-7.28.6.tgz#d267a43cb1836dc4d182cce93ae75ba954ef6d2b"
integrity sha512-05WQkdpL9COIMz4LjTxGpPNCdlpyimKppYNoJ5Di5EUObifl8t4tuLuUBBZEpoLYOmfvIWrsp9fCl0HoPRVTdA==
"@babel/template@^7.27.1", "@babel/template@^7.27.2":
version "7.27.2"
resolved "https://registry.yarnpkg.com/@babel/template/-/template-7.27.2.tgz#fa78ceed3c4e7b63ebf6cb39e5852fca45f6809d"
@@ -13207,7 +13212,7 @@ playwright-core@1.56.1:
resolved "https://registry.yarnpkg.com/playwright-core/-/playwright-core-1.56.1.tgz#24a66481e5cd33a045632230aa2c4f0cb6b1db3d"
integrity sha512-hutraynyn31F+Bifme+Ps9Vq59hKuUCz7H1kDOcBs+2oGguKkWTU50bBWrtz34OUWmIwpBTWDxaRPXrIXkgvmQ==
playwright@^1.49.0:
playwright@1.56.1, playwright@^1.49.0:
version "1.56.1"
resolved "https://registry.yarnpkg.com/playwright/-/playwright-1.56.1.tgz#62e3b99ddebed0d475e5936a152c88e68be55fbf"
integrity sha512-aFi5B0WovBHTEvpM3DzXTUaeN6eN0qWnTkKx4NQaH4Wvcmc153PdaY2UBdSYKaGYw+UyWXSVyxDUg5DoPEttjw==
@@ -13581,12 +13586,12 @@ react-grab@0.0.98:
bippy "^0.5.28"
solid-js "^1.9.10"
react-i18next@13.2.2:
version "13.2.2"
resolved "https://registry.yarnpkg.com/react-i18next/-/react-i18next-13.2.2.tgz#b1e78ed66a54f4bc819616f68b98221e1b1a1936"
integrity sha512-+nFUkbRByFwnrfDcYqvzBuaeZb+nACHx+fAWN/pZMddWOCJH5hoc21+Sa/N/Lqi6ne6/9wC/qRGOoQhJa6IkEQ==
react-i18next@15.7.4:
version "15.7.4"
resolved "https://registry.yarnpkg.com/react-i18next/-/react-i18next-15.7.4.tgz#146e50f220d204b842e22c75d1a3d23c6c589a30"
integrity sha512-nyU8iKNrI5uDJch0z9+Y5XEr34b0wkyYj3Rp+tfbahxtlswxSCjcUL9H0nqXo9IR3/t5Y5PKIA3fx3MfUyR9Xw==
dependencies:
"@babel/runtime" "^7.22.5"
"@babel/runtime" "^7.27.6"
html-parse-stringify "^3.0.1"
react-is@^16.13.1: