External/SnapOtter
1
0
Fork 0
mirror of https://github.com/snapotter-hq/SnapOtter.git synced 2026-08-03 07:46:42 +02:00
Code Issues Packages Projects Releases Wiki Activity

fix(ai-bundles): lock the numpy-1.x ABI closure so the OCR bundle can't strand scipy

Browse Source 
The OCR bundle installs paddleocr[doc-parser] 3.4, whose dependency closure drags
numpy 1.26.4 up to 2.5.1 and pulls scipy/scikit-learn/pandas wheels built against
the numpy 2.x ABI. build-bundle.sh re-pinned only numpy (basePackages), so those
numpy-2.x wheels stayed behind; the by-dir-name site-packages diff then shipped
them, and once merged onto the numpy==1.26.4 base they raise "numpy.dtype size
changed" on import.

Because the dispatcher pre-imports every ML library at startup and disables all AI
after 5 crashes in 60s, one stranded scipy takes down every AI tool, not just OCR
(observed on a CPU host: remove-background worked before the OCR bundle and broke
after). All-7 installs escaped it through last-writer-wins ordering; a subset
install did not, which is why it surfaced only intermittently.

Fix: add a manifest "constraints" list (numpy, scipy, scikit-learn, scikit-image,
pandas pinned to numpy-1.x-ABI versions) and apply it via PIP_CONSTRAINT to every
bundle pip install, so no bundle can pull a numpy-2.x wheel. paddleocr 3.4.1 still
resolves cleanly under the lock and the pinned stack imports without ABI error on
numpy 1.26.4 (validated on py3.12). Also import scipy/sklearn in the OCR path of
verify-bundle.sh so CI catches this class in isolation, and add a manifest
regression test.

Note: the published bundles must be rebuilt and republished (ai-bundles.yml) for
this to reach already-installed bases.

Claude-Session: https://claude.ai/code/session_01UvVCMNUBrgpghk8gye5gav
This commit is contained in:
SnapOtter
2026-07-05 18:31:30 +08:00
parent 47a60e7fad
commit 67c55669d6
4 changed files with 74 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docker/build-bundle.sh
+26
View File
@@ -65,6 +65,32 @@ if '${BUNDLE_ID}' not in m['bundles']:
sys.exit(1)
"
# ── Step 0: Write pip constraints (numpy-1.x-ABI closure lock) ────────────
# Bundle installs below can otherwise pull the latest transitive scientific
# stack: numpy 2.x plus scipy/scikit-learn/scikit-image/pandas wheels built
# against the numpy 2.x ABI (e.g. paddleocr[doc-parser] drags numpy 1.26.4 ->
# 2.5.1 + scipy 1.18). The Step 4 re-pin snaps numpy back to 1.26.4 but leaves
# those numpy-2.x wheels behind; the Step 6 diff (by dir name, not version)
# then ships them, and once installed they strand on the numpy==1.26.4 base and
# raise "numpy.dtype size changed" on import, crashing the dispatcher for
# EVERY AI tool (e.g. rembg's scipy), not just this bundle. Applying the
# manifest's constraints to every pip install keeps the whole closure on
# numpy-1.x-ABI versions so no bundle can strand a numpy-2.x wheel.
CONSTRAINTS_FILE="/tmp/bundle-constraints.txt"
python3 -c "
import json
with open('${MANIFEST}') as f:
m = json.load(f)
with open('${CONSTRAINTS_FILE}', 'w') as f:
f.write('\n'.join(m.get('constraints', [])) + '\n')
"
if [[ -s "${CONSTRAINTS_FILE}" ]] && [[ -n "$(tr -d '[:space:]' < "${CONSTRAINTS_FILE}")" ]]; then
export PIP_CONSTRAINT="${CONSTRAINTS_FILE}"
echo " Build constraints: $(tr '\n' ' ' < "${CONSTRAINTS_FILE}")"
else
echo " No build constraints in manifest"
fi
# Clean previous build artifacts
rm -rf "${MODELS_DIR}" "${BUILD_DIR}"
mkdir -p "${MODELS_DIR}" "${BUILD_DIR}/site-packages" "${BUILD_DIR}/models" "${OUTPUT_DIR}"