Readiness
- Readiness
- Production Ready
- Smoke
- passed
- Artifacts
- passed
- Production
- production_ready_api_slurm_amplify_120m_embedding
- Storage
- <5
- Strategy
- shared python312_slim.sif + tool ext3 overlay
- Next
- Production Ready. Optional hardening: add batching, expose per-residue embeddings, and replace CPU xformers shim with a pinned native xformers runtime if needed.
AMPLIFY source cloned from chandar-lab/AMPLIFY rev 7fb657a6bd411e0d51517ccde7cc35391c072b65; MIT license. Public AMPLIFY 350M/120M/base checkpoints listed on HF/Zenodo; no token requirement found and no checkpoints downloaded. Repo includes 30 MB synthetic reference model under tests/reference-model. Existing closest shared env has torch 2.4.1+cpu/transformers 4.56.2 but lacks xformers/accelerate/datasets; AMPLIFY imports xformers at module import. Package pins expect torch 2.2.*, transformers 4.38.*, accelerate 0.27.*, xformers 0.0.24.*, datasets 2.17.*. No Slurm smoke yet. 2026-07-05 SOP recheck: public checkpoints do not require token, but no checkpoint is cached and current shared env lacks xformers/accelerate/datasets with AMPLIFY-compatible pinning. Kept Smoke Pending. 2026-07-05 SOP batch recheck: public checkpoints are token-free, but no checkpoint is cached and current env lacks xformers/accelerate/datasets for AMPLIFY-compatible pinning. Kept Smoke Pending. 2026-07-05 SOP recheck: public checkpoints are token-free, but no checkpoint is cached and current env lacks xformers/accelerate/datasets for AMPLIFY-compatible pinning. Kept Smoke Pending. 2026-07-05 SOP batch recheck: public AMPLIFY checkpoints are token-free, but no checkpoint is cached and current shared env lacks xformers/accelerate/datasets with AMPLIFY-compatible pinning. Kept Smoke Pending. 2026-07-05 18:19 +08:00 backend re-audit: AMPLIFY source remains staged (~132 MB) and public checkpoints are token-free, but no checkpoint cache and no xformers/accelerate/datasets-compatible shared PLM runtime exists. Agents agreed Smoke Pending remains correct. 2026-07-06 21:01 +08:00 implementation attempt: source staged at /media/nik/seagate_nik/bio_server/tools/amplify/source and AMPLIFY public checkpoint URLs confirmed from upstream docs, but shared python312_slim plus palm_py312_site cannot import AMPLIFY because the package imports xformers at module load. A no-deps accelerate overlay was staged, but no real public AMPLIFY_120M/350M checkpoint smoke was submitted; bundled tests/reference-model is synthetic and is not SOP-valid production evidence. 2026-07-06 21:34 +08:00 recheck: sidecar review confirmed current blocker remains xformers/pinned PLM runtime plus real AMPLIFY_120M checkpoint; no valid smoke was submitted on the synthetic reference model. 2026-07-07 00:14 +08:00 continuation: included in next-10 Smoke Pending review; no SOP-valid direct Slurm smoke completed in this pass. Status unchanged pending the previously identified runtime/model/container blocker. 2026-07-07 00:58 +08:00 continuation: reviewed in the remaining Smoke Pending batch; no SOP-valid direct Slurm smoke completed and status remains unchanged pending the established runtime/model/container blocker. 2026-07-07 01:16 +08:00 continuation: downloaded real public Hugging Face chandar-lab/AMPLIFY_120M checkpoint to /media/nik/seagate_nik/bio_server/models/amplify/AMPLIFY_120M (model.safetensors 473,126,988 bytes). Direct Slurm job 486 at /media/nik/seagate_nik/bio_server/runs/amplify/smoke_20260707_011607 loaded the custom AMPLIFY AutoModel with trust_remote_code, used a CPU compatibility shim for xformers SwiGLU/memory_efficient_attention and AMPLIFY additive attention mask, and produced finite embeddings shape [1, 44, 640] with hidden_mean -1.3412952423095703 and hidden_std 76.60812377929688. Promoted to Runner Adapter Pending; runner adapter must preserve the CPU xformers compatibility shim or use a proper xformers runtime. 2026-07-08T21:08:30+08:00: Production Ready promotion: backend runner API job 0248c563269f / Slurm job 569 completed through the web runner API using shared python312_slim.sif, palm_py312_site, public AMPLIFY_120M checkpoint, and the CPU xformers compatibility shim from the direct smoke. Produced zero-byte slurm.out/slurm.err, empty model_load_stderr.txt, summary.json, results.csv, and embedding_mean.npy. Scientific sanity matched direct Slurm smoke: input length 44, hidden shape [1, 44, 640], finite hidden states, hidden_mean -1.3412952423095703, hidden_std 76.60812377929688, checkpoint size 473126988 bytes.