mRNABERT

1-5 minutesPaperCodeProduction Ready

Pretrained mRNA language model for sequence scoring, codon optimization, and embeddings.

Readiness

Readiness: Production Ready
Smoke: passed
Artifacts: passed
Production: production_pass
Storage: <5
Strategy: shared Torch/Transformers runtime layer with mRNABERT checkpoint cache and CPU flash-attention fallback, or a CUDA/Triton runtime for production throughput
Next: Production Ready. Optional hardening: expose masked-token scoring outputs and longer batch-size/runtime limits.

No obvious large model/database dependency identified; validate upstream docs before install. 2026-07-05 SOP batch recheck: backend inventory found no staged mRNABERT source/model/SIF/artifacts/runner. Public Apache-2.0 HF model is about 456 MB and full datasets are not needed for an embedding smoke, but a PyTorch/Transformers runtime and model cache must be staged first. Kept Smoke Pending. 2026-07-05 18:21 +08:00 backend re-audit: no staged mRNABERT source/model/runtime/SIF/artifacts/runner found; public model appears small enough but no cache or smoke exists. Kept Smoke Pending. 2026-07-06 implementation attempt: cloned yyly6/mRNABERT at /media/nik/seagate_nik/bio_server/tools/mrnabert/source/mRNABERT rev 893ccc920bb9be02a4677d14d96b03126da17689 and partially cached YYLY66/mRNABERT under /media/nik/seagate_nik/bio_server/tools/mrnabert/hf_cache (~1.3 GB after degraded exFAT cache). Primary-source check confirmed official GitHub/Hugging Face model and feature-extraction use. Shared python312_slim.sif plus palm_py312_site Transformers reached config/tokenizer, but official trust_remote_code model requires einops and triton; einops was staged in /media/nik/seagate_nik/bio_server/envs/mrnabert_py312_site, then import stopped on missing triton. Standard BertModel loading is not acceptable: the default .bin is refused by Transformers with Torch 2.4 due the torch.load CVE gate requiring Torch >=2.6, and the cached safetensors snapshot maps to custom fused mRNABERT keys (Wqkv/gated MLP), leaving standard-BERT encoder weights randomly initialized. No Slurm inference smoke was submitted; kept Smoke Pending. 2026-07-06 21:01 +08:00 recheck: prior staged yyly6/mRNABERT source and partial HF cache remain insufficient. Official remote-code model still needs Triton and Torch >=2.6-safe loading; standard BertModel/safetensors workaround would leave custom fused weights unmapped/randomly initialized, so no SOP-valid smoke was submitted. 2026-07-06 21:34 +08:00 recheck: official remote-code path still needs Triton and Torch >=2.6-safe loading; no valid embedding smoke submitted. 2026-07-07 00:14 +08:00 continuation: included in next-10 Smoke Pending review; no SOP-valid direct Slurm smoke completed in this pass. Status unchanged pending the previously identified runtime/model/container blocker. 2026-07-07 00:58 +08:00 continuation: reviewed in the remaining Smoke Pending batch; no SOP-valid direct Slurm smoke completed and status remains unchanged pending the established runtime/model/container blocker. 2026-07-07 01:16 +08:00 continuation: reviewed in remaining Smoke Pending batch; no SOP-valid direct Slurm smoke completed and status remains unchanged pending the established runtime/model/container blocker. 2026-07-07 01:31 +08:00 promotion: direct Slurm job 487 passed using official YYLY66/mRNABERT Hugging Face snapshot a1eb7df25804d23f08646e1cb996b234d7208a40 and real pytorch_model.bin checkpoint (455,973,118 bytes, sha256 cb2eb64831a494d4cac14acb5df908f734e088c4d62256ac3e42cada60c3bf75). Run /media/nik/seagate_nik/bio_server/runs/mrnabert/smoke_20260707_013015 loaded BertForMaskedLM with zero missing/unexpected state-dict keys, forced the upstream CPU PyTorch attention fallback by disabling flash_attn_triton, and produced finite logits/hidden states for two toy mRNA sequences. Artifacts: summary.json, results.csv, Slurm stdout/stderr, and run_mrnabert_smoke.sh. Runner/API adapter remains pending. 2026-07-08T02:17:48+08:00: Production Ready promotion: backend runner API job 4eae62b95b62 / Slurm job 537 completed cleanly using shared python312_slim.sif with mrnabert_runtime/deeprank overlays and official YYLY66/mRNABERT snapshot. Loaded pytorch_model.bin sha256 cb2eb64831a494d4cac14acb5df908f734e088c4d62256ac3e42cada60c3bf75 with zero missing/unexpected state-dict keys, flash attention disabled for CPU fallback, zero-byte slurm.out/slurm.err, and finite logits/hidden states for two mRNA sequences.