📚 Data Science Riddle
Why do we use Batch Normalization?
Why do we use Batch Normalization?
Anonymous Quiz
30%
Speeds up training
41%
Prevents overfitting
8%
Adds non-linearity
21%
Reduces dataset size
❤3
📚 Data Science Riddle
Your object detection model misses small objects. Easiest fix?
Your object detection model misses small objects. Easiest fix?
Anonymous Quiz
21%
Use larger input images
30%
Add more classes
34%
Reduce learning rate
15%
Train longer
🤖 AI that creates AI: ASI-ARCH finds 106 new SOTA architectures
ASI-ARCH — experimental ASI that autonomously researches and designs neural nets. It hypothesizes, codes, trains & tests models.
💡 Scale:
1,773 experiments → 20,000+ GPU-hours.
Stage 1 (20M params, 1B tokens): 1,350 candidates beat DeltaNet.
Stage 2 (340M params): 400 models → 106 SOTA winners.
Top 5 trained on 15B tokens vs Mamba2 & Gated DeltaNet.
📊 Results:
PathGateFusionNet: 48.51 avg (Mamba2: 47.84, Gated DeltaNet: 47.32).
BoolQ: 60.58 vs 60.12 (Gated DeltaNet).
Consistent gains across tasks.
🔍 Insights:
Prefers proven tools (gating, convs), refines them iteratively.
Ideas come from: 51.7% literature, 38.2% self-analysis, 10.1% originality.
SOTA share: self-analysis ↑ to 44.8%, literature ↓ to 48.6%.
@datascience_bds
ASI-ARCH — experimental ASI that autonomously researches and designs neural nets. It hypothesizes, codes, trains & tests models.
💡 Scale:
1,773 experiments → 20,000+ GPU-hours.
Stage 1 (20M params, 1B tokens): 1,350 candidates beat DeltaNet.
Stage 2 (340M params): 400 models → 106 SOTA winners.
Top 5 trained on 15B tokens vs Mamba2 & Gated DeltaNet.
📊 Results:
PathGateFusionNet: 48.51 avg (Mamba2: 47.84, Gated DeltaNet: 47.32).
BoolQ: 60.58 vs 60.12 (Gated DeltaNet).
Consistent gains across tasks.
🔍 Insights:
Prefers proven tools (gating, convs), refines them iteratively.
Ideas come from: 51.7% literature, 38.2% self-analysis, 10.1% originality.
SOTA share: self-analysis ↑ to 44.8%, literature ↓ to 48.6%.
@datascience_bds
❤1