I finally changed the architecture of my 15M French LLM. It worked. Then I almost fooled myself about how much and catching that was the real win.
After proving last time that architecture is a threshold, not a lever, I got stubborn: could I change how the model learns? Four honest attempts, Lion, a sharper AdamW β2, multi-token prediction, LayerScale. Four failures. The bottleneck wasn't the learning rule either.
So I changed the shape of the computation instead: loop the same transformer blocks 4×, deeper reasoning, zero added parameters. It beat the baseline on perplexity, the first thing in the whole project to move that number. Then I added my own twist: let each token decide how deep to think, halting on its own entropy.
My first evaluation was spectacular. Coherence up 65%. Hallucinated names down 62%.
It was noise.
Eight prompts, one seed. I re-ran on 50 prompts × 200 tokens and watched the gains shrink to "modest" and on out-of-domain prompts, recurrence actually made things worse. No universal winner. And none of it is new: it's Adaptive Computation Time (2016), the Universal Transformer (2018), and LoopViT (2026), recombined and measured honestly.
The real lesson:
A number from 8 prompts is a rumor. The eval harness that kills your own best result is worth more than the result it kills. Cite your lineage. Stay preliminary until multiple seeds say otherwise.
The three models are live. The write-up is honest about every caveat 👇
Greek Corpus 150B is now live on the Hub. A deduplicated, ~146B-token Greek dataset for pretraining and fine-tuning foundation models — a pretrain layer + an instruction (SFT) layer, one unified schema, globally deduplicated. 📊 49.6M documents / ~146B pretrain tokens
📚 Web (FineWeb-2) + long-form PDFs (FinePDFs) + FineWiki + native Greek legislation (47k statutes from the Government Gazette)
💬 ~10B-token SFT layer (9.9M conversations) The newest in my Global Corpus family — Dutch, Turkish, Bulgarian, Greek — built on a consistent, reproducible pipeline. 🔗 hasankursun/greek-corpus-150b #greek #llm #dataset #multilingual
Precision is very important in AI as it shapes how accurate and efficient models are. It controls how finely numbers are represented, approximating real-world values with formats like fixed-point and floating-point. A recent BF16 → FP16 study renewed attention to precision impact. Here are the main precision types used in AI, from full precision for training to ultra-low precision for inference:
1. FP32 (Float32): Standard full-precision float used in most training: 1 sign bit, 8 exponent bits, 23 mantissa bits. Default for backward-compatible training and baseline numerical stability
2. FP16 (Float16) → https://arxiv.org/abs/2305.10947v6 Half-precision float. It balances accuracy and efficiency. 1 sign bit, 5 exponent bits, 10 mantissa bits. Common on NVIDIA Tensor Cores and mixed-precision setups. There’s now a new wave of using it in reinforcement learning: https://www.turingpost.com/p/fp16
I am excited to announce the launch of research on the Digital Heart Model (DHM), an AI-driven digital twin designed to transform personalized cardiovascular care. DHM integrates multimodal data, focusing initially on cardiac imaging, histopathological imaging, and ECG data, to predict patient outcomes and optimize interventions.
Initial Model and Dataset Overview:
Base Model: Multimodal AI foundation combining Convolutional Neural Networks (CNN), Vision Transformers (ViT), and Graph Neural Networks (GNN).
Datasets: Cardiac MRI and CT imaging datasets, histopathological cardiac tissue images, and extensive ECG waveform data.
