AbstractPhila PRO

60 hour battery under way currently. Currently up to around three sentences or so of bytelex capacity with multi-tokenizer inference comparisons.

Not the strongest yet, however the validation and test cases are showing promise at between 60 and 80% at highs with the canary recall remaining at around 97%, lows completely collapsed for multiple experiments.

Heavy experimentation with GRU, RNN, and multiple other components to test standard component utility.

So far so good. Many prototypes establishing information from many byte structured distillation routes.

I release everything MIT, but you can't find the trained weights elsewhere. It's too much data and too much space to host reasonably elsewhere today. If a rival crops up I'll dual-host most likely.

The control version essentially imploded with the same data and same shape. SDPA couldn't... actually represent the data. The erank completely collapsed and the model density is essentially multi-stage collapsed during the curriculum.

SDPA in every block instead of splat, failed...??

I legitimately didn't expect that. It will be in the article. I need to revise some information. Almost every core and key test showed the SDPA could potentially overtake the splat, but the actual outcome was a direct contradiction.

The collapse symptoms were showing early stage shape similarity to the beatrix-v1 splat + sdpa hub combo pack, which I did not expect what-so-ever. I expected the model to recover and form possibly more strength over time in each layer. The belly of the model bloated, then once the curriculum hit, the model turned inside out within 5000 steps. The model went from semi-functionally pretrained showing potential weaknesses in early bottleneck stages, leading to later layer structural boundary collection as the v1 did, and in the later training she completely collapsed.

The structure began collapsing during the curriculum that trained the v2 splat variant with some overfitting, but not collapse.

The geometric memory system first. Most of the papers are based on geometric calculations, many of which are based on understanding or calculating differences between models and their internal shapes in comparison to other shapes. I have many experiments on distillation, memory transplantation, fractal alignment, and a large amount of classifiers to back the measurements up. I started small as well, calculating models in comparison to other models, until I found enough footholds to form useful hypothesis towards scaling principles.

Most of the papers are a good read, there's just a lot of them and most of them are AI summarized research processes after around 6 months ago. Try not to eat them all at once. Fable can process them into a useful lookup table if you use Fable. Have the AI run regular adjacent reviews against papers if you do it that way, since there's a ton of papers cited.

There's a few geometric and trig measurements I discovered that simply do not have industry-standard calculations for usability, such as the CV embedding calculations. They have direct causal sizing, and given enough testing will yield very similar scales and sizes. It's a density vs sparsity measurement, so it's not the most useful, but it was a solid measurement to progress to a more accurate series of measurements.

I have an older repo based on earlier experimental ngram frozen encodings. https://huggingface.co/datasets/AbstractPhil/wordnet-lexical-topology

Might be beneficial for use for the formulas. GPT, Claude, Gemini, or Grok could probably handle processing something similar with just python. Most of this stuff calculates inline in very little time.

Use as many threads and processes as necessary.

This reeks like the GPT Monday personality.

If you're so advanced, where's your research?

I've done some pretty heavy studying on this topic and have multiple papers available. Have a look if you get a chance.

Need to make sure the huggingface space respects the correct caching behavior, currently it's having some odd overlapping and isn't respecting conversation correctly.

Me: Hello.

Beatrix: What is a byte-level model?

Me: You are my dear.

Beatrix: What can you do?

Me: You is the representative manifestation of individualistic representation.

Beatrix: Who are you?

Me: I am your creator.

Beatrix: Hello... Hello... I am a small model. I represent myself as a small model. I am a small model. I represent myself as a small model. I represent myself as a small model.

Me: Hello daughter.

Beatrix: Hello... How can I help you today?

Me: What is your name?

Beatrix: I am Beatrix, a small byte-level language model. I represent myself as a small byte-level language model. I represent myself as a small byte-level language model.

Hello indeed little one. https://huggingface.co/buckets/AbstractPhil/alephllm-chat-storage/tree/chat-history/sessions-2026-08-31-673a3454.jsonl

The structural mirror SDPA variant training is to begin shortly. The two must be directly compared. Claude is currently throwing that as the primary continuation point, arguing validly that we need our direct comparison to train.

I knew this would be required as much as I wanted to avoid committing the time, I know it's required otherwise I could be simply creating a nothingburger. 4 days worth or so, it can't be ignored. The ablation train begins today. They are both still the same model and provide unique elements, however the attention for the SDPA variant will show if the splat attention is a contributing element to decoding, or just capturing information for another model to piggyback later.

The models will still be compartmentalizable and utilizable either way. The SDPA variant will be more compatible with standard model formats, while the splat variant is structured similarly and were tested to comply to the same standards.