Older models that are no longer useful for anything in RL1 or RL2, or are now unused as experimentation discontinued.
AI & ML interests
Singular Learning Theory & Developmental Interpretability
Recent Activity
100k-row datasets filtered from https://huggingface.co/datasets/monology/pile-uncopyrighted. Doesn't include Books3, BookCorpus2, OpenSubtitles, YTSub
Attention-only transformers, sweep over head dimension
Attention-only transformers, sweep over number of heads (for fixed head dimension)
Attention-only transformers, sweep over model dimension
Models in use for RL1 + RL2 + susceptibility html plots + susceptibility viewer + action probs viewer. RL1 experiments redone with these models.
Attention-only transformers, sweep over number of heads (variable head dimension)
Attention-only transformers, sweep over number of layers
Missing square-brackets
Older models that are no longer useful for anything in RL1 or RL2, or are now unused as experimentation discontinued.
Models in use for RL1 + RL2 + susceptibility html plots + susceptibility viewer + action probs viewer. RL1 experiments redone with these models.
100k-row datasets filtered from https://huggingface.co/datasets/monology/pile-uncopyrighted. Doesn't include Books3, BookCorpus2, OpenSubtitles, YTSub
Attention-only transformers, sweep over head dimension
Attention-only transformers, sweep over number of heads (variable head dimension)
Attention-only transformers, sweep over number of heads (for fixed head dimension)
Attention-only transformers, sweep over number of layers
Attention-only transformers, sweep over model dimension
Missing square-brackets