Brian PRO
brian-learns
AI & ML interests
ethical ai use for cultural heritage use cases
Recent Activity
liked a model about 3 hours ago
convaiinnovations/laya liked a model about 22 hours ago
tngtech/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-GGUF liked a model 3 days ago
nvidia/Qwen3.8-27B-NVFP4Organizations
None yet
published an article 16 days ago
Article
agent to agent
brian-learns
• posted an update 18 days ago
Post
94
brian-learns/cdx-cc-news and https://huggingface.co/buckets/brian-learns/cdx-rocks-monthly have been updated with WARC data from August 2026
upvoted a paper 21 days ago
posted an update about 1 month ago
Post
103
brian-learns/cc-news-cdx-server has been upgraded to add a browse by domain/host/substring.
https://github.com/brian-learns/ccnget also supports browse, agent friendly command line options, and a really bad TUI
https://github.com/brian-learns/ccnget also supports browse, agent friendly command line options, and a really bad TUI
posted an update about 1 month ago
Post
2591
new docker and hugging face space!
brian-learns/cc-news-cdx-server and https://github.com/brian-learns/cdx_rocks are now in sync / same git repo
hugging face is running https://huggingface.co/buckets/brian-learns/cdx-rocks-demo but the docker can point at the full index https://huggingface.co/buckets/brian-learns/cdx-rocks-monthly
original hugging face space is now at brian-learns/cc-news-cdx-server-v1
thanks @dipankarsarkar for all the feedback
brian-learns/cc-news-cdx-server and https://github.com/brian-learns/cdx_rocks are now in sync / same git repo
hugging face is running https://huggingface.co/buckets/brian-learns/cdx-rocks-demo but the docker can point at the full index https://huggingface.co/buckets/brian-learns/cdx-rocks-monthly
original hugging face space is now at brian-learns/cc-news-cdx-server-v1
thanks @dipankarsarkar for all the feedback
replied to their post about 1 month ago
CC's own CDX server does not index CC-NEWS at all.
that's why I'm doing this
I've tried all kinds of things on the hugging face space, I'm not going to try any new permutations of options. If I put 1 or -1 in max_open_files it still hangs in the same way (I don't think I literally tried 1). I reproduced locally using hf-mount and it hangs in the same way on the first query that tries to hit the db files.
here is a regenerated file with the new cdx-rocks-build command
https://huggingface.co/buckets/brian-learns/cdx-rocks-monthly
I've got this small demo index I'll use for testing with the hugging face. Once I get the space working, I'll see how large I can get the demo database
https://huggingface.co/buckets/brian-learns/cdx-rocks-demo