| --- |
| license: cc-by-nc-sa-4.0 |
| base_model: stanford-star/rt-j |
| tags: |
| - relational-deep-learning |
| - relational-databases |
| - tabular |
| - tabular-classification |
| - tabular-regression |
| - foundation-model |
| - in-context-learning |
| - fp16 |
| - relational-transformer |
| datasets: |
| - stanford-star/the-join |
| - stanford-star/relbench |
| --- |
| |
| # RT-J — fp16 checkpoints |
|
|
| Half-precision conversions of [stanford-star/rt-j](https://huggingface.co/stanford-star/rt-j), |
| the Relational Transformer foundation model for in-context learning over |
| relational databases. Produced and consumed by the |
| [RelativeDB](https://github.com/RelativeDB) native inference engine |
| (`cpp/rt_quantize --type f16`). Weights stay **f16-resident** at inference: |
| the engine's CPU (Accelerate / portable SIMD) and Metal/MPS kernels convert |
| to fp32 inside the GEMM, halving weight memory and DRAM traffic vs. fp32 |
| with **no measurable accuracy loss** (drift vs. the PyTorch reference is |
| identical to the fp32 checkpoint on the golden batch). |
|
|
| Sibling repos: [rt-j-int8](https://huggingface.co/RelativeDB/rt-j-int8) · |
| [rt-j-int4](https://huggingface.co/RelativeDB/rt-j-int4) |
|
|
| | File | Task head | Size | |
| |---|---|---| |
| | `classification/model.f16.safetensors` | classification / ranking (logits — apply sigmoid) | 172 MB | |
| | `regression/model.f16.safetensors` | regression / forecasting (normalized values) | 172 MB | |
|
|
| ## Format |
|
|
| Plain safetensors: every transformer-block projection (`wq/wk/wv/wg`, `wo`, |
| `ffn.w1/w2/w3` — ~99% of parameters) is stored as `F16` (IEEE half, |
| round-to-nearest). The value/col-name encoders, decoder head, norms, biases |
| and mask embeddings stay fp32. Any safetensors reader can load these files. |
|
|
| ## Accuracy |
|
|
| Golden batch (B=5, S=16) vs. the PyTorch reference, identical on CPU and |
| Metal/MPS: `yhat` max abs. error 3.9e-3 — the same as the fp32 checkpoint |
| (upstream weights are bf16, so f16 storage adds no error above fp32 |
| op-ordering drift). |
|
|
| ## Usage (RelativeDB native engine) |
|
|
| ```bash |
| ./build/rt_test testdata classification/model.f16.safetensors --quantized --device mps |
| |
| # via the Java / Python / Rust bindings: place the .f16 file next to the fp32 |
| # checkpoint (or point at a directory containing it) and opt in with |
| export RELATIVEDB_RT_QUANTIZED=f16 |
| ``` |
|
|
| The C ABI (`rt_model_load`) accepts these files directly — the format is |
| auto-detected from the tensor dtypes. f16 checkpoints currently run on the |
| CPU and Metal/MPS backends (CUDA is fp32-only). |
|
|
| ## Reproduce |
|
|
| ```bash |
| cmake -B build -S cpp && cmake --build build -j |
| ./build/rt_quantize <rt-j>/classification/model.safetensors classification/model.f16.safetensors --type f16 |
| ./build/rt_quantize <rt-j>/regression/model.safetensors regression/model.f16.safetensors --type f16 |
| ``` |
|
|
| ## License & attribution |
|
|
| Derivative of [stanford-star/rt-j](https://huggingface.co/stanford-star/rt-j) |
| (Stanford STAR lab), redistributed under the same **CC-BY-NC-SA-4.0** |
| license. Architecture and training details are described in the upstream |
| model card; only the weight storage format differs here. |
|
|
