Add LabHorizon Qwen LoRA adapter

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,97 @@
 ---
 license: mit
+base_model: Qwen/Qwen3.6-35B-A3B
+library_name: peft
+pipeline_tag: image-text-to-text
+tags:
+- laboratory
+- protocol-conditioned-action-prediction
+- lora
+- qwen
+- long-horizon-planning
 ---
+
+<div align="center">
+  <div style="font-size: 2em; font-weight: bold;">
+    LabHorizon Model
+  </div>
+</div>
+
+<div align="center">
+
+[![Website](https://img.shields.io/badge/%F0%9F%9A%80%20Website-LabHorizon-00c2a8)](https://conglab-research.github.io/LabHorizon/)&nbsp;
+![arXiv](https://img.shields.io/badge/arXiv-coming%20soon-b31b1b?logo=arxiv&logoColor=white)&nbsp;
+[![Code](https://img.shields.io/badge/Code-LabHorizon-000000?logo=github&logoColor=white)](https://github.com/CongLab-Research/LabHorizon)&nbsp;
+[![Data L1](https://img.shields.io/badge/%F0%9F%A4%97%20Data-L1-blue)](https://huggingface.co/datasets/CongLab-Research/LabHorizon-3D-Asset-Perception)&nbsp;
+[![Data L2](https://img.shields.io/badge/%F0%9F%A4%97%20Data-L2-purple)](https://huggingface.co/datasets/CongLab-Research/LabHorizon-Protocol-Conditioned-Planning)&nbsp;
+[![Model](https://img.shields.io/badge/%F0%9F%A4%97%20Model-LoRA-orange)](https://huggingface.co/CongLab-Research/LabHorizon-Model)
+
+**Qwen3.6-35B-A3B LoRA for protocol-conditioned laboratory action prediction**
+
+</div>
+
+---
+
+## 🔎 Overview
+
+This repository releases the LabHorizon LoRA adapter trained from `Qwen/Qwen3.6-35B-A3B` on the 6,000-sample LabHorizon training split. The model is optimized for **Protocol-Conditioned Action Prediction**:
+
+- **Level 1:** connect multi-view laboratory assets and historical actions to the gold next action.
+- **Level 2:** produce a structured long-horizon experimental action sequence from context, constraints, available inputs, and an action pool.
+
+The released weights are an adapter, not the base model. Load them with the corresponding Qwen3.6-35B-A3B base model.
+
+## 📦 Files
+
+| File | Meaning |
+|:---|:---|
+| `adapter_model.safetensors` | LoRA adapter weights. |
+| `adapter_config.json` | PEFT adapter configuration. |
+| `tokenizer.json`, `tokenizer_config.json`, `chat_template.jinja` | Tokenizer and chat template files used for training/evaluation. |
+| `processor_config.json` | Processor configuration. |
+| `train_results.json`, `eval_results.json`, `all_results.json` | Training and evaluation summaries from the LoRA run. |
+| `trainer_state.json`, `trainer_log.jsonl`, `training_args.bin` | Training state and arguments for reproducibility. |
+| `training_loss.png`, `training_eval_loss.png` | Loss curves. |
+
+## 🧠 Training Result
+
+The table compares direct-prompting SOTA/baseline systems, the base Qwen model, this trained LoRA adapter, and the trained+agents system evaluated on the same LabHorizon test splits.
+
+| System | Level 1 Next Action Accuracy | Level 2 Action Sequence Similarity | Level 2 Parameter Accuracy | Level 2 Final Score |
+|:---|---:|---:|---:|---:|
+| Grok 4.3 | 0.555 | 0.3339 | 0.3148 | 0.3244 |
+| Gemini 3.1 Pro Preview | 0.465 | 0.3195 | 0.3331 | 0.3263 |
+| GPT-5.5 | 0.535 | 0.2092 | 0.2459 | 0.2276 |
+| Kimi K2.6 | 0.550 | 0.2845 | 0.3456 | 0.3150 |
+| Qwen3.6-35B-A3B | 0.475 | 0.2585 | 0.2483 | 0.2534 |
+| Qwen3.6-35B-A3B(trained) | 0.635 | 0.4030 | 0.4170 | 0.4100 |
+| Qwen3.6-35B-A3B(trained+agents*) | **0.665** | **0.4485** | **0.4580** | **0.4532** |
+
+`*` uses `Qwen3.6-35B-A3B(trained)` as Actor and Gemini 3.1 Pro Preview as Simulator/Selector. The Simulator/Selector choice is the current setting and has not been exhaustively ablated.
+
+## ⚙️ Loading
+
+```python
+from transformers import AutoModelForCausalLM, AutoProcessor
+from peft import PeftModel
+
+base_id = "Qwen/Qwen3.6-35B-A3B"
+adapter_id = "CongLab-Research/LabHorizon-Model"
+
+processor = AutoProcessor.from_pretrained(adapter_id, trust_remote_code=True)
+base = AutoModelForCausalLM.from_pretrained(
+    base_id,
+    device_map="auto",
+    torch_dtype="auto",
+    trust_remote_code=True,
+)
+model = PeftModel.from_pretrained(base, adapter_id)
+```
+
+## ⚠️ Intended Use
+
+This adapter is intended for academic research on laboratory action prediction, experimental planning, and AI scientist systems. It should not be used as an autonomous wet-lab controller or for safety-critical experimental decisions without expert review.
+
+## 📜 Citation
+
+Coming soon...

adapter_config.json

@@ -0,0 +1,56 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.6-35B-A3B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "shared_expert_gate",
+    "v_proj",
+    "k_proj",
+    "in_proj_b",
+    "linear_fc1",
+    "up_proj",
+    "in_proj_qkv",
+    "attn.proj",
+    "in_proj_z",
+    "gate_proj",
+    "out_proj",
+    "q_proj",
+    "qkv",
+    "linear_fc2",
+    "o_proj",
+    "in_proj_a"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1a706ab043f27c19ad89d5ae4acf81c304b603b3ac9926105f34d12dc3173cbf
+size 243590944

all_results.json

@@ -0,0 +1,12 @@
+{
+    "epoch": 10.0,
+    "eval_loss": 0.44259119033813477,
+    "eval_runtime": 27.1503,
+    "eval_samples_per_second": 14.733,
+    "eval_steps_per_second": 2.468,
+    "total_flos": 3.634151342457697e+19,
+    "train_loss": 0.2690703985452652,
+    "train_runtime": 10014.7733,
+    "train_samples_per_second": 5.991,
+    "train_steps_per_second": 0.25
+}

chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if (preserve_thinking is defined and preserve_thinking is true) or (loop.index0 > ns.last_query_index) %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | string if args_value is string else args_value | tojson | safe %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

eval_results.json

@@ -0,0 +1,7 @@
+{
+    "epoch": 10.0,
+    "eval_loss": 0.44259119033813477,
+    "eval_runtime": 27.1503,
+    "eval_samples_per_second": 14.733,
+    "eval_steps_per_second": 2.468
+}

processor_config.json

@@ -0,0 +1,60 @@
+{
+  "image_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "image_processor_type": "Qwen2VLImageProcessor",
+    "image_std": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "merge_size": 2,
+    "patch_size": 16,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "size": {
+      "longest_edge": 16777216,
+      "shortest_edge": 65536
+    },
+    "temporal_patch_size": 2
+  },
+  "processor_class": "Qwen3VLProcessor",
+  "video_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "do_sample_frames": true,
+    "fps": 2,
+    "image_mean": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "image_std": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "max_frames": 768,
+    "merge_size": 2,
+    "min_frames": 4,
+    "patch_size": 16,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "return_metadata": false,
+    "size": {
+      "longest_edge": 25165824,
+      "shortest_edge": 4096
+    },
+    "temporal_patch_size": 2,
+    "video_processor_type": "Qwen3VLVideoProcessor"
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523
+size 19989325

tokenizer_config.json

@@ -0,0 +1,34 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "padding_side": "right",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "processor_class": "Qwen3VLProcessor",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

train_results.json

@@ -0,0 +1,8 @@
+{
+    "epoch": 10.0,
+    "total_flos": 3.634151342457697e+19,
+    "train_loss": 0.2690703985452652,
+    "train_runtime": 10014.7733,
+    "train_samples_per_second": 5.991,
+    "train_steps_per_second": 0.25
+}

trainer_log.jsonl

@@ -0,0 +1,276 @@
+{"current_steps": 10, "total_steps": 2500, "loss": 1.1145790100097657, "lr": 3.6e-06, "epoch": 0.04, "percentage": 0.4, "elapsed_time": "0:01:53", "remaining_time": "7:49:28"}
+{"current_steps": 20, "total_steps": 2500, "loss": 1.2167404174804688, "lr": 7.6e-06, "epoch": 0.08, "percentage": 0.8, "elapsed_time": "0:02:32", "remaining_time": "5:15:11"}
+{"current_steps": 30, "total_steps": 2500, "loss": 1.0437713623046876, "lr": 1.16e-05, "epoch": 0.12, "percentage": 1.2, "elapsed_time": "0:03:15", "remaining_time": "4:27:50"}
+{"current_steps": 40, "total_steps": 2500, "loss": 0.9282869338989258, "lr": 1.56e-05, "epoch": 0.16, "percentage": 1.6, "elapsed_time": "0:03:58", "remaining_time": "4:04:27"}
+{"current_steps": 50, "total_steps": 2500, "loss": 0.8799624443054199, "lr": 1.9600000000000002e-05, "epoch": 0.2, "percentage": 2.0, "elapsed_time": "0:04:40", "remaining_time": "3:48:42"}
+{"current_steps": 60, "total_steps": 2500, "loss": 0.7062759399414062, "lr": 2.36e-05, "epoch": 0.24, "percentage": 2.4, "elapsed_time": "0:05:21", "remaining_time": "3:37:36"}
+{"current_steps": 70, "total_steps": 2500, "loss": 0.7228042602539062, "lr": 2.7600000000000003e-05, "epoch": 0.28, "percentage": 2.8, "elapsed_time": "0:06:01", "remaining_time": "3:29:20"}
+{"current_steps": 80, "total_steps": 2500, "loss": 0.6257906913757324, "lr": 3.16e-05, "epoch": 0.32, "percentage": 3.2, "elapsed_time": "0:06:42", "remaining_time": "3:23:02"}
+{"current_steps": 90, "total_steps": 2500, "loss": 0.5399329185485839, "lr": 3.56e-05, "epoch": 0.36, "percentage": 3.6, "elapsed_time": "0:07:23", "remaining_time": "3:17:44"}
+{"current_steps": 100, "total_steps": 2500, "loss": 0.5184461116790772, "lr": 3.960000000000001e-05, "epoch": 0.4, "percentage": 4.0, "elapsed_time": "0:08:03", "remaining_time": "3:13:14"}
+{"current_steps": 100, "total_steps": 2500, "eval_loss": 0.5476460456848145, "epoch": 0.4, "percentage": 4.0, "elapsed_time": "0:08:24", "remaining_time": "3:21:50"}
+{"current_steps": 110, "total_steps": 2500, "loss": 0.5210700988769531, "lr": 4.36e-05, "epoch": 0.44, "percentage": 4.4, "elapsed_time": "0:09:05", "remaining_time": "3:17:37"}
+{"current_steps": 120, "total_steps": 2500, "loss": 0.5155693531036377, "lr": 4.76e-05, "epoch": 0.48, "percentage": 4.8, "elapsed_time": "0:09:46", "remaining_time": "3:13:46"}
+{"current_steps": 130, "total_steps": 2500, "loss": 0.45534143447875974, "lr": 5.16e-05, "epoch": 0.52, "percentage": 5.2, "elapsed_time": "0:10:26", "remaining_time": "3:10:12"}
+{"current_steps": 140, "total_steps": 2500, "loss": 0.45524797439575193, "lr": 5.560000000000001e-05, "epoch": 0.56, "percentage": 5.6, "elapsed_time": "0:11:06", "remaining_time": "3:07:08"}
+{"current_steps": 150, "total_steps": 2500, "loss": 0.47152209281921387, "lr": 5.96e-05, "epoch": 0.6, "percentage": 6.0, "elapsed_time": "0:11:45", "remaining_time": "3:04:10"}
+{"current_steps": 160, "total_steps": 2500, "loss": 0.4532940864562988, "lr": 6.36e-05, "epoch": 0.64, "percentage": 6.4, "elapsed_time": "0:12:23", "remaining_time": "3:01:14"}
+{"current_steps": 170, "total_steps": 2500, "loss": 0.48988704681396483, "lr": 6.76e-05, "epoch": 0.68, "percentage": 6.8, "elapsed_time": "0:12:59", "remaining_time": "2:58:08"}
+{"current_steps": 180, "total_steps": 2500, "loss": 0.46865572929382326, "lr": 7.16e-05, "epoch": 0.72, "percentage": 7.2, "elapsed_time": "0:13:37", "remaining_time": "2:55:41"}
+{"current_steps": 190, "total_steps": 2500, "loss": 0.45577139854431153, "lr": 7.560000000000001e-05, "epoch": 0.76, "percentage": 7.6, "elapsed_time": "0:14:15", "remaining_time": "2:53:22"}
+{"current_steps": 200, "total_steps": 2500, "loss": 0.4559042453765869, "lr": 7.960000000000001e-05, "epoch": 0.8, "percentage": 8.0, "elapsed_time": "0:14:52", "remaining_time": "2:50:58"}
+{"current_steps": 200, "total_steps": 2500, "eval_loss": 0.485470175743103, "epoch": 0.8, "percentage": 8.0, "elapsed_time": "0:15:09", "remaining_time": "2:54:18"}
+{"current_steps": 210, "total_steps": 2500, "loss": 0.45926451683044434, "lr": 8.36e-05, "epoch": 0.84, "percentage": 8.4, "elapsed_time": "0:15:49", "remaining_time": "2:52:34"}
+{"current_steps": 220, "total_steps": 2500, "loss": 0.4545453548431396, "lr": 8.76e-05, "epoch": 0.88, "percentage": 8.8, "elapsed_time": "0:16:33", "remaining_time": "2:51:32"}
+{"current_steps": 230, "total_steps": 2500, "loss": 0.47637343406677246, "lr": 9.16e-05, "epoch": 0.92, "percentage": 9.2, "elapsed_time": "0:17:13", "remaining_time": "2:49:56"}
+{"current_steps": 240, "total_steps": 2500, "loss": 0.43120541572570803, "lr": 9.56e-05, "epoch": 0.96, "percentage": 9.6, "elapsed_time": "0:17:50", "remaining_time": "2:48:02"}
+{"current_steps": 250, "total_steps": 2500, "loss": 0.4153712272644043, "lr": 9.960000000000001e-05, "epoch": 1.0, "percentage": 10.0, "elapsed_time": "0:18:29", "remaining_time": "2:46:26"}
+{"current_steps": 260, "total_steps": 2500, "loss": 0.44300012588500975, "lr": 9.999605221019081e-05, "epoch": 1.04, "percentage": 10.4, "elapsed_time": "0:19:08", "remaining_time": "2:44:53"}
+{"current_steps": 270, "total_steps": 2500, "loss": 0.462084436416626, "lr": 9.998240632972073e-05, "epoch": 1.08, "percentage": 10.8, "elapsed_time": "0:19:48", "remaining_time": "2:43:34"}
+{"current_steps": 280, "total_steps": 2500, "loss": 0.39808471202850343, "lr": 9.995901628010196e-05, "epoch": 1.12, "percentage": 11.2, "elapsed_time": "0:20:26", "remaining_time": "2:42:01"}
+{"current_steps": 290, "total_steps": 2500, "loss": 0.423044490814209, "lr": 9.9925886621271e-05, "epoch": 1.16, "percentage": 11.6, "elapsed_time": "0:21:04", "remaining_time": "2:40:38"}
+{"current_steps": 300, "total_steps": 2500, "loss": 0.41622562408447267, "lr": 9.98830238119205e-05, "epoch": 1.2, "percentage": 12.0, "elapsed_time": "0:21:43", "remaining_time": "2:39:22"}
+{"current_steps": 300, "total_steps": 2500, "eval_loss": 0.4695434272289276, "epoch": 1.2, "percentage": 12.0, "elapsed_time": "0:22:03", "remaining_time": "2:41:43"}
+{"current_steps": 310, "total_steps": 2500, "loss": 0.4166346549987793, "lr": 9.983043620824005e-05, "epoch": 1.24, "percentage": 12.4, "elapsed_time": "0:22:45", "remaining_time": "2:40:48"}
+{"current_steps": 320, "total_steps": 2500, "loss": 0.43734130859375, "lr": 9.97681340622872e-05, "epoch": 1.28, "percentage": 12.8, "elapsed_time": "0:23:23", "remaining_time": "2:39:19"}
+{"current_steps": 330, "total_steps": 2500, "loss": 0.3747305631637573, "lr": 9.969612951998874e-05, "epoch": 1.32, "percentage": 13.2, "elapsed_time": "0:23:57", "remaining_time": "2:37:32"}
+{"current_steps": 340, "total_steps": 2500, "loss": 0.42578792572021484, "lr": 9.961443661877289e-05, "epoch": 1.3599999999999999, "percentage": 13.6, "elapsed_time": "0:24:31", "remaining_time": "2:35:46"}
+{"current_steps": 350, "total_steps": 2500, "loss": 0.39537777900695803, "lr": 9.952307128483256e-05, "epoch": 1.4, "percentage": 14.0, "elapsed_time": "0:25:06", "remaining_time": "2:34:17"}
+{"current_steps": 360, "total_steps": 2500, "loss": 0.4084367275238037, "lr": 9.942205133002068e-05, "epoch": 1.44, "percentage": 14.4, "elapsed_time": "0:25:44", "remaining_time": "2:33:03"}
+{"current_steps": 370, "total_steps": 2500, "loss": 0.3781426906585693, "lr": 9.931139644837754e-05, "epoch": 1.48, "percentage": 14.8, "elapsed_time": "0:26:21", "remaining_time": "2:31:45"}
+{"current_steps": 380, "total_steps": 2500, "loss": 0.3952002048492432, "lr": 9.919112821229163e-05, "epoch": 1.52, "percentage": 15.2, "elapsed_time": "0:27:00", "remaining_time": "2:30:37"}
+{"current_steps": 390, "total_steps": 2500, "loss": 0.4087832927703857, "lr": 9.906127006829384e-05, "epoch": 1.56, "percentage": 15.6, "elapsed_time": "0:27:37", "remaining_time": "2:29:29"}
+{"current_steps": 400, "total_steps": 2500, "loss": 0.3861570119857788, "lr": 9.892184733248666e-05, "epoch": 1.6, "percentage": 16.0, "elapsed_time": "0:28:16", "remaining_time": "2:28:27"}
+{"current_steps": 400, "total_steps": 2500, "eval_loss": 0.45406103134155273, "epoch": 1.6, "percentage": 16.0, "elapsed_time": "0:28:36", "remaining_time": "2:30:10"}
+{"current_steps": 410, "total_steps": 2500, "loss": 0.39033331871032717, "lr": 9.877288718560866e-05, "epoch": 1.6400000000000001, "percentage": 16.4, "elapsed_time": "0:29:17", "remaining_time": "2:29:19"}
+{"current_steps": 420, "total_steps": 2500, "loss": 0.43663845062255857, "lr": 9.861441866773564e-05, "epoch": 1.6800000000000002, "percentage": 16.8, "elapsed_time": "0:29:55", "remaining_time": "2:28:12"}
+{"current_steps": 430, "total_steps": 2500, "loss": 0.43364706039428713, "lr": 9.844647267261916e-05, "epoch": 1.72, "percentage": 17.2, "elapsed_time": "0:30:30", "remaining_time": "2:26:53"}
+{"current_steps": 440, "total_steps": 2500, "loss": 0.409498929977417, "lr": 9.82690819416637e-05, "epoch": 1.76, "percentage": 17.6, "elapsed_time": "0:31:04", "remaining_time": "2:25:28"}
+{"current_steps": 450, "total_steps": 2500, "loss": 0.4264820098876953, "lr": 9.808228105754376e-05, "epoch": 1.8, "percentage": 18.0, "elapsed_time": "0:31:38", "remaining_time": "2:24:06"}
+{"current_steps": 460, "total_steps": 2500, "loss": 0.417040491104126, "lr": 9.788610643746184e-05, "epoch": 1.8399999999999999, "percentage": 18.4, "elapsed_time": "0:32:11", "remaining_time": "2:22:47"}
+{"current_steps": 470, "total_steps": 2500, "loss": 0.3749807357788086, "lr": 9.76805963260488e-05, "epoch": 1.88, "percentage": 18.8, "elapsed_time": "0:32:48", "remaining_time": "2:21:43"}
+{"current_steps": 480, "total_steps": 2500, "loss": 0.4022481918334961, "lr": 9.746579078790807e-05, "epoch": 1.92, "percentage": 19.2, "elapsed_time": "0:33:25", "remaining_time": "2:20:40"}
+{"current_steps": 490, "total_steps": 2500, "loss": 0.38319835662841795, "lr": 9.724173169980491e-05, "epoch": 1.96, "percentage": 19.6, "elapsed_time": "0:34:02", "remaining_time": "2:19:40"}
+{"current_steps": 500, "total_steps": 2500, "loss": 0.4122174263000488, "lr": 9.700846274250251e-05, "epoch": 2.0, "percentage": 20.0, "elapsed_time": "0:34:41", "remaining_time": "2:18:46"}
+{"current_steps": 500, "total_steps": 2500, "eval_loss": 0.44415727257728577, "epoch": 2.0, "percentage": 20.0, "elapsed_time": "0:35:00", "remaining_time": "2:20:01"}
+{"current_steps": 510, "total_steps": 2500, "loss": 0.3524669408798218, "lr": 9.676602939224629e-05, "epoch": 2.04, "percentage": 20.4, "elapsed_time": "0:35:41", "remaining_time": "2:19:16"}
+{"current_steps": 520, "total_steps": 2500, "loss": 0.3717231273651123, "lr": 9.651447891189825e-05, "epoch": 2.08, "percentage": 20.8, "elapsed_time": "0:36:17", "remaining_time": "2:18:10"}
+{"current_steps": 530, "total_steps": 2500, "loss": 0.40065832138061525, "lr": 9.62538603417229e-05, "epoch": 2.12, "percentage": 21.2, "elapsed_time": "0:36:49", "remaining_time": "2:16:53"}
+{"current_steps": 540, "total_steps": 2500, "loss": 0.33635973930358887, "lr": 9.598422448982696e-05, "epoch": 2.16, "percentage": 21.6, "elapsed_time": "0:37:25", "remaining_time": "2:15:49"}
+{"current_steps": 550, "total_steps": 2500, "loss": 0.3708656787872314, "lr": 9.570562392225396e-05, "epoch": 2.2, "percentage": 22.0, "elapsed_time": "0:38:03", "remaining_time": "2:14:55"}
+{"current_steps": 560, "total_steps": 2500, "loss": 0.35284056663513186, "lr": 9.541811295273656e-05, "epoch": 2.24, "percentage": 22.4, "elapsed_time": "0:38:38", "remaining_time": "2:13:51"}
+{"current_steps": 570, "total_steps": 2500, "loss": 0.3429510831832886, "lr": 9.512174763210797e-05, "epoch": 2.2800000000000002, "percentage": 22.8, "elapsed_time": "0:39:15", "remaining_time": "2:12:55"}
+{"current_steps": 580, "total_steps": 2500, "loss": 0.36770102977752683, "lr": 9.481658573737465e-05, "epoch": 2.32, "percentage": 23.2, "elapsed_time": "0:39:52", "remaining_time": "2:12:01"}
+{"current_steps": 590, "total_steps": 2500, "loss": 0.3684037208557129, "lr": 9.450268676045262e-05, "epoch": 2.36, "percentage": 23.6, "elapsed_time": "0:40:31", "remaining_time": "2:11:12"}
+{"current_steps": 600, "total_steps": 2500, "loss": 0.3221792697906494, "lr": 9.418011189656941e-05, "epoch": 2.4, "percentage": 24.0, "elapsed_time": "0:41:11", "remaining_time": "2:10:25"}
+{"current_steps": 600, "total_steps": 2500, "eval_loss": 0.44748273491859436, "epoch": 2.4, "percentage": 24.0, "elapsed_time": "0:41:30", "remaining_time": "2:11:25"}
+{"current_steps": 610, "total_steps": 2500, "loss": 0.40174164772033694, "lr": 9.384892403233384e-05, "epoch": 2.44, "percentage": 24.4, "elapsed_time": "0:42:07", "remaining_time": "2:10:30"}
+{"current_steps": 620, "total_steps": 2500, "loss": 0.3701002836227417, "lr": 9.35091877334763e-05, "epoch": 2.48, "percentage": 24.8, "elapsed_time": "0:42:41", "remaining_time": "2:09:28"}
+{"current_steps": 630, "total_steps": 2500, "loss": 0.3759175777435303, "lr": 9.316096923226135e-05, "epoch": 2.52, "percentage": 25.2, "elapsed_time": "0:43:18", "remaining_time": "2:08:33"}
+{"current_steps": 640, "total_steps": 2500, "loss": 0.3581662178039551, "lr": 9.28043364145758e-05, "epoch": 2.56, "percentage": 25.6, "elapsed_time": "0:43:53", "remaining_time": "2:07:33"}
+{"current_steps": 650, "total_steps": 2500, "loss": 0.35065665245056155, "lr": 9.24393588066941e-05, "epoch": 2.6, "percentage": 26.0, "elapsed_time": "0:44:26", "remaining_time": "2:06:28"}
+{"current_steps": 660, "total_steps": 2500, "loss": 0.36879355907440187, "lr": 9.206610756172402e-05, "epoch": 2.64, "percentage": 26.4, "elapsed_time": "0:45:03", "remaining_time": "2:05:37"}
+{"current_steps": 670, "total_steps": 2500, "loss": 0.3592060565948486, "lr": 9.168465544573536e-05, "epoch": 2.68, "percentage": 26.8, "elapsed_time": "0:45:43", "remaining_time": "2:04:53"}
+{"current_steps": 680, "total_steps": 2500, "loss": 0.36156315803527833, "lr": 9.129507682357394e-05, "epoch": 2.7199999999999998, "percentage": 27.2, "elapsed_time": "0:46:22", "remaining_time": "2:04:08"}
+{"current_steps": 690, "total_steps": 2500, "loss": 0.34445748329162595, "lr": 9.089744764436403e-05, "epoch": 2.76, "percentage": 27.6, "elapsed_time": "0:47:02", "remaining_time": "2:03:23"}
+{"current_steps": 700, "total_steps": 2500, "loss": 0.3526463985443115, "lr": 9.049184542670199e-05, "epoch": 2.8, "percentage": 28.0, "elapsed_time": "0:47:37", "remaining_time": "2:02:27"}
+{"current_steps": 700, "total_steps": 2500, "eval_loss": 0.44259119033813477, "epoch": 2.8, "percentage": 28.0, "elapsed_time": "0:47:54", "remaining_time": "2:03:10"}
+{"current_steps": 710, "total_steps": 2500, "loss": 0.3458081245422363, "lr": 9.007834924354383e-05, "epoch": 2.84, "percentage": 28.4, "elapsed_time": "0:48:32", "remaining_time": "2:02:23"}
+{"current_steps": 720, "total_steps": 2500, "loss": 0.3651163101196289, "lr": 8.965703970678974e-05, "epoch": 2.88, "percentage": 28.8, "elapsed_time": "0:49:08", "remaining_time": "2:01:28"}
+{"current_steps": 730, "total_steps": 2500, "loss": 0.3218229293823242, "lr": 8.922799895156867e-05, "epoch": 2.92, "percentage": 29.2, "elapsed_time": "0:49:42", "remaining_time": "2:00:31"}
+{"current_steps": 740, "total_steps": 2500, "loss": 0.3561582088470459, "lr": 8.879131062022598e-05, "epoch": 2.96, "percentage": 29.6, "elapsed_time": "0:50:18", "remaining_time": "1:59:39"}
+{"current_steps": 750, "total_steps": 2500, "loss": 0.36128854751586914, "lr": 8.834705984601708e-05, "epoch": 3.0, "percentage": 30.0, "elapsed_time": "0:50:55", "remaining_time": "1:58:48"}
+{"current_steps": 760, "total_steps": 2500, "loss": 0.31422438621521, "lr": 8.789533323651066e-05, "epoch": 3.04, "percentage": 30.4, "elapsed_time": "0:51:32", "remaining_time": "1:57:59"}
+{"current_steps": 770, "total_steps": 2500, "loss": 0.29355826377868655, "lr": 8.74362188567043e-05, "epoch": 3.08, "percentage": 30.8, "elapsed_time": "0:52:11", "remaining_time": "1:57:15"}
+{"current_steps": 780, "total_steps": 2500, "loss": 0.3185117721557617, "lr": 8.696980621185602e-05, "epoch": 3.12, "percentage": 31.2, "elapsed_time": "0:52:51", "remaining_time": "1:56:32"}
+{"current_steps": 790, "total_steps": 2500, "loss": 0.28971233367919924, "lr": 8.649618623003508e-05, "epoch": 3.16, "percentage": 31.6, "elapsed_time": "0:53:28", "remaining_time": "1:55:45"}
+{"current_steps": 800, "total_steps": 2500, "loss": 0.3055370092391968, "lr": 8.601545124439535e-05, "epoch": 3.2, "percentage": 32.0, "elapsed_time": "0:54:06", "remaining_time": "1:54:58"}
+{"current_steps": 800, "total_steps": 2500, "eval_loss": 0.4529191255569458, "epoch": 3.2, "percentage": 32.0, "elapsed_time": "0:54:24", "remaining_time": "1:55:37"}
+{"current_steps": 810, "total_steps": 2500, "loss": 0.28035550117492675, "lr": 8.552769497517482e-05, "epoch": 3.24, "percentage": 32.4, "elapsed_time": "0:55:03", "remaining_time": "1:54:52"}
+{"current_steps": 820, "total_steps": 2500, "loss": 0.3199602603912354, "lr": 8.503301251142459e-05, "epoch": 3.2800000000000002, "percentage": 32.8, "elapsed_time": "0:55:37", "remaining_time": "1:53:57"}
+{"current_steps": 830, "total_steps": 2500, "loss": 0.29444499015808107, "lr": 8.453150029247114e-05, "epoch": 3.32, "percentage": 33.2, "elapsed_time": "0:56:12", "remaining_time": "1:53:04"}
+{"current_steps": 840, "total_steps": 2500, "loss": 0.30467259883880615, "lr": 8.402325608911526e-05, "epoch": 3.36, "percentage": 33.6, "elapsed_time": "0:56:45", "remaining_time": "1:52:10"}
+{"current_steps": 850, "total_steps": 2500, "loss": 0.3117033004760742, "lr": 8.350837898457143e-05, "epoch": 3.4, "percentage": 34.0, "elapsed_time": "0:57:19", "remaining_time": "1:51:15"}
+{"current_steps": 860, "total_steps": 2500, "loss": 0.34261503219604494, "lr": 8.298696935515132e-05, "epoch": 3.44, "percentage": 34.4, "elapsed_time": "0:57:55", "remaining_time": "1:50:27"}
+{"current_steps": 870, "total_steps": 2500, "loss": 0.3159458637237549, "lr": 8.245912885069531e-05, "epoch": 3.48, "percentage": 34.8, "elapsed_time": "0:58:33", "remaining_time": "1:49:42"}
+{"current_steps": 880, "total_steps": 2500, "loss": 0.2982481002807617, "lr": 8.192496037475562e-05, "epoch": 3.52, "percentage": 35.2, "elapsed_time": "0:59:13", "remaining_time": "1:49:01"}
+{"current_steps": 890, "total_steps": 2500, "loss": 0.3232215404510498, "lr": 8.138456806453503e-05, "epoch": 3.56, "percentage": 35.6, "elapsed_time": "0:59:53", "remaining_time": "1:48:20"}
+{"current_steps": 900, "total_steps": 2500, "loss": 0.3305091381072998, "lr": 8.083805727058513e-05, "epoch": 3.6, "percentage": 36.0, "elapsed_time": "1:00:32", "remaining_time": "1:47:38"}
+{"current_steps": 900, "total_steps": 2500, "eval_loss": 0.44760578870773315, "epoch": 3.6, "percentage": 36.0, "elapsed_time": "1:00:52", "remaining_time": "1:48:13"}
+{"current_steps": 910, "total_steps": 2500, "loss": 0.35752732753753663, "lr": 8.028553453626808e-05, "epoch": 3.64, "percentage": 36.4, "elapsed_time": "1:01:35", "remaining_time": "1:47:37"}
+{"current_steps": 920, "total_steps": 2500, "loss": 0.3292932271957397, "lr": 7.972710757698567e-05, "epoch": 3.68, "percentage": 36.8, "elapsed_time": "1:02:16", "remaining_time": "1:46:56"}
+{"current_steps": 930, "total_steps": 2500, "loss": 0.28986682891845705, "lr": 7.916288525918007e-05, "epoch": 3.7199999999999998, "percentage": 37.2, "elapsed_time": "1:02:55", "remaining_time": "1:46:13"}
+{"current_steps": 940, "total_steps": 2500, "loss": 0.3027395725250244, "lr": 7.859297757911013e-05, "epoch": 3.76, "percentage": 37.6, "elapsed_time": "1:03:33", "remaining_time": "1:45:28"}
+{"current_steps": 950, "total_steps": 2500, "loss": 0.3238774061203003, "lr": 7.801749564140724e-05, "epoch": 3.8, "percentage": 38.0, "elapsed_time": "1:04:12", "remaining_time": "1:44:46"}
+{"current_steps": 960, "total_steps": 2500, "loss": 0.34537086486816404, "lr": 7.743655163741543e-05, "epoch": 3.84, "percentage": 38.4, "elapsed_time": "1:04:51", "remaining_time": "1:44:02"}
+{"current_steps": 970, "total_steps": 2500, "loss": 0.3292637825012207, "lr": 7.685025882331936e-05, "epoch": 3.88, "percentage": 38.8, "elapsed_time": "1:05:30", "remaining_time": "1:43:20"}
+{"current_steps": 980, "total_steps": 2500, "loss": 0.32722015380859376, "lr": 7.62587314980648e-05, "epoch": 3.92, "percentage": 39.2, "elapsed_time": "1:06:11", "remaining_time": "1:42:39"}
+{"current_steps": 990, "total_steps": 2500, "loss": 0.29880056381225584, "lr": 7.566208498107585e-05, "epoch": 3.96, "percentage": 39.6, "elapsed_time": "1:06:50", "remaining_time": "1:41:56"}
+{"current_steps": 1000, "total_steps": 2500, "loss": 0.2978524684906006, "lr": 7.506043558977321e-05, "epoch": 4.0, "percentage": 40.0, "elapsed_time": "1:07:29", "remaining_time": "1:41:14"}
+{"current_steps": 1000, "total_steps": 2500, "eval_loss": 0.44613513350486755, "epoch": 4.0, "percentage": 40.0, "elapsed_time": "1:07:49", "remaining_time": "1:41:43"}
+{"current_steps": 1010, "total_steps": 2500, "loss": 0.27530927658081056, "lr": 7.445390061689782e-05, "epoch": 4.04, "percentage": 40.4, "elapsed_time": "1:08:31", "remaining_time": "1:41:05"}
+{"current_steps": 1020, "total_steps": 2500, "loss": 0.2517704486846924, "lr": 7.38425983076444e-05, "epoch": 4.08, "percentage": 40.8, "elapsed_time": "1:09:11", "remaining_time": "1:40:23"}
+{"current_steps": 1030, "total_steps": 2500, "loss": 0.28200175762176516, "lr": 7.32266478366094e-05, "epoch": 4.12, "percentage": 41.2, "elapsed_time": "1:09:50", "remaining_time": "1:39:40"}
+{"current_steps": 1040, "total_steps": 2500, "loss": 0.2569046258926392, "lr": 7.260616928455754e-05, "epoch": 4.16, "percentage": 41.6, "elapsed_time": "1:10:30", "remaining_time": "1:38:58"}
+{"current_steps": 1050, "total_steps": 2500, "loss": 0.2665576696395874, "lr": 7.1981283615012e-05, "epoch": 4.2, "percentage": 42.0, "elapsed_time": "1:11:09", "remaining_time": "1:38:15"}
+{"current_steps": 1060, "total_steps": 2500, "loss": 0.2635650634765625, "lr": 7.135211265067216e-05, "epoch": 4.24, "percentage": 42.4, "elapsed_time": "1:11:47", "remaining_time": "1:37:31"}
+{"current_steps": 1070, "total_steps": 2500, "loss": 0.26842334270477297, "lr": 7.071877904966423e-05, "epoch": 4.28, "percentage": 42.8, "elapsed_time": "1:12:27", "remaining_time": "1:36:49"}
+{"current_steps": 1080, "total_steps": 2500, "loss": 0.2633937358856201, "lr": 7.00814062816285e-05, "epoch": 4.32, "percentage": 43.2, "elapsed_time": "1:13:07", "remaining_time": "1:36:08"}
+{"current_steps": 1090, "total_steps": 2500, "loss": 0.2895397186279297, "lr": 6.944011860364905e-05, "epoch": 4.36, "percentage": 43.6, "elapsed_time": "1:13:45", "remaining_time": "1:35:24"}
+{"current_steps": 1100, "total_steps": 2500, "loss": 0.27405414581298826, "lr": 6.879504103602935e-05, "epoch": 4.4, "percentage": 44.0, "elapsed_time": "1:14:25", "remaining_time": "1:34:43"}
+{"current_steps": 1100, "total_steps": 2500, "eval_loss": 0.46795058250427246, "epoch": 4.4, "percentage": 44.0, "elapsed_time": "1:14:42", "remaining_time": "1:35:05"}
+{"current_steps": 1110, "total_steps": 2500, "loss": 0.2581511974334717, "lr": 6.814629933791931e-05, "epoch": 4.44, "percentage": 44.4, "elapsed_time": "1:15:21", "remaining_time": "1:34:22"}
+{"current_steps": 1120, "total_steps": 2500, "loss": 0.2689012050628662, "lr": 6.749401998279846e-05, "epoch": 4.48, "percentage": 44.8, "elapsed_time": "1:16:00", "remaining_time": "1:33:39"}
+{"current_steps": 1130, "total_steps": 2500, "loss": 0.27230424880981446, "lr": 6.683833013381941e-05, "epoch": 4.52, "percentage": 45.2, "elapsed_time": "1:16:39", "remaining_time": "1:32:56"}
+{"current_steps": 1140, "total_steps": 2500, "loss": 0.2903036594390869, "lr": 6.617935761901748e-05, "epoch": 4.5600000000000005, "percentage": 45.6, "elapsed_time": "1:17:18", "remaining_time": "1:32:13"}
+{"current_steps": 1150, "total_steps": 2500, "loss": 0.2551115989685059, "lr": 6.551723090639007e-05, "epoch": 4.6, "percentage": 46.0, "elapsed_time": "1:17:57", "remaining_time": "1:31:31"}
+{"current_steps": 1160, "total_steps": 2500, "loss": 0.2783109188079834, "lr": 6.485207907885175e-05, "epoch": 4.64, "percentage": 46.4, "elapsed_time": "1:18:37", "remaining_time": "1:30:49"}
+{"current_steps": 1170, "total_steps": 2500, "loss": 0.29131503105163575, "lr": 6.418403180906922e-05, "epoch": 4.68, "percentage": 46.8, "elapsed_time": "1:19:17", "remaining_time": "1:30:08"}
+{"current_steps": 1180, "total_steps": 2500, "loss": 0.2730400085449219, "lr": 6.351321933418139e-05, "epoch": 4.72, "percentage": 47.2, "elapsed_time": "1:19:57", "remaining_time": "1:29:26"}
+{"current_steps": 1190, "total_steps": 2500, "loss": 0.2572148323059082, "lr": 6.283977243040939e-05, "epoch": 4.76, "percentage": 47.6, "elapsed_time": "1:20:37", "remaining_time": "1:28:44"}
+{"current_steps": 1200, "total_steps": 2500, "loss": 0.27444655895233155, "lr": 6.216382238756146e-05, "epoch": 4.8, "percentage": 48.0, "elapsed_time": "1:21:16", "remaining_time": "1:28:03"}
+{"current_steps": 1200, "total_steps": 2500, "eval_loss": 0.466619610786438, "epoch": 4.8, "percentage": 48.0, "elapsed_time": "1:21:36", "remaining_time": "1:28:24"}
+{"current_steps": 1210, "total_steps": 2500, "loss": 0.27054529190063475, "lr": 6.148550098343778e-05, "epoch": 4.84, "percentage": 48.4, "elapsed_time": "1:22:17", "remaining_time": "1:27:43"}
+{"current_steps": 1220, "total_steps": 2500, "loss": 0.26785056591033934, "lr": 6.080494045814011e-05, "epoch": 4.88, "percentage": 48.8, "elapsed_time": "1:22:55", "remaining_time": "1:26:59"}
+{"current_steps": 1230, "total_steps": 2500, "loss": 0.26335647106170657, "lr": 6.0122273488291304e-05, "epoch": 4.92, "percentage": 49.2, "elapsed_time": "1:23:34", "remaining_time": "1:26:17"}
+{"current_steps": 1240, "total_steps": 2500, "loss": 0.2614041090011597, "lr": 5.943763316116977e-05, "epoch": 4.96, "percentage": 49.6, "elapsed_time": "1:24:13", "remaining_time": "1:25:34"}
+{"current_steps": 1250, "total_steps": 2500, "loss": 0.24768717288970948, "lr": 5.875115294876381e-05, "epoch": 5.0, "percentage": 50.0, "elapsed_time": "1:24:52", "remaining_time": "1:24:52"}
+{"current_steps": 1260, "total_steps": 2500, "loss": 0.21707432270050048, "lr": 5.806296668175104e-05, "epoch": 5.04, "percentage": 50.4, "elapsed_time": "1:25:31", "remaining_time": "1:24:10"}
+{"current_steps": 1270, "total_steps": 2500, "loss": 0.2139519214630127, "lr": 5.737320852340775e-05, "epoch": 5.08, "percentage": 50.8, "elapsed_time": "1:26:06", "remaining_time": "1:23:23"}
+{"current_steps": 1280, "total_steps": 2500, "loss": 0.20998594760894776, "lr": 5.668201294345363e-05, "epoch": 5.12, "percentage": 51.2, "elapsed_time": "1:26:39", "remaining_time": "1:22:35"}
+{"current_steps": 1290, "total_steps": 2500, "loss": 0.23306002616882324, "lr": 5.598951469183649e-05, "epoch": 5.16, "percentage": 51.6, "elapsed_time": "1:27:14", "remaining_time": "1:21:49"}
+{"current_steps": 1300, "total_steps": 2500, "loss": 0.2262401580810547, "lr": 5.52958487724626e-05, "epoch": 5.2, "percentage": 52.0, "elapsed_time": "1:27:49", "remaining_time": "1:21:04"}
+{"current_steps": 1300, "total_steps": 2500, "eval_loss": 0.49972543120384216, "epoch": 5.2, "percentage": 52.0, "elapsed_time": "1:28:08", "remaining_time": "1:21:22"}
+{"current_steps": 1310, "total_steps": 2500, "loss": 0.21100988388061523, "lr": 5.4601150416877367e-05, "epoch": 5.24, "percentage": 52.4, "elapsed_time": "1:28:47", "remaining_time": "1:20:39"}
+{"current_steps": 1320, "total_steps": 2500, "loss": 0.23542592525482178, "lr": 5.390555505790168e-05, "epoch": 5.28, "percentage": 52.8, "elapsed_time": "1:29:20", "remaining_time": "1:19:52"}
+{"current_steps": 1330, "total_steps": 2500, "loss": 0.2095633029937744, "lr": 5.3209198303229027e-05, "epoch": 5.32, "percentage": 53.2, "elapsed_time": "1:29:54", "remaining_time": "1:19:05"}
+{"current_steps": 1340, "total_steps": 2500, "loss": 0.21693904399871827, "lr": 5.2512215908988484e-05, "epoch": 5.36, "percentage": 53.6, "elapsed_time": "1:30:31", "remaining_time": "1:18:21"}
+{"current_steps": 1350, "total_steps": 2500, "loss": 0.2076347827911377, "lr": 5.1814743753278795e-05, "epoch": 5.4, "percentage": 54.0, "elapsed_time": "1:31:11", "remaining_time": "1:17:40"}
+{"current_steps": 1360, "total_steps": 2500, "loss": 0.22539749145507812, "lr": 5.111691780967869e-05, "epoch": 5.44, "percentage": 54.4, "elapsed_time": "1:31:50", "remaining_time": "1:16:58"}
+{"current_steps": 1370, "total_steps": 2500, "loss": 0.2077547550201416, "lr": 5.041887412073854e-05, "epoch": 5.48, "percentage": 54.8, "elapsed_time": "1:32:27", "remaining_time": "1:16:15"}
+{"current_steps": 1380, "total_steps": 2500, "loss": 0.21558783054351807, "lr": 4.97207487714586e-05, "epoch": 5.52, "percentage": 55.2, "elapsed_time": "1:33:04", "remaining_time": "1:15:32"}
+{"current_steps": 1390, "total_steps": 2500, "loss": 0.21069679260253907, "lr": 4.9022677862758945e-05, "epoch": 5.5600000000000005, "percentage": 55.6, "elapsed_time": "1:33:39", "remaining_time": "1:14:47"}
+{"current_steps": 1400, "total_steps": 2500, "loss": 0.21843309402465821, "lr": 4.832479748494643e-05, "epoch": 5.6, "percentage": 56.0, "elapsed_time": "1:34:13", "remaining_time": "1:14:02"}
+{"current_steps": 1400, "total_steps": 2500, "eval_loss": 0.49576279520988464, "epoch": 5.6, "percentage": 56.0, "elapsed_time": "1:34:31", "remaining_time": "1:14:16"}
+{"current_steps": 1410, "total_steps": 2500, "loss": 0.22310276031494142, "lr": 4.7627243691183453e-05, "epoch": 5.64, "percentage": 56.4, "elapsed_time": "1:35:09", "remaining_time": "1:13:33"}
+{"current_steps": 1420, "total_steps": 2500, "loss": 0.22056117057800292, "lr": 4.693015247096423e-05, "epoch": 5.68, "percentage": 56.8, "elapsed_time": "1:35:47", "remaining_time": "1:12:51"}
+{"current_steps": 1430, "total_steps": 2500, "loss": 0.2241537094116211, "lr": 4.623365972360337e-05, "epoch": 5.72, "percentage": 57.2, "elapsed_time": "1:36:25", "remaining_time": "1:12:09"}
+{"current_steps": 1440, "total_steps": 2500, "loss": 0.21514451503753662, "lr": 4.553790123174197e-05, "epoch": 5.76, "percentage": 57.6, "elapsed_time": "1:37:04", "remaining_time": "1:11:27"}
+{"current_steps": 1450, "total_steps": 2500, "loss": 0.21031346321105956, "lr": 4.484301263487665e-05, "epoch": 5.8, "percentage": 58.0, "elapsed_time": "1:37:42", "remaining_time": "1:10:45"}
+{"current_steps": 1460, "total_steps": 2500, "loss": 0.2312474489212036, "lr": 4.414912940291613e-05, "epoch": 5.84, "percentage": 58.4, "elapsed_time": "1:38:21", "remaining_time": "1:10:03"}
+{"current_steps": 1470, "total_steps": 2500, "loss": 0.22380952835083007, "lr": 4.345638680977139e-05, "epoch": 5.88, "percentage": 58.8, "elapsed_time": "1:39:00", "remaining_time": "1:09:22"}
+{"current_steps": 1480, "total_steps": 2500, "loss": 0.22706894874572753, "lr": 4.276491990698355e-05, "epoch": 5.92, "percentage": 59.2, "elapsed_time": "1:39:39", "remaining_time": "1:08:40"}
+{"current_steps": 1490, "total_steps": 2500, "loss": 0.2103546142578125, "lr": 4.2074863497395377e-05, "epoch": 5.96, "percentage": 59.6, "elapsed_time": "1:40:15", "remaining_time": "1:07:57"}
+{"current_steps": 1500, "total_steps": 2500, "loss": 0.2276217222213745, "lr": 4.1386352108871174e-05, "epoch": 6.0, "percentage": 60.0, "elapsed_time": "1:40:50", "remaining_time": "1:07:13"}
+{"current_steps": 1500, "total_steps": 2500, "eval_loss": 0.4966464042663574, "epoch": 6.0, "percentage": 60.0, "elapsed_time": "1:41:07", "remaining_time": "1:07:25"}
+{"current_steps": 1510, "total_steps": 2500, "loss": 0.16540236473083497, "lr": 4.069951996807034e-05, "epoch": 6.04, "percentage": 60.4, "elapsed_time": "1:41:49", "remaining_time": "1:06:45"}
+{"current_steps": 1520, "total_steps": 2500, "loss": 0.1638352394104004, "lr": 4.001450097427966e-05, "epoch": 6.08, "percentage": 60.8, "elapsed_time": "1:42:26", "remaining_time": "1:06:03"}
+{"current_steps": 1530, "total_steps": 2500, "loss": 0.1719011664390564, "lr": 3.9331428673309204e-05, "epoch": 6.12, "percentage": 61.2, "elapsed_time": "1:43:05", "remaining_time": "1:05:21"}
+{"current_steps": 1540, "total_steps": 2500, "loss": 0.1651092290878296, "lr": 3.865043623145751e-05, "epoch": 6.16, "percentage": 61.6, "elapsed_time": "1:43:41", "remaining_time": "1:04:38"}
+{"current_steps": 1550, "total_steps": 2500, "loss": 0.1746900796890259, "lr": 3.797165640955041e-05, "epoch": 6.2, "percentage": 62.0, "elapsed_time": "1:44:21", "remaining_time": "1:03:57"}
+{"current_steps": 1560, "total_steps": 2500, "loss": 0.16637682914733887, "lr": 3.729522153705916e-05, "epoch": 6.24, "percentage": 62.4, "elapsed_time": "1:44:59", "remaining_time": "1:03:15"}
+{"current_steps": 1570, "total_steps": 2500, "loss": 0.1709848165512085, "lr": 3.662126348630237e-05, "epoch": 6.28, "percentage": 62.8, "elapsed_time": "1:45:38", "remaining_time": "1:02:34"}
+{"current_steps": 1580, "total_steps": 2500, "loss": 0.18107957839965821, "lr": 3.594991364673745e-05, "epoch": 6.32, "percentage": 63.2, "elapsed_time": "1:46:17", "remaining_time": "1:01:53"}
+{"current_steps": 1590, "total_steps": 2500, "loss": 0.16225044727325438, "lr": 3.528130289934583e-05, "epoch": 6.36, "percentage": 63.6, "elapsed_time": "1:46:54", "remaining_time": "1:01:11"}
+{"current_steps": 1600, "total_steps": 2500, "loss": 0.17544152736663818, "lr": 3.461556159111748e-05, "epoch": 6.4, "percentage": 64.0, "elapsed_time": "1:47:32", "remaining_time": "1:00:29"}
+{"current_steps": 1600, "total_steps": 2500, "eval_loss": 0.5342507362365723, "epoch": 6.4, "percentage": 64.0, "elapsed_time": "1:47:52", "remaining_time": "1:00:40"}
+{"current_steps": 1610, "total_steps": 2500, "loss": 0.17091144323349, "lr": 3.3952819509639534e-05, "epoch": 6.44, "percentage": 64.4, "elapsed_time": "1:48:32", "remaining_time": "0:59:59"}
+{"current_steps": 1620, "total_steps": 2500, "loss": 0.17765278816223146, "lr": 3.329320585779393e-05, "epoch": 6.48, "percentage": 64.8, "elapsed_time": "1:49:11", "remaining_time": "0:59:18"}
+{"current_steps": 1630, "total_steps": 2500, "loss": 0.16475566625595092, "lr": 3.263684922856905e-05, "epoch": 6.52, "percentage": 65.2, "elapsed_time": "1:49:51", "remaining_time": "0:58:38"}
+{"current_steps": 1640, "total_steps": 2500, "loss": 0.172060227394104, "lr": 3.1983877579990274e-05, "epoch": 6.5600000000000005, "percentage": 65.6, "elapsed_time": "1:50:30", "remaining_time": "0:57:57"}
+{"current_steps": 1650, "total_steps": 2500, "loss": 0.16673840284347535, "lr": 3.1334418210174263e-05, "epoch": 6.6, "percentage": 66.0, "elapsed_time": "1:51:09", "remaining_time": "0:57:15"}
+{"current_steps": 1660, "total_steps": 2500, "loss": 0.17414634227752684, "lr": 3.0688597732512e-05, "epoch": 6.64, "percentage": 66.4, "elapsed_time": "1:51:47", "remaining_time": "0:56:34"}
+{"current_steps": 1670, "total_steps": 2500, "loss": 0.1620783567428589, "lr": 3.0046542050985237e-05, "epoch": 6.68, "percentage": 66.8, "elapsed_time": "1:52:28", "remaining_time": "0:55:53"}
+{"current_steps": 1680, "total_steps": 2500, "loss": 0.17428462505340575, "lr": 2.940837633562127e-05, "epoch": 6.72, "percentage": 67.2, "elapsed_time": "1:53:08", "remaining_time": "0:55:13"}
+{"current_steps": 1690, "total_steps": 2500, "loss": 0.19050977230072022, "lr": 2.877422499809072e-05, "epoch": 6.76, "percentage": 67.6, "elapsed_time": "1:53:46", "remaining_time": "0:54:31"}
+{"current_steps": 1700, "total_steps": 2500, "loss": 0.16926174163818358, "lr": 2.8144211667453368e-05, "epoch": 6.8, "percentage": 68.0, "elapsed_time": "1:54:23", "remaining_time": "0:53:50"}
+{"current_steps": 1700, "total_steps": 2500, "eval_loss": 0.5441356301307678, "epoch": 6.8, "percentage": 68.0, "elapsed_time": "1:54:41", "remaining_time": "0:53:58"}
+{"current_steps": 1710, "total_steps": 2500, "loss": 0.1793771743774414, "lr": 2.75184591660563e-05, "epoch": 6.84, "percentage": 68.4, "elapsed_time": "1:55:20", "remaining_time": "0:53:17"}
+{"current_steps": 1720, "total_steps": 2500, "loss": 0.1647491931915283, "lr": 2.6897089485589583e-05, "epoch": 6.88, "percentage": 68.8, "elapsed_time": "1:55:56", "remaining_time": "0:52:34"}
+{"current_steps": 1730, "total_steps": 2500, "loss": 0.17397019863128663, "lr": 2.6280223763303546e-05, "epoch": 6.92, "percentage": 69.2, "elapsed_time": "1:56:35", "remaining_time": "0:51:53"}
+{"current_steps": 1740, "total_steps": 2500, "loss": 0.17107686996459961, "lr": 2.5667982258393014e-05, "epoch": 6.96, "percentage": 69.6, "elapsed_time": "1:57:11", "remaining_time": "0:51:11"}
+{"current_steps": 1750, "total_steps": 2500, "loss": 0.1730511426925659, "lr": 2.506048432855247e-05, "epoch": 7.0, "percentage": 70.0, "elapsed_time": "1:57:50", "remaining_time": "0:50:30"}
+{"current_steps": 1760, "total_steps": 2500, "loss": 0.13950222730636597, "lr": 2.4457848406707013e-05, "epoch": 7.04, "percentage": 70.4, "elapsed_time": "1:58:29", "remaining_time": "0:49:49"}
+{"current_steps": 1770, "total_steps": 2500, "loss": 0.1326605796813965, "lr": 2.3860191977923672e-05, "epoch": 7.08, "percentage": 70.8, "elapsed_time": "1:59:08", "remaining_time": "0:49:08"}
+{"current_steps": 1780, "total_steps": 2500, "loss": 0.1265331983566284, "lr": 2.326763155650744e-05, "epoch": 7.12, "percentage": 71.2, "elapsed_time": "1:59:47", "remaining_time": "0:48:27"}
+{"current_steps": 1790, "total_steps": 2500, "loss": 0.12731509208679198, "lr": 2.2680282663286552e-05, "epoch": 7.16, "percentage": 71.6, "elapsed_time": "2:00:26", "remaining_time": "0:47:46"}
+{"current_steps": 1800, "total_steps": 2500, "loss": 0.13114826679229735, "lr": 2.209825980309151e-05, "epoch": 7.2, "percentage": 72.0, "elapsed_time": "2:01:03", "remaining_time": "0:47:04"}
+{"current_steps": 1800, "total_steps": 2500, "eval_loss": 0.5847110748291016, "epoch": 7.2, "percentage": 72.0, "elapsed_time": "2:01:22", "remaining_time": "0:47:12"}
+{"current_steps": 1810, "total_steps": 2500, "loss": 0.12906957864761354, "lr": 2.152167644243213e-05, "epoch": 7.24, "percentage": 72.4, "elapsed_time": "2:02:00", "remaining_time": "0:46:30"}
+{"current_steps": 1820, "total_steps": 2500, "loss": 0.133590030670166, "lr": 2.095064498737701e-05, "epoch": 7.28, "percentage": 72.8, "elapsed_time": "2:02:38", "remaining_time": "0:45:49"}
+{"current_steps": 1830, "total_steps": 2500, "loss": 0.13653848171234131, "lr": 2.0385276761639765e-05, "epoch": 7.32, "percentage": 73.2, "elapsed_time": "2:03:18", "remaining_time": "0:45:08"}
+{"current_steps": 1840, "total_steps": 2500, "loss": 0.12472724914550781, "lr": 1.9825681984876172e-05, "epoch": 7.36, "percentage": 73.6, "elapsed_time": "2:03:56", "remaining_time": "0:44:27"}
+{"current_steps": 1850, "total_steps": 2500, "loss": 0.13255125284194946, "lr": 1.9271969751196776e-05, "epoch": 7.4, "percentage": 74.0, "elapsed_time": "2:04:32", "remaining_time": "0:43:45"}
+{"current_steps": 1860, "total_steps": 2500, "loss": 0.13693161010742189, "lr": 1.8724248007898647e-05, "epoch": 7.44, "percentage": 74.4, "elapsed_time": "2:05:09", "remaining_time": "0:43:03"}
+{"current_steps": 1870, "total_steps": 2500, "loss": 0.13425672054290771, "lr": 1.8182623534420907e-05, "epoch": 7.48, "percentage": 74.8, "elapsed_time": "2:05:43", "remaining_time": "0:42:21"}
+{"current_steps": 1880, "total_steps": 2500, "loss": 0.13668575286865234, "lr": 1.76472019215278e-05, "epoch": 7.52, "percentage": 75.2, "elapsed_time": "2:06:19", "remaining_time": "0:41:39"}
+{"current_steps": 1890, "total_steps": 2500, "loss": 0.1317702889442444, "lr": 1.7118087550723633e-05, "epoch": 7.5600000000000005, "percentage": 75.6, "elapsed_time": "2:06:53", "remaining_time": "0:40:57"}
+{"current_steps": 1900, "total_steps": 2500, "loss": 0.14458621740341188, "lr": 1.659538357390341e-05, "epoch": 7.6, "percentage": 76.0, "elapsed_time": "2:07:26", "remaining_time": "0:40:14"}
+{"current_steps": 1900, "total_steps": 2500, "eval_loss": 0.5830516219139099, "epoch": 7.6, "percentage": 76.0, "elapsed_time": "2:07:44", "remaining_time": "0:40:20"}
+{"current_steps": 1910, "total_steps": 2500, "loss": 0.13126691579818725, "lr": 1.60791918932431e-05, "epoch": 7.64, "percentage": 76.4, "elapsed_time": "2:08:25", "remaining_time": "0:39:40"}
+{"current_steps": 1920, "total_steps": 2500, "loss": 0.12600460052490234, "lr": 1.556961314133359e-05, "epoch": 7.68, "percentage": 76.8, "elapsed_time": "2:09:04", "remaining_time": "0:38:59"}
+{"current_steps": 1930, "total_steps": 2500, "loss": 0.12453792095184327, "lr": 1.5066746661562253e-05, "epoch": 7.72, "percentage": 77.2, "elapsed_time": "2:09:40", "remaining_time": "0:38:17"}
+{"current_steps": 1940, "total_steps": 2500, "loss": 0.14839541912078857, "lr": 1.4570690488745687e-05, "epoch": 7.76, "percentage": 77.6, "elapsed_time": "2:10:17", "remaining_time": "0:37:36"}
+{"current_steps": 1950, "total_steps": 2500, "loss": 0.1321096420288086, "lr": 1.4081541330017705e-05, "epoch": 7.8, "percentage": 78.0, "elapsed_time": "2:10:53", "remaining_time": "0:36:55"}
+{"current_steps": 1960, "total_steps": 2500, "loss": 0.1317069411277771, "lr": 1.3599394545975951e-05, "epoch": 7.84, "percentage": 78.4, "elapsed_time": "2:11:29", "remaining_time": "0:36:13"}
+{"current_steps": 1970, "total_steps": 2500, "loss": 0.13362932205200195, "lr": 1.312434413209131e-05, "epoch": 7.88, "percentage": 78.8, "elapsed_time": "2:12:06", "remaining_time": "0:35:32"}
+{"current_steps": 1980, "total_steps": 2500, "loss": 0.12677763700485228, "lr": 1.2656482700383237e-05, "epoch": 7.92, "percentage": 79.2, "elapsed_time": "2:12:43", "remaining_time": "0:34:51"}
+{"current_steps": 1990, "total_steps": 2500, "loss": 0.1382434129714966, "lr": 1.219590146136485e-05, "epoch": 7.96, "percentage": 79.6, "elapsed_time": "2:13:22", "remaining_time": "0:34:10"}
+{"current_steps": 2000, "total_steps": 2500, "loss": 0.12519369125366211, "lr": 1.1742690206261292e-05, "epoch": 8.0, "percentage": 80.0, "elapsed_time": "2:13:59", "remaining_time": "0:33:29"}
+{"current_steps": 2000, "total_steps": 2500, "eval_loss": 0.5840195417404175, "epoch": 8.0, "percentage": 80.0, "elapsed_time": "2:14:18", "remaining_time": "0:33:34"}
+{"current_steps": 2010, "total_steps": 2500, "loss": 0.10409053564071655, "lr": 1.129693728950474e-05, "epoch": 8.04, "percentage": 80.4, "elapsed_time": "2:15:02", "remaining_time": "0:32:55"}
+{"current_steps": 2020, "total_steps": 2500, "loss": 0.10310100317001343, "lr": 1.0858729611509516e-05, "epoch": 8.08, "percentage": 80.8, "elapsed_time": "2:15:43", "remaining_time": "0:32:15"}
+{"current_steps": 2030, "total_steps": 2500, "loss": 0.09960774183273316, "lr": 1.0428152601730718e-05, "epoch": 8.12, "percentage": 81.2, "elapsed_time": "2:16:20", "remaining_time": "0:31:34"}
+{"current_steps": 2040, "total_steps": 2500, "loss": 0.09982571601867676, "lr": 1.0005290202009531e-05, "epoch": 8.16, "percentage": 81.6, "elapsed_time": "2:16:58", "remaining_time": "0:30:53"}
+{"current_steps": 2050, "total_steps": 2500, "loss": 0.11322143077850341, "lr": 9.590224850208646e-06, "epoch": 8.2, "percentage": 82.0, "elapsed_time": "2:17:36", "remaining_time": "0:30:12"}
+{"current_steps": 2060, "total_steps": 2500, "loss": 0.10006082057952881, "lr": 9.183037464140804e-06, "epoch": 8.24, "percentage": 82.4, "elapsed_time": "2:18:13", "remaining_time": "0:29:31"}
+{"current_steps": 2070, "total_steps": 2500, "loss": 0.11560235023498536, "lr": 8.783807425793721e-06, "epoch": 8.28, "percentage": 82.8, "elapsed_time": "2:18:46", "remaining_time": "0:28:49"}
+{"current_steps": 2080, "total_steps": 2500, "loss": 0.10931503772735596, "lr": 8.392612565854375e-06, "epoch": 8.32, "percentage": 83.2, "elapsed_time": "2:19:21", "remaining_time": "0:28:08"}
+{"current_steps": 2090, "total_steps": 2500, "loss": 0.10900030136108399, "lr": 8.009529148535855e-06, "epoch": 8.36, "percentage": 83.6, "elapsed_time": "2:19:58", "remaining_time": "0:27:27"}
+{"current_steps": 2100, "total_steps": 2500, "loss": 0.1069128155708313, "lr": 7.63463185670939e-06, "epoch": 8.4, "percentage": 84.0, "elapsed_time": "2:20:34", "remaining_time": "0:26:46"}
+{"current_steps": 2100, "total_steps": 2500, "eval_loss": 0.6247864961624146, "epoch": 8.4, "percentage": 84.0, "elapsed_time": "2:20:53", "remaining_time": "0:26:50"}
+{"current_steps": 2110, "total_steps": 2500, "loss": 0.09856721758842468, "lr": 7.267993777344856e-06, "epoch": 8.44, "percentage": 84.4, "elapsed_time": "2:21:31", "remaining_time": "0:26:09"}
+{"current_steps": 2120, "total_steps": 2500, "loss": 0.10609345436096192, "lr": 6.909686387262254e-06, "epoch": 8.48, "percentage": 84.8, "elapsed_time": "2:22:07", "remaining_time": "0:25:28"}
+{"current_steps": 2130, "total_steps": 2500, "loss": 0.105103600025177, "lr": 6.559779539197231e-06, "epoch": 8.52, "percentage": 85.2, "elapsed_time": "2:22:41", "remaining_time": "0:24:47"}
+{"current_steps": 2140, "total_steps": 2500, "loss": 0.10853493213653564, "lr": 6.21834144818314e-06, "epoch": 8.56, "percentage": 85.6, "elapsed_time": "2:23:18", "remaining_time": "0:24:06"}
+{"current_steps": 2150, "total_steps": 2500, "loss": 0.11464111804962158, "lr": 5.885438678252342e-06, "epoch": 8.6, "percentage": 86.0, "elapsed_time": "2:23:56", "remaining_time": "0:23:25"}
+{"current_steps": 2160, "total_steps": 2500, "loss": 0.10765299797058106, "lr": 5.5611361294594325e-06, "epoch": 8.64, "percentage": 86.4, "elapsed_time": "2:24:33", "remaining_time": "0:22:45"}
+{"current_steps": 2170, "total_steps": 2500, "loss": 0.10699164867401123, "lr": 5.245497025228874e-06, "epoch": 8.68, "percentage": 86.8, "elapsed_time": "2:25:12", "remaining_time": "0:22:04"}
+{"current_steps": 2180, "total_steps": 2500, "loss": 0.10728691816329956, "lr": 4.938582900029437e-06, "epoch": 8.72, "percentage": 87.2, "elapsed_time": "2:25:52", "remaining_time": "0:21:24"}
+{"current_steps": 2190, "total_steps": 2500, "loss": 0.11177785396575927, "lr": 4.640453587377957e-06, "epoch": 8.76, "percentage": 87.6, "elapsed_time": "2:26:32", "remaining_time": "0:20:44"}
+{"current_steps": 2200, "total_steps": 2500, "loss": 0.11041848659515381, "lr": 4.351167208174639e-06, "epoch": 8.8, "percentage": 88.0, "elapsed_time": "2:27:11", "remaining_time": "0:20:04"}
+{"current_steps": 2200, "total_steps": 2500, "eval_loss": 0.6235533356666565, "epoch": 8.8, "percentage": 88.0, "elapsed_time": "2:27:30", "remaining_time": "0:20:06"}
+{"current_steps": 2210, "total_steps": 2500, "loss": 0.1085782766342163, "lr": 4.0707801593723e-06, "epoch": 8.84, "percentage": 88.4, "elapsed_time": "2:28:10", "remaining_time": "0:19:26"}
+{"current_steps": 2220, "total_steps": 2500, "loss": 0.11138873100280762, "lr": 3.799347102981665e-06, "epoch": 8.88, "percentage": 88.8, "elapsed_time": "2:28:48", "remaining_time": "0:18:46"}
+{"current_steps": 2230, "total_steps": 2500, "loss": 0.10770895481109619, "lr": 3.536920955414885e-06, "epoch": 8.92, "percentage": 89.2, "elapsed_time": "2:29:26", "remaining_time": "0:18:05"}
+{"current_steps": 2240, "total_steps": 2500, "loss": 0.11167995929718018, "lr": 3.2835528771693992e-06, "epoch": 8.96, "percentage": 89.6, "elapsed_time": "2:30:02", "remaining_time": "0:17:24"}
+{"current_steps": 2250, "total_steps": 2500, "loss": 0.11738998889923095, "lr": 3.039292262854088e-06, "epoch": 9.0, "percentage": 90.0, "elapsed_time": "2:30:38", "remaining_time": "0:16:44"}
+{"current_steps": 2260, "total_steps": 2500, "loss": 0.10072145462036133, "lr": 2.804186731559677e-06, "epoch": 9.04, "percentage": 90.4, "elapsed_time": "2:31:15", "remaining_time": "0:16:03"}
+{"current_steps": 2270, "total_steps": 2500, "loss": 0.09228388667106628, "lr": 2.5782821175753422e-06, "epoch": 9.08, "percentage": 90.8, "elapsed_time": "2:31:51", "remaining_time": "0:15:23"}
+{"current_steps": 2280, "total_steps": 2500, "loss": 0.09626876711845397, "lr": 2.361622461453178e-06, "epoch": 9.12, "percentage": 91.2, "elapsed_time": "2:32:28", "remaining_time": "0:14:42"}
+{"current_steps": 2290, "total_steps": 2500, "loss": 0.0960278868675232, "lr": 2.154250001422431e-06, "epoch": 9.16, "percentage": 91.6, "elapsed_time": "2:33:07", "remaining_time": "0:14:02"}
+{"current_steps": 2300, "total_steps": 2500, "loss": 0.0941778838634491, "lr": 1.956205165155078e-06, "epoch": 9.2, "percentage": 92.0, "elapsed_time": "2:33:45", "remaining_time": "0:13:22"}
+{"current_steps": 2300, "total_steps": 2500, "eval_loss": 0.6419874429702759, "epoch": 9.2, "percentage": 92.0, "elapsed_time": "2:34:05", "remaining_time": "0:13:23"}
+{"current_steps": 2310, "total_steps": 2500, "loss": 0.09725146293640137, "lr": 1.7675265618843362e-06, "epoch": 9.24, "percentage": 92.4, "elapsed_time": "2:34:46", "remaining_time": "0:12:43"}
+{"current_steps": 2320, "total_steps": 2500, "loss": 0.09353782534599304, "lr": 1.5882509748777808e-06, "epoch": 9.28, "percentage": 92.8, "elapsed_time": "2:35:21", "remaining_time": "0:12:03"}
+{"current_steps": 2330, "total_steps": 2500, "loss": 0.09848537445068359, "lr": 1.4184133542663014e-06, "epoch": 9.32, "percentage": 93.2, "elapsed_time": "2:35:54", "remaining_time": "0:11:22"}
+{"current_steps": 2340, "total_steps": 2500, "loss": 0.10164464712142944, "lr": 1.258046810230562e-06, "epoch": 9.36, "percentage": 93.6, "elapsed_time": "2:36:32", "remaining_time": "0:10:42"}
+{"current_steps": 2350, "total_steps": 2500, "loss": 0.0934177041053772, "lr": 1.1071826065460588e-06, "epoch": 9.4, "percentage": 94.0, "elapsed_time": "2:37:08", "remaining_time": "0:10:01"}
+{"current_steps": 2360, "total_steps": 2500, "loss": 0.1012031078338623, "lr": 9.65850154488218e-07, "epoch": 9.44, "percentage": 94.4, "elapsed_time": "2:37:43", "remaining_time": "0:09:21"}
+{"current_steps": 2370, "total_steps": 2500, "loss": 0.09371918439865112, "lr": 8.340770070986214e-07, "epoch": 9.48, "percentage": 94.8, "elapsed_time": "2:38:22", "remaining_time": "0:08:41"}
+{"current_steps": 2380, "total_steps": 2500, "loss": 0.09450345039367676, "lr": 7.11888853813436e-07, "epoch": 9.52, "percentage": 95.2, "elapsed_time": "2:39:01", "remaining_time": "0:08:01"}
+{"current_steps": 2390, "total_steps": 2500, "loss": 0.09499152898788452, "lr": 5.993095154552431e-07, "epoch": 9.56, "percentage": 95.6, "elapsed_time": "2:39:40", "remaining_time": "0:07:20"}
+{"current_steps": 2400, "total_steps": 2500, "loss": 0.10716021060943604, "lr": 4.963609395891299e-07, "epoch": 9.6, "percentage": 96.0, "elapsed_time": "2:40:19", "remaining_time": "0:06:40"}
+{"current_steps": 2400, "total_steps": 2500, "eval_loss": 0.6402375102043152, "epoch": 9.6, "percentage": 96.0, "elapsed_time": "2:40:38", "remaining_time": "0:06:41"}
+{"current_steps": 2410, "total_steps": 2500, "loss": 0.09596163630485535, "lr": 4.030631962439302e-07, "epoch": 9.64, "percentage": 96.4, "elapsed_time": "2:41:17", "remaining_time": "0:06:01"}
+{"current_steps": 2420, "total_steps": 2500, "loss": 0.09645589590072631, "lr": 3.1943447399958027e-07, "epoch": 9.68, "percentage": 96.8, "elapsed_time": "2:41:52", "remaining_time": "0:05:21"}
+{"current_steps": 2430, "total_steps": 2500, "loss": 0.09415926933288574, "lr": 2.4549107644117885e-07, "epoch": 9.72, "percentage": 97.2, "elapsed_time": "2:42:26", "remaining_time": "0:04:40"}
+{"current_steps": 2440, "total_steps": 2500, "loss": 0.10026730298995971, "lr": 1.8124741898058462e-07, "epoch": 9.76, "percentage": 97.6, "elapsed_time": "2:42:59", "remaining_time": "0:04:00"}
+{"current_steps": 2450, "total_steps": 2500, "loss": 0.09711679220199584, "lr": 1.267160260461253e-07, "epoch": 9.8, "percentage": 98.0, "elapsed_time": "2:43:33", "remaining_time": "0:03:20"}
+{"current_steps": 2460, "total_steps": 2500, "loss": 0.09345818758010864, "lr": 8.190752864088436e-08, "epoch": 9.84, "percentage": 98.4, "elapsed_time": "2:44:08", "remaining_time": "0:02:40"}
+{"current_steps": 2470, "total_steps": 2500, "loss": 0.102751624584198, "lr": 4.683066227023081e-08, "epoch": 9.88, "percentage": 98.8, "elapsed_time": "2:44:45", "remaining_time": "0:02:00"}
+{"current_steps": 2480, "total_steps": 2500, "loss": 0.0988599717617035, "lr": 2.1492265238748366e-08, "epoch": 9.92, "percentage": 99.2, "elapsed_time": "2:45:21", "remaining_time": "0:01:20"}
+{"current_steps": 2490, "total_steps": 2500, "loss": 0.09828301668167114, "lr": 5.897277317157279e-09, "epoch": 9.96, "percentage": 99.6, "elapsed_time": "2:45:56", "remaining_time": "0:00:39"}
+{"current_steps": 2500, "total_steps": 2500, "loss": 0.0937616467475891, "lr": 4.873877924582715e-11, "epoch": 10.0, "percentage": 100.0, "elapsed_time": "2:46:32", "remaining_time": "0:00:00"}
+{"current_steps": 2500, "total_steps": 2500, "eval_loss": 0.6409608721733093, "epoch": 10.0, "percentage": 100.0, "elapsed_time": "2:46:50", "remaining_time": "0:00:00"}
+{"current_steps": 2500, "total_steps": 2500, "epoch": 10.0, "percentage": 100.0, "elapsed_time": "2:46:51", "remaining_time": "0:00:00"}

trainer_state.json

@@ -0,0 +1,1993 @@
+{
+  "best_global_step": 700,
+  "best_metric": 0.44259119033813477,
+  "best_model_checkpoint": "/data/taoyong/LabOS/QWEN-36/checkpoints/qwen3.6-35b-a3b-lora-lf/checkpoint-700",
+  "epoch": 10.0,
+  "eval_steps": 100,
+  "global_step": 2500,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.04,
+      "grad_norm": 1.3030611276626587,
+      "learning_rate": 3.6e-06,
+      "loss": 1.1145790100097657,
+      "step": 10
+    },
+    {
+      "epoch": 0.08,
+      "grad_norm": 1.540786623954773,
+      "learning_rate": 7.6e-06,
+      "loss": 1.2167404174804688,
+      "step": 20
+    },
+    {
+      "epoch": 0.12,
+      "grad_norm": 1.0591915845870972,
+      "learning_rate": 1.16e-05,
+      "loss": 1.0437713623046876,
+      "step": 30
+    },
+    {
+      "epoch": 0.16,
+      "grad_norm": 0.6695119142532349,
+      "learning_rate": 1.56e-05,
+      "loss": 0.9282869338989258,
+      "step": 40
+    },
+    {
+      "epoch": 0.2,
+      "grad_norm": 0.7912387847900391,
+      "learning_rate": 1.9600000000000002e-05,
+      "loss": 0.8799624443054199,
+      "step": 50
+    },
+    {
+      "epoch": 0.24,
+      "grad_norm": 0.7810359001159668,
+      "learning_rate": 2.36e-05,
+      "loss": 0.7062759399414062,
+      "step": 60
+    },
+    {
+      "epoch": 0.28,
+      "grad_norm": 0.7185921669006348,
+      "learning_rate": 2.7600000000000003e-05,
+      "loss": 0.7228042602539062,
+      "step": 70
+    },
+    {
+      "epoch": 0.32,
+      "grad_norm": 0.7974339723587036,
+      "learning_rate": 3.16e-05,
+      "loss": 0.6257906913757324,
+      "step": 80
+    },
+    {
+      "epoch": 0.36,
+      "grad_norm": 0.7850703597068787,
+      "learning_rate": 3.56e-05,
+      "loss": 0.5399329185485839,
+      "step": 90
+    },
+    {
+      "epoch": 0.4,
+      "grad_norm": 0.7295215129852295,
+      "learning_rate": 3.960000000000001e-05,
+      "loss": 0.5184461116790772,
+      "step": 100
+    },
+    {
+      "epoch": 0.4,
+      "eval_loss": 0.5476460456848145,
+      "eval_runtime": 21.5181,
+      "eval_samples_per_second": 18.589,
+      "eval_steps_per_second": 3.114,
+      "step": 100
+    },
+    {
+      "epoch": 0.44,
+      "grad_norm": 1.0682953596115112,
+      "learning_rate": 4.36e-05,
+      "loss": 0.5210700988769531,
+      "step": 110
+    },
+    {
+      "epoch": 0.48,
+      "grad_norm": 0.9108087420463562,
+      "learning_rate": 4.76e-05,
+      "loss": 0.5155693531036377,
+      "step": 120
+    },
+    {
+      "epoch": 0.52,
+      "grad_norm": 1.0037930011749268,
+      "learning_rate": 5.16e-05,
+      "loss": 0.45534143447875974,
+      "step": 130
+    },
+    {
+      "epoch": 0.56,
+      "grad_norm": 0.9430785179138184,
+      "learning_rate": 5.560000000000001e-05,
+      "loss": 0.45524797439575193,
+      "step": 140
+    },
+    {
+      "epoch": 0.6,
+      "grad_norm": 0.9689427614212036,
+      "learning_rate": 5.96e-05,
+      "loss": 0.47152209281921387,
+      "step": 150
+    },
+    {
+      "epoch": 0.64,
+      "grad_norm": 0.7584393620491028,
+      "learning_rate": 6.36e-05,
+      "loss": 0.4532940864562988,
+      "step": 160
+    },
+    {
+      "epoch": 0.68,
+      "grad_norm": 0.7581620216369629,
+      "learning_rate": 6.76e-05,
+      "loss": 0.48988704681396483,
+      "step": 170
+    },
+    {
+      "epoch": 0.72,
+      "grad_norm": 0.9882776141166687,
+      "learning_rate": 7.16e-05,
+      "loss": 0.46865572929382326,
+      "step": 180
+    },
+    {
+      "epoch": 0.76,
+      "grad_norm": 0.743236780166626,
+      "learning_rate": 7.560000000000001e-05,
+      "loss": 0.45577139854431153,
+      "step": 190
+    },
+    {
+      "epoch": 0.8,
+      "grad_norm": 0.6103836894035339,
+      "learning_rate": 7.960000000000001e-05,
+      "loss": 0.4559042453765869,
+      "step": 200
+    },
+    {
+      "epoch": 0.8,
+      "eval_loss": 0.485470175743103,
+      "eval_runtime": 17.4199,
+      "eval_samples_per_second": 22.962,
+      "eval_steps_per_second": 3.846,
+      "step": 200
+    },
+    {
+      "epoch": 0.84,
+      "grad_norm": 0.8245580792427063,
+      "learning_rate": 8.36e-05,
+      "loss": 0.45926451683044434,
+      "step": 210
+    },
+    {
+      "epoch": 0.88,
+      "grad_norm": 0.6920369267463684,
+      "learning_rate": 8.76e-05,
+      "loss": 0.4545453548431396,
+      "step": 220
+    },
+    {
+      "epoch": 0.92,
+      "grad_norm": 0.6936920881271362,
+      "learning_rate": 9.16e-05,
+      "loss": 0.47637343406677246,
+      "step": 230
+    },
+    {
+      "epoch": 0.96,
+      "grad_norm": 0.6694210767745972,
+      "learning_rate": 9.56e-05,
+      "loss": 0.43120541572570803,
+      "step": 240
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 0.583095133304596,
+      "learning_rate": 9.960000000000001e-05,
+      "loss": 0.4153712272644043,
+      "step": 250
+    },
+    {
+      "epoch": 1.04,
+      "grad_norm": 0.6926116943359375,
+      "learning_rate": 9.999605221019081e-05,
+      "loss": 0.44300012588500975,
+      "step": 260
+    },
+    {
+      "epoch": 1.08,
+      "grad_norm": 0.761324405670166,
+      "learning_rate": 9.998240632972073e-05,
+      "loss": 0.462084436416626,
+      "step": 270
+    },
+    {
+      "epoch": 1.12,
+      "grad_norm": 0.5191273093223572,
+      "learning_rate": 9.995901628010196e-05,
+      "loss": 0.39808471202850343,
+      "step": 280
+    },
+    {
+      "epoch": 1.16,
+      "grad_norm": 0.8463711738586426,
+      "learning_rate": 9.9925886621271e-05,
+      "loss": 0.423044490814209,
+      "step": 290
+    },
+    {
+      "epoch": 1.2,
+      "grad_norm": 0.8373249769210815,
+      "learning_rate": 9.98830238119205e-05,
+      "loss": 0.41622562408447267,
+      "step": 300
+    },
+    {
+      "epoch": 1.2,
+      "eval_loss": 0.4695434272289276,
+      "eval_runtime": 19.2419,
+      "eval_samples_per_second": 20.788,
+      "eval_steps_per_second": 3.482,
+      "step": 300
+    },
+    {
+      "epoch": 1.24,
+      "grad_norm": 0.6290304064750671,
+      "learning_rate": 9.983043620824005e-05,
+      "loss": 0.4166346549987793,
+      "step": 310
+    },
+    {
+      "epoch": 1.28,
+      "grad_norm": 0.6189863681793213,
+      "learning_rate": 9.97681340622872e-05,
+      "loss": 0.43734130859375,
+      "step": 320
+    },
+    {
+      "epoch": 1.32,
+      "grad_norm": 0.5579029321670532,
+      "learning_rate": 9.969612951998874e-05,
+      "loss": 0.3747305631637573,
+      "step": 330
+    },
+    {
+      "epoch": 1.3599999999999999,
+      "grad_norm": 1.1675549745559692,
+      "learning_rate": 9.961443661877289e-05,
+      "loss": 0.42578792572021484,
+      "step": 340
+    },
+    {
+      "epoch": 1.4,
+      "grad_norm": 0.6578675508499146,
+      "learning_rate": 9.952307128483256e-05,
+      "loss": 0.39537777900695803,
+      "step": 350
+    },
+    {
+      "epoch": 1.44,
+      "grad_norm": 0.8092941045761108,
+      "learning_rate": 9.942205133002068e-05,
+      "loss": 0.4084367275238037,
+      "step": 360
+    },
+    {
+      "epoch": 1.48,
+      "grad_norm": 0.6226063370704651,
+      "learning_rate": 9.931139644837754e-05,
+      "loss": 0.3781426906585693,
+      "step": 370
+    },
+    {
+      "epoch": 1.52,
+      "grad_norm": 0.7148721218109131,
+      "learning_rate": 9.919112821229163e-05,
+      "loss": 0.3952002048492432,
+      "step": 380
+    },
+    {
+      "epoch": 1.56,
+      "grad_norm": 0.5743547081947327,
+      "learning_rate": 9.906127006829384e-05,
+      "loss": 0.4087832927703857,
+      "step": 390
+    },
+    {
+      "epoch": 1.6,
+      "grad_norm": 0.6315461993217468,
+      "learning_rate": 9.892184733248666e-05,
+      "loss": 0.3861570119857788,
+      "step": 400
+    },
+    {
+      "epoch": 1.6,
+      "eval_loss": 0.45406103134155273,
+      "eval_runtime": 19.7154,
+      "eval_samples_per_second": 20.289,
+      "eval_steps_per_second": 3.398,
+      "step": 400
+    },
+    {
+      "epoch": 1.6400000000000001,
+      "grad_norm": 0.6243694424629211,
+      "learning_rate": 9.877288718560866e-05,
+      "loss": 0.39033331871032717,
+      "step": 410
+    },
+    {
+      "epoch": 1.6800000000000002,
+      "grad_norm": 0.6677294969558716,
+      "learning_rate": 9.861441866773564e-05,
+      "loss": 0.43663845062255857,
+      "step": 420
+    },
+    {
+      "epoch": 1.72,
+      "grad_norm": 0.6460554599761963,
+      "learning_rate": 9.844647267261916e-05,
+      "loss": 0.43364706039428713,
+      "step": 430
+    },
+    {
+      "epoch": 1.76,
+      "grad_norm": 0.570160984992981,
+      "learning_rate": 9.82690819416637e-05,
+      "loss": 0.409498929977417,
+      "step": 440
+    },
+    {
+      "epoch": 1.8,
+      "grad_norm": 0.5696760416030884,
+      "learning_rate": 9.808228105754376e-05,
+      "loss": 0.4264820098876953,
+      "step": 450
+    },
+    {
+      "epoch": 1.8399999999999999,
+      "grad_norm": 0.583260715007782,
+      "learning_rate": 9.788610643746184e-05,
+      "loss": 0.417040491104126,
+      "step": 460
+    },
+    {
+      "epoch": 1.88,
+      "grad_norm": 0.6025984287261963,
+      "learning_rate": 9.76805963260488e-05,
+      "loss": 0.3749807357788086,
+      "step": 470
+    },
+    {
+      "epoch": 1.92,
+      "grad_norm": 0.5953373312950134,
+      "learning_rate": 9.746579078790807e-05,
+      "loss": 0.4022481918334961,
+      "step": 480
+    },
+    {
+      "epoch": 1.96,
+      "grad_norm": 0.4357820153236389,
+      "learning_rate": 9.724173169980491e-05,
+      "loss": 0.38319835662841795,
+      "step": 490
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.5152677297592163,
+      "learning_rate": 9.700846274250251e-05,
+      "loss": 0.4122174263000488,
+      "step": 500
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 0.44415727257728577,
+      "eval_runtime": 18.9015,
+      "eval_samples_per_second": 21.162,
+      "eval_steps_per_second": 3.545,
+      "step": 500
+    },
+    {
+      "epoch": 2.04,
+      "grad_norm": 0.38848409056663513,
+      "learning_rate": 9.676602939224629e-05,
+      "loss": 0.3524669408798218,
+      "step": 510
+    },
+    {
+      "epoch": 2.08,
+      "grad_norm": 0.5285012125968933,
+      "learning_rate": 9.651447891189825e-05,
+      "loss": 0.3717231273651123,
+      "step": 520
+    },
+    {
+      "epoch": 2.12,
+      "grad_norm": 0.6452465653419495,
+      "learning_rate": 9.62538603417229e-05,
+      "loss": 0.40065832138061525,
+      "step": 530
+    },
+    {
+      "epoch": 2.16,
+      "grad_norm": 0.48196467757225037,
+      "learning_rate": 9.598422448982696e-05,
+      "loss": 0.33635973930358887,
+      "step": 540
+    },
+    {
+      "epoch": 2.2,
+      "grad_norm": 0.563376247882843,
+      "learning_rate": 9.570562392225396e-05,
+      "loss": 0.3708656787872314,
+      "step": 550
+    },
+    {
+      "epoch": 2.24,
+      "grad_norm": 0.6459429860115051,
+      "learning_rate": 9.541811295273656e-05,
+      "loss": 0.35284056663513186,
+      "step": 560
+    },
+    {
+      "epoch": 2.2800000000000002,
+      "grad_norm": 0.5247339606285095,
+      "learning_rate": 9.512174763210797e-05,
+      "loss": 0.3429510831832886,
+      "step": 570
+    },
+    {
+      "epoch": 2.32,
+      "grad_norm": 0.5456256866455078,
+      "learning_rate": 9.481658573737465e-05,
+      "loss": 0.36770102977752683,
+      "step": 580
+    },
+    {
+      "epoch": 2.36,
+      "grad_norm": 0.5435087084770203,
+      "learning_rate": 9.450268676045262e-05,
+      "loss": 0.3684037208557129,
+      "step": 590
+    },
+    {
+      "epoch": 2.4,
+      "grad_norm": 0.5584478974342346,
+      "learning_rate": 9.418011189656941e-05,
+      "loss": 0.3221792697906494,
+      "step": 600
+    },
+    {
+      "epoch": 2.4,
+      "eval_loss": 0.44748273491859436,
+      "eval_runtime": 18.8521,
+      "eval_samples_per_second": 21.218,
+      "eval_steps_per_second": 3.554,
+      "step": 600
+    },
+    {
+      "epoch": 2.44,
+      "grad_norm": 0.7217129468917847,
+      "learning_rate": 9.384892403233384e-05,
+      "loss": 0.40174164772033694,
+      "step": 610
+    },
+    {
+      "epoch": 2.48,
+      "grad_norm": 0.5068971514701843,
+      "learning_rate": 9.35091877334763e-05,
+      "loss": 0.3701002836227417,
+      "step": 620
+    },
+    {
+      "epoch": 2.52,
+      "grad_norm": 0.4331487715244293,
+      "learning_rate": 9.316096923226135e-05,
+      "loss": 0.3759175777435303,
+      "step": 630
+    },
+    {
+      "epoch": 2.56,
+      "grad_norm": 0.5161293148994446,
+      "learning_rate": 9.28043364145758e-05,
+      "loss": 0.3581662178039551,
+      "step": 640
+    },
+    {
+      "epoch": 2.6,
+      "grad_norm": 0.709299623966217,
+      "learning_rate": 9.24393588066941e-05,
+      "loss": 0.35065665245056155,
+      "step": 650
+    },
+    {
+      "epoch": 2.64,
+      "grad_norm": 0.6004891991615295,
+      "learning_rate": 9.206610756172402e-05,
+      "loss": 0.36879355907440187,
+      "step": 660
+    },
+    {
+      "epoch": 2.68,
+      "grad_norm": 0.4662474989891052,
+      "learning_rate": 9.168465544573536e-05,
+      "loss": 0.3592060565948486,
+      "step": 670
+    },
+    {
+      "epoch": 2.7199999999999998,
+      "grad_norm": 0.5826489329338074,
+      "learning_rate": 9.129507682357394e-05,
+      "loss": 0.36156315803527833,
+      "step": 680
+    },
+    {
+      "epoch": 2.76,
+      "grad_norm": 0.48988744616508484,
+      "learning_rate": 9.089744764436403e-05,
+      "loss": 0.34445748329162595,
+      "step": 690
+    },
+    {
+      "epoch": 2.8,
+      "grad_norm": 0.4443361163139343,
+      "learning_rate": 9.049184542670199e-05,
+      "loss": 0.3526463985443115,
+      "step": 700
+    },
+    {
+      "epoch": 2.8,
+      "eval_loss": 0.44259119033813477,
+      "eval_runtime": 16.8228,
+      "eval_samples_per_second": 23.777,
+      "eval_steps_per_second": 3.983,
+      "step": 700
+    },
+    {
+      "epoch": 2.84,
+      "grad_norm": 0.5471161007881165,
+      "learning_rate": 9.007834924354383e-05,
+      "loss": 0.3458081245422363,
+      "step": 710
+    },
+    {
+      "epoch": 2.88,
+      "grad_norm": 0.5264748930931091,
+      "learning_rate": 8.965703970678974e-05,
+      "loss": 0.3651163101196289,
+      "step": 720
+    },
+    {
+      "epoch": 2.92,
+      "grad_norm": 0.48987507820129395,
+      "learning_rate": 8.922799895156867e-05,
+      "loss": 0.3218229293823242,
+      "step": 730
+    },
+    {
+      "epoch": 2.96,
+      "grad_norm": 0.5640589594841003,
+      "learning_rate": 8.879131062022598e-05,
+      "loss": 0.3561582088470459,
+      "step": 740
+    },
+    {
+      "epoch": 3.0,
+      "grad_norm": 0.7934619784355164,
+      "learning_rate": 8.834705984601708e-05,
+      "loss": 0.36128854751586914,
+      "step": 750
+    },
+    {
+      "epoch": 3.04,
+      "grad_norm": 1.0869489908218384,
+      "learning_rate": 8.789533323651066e-05,
+      "loss": 0.31422438621521,
+      "step": 760
+    },
+    {
+      "epoch": 3.08,
+      "grad_norm": 0.4695897102355957,
+      "learning_rate": 8.74362188567043e-05,
+      "loss": 0.29355826377868655,
+      "step": 770
+    },
+    {
+      "epoch": 3.12,
+      "grad_norm": 0.5532680153846741,
+      "learning_rate": 8.696980621185602e-05,
+      "loss": 0.3185117721557617,
+      "step": 780
+    },
+    {
+      "epoch": 3.16,
+      "grad_norm": 0.5760806202888489,
+      "learning_rate": 8.649618623003508e-05,
+      "loss": 0.28971233367919924,
+      "step": 790
+    },
+    {
+      "epoch": 3.2,
+      "grad_norm": 0.5517900586128235,
+      "learning_rate": 8.601545124439535e-05,
+      "loss": 0.3055370092391968,
+      "step": 800
+    },
+    {
+      "epoch": 3.2,
+      "eval_loss": 0.4529191255569458,
+      "eval_runtime": 18.5382,
+      "eval_samples_per_second": 21.577,
+      "eval_steps_per_second": 3.614,
+      "step": 800
+    },
+    {
+      "epoch": 3.24,
+      "grad_norm": 0.5356678366661072,
+      "learning_rate": 8.552769497517482e-05,
+      "loss": 0.28035550117492675,
+      "step": 810
+    },
+    {
+      "epoch": 3.2800000000000002,
+      "grad_norm": 0.5985352993011475,
+      "learning_rate": 8.503301251142459e-05,
+      "loss": 0.3199602603912354,
+      "step": 820
+    },
+    {
+      "epoch": 3.32,
+      "grad_norm": 0.5187913179397583,
+      "learning_rate": 8.453150029247114e-05,
+      "loss": 0.29444499015808107,
+      "step": 830
+    },
+    {
+      "epoch": 3.36,
+      "grad_norm": 0.5703292489051819,
+      "learning_rate": 8.402325608911526e-05,
+      "loss": 0.30467259883880615,
+      "step": 840
+    },
+    {
+      "epoch": 3.4,
+      "grad_norm": 0.9323157072067261,
+      "learning_rate": 8.350837898457143e-05,
+      "loss": 0.3117033004760742,
+      "step": 850
+    },
+    {
+      "epoch": 3.44,
+      "grad_norm": 0.628546953201294,
+      "learning_rate": 8.298696935515132e-05,
+      "loss": 0.34261503219604494,
+      "step": 860
+    },
+    {
+      "epoch": 3.48,
+      "grad_norm": 0.5379561185836792,
+      "learning_rate": 8.245912885069531e-05,
+      "loss": 0.3159458637237549,
+      "step": 870
+    },
+    {
+      "epoch": 3.52,
+      "grad_norm": 0.6575730443000793,
+      "learning_rate": 8.192496037475562e-05,
+      "loss": 0.2982481002807617,
+      "step": 880
+    },
+    {
+      "epoch": 3.56,
+      "grad_norm": 0.5830497145652771,
+      "learning_rate": 8.138456806453503e-05,
+      "loss": 0.3232215404510498,
+      "step": 890
+    },
+    {
+      "epoch": 3.6,
+      "grad_norm": 0.5474710464477539,
+      "learning_rate": 8.083805727058513e-05,
+      "loss": 0.3305091381072998,
+      "step": 900
+    },
+    {
+      "epoch": 3.6,
+      "eval_loss": 0.44760578870773315,
+      "eval_runtime": 19.5159,
+      "eval_samples_per_second": 20.496,
+      "eval_steps_per_second": 3.433,
+      "step": 900
+    },
+    {
+      "epoch": 3.64,
+      "grad_norm": 0.5096336007118225,
+      "learning_rate": 8.028553453626808e-05,
+      "loss": 0.35752732753753663,
+      "step": 910
+    },
+    {
+      "epoch": 3.68,
+      "grad_norm": 0.5023341774940491,
+      "learning_rate": 7.972710757698567e-05,
+      "loss": 0.3292932271957397,
+      "step": 920
+    },
+    {
+      "epoch": 3.7199999999999998,
+      "grad_norm": 0.5277951955795288,
+      "learning_rate": 7.916288525918007e-05,
+      "loss": 0.28986682891845705,
+      "step": 930
+    },
+    {
+      "epoch": 3.76,
+      "grad_norm": 0.600412905216217,
+      "learning_rate": 7.859297757911013e-05,
+      "loss": 0.3027395725250244,
+      "step": 940
+    },
+    {
+      "epoch": 3.8,
+      "grad_norm": 0.6396210193634033,
+      "learning_rate": 7.801749564140724e-05,
+      "loss": 0.3238774061203003,
+      "step": 950
+    },
+    {
+      "epoch": 3.84,
+      "grad_norm": 0.628635585308075,
+      "learning_rate": 7.743655163741543e-05,
+      "loss": 0.34537086486816404,
+      "step": 960
+    },
+    {
+      "epoch": 3.88,
+      "grad_norm": 0.49822649359703064,
+      "learning_rate": 7.685025882331936e-05,
+      "loss": 0.3292637825012207,
+      "step": 970
+    },
+    {
+      "epoch": 3.92,
+      "grad_norm": 0.5356727242469788,
+      "learning_rate": 7.62587314980648e-05,
+      "loss": 0.32722015380859376,
+      "step": 980
+    },
+    {
+      "epoch": 3.96,
+      "grad_norm": 0.6211317777633667,
+      "learning_rate": 7.566208498107585e-05,
+      "loss": 0.29880056381225584,
+      "step": 990
+    },
+    {
+      "epoch": 4.0,
+      "grad_norm": 0.5336779356002808,
+      "learning_rate": 7.506043558977321e-05,
+      "loss": 0.2978524684906006,
+      "step": 1000
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 0.44613513350486755,
+      "eval_runtime": 19.2382,
+      "eval_samples_per_second": 20.792,
+      "eval_steps_per_second": 3.483,
+      "step": 1000
+    },
+    {
+      "epoch": 4.04,
+      "grad_norm": 0.6681120991706848,
+      "learning_rate": 7.445390061689782e-05,
+      "loss": 0.27530927658081056,
+      "step": 1010
+    },
+    {
+      "epoch": 4.08,
+      "grad_norm": 0.6299528479576111,
+      "learning_rate": 7.38425983076444e-05,
+      "loss": 0.2517704486846924,
+      "step": 1020
+    },
+    {
+      "epoch": 4.12,
+      "grad_norm": 0.5211061239242554,
+      "learning_rate": 7.32266478366094e-05,
+      "loss": 0.28200175762176516,
+      "step": 1030
+    },
+    {
+      "epoch": 4.16,
+      "grad_norm": 0.5778363347053528,
+      "learning_rate": 7.260616928455754e-05,
+      "loss": 0.2569046258926392,
+      "step": 1040
+    },
+    {
+      "epoch": 4.2,
+      "grad_norm": 0.6715266108512878,
+      "learning_rate": 7.1981283615012e-05,
+      "loss": 0.2665576696395874,
+      "step": 1050
+    },
+    {
+      "epoch": 4.24,
+      "grad_norm": 0.6580007672309875,
+      "learning_rate": 7.135211265067216e-05,
+      "loss": 0.2635650634765625,
+      "step": 1060
+    },
+    {
+      "epoch": 4.28,
+      "grad_norm": 0.6889304518699646,
+      "learning_rate": 7.071877904966423e-05,
+      "loss": 0.26842334270477297,
+      "step": 1070
+    },
+    {
+      "epoch": 4.32,
+      "grad_norm": 0.5896309018135071,
+      "learning_rate": 7.00814062816285e-05,
+      "loss": 0.2633937358856201,
+      "step": 1080
+    },
+    {
+      "epoch": 4.36,
+      "grad_norm": 0.6062363386154175,
+      "learning_rate": 6.944011860364905e-05,
+      "loss": 0.2895397186279297,
+      "step": 1090
+    },
+    {
+      "epoch": 4.4,
+      "grad_norm": 0.6124110817909241,
+      "learning_rate": 6.879504103602935e-05,
+      "loss": 0.27405414581298826,
+      "step": 1100
+    },
+    {
+      "epoch": 4.4,
+      "eval_loss": 0.46795058250427246,
+      "eval_runtime": 17.2143,
+      "eval_samples_per_second": 23.237,
+      "eval_steps_per_second": 3.892,
+      "step": 1100
+    },
+    {
+      "epoch": 4.44,
+      "grad_norm": 0.8100364208221436,
+      "learning_rate": 6.814629933791931e-05,
+      "loss": 0.2581511974334717,
+      "step": 1110
+    },
+    {
+      "epoch": 4.48,
+      "grad_norm": 0.6187950372695923,
+      "learning_rate": 6.749401998279846e-05,
+      "loss": 0.2689012050628662,
+      "step": 1120
+    },
+    {
+      "epoch": 4.52,
+      "grad_norm": 0.6595885157585144,
+      "learning_rate": 6.683833013381941e-05,
+      "loss": 0.27230424880981446,
+      "step": 1130
+    },
+    {
+      "epoch": 4.5600000000000005,
+      "grad_norm": 0.6320788860321045,
+      "learning_rate": 6.617935761901748e-05,
+      "loss": 0.2903036594390869,
+      "step": 1140
+    },
+    {
+      "epoch": 4.6,
+      "grad_norm": 0.6367589831352234,
+      "learning_rate": 6.551723090639007e-05,
+      "loss": 0.2551115989685059,
+      "step": 1150
+    },
+    {
+      "epoch": 4.64,
+      "grad_norm": 0.5754795670509338,
+      "learning_rate": 6.485207907885175e-05,
+      "loss": 0.2783109188079834,
+      "step": 1160
+    },
+    {
+      "epoch": 4.68,
+      "grad_norm": 0.6343188881874084,
+      "learning_rate": 6.418403180906922e-05,
+      "loss": 0.29131503105163575,
+      "step": 1170
+    },
+    {
+      "epoch": 4.72,
+      "grad_norm": 0.6726956963539124,
+      "learning_rate": 6.351321933418139e-05,
+      "loss": 0.2730400085449219,
+      "step": 1180
+    },
+    {
+      "epoch": 4.76,
+      "grad_norm": 0.5498913526535034,
+      "learning_rate": 6.283977243040939e-05,
+      "loss": 0.2572148323059082,
+      "step": 1190
+    },
+    {
+      "epoch": 4.8,
+      "grad_norm": 0.6083167195320129,
+      "learning_rate": 6.216382238756146e-05,
+      "loss": 0.27444655895233155,
+      "step": 1200
+    },
+    {
+      "epoch": 4.8,
+      "eval_loss": 0.466619610786438,
+      "eval_runtime": 19.9505,
+      "eval_samples_per_second": 20.05,
+      "eval_steps_per_second": 3.358,
+      "step": 1200
+    },
+    {
+      "epoch": 4.84,
+      "grad_norm": 0.5861450433731079,
+      "learning_rate": 6.148550098343778e-05,
+      "loss": 0.27054529190063475,
+      "step": 1210
+    },
+    {
+      "epoch": 4.88,
+      "grad_norm": 0.7090939879417419,
+      "learning_rate": 6.080494045814011e-05,
+      "loss": 0.26785056591033934,
+      "step": 1220
+    },
+    {
+      "epoch": 4.92,
+      "grad_norm": 0.5825073719024658,
+      "learning_rate": 6.0122273488291304e-05,
+      "loss": 0.26335647106170657,
+      "step": 1230
+    },
+    {
+      "epoch": 4.96,
+      "grad_norm": 0.5506169199943542,
+      "learning_rate": 5.943763316116977e-05,
+      "loss": 0.2614041090011597,
+      "step": 1240
+    },
+    {
+      "epoch": 5.0,
+      "grad_norm": 0.6169804930686951,
+      "learning_rate": 5.875115294876381e-05,
+      "loss": 0.24768717288970948,
+      "step": 1250
+    },
+    {
+      "epoch": 5.04,
+      "grad_norm": 0.8200834393501282,
+      "learning_rate": 5.806296668175104e-05,
+      "loss": 0.21707432270050048,
+      "step": 1260
+    },
+    {
+      "epoch": 5.08,
+      "grad_norm": 1.5680038928985596,
+      "learning_rate": 5.737320852340775e-05,
+      "loss": 0.2139519214630127,
+      "step": 1270
+    },
+    {
+      "epoch": 5.12,
+      "grad_norm": 0.6845637559890747,
+      "learning_rate": 5.668201294345363e-05,
+      "loss": 0.20998594760894776,
+      "step": 1280
+    },
+    {
+      "epoch": 5.16,
+      "grad_norm": 0.8293268084526062,
+      "learning_rate": 5.598951469183649e-05,
+      "loss": 0.23306002616882324,
+      "step": 1290
+    },
+    {
+      "epoch": 5.2,
+      "grad_norm": 0.7228839993476868,
+      "learning_rate": 5.52958487724626e-05,
+      "loss": 0.2262401580810547,
+      "step": 1300
+    },
+    {
+      "epoch": 5.2,
+      "eval_loss": 0.49972543120384216,
+      "eval_runtime": 18.926,
+      "eval_samples_per_second": 21.135,
+      "eval_steps_per_second": 3.54,
+      "step": 1300
+    },
+    {
+      "epoch": 5.24,
+      "grad_norm": 0.6243706345558167,
+      "learning_rate": 5.4601150416877367e-05,
+      "loss": 0.21100988388061523,
+      "step": 1310
+    },
+    {
+      "epoch": 5.28,
+      "grad_norm": 1.0553343296051025,
+      "learning_rate": 5.390555505790168e-05,
+      "loss": 0.23542592525482178,
+      "step": 1320
+    },
+    {
+      "epoch": 5.32,
+      "grad_norm": 0.6127402186393738,
+      "learning_rate": 5.3209198303229027e-05,
+      "loss": 0.2095633029937744,
+      "step": 1330
+    },
+    {
+      "epoch": 5.36,
+      "grad_norm": 0.7463288903236389,
+      "learning_rate": 5.2512215908988484e-05,
+      "loss": 0.21693904399871827,
+      "step": 1340
+    },
+    {
+      "epoch": 5.4,
+      "grad_norm": 0.8020226955413818,
+      "learning_rate": 5.1814743753278795e-05,
+      "loss": 0.2076347827911377,
+      "step": 1350
+    },
+    {
+      "epoch": 5.44,
+      "grad_norm": 0.6652446389198303,
+      "learning_rate": 5.111691780967869e-05,
+      "loss": 0.22539749145507812,
+      "step": 1360
+    },
+    {
+      "epoch": 5.48,
+      "grad_norm": 0.6378898620605469,
+      "learning_rate": 5.041887412073854e-05,
+      "loss": 0.2077547550201416,
+      "step": 1370
+    },
+    {
+      "epoch": 5.52,
+      "grad_norm": 0.7381134033203125,
+      "learning_rate": 4.97207487714586e-05,
+      "loss": 0.21558783054351807,
+      "step": 1380
+    },
+    {
+      "epoch": 5.5600000000000005,
+      "grad_norm": 0.6613102555274963,
+      "learning_rate": 4.9022677862758945e-05,
+      "loss": 0.21069679260253907,
+      "step": 1390
+    },
+    {
+      "epoch": 5.6,
+      "grad_norm": 0.7527480721473694,
+      "learning_rate": 4.832479748494643e-05,
+      "loss": 0.21843309402465821,
+      "step": 1400
+    },
+    {
+      "epoch": 5.6,
+      "eval_loss": 0.49576279520988464,
+      "eval_runtime": 18.3368,
+      "eval_samples_per_second": 21.814,
+      "eval_steps_per_second": 3.654,
+      "step": 1400
+    },
+    {
+      "epoch": 5.64,
+      "grad_norm": 0.5983570218086243,
+      "learning_rate": 4.7627243691183453e-05,
+      "loss": 0.22310276031494142,
+      "step": 1410
+    },
+    {
+      "epoch": 5.68,
+      "grad_norm": 0.6202098727226257,
+      "learning_rate": 4.693015247096423e-05,
+      "loss": 0.22056117057800292,
+      "step": 1420
+    },
+    {
+      "epoch": 5.72,
+      "grad_norm": 0.7730934023857117,
+      "learning_rate": 4.623365972360337e-05,
+      "loss": 0.2241537094116211,
+      "step": 1430
+    },
+    {
+      "epoch": 5.76,
+      "grad_norm": 0.6262892484664917,
+      "learning_rate": 4.553790123174197e-05,
+      "loss": 0.21514451503753662,
+      "step": 1440
+    },
+    {
+      "epoch": 5.8,
+      "grad_norm": 0.646507203578949,
+      "learning_rate": 4.484301263487665e-05,
+      "loss": 0.21031346321105956,
+      "step": 1450
+    },
+    {
+      "epoch": 5.84,
+      "grad_norm": 0.8227706551551819,
+      "learning_rate": 4.414912940291613e-05,
+      "loss": 0.2312474489212036,
+      "step": 1460
+    },
+    {
+      "epoch": 5.88,
+      "grad_norm": 0.6932390332221985,
+      "learning_rate": 4.345638680977139e-05,
+      "loss": 0.22380952835083007,
+      "step": 1470
+    },
+    {
+      "epoch": 5.92,
+      "grad_norm": 0.7352316379547119,
+      "learning_rate": 4.276491990698355e-05,
+      "loss": 0.22706894874572753,
+      "step": 1480
+    },
+    {
+      "epoch": 5.96,
+      "grad_norm": 0.6953718066215515,
+      "learning_rate": 4.2074863497395377e-05,
+      "loss": 0.2103546142578125,
+      "step": 1490
+    },
+    {
+      "epoch": 6.0,
+      "grad_norm": 0.661618709564209,
+      "learning_rate": 4.1386352108871174e-05,
+      "loss": 0.2276217222213745,
+      "step": 1500
+    },
+    {
+      "epoch": 6.0,
+      "eval_loss": 0.4966464042663574,
+      "eval_runtime": 17.2948,
+      "eval_samples_per_second": 23.128,
+      "eval_steps_per_second": 3.874,
+      "step": 1500
+    },
+    {
+      "epoch": 6.04,
+      "grad_norm": 0.8837434649467468,
+      "learning_rate": 4.069951996807034e-05,
+      "loss": 0.16540236473083497,
+      "step": 1510
+    },
+    {
+      "epoch": 6.08,
+      "grad_norm": 1.3857215642929077,
+      "learning_rate": 4.001450097427966e-05,
+      "loss": 0.1638352394104004,
+      "step": 1520
+    },
+    {
+      "epoch": 6.12,
+      "grad_norm": 0.8306711912155151,
+      "learning_rate": 3.9331428673309204e-05,
+      "loss": 0.1719011664390564,
+      "step": 1530
+    },
+    {
+      "epoch": 6.16,
+      "grad_norm": 0.8509021997451782,
+      "learning_rate": 3.865043623145751e-05,
+      "loss": 0.1651092290878296,
+      "step": 1540
+    },
+    {
+      "epoch": 6.2,
+      "grad_norm": 0.7507994174957275,
+      "learning_rate": 3.797165640955041e-05,
+      "loss": 0.1746900796890259,
+      "step": 1550
+    },
+    {
+      "epoch": 6.24,
+      "grad_norm": 0.740626335144043,
+      "learning_rate": 3.729522153705916e-05,
+      "loss": 0.16637682914733887,
+      "step": 1560
+    },
+    {
+      "epoch": 6.28,
+      "grad_norm": 0.6479809880256653,
+      "learning_rate": 3.662126348630237e-05,
+      "loss": 0.1709848165512085,
+      "step": 1570
+    },
+    {
+      "epoch": 6.32,
+      "grad_norm": 0.6932395100593567,
+      "learning_rate": 3.594991364673745e-05,
+      "loss": 0.18107957839965821,
+      "step": 1580
+    },
+    {
+      "epoch": 6.36,
+      "grad_norm": 0.8027141690254211,
+      "learning_rate": 3.528130289934583e-05,
+      "loss": 0.16225044727325438,
+      "step": 1590
+    },
+    {
+      "epoch": 6.4,
+      "grad_norm": 0.5781376957893372,
+      "learning_rate": 3.461556159111748e-05,
+      "loss": 0.17544152736663818,
+      "step": 1600
+    },
+    {
+      "epoch": 6.4,
+      "eval_loss": 0.5342507362365723,
+      "eval_runtime": 19.471,
+      "eval_samples_per_second": 20.543,
+      "eval_steps_per_second": 3.441,
+      "step": 1600
+    },
+    {
+      "epoch": 6.44,
+      "grad_norm": 0.7642867565155029,
+      "learning_rate": 3.3952819509639534e-05,
+      "loss": 0.17091144323349,
+      "step": 1610
+    },
+    {
+      "epoch": 6.48,
+      "grad_norm": 0.7651257514953613,
+      "learning_rate": 3.329320585779393e-05,
+      "loss": 0.17765278816223146,
+      "step": 1620
+    },
+    {
+      "epoch": 6.52,
+      "grad_norm": 0.6956056356430054,
+      "learning_rate": 3.263684922856905e-05,
+      "loss": 0.16475566625595092,
+      "step": 1630
+    },
+    {
+      "epoch": 6.5600000000000005,
+      "grad_norm": 0.7344402074813843,
+      "learning_rate": 3.1983877579990274e-05,
+      "loss": 0.172060227394104,
+      "step": 1640
+    },
+    {
+      "epoch": 6.6,
+      "grad_norm": 0.7196578979492188,
+      "learning_rate": 3.1334418210174263e-05,
+      "loss": 0.16673840284347535,
+      "step": 1650
+    },
+    {
+      "epoch": 6.64,
+      "grad_norm": 0.7540257573127747,
+      "learning_rate": 3.0688597732512e-05,
+      "loss": 0.17414634227752684,
+      "step": 1660
+    },
+    {
+      "epoch": 6.68,
+      "grad_norm": 0.5103999972343445,
+      "learning_rate": 3.0046542050985237e-05,
+      "loss": 0.1620783567428589,
+      "step": 1670
+    },
+    {
+      "epoch": 6.72,
+      "grad_norm": 0.8846920132637024,
+      "learning_rate": 2.940837633562127e-05,
+      "loss": 0.17428462505340575,
+      "step": 1680
+    },
+    {
+      "epoch": 6.76,
+      "grad_norm": 0.8017328381538391,
+      "learning_rate": 2.877422499809072e-05,
+      "loss": 0.19050977230072022,
+      "step": 1690
+    },
+    {
+      "epoch": 6.8,
+      "grad_norm": 0.8515416383743286,
+      "learning_rate": 2.8144211667453368e-05,
+      "loss": 0.16926174163818358,
+      "step": 1700
+    },
+    {
+      "epoch": 6.8,
+      "eval_loss": 0.5441356301307678,
+      "eval_runtime": 17.5836,
+      "eval_samples_per_second": 22.749,
+      "eval_steps_per_second": 3.81,
+      "step": 1700
+    },
+    {
+      "epoch": 6.84,
+      "grad_norm": 0.7547643184661865,
+      "learning_rate": 2.75184591660563e-05,
+      "loss": 0.1793771743774414,
+      "step": 1710
+    },
+    {
+      "epoch": 6.88,
+      "grad_norm": 0.7164461016654968,
+      "learning_rate": 2.6897089485589583e-05,
+      "loss": 0.1647491931915283,
+      "step": 1720
+    },
+    {
+      "epoch": 6.92,
+      "grad_norm": 1.1592035293579102,
+      "learning_rate": 2.6280223763303546e-05,
+      "loss": 0.17397019863128663,
+      "step": 1730
+    },
+    {
+      "epoch": 6.96,
+      "grad_norm": 0.9889470934867859,
+      "learning_rate": 2.5667982258393014e-05,
+      "loss": 0.17107686996459961,
+      "step": 1740
+    },
+    {
+      "epoch": 7.0,
+      "grad_norm": 0.7448652982711792,
+      "learning_rate": 2.506048432855247e-05,
+      "loss": 0.1730511426925659,
+      "step": 1750
+    },
+    {
+      "epoch": 7.04,
+      "grad_norm": 0.6695497632026672,
+      "learning_rate": 2.4457848406707013e-05,
+      "loss": 0.13950222730636597,
+      "step": 1760
+    },
+    {
+      "epoch": 7.08,
+      "grad_norm": 0.7200675010681152,
+      "learning_rate": 2.3860191977923672e-05,
+      "loss": 0.1326605796813965,
+      "step": 1770
+    },
+    {
+      "epoch": 7.12,
+      "grad_norm": 0.6615055799484253,
+      "learning_rate": 2.326763155650744e-05,
+      "loss": 0.1265331983566284,
+      "step": 1780
+    },
+    {
+      "epoch": 7.16,
+      "grad_norm": 0.8998573422431946,
+      "learning_rate": 2.2680282663286552e-05,
+      "loss": 0.12731509208679198,
+      "step": 1790
+    },
+    {
+      "epoch": 7.2,
+      "grad_norm": 0.808588981628418,
+      "learning_rate": 2.209825980309151e-05,
+      "loss": 0.13114826679229735,
+      "step": 1800
+    },
+    {
+      "epoch": 7.2,
+      "eval_loss": 0.5847110748291016,
+      "eval_runtime": 18.9921,
+      "eval_samples_per_second": 21.061,
+      "eval_steps_per_second": 3.528,
+      "step": 1800
+    },
+    {
+      "epoch": 7.24,
+      "grad_norm": 0.951817512512207,
+      "learning_rate": 2.152167644243213e-05,
+      "loss": 0.12906957864761354,
+      "step": 1810
+    },
+    {
+      "epoch": 7.28,
+      "grad_norm": 0.8695458173751831,
+      "learning_rate": 2.095064498737701e-05,
+      "loss": 0.133590030670166,
+      "step": 1820
+    },
+    {
+      "epoch": 7.32,
+      "grad_norm": 0.7357354760169983,
+      "learning_rate": 2.0385276761639765e-05,
+      "loss": 0.13653848171234131,
+      "step": 1830
+    },
+    {
+      "epoch": 7.36,
+      "grad_norm": 0.7873698472976685,
+      "learning_rate": 1.9825681984876172e-05,
+      "loss": 0.12472724914550781,
+      "step": 1840
+    },
+    {
+      "epoch": 7.4,
+      "grad_norm": 0.873921811580658,
+      "learning_rate": 1.9271969751196776e-05,
+      "loss": 0.13255125284194946,
+      "step": 1850
+    },
+    {
+      "epoch": 7.44,
+      "grad_norm": 0.7591536045074463,
+      "learning_rate": 1.8724248007898647e-05,
+      "loss": 0.13693161010742189,
+      "step": 1860
+    },
+    {
+      "epoch": 7.48,
+      "grad_norm": 1.0509488582611084,
+      "learning_rate": 1.8182623534420907e-05,
+      "loss": 0.13425672054290771,
+      "step": 1870
+    },
+    {
+      "epoch": 7.52,
+      "grad_norm": 0.8472399711608887,
+      "learning_rate": 1.76472019215278e-05,
+      "loss": 0.13668575286865234,
+      "step": 1880
+    },
+    {
+      "epoch": 7.5600000000000005,
+      "grad_norm": 0.911901593208313,
+      "learning_rate": 1.7118087550723633e-05,
+      "loss": 0.1317702889442444,
+      "step": 1890
+    },
+    {
+      "epoch": 7.6,
+      "grad_norm": 0.9731144309043884,
+      "learning_rate": 1.659538357390341e-05,
+      "loss": 0.14458621740341188,
+      "step": 1900
+    },
+    {
+      "epoch": 7.6,
+      "eval_loss": 0.5830516219139099,
+      "eval_runtime": 18.7747,
+      "eval_samples_per_second": 21.305,
+      "eval_steps_per_second": 3.569,
+      "step": 1900
+    },
+    {
+      "epoch": 7.64,
+      "grad_norm": 0.5515460968017578,
+      "learning_rate": 1.60791918932431e-05,
+      "loss": 0.13126691579818725,
+      "step": 1910
+    },
+    {
+      "epoch": 7.68,
+      "grad_norm": 0.7286776304244995,
+      "learning_rate": 1.556961314133359e-05,
+      "loss": 0.12600460052490234,
+      "step": 1920
+    },
+    {
+      "epoch": 7.72,
+      "grad_norm": 0.95229572057724,
+      "learning_rate": 1.5066746661562253e-05,
+      "loss": 0.12453792095184327,
+      "step": 1930
+    },
+    {
+      "epoch": 7.76,
+      "grad_norm": 0.7712796330451965,
+      "learning_rate": 1.4570690488745687e-05,
+      "loss": 0.14839541912078857,
+      "step": 1940
+    },
+    {
+      "epoch": 7.8,
+      "grad_norm": 0.8011840581893921,
+      "learning_rate": 1.4081541330017705e-05,
+      "loss": 0.1321096420288086,
+      "step": 1950
+    },
+    {
+      "epoch": 7.84,
+      "grad_norm": 0.936607301235199,
+      "learning_rate": 1.3599394545975951e-05,
+      "loss": 0.1317069411277771,
+      "step": 1960
+    },
+    {
+      "epoch": 7.88,
+      "grad_norm": 0.9034994840621948,
+      "learning_rate": 1.312434413209131e-05,
+      "loss": 0.13362932205200195,
+      "step": 1970
+    },
+    {
+      "epoch": 7.92,
+      "grad_norm": 0.9586318731307983,
+      "learning_rate": 1.2656482700383237e-05,
+      "loss": 0.12677763700485228,
+      "step": 1980
+    },
+    {
+      "epoch": 7.96,
+      "grad_norm": 0.9358674883842468,
+      "learning_rate": 1.219590146136485e-05,
+      "loss": 0.1382434129714966,
+      "step": 1990
+    },
+    {
+      "epoch": 8.0,
+      "grad_norm": 0.8410677313804626,
+      "learning_rate": 1.1742690206261292e-05,
+      "loss": 0.12519369125366211,
+      "step": 2000
+    },
+    {
+      "epoch": 8.0,
+      "eval_loss": 0.5840195417404175,
+      "eval_runtime": 18.625,
+      "eval_samples_per_second": 21.477,
+      "eval_steps_per_second": 3.597,
+      "step": 2000
+    },
+    {
+      "epoch": 8.04,
+      "grad_norm": 0.6319883465766907,
+      "learning_rate": 1.129693728950474e-05,
+      "loss": 0.10409053564071655,
+      "step": 2010
+    },
+    {
+      "epoch": 8.08,
+      "grad_norm": 0.7751646041870117,
+      "learning_rate": 1.0858729611509516e-05,
+      "loss": 0.10310100317001343,
+      "step": 2020
+    },
+    {
+      "epoch": 8.12,
+      "grad_norm": 0.9277542233467102,
+      "learning_rate": 1.0428152601730718e-05,
+      "loss": 0.09960774183273316,
+      "step": 2030
+    },
+    {
+      "epoch": 8.16,
+      "grad_norm": 0.8381429314613342,
+      "learning_rate": 1.0005290202009531e-05,
+      "loss": 0.09982571601867676,
+      "step": 2040
+    },
+    {
+      "epoch": 8.2,
+      "grad_norm": 0.7726228833198547,
+      "learning_rate": 9.590224850208646e-06,
+      "loss": 0.11322143077850341,
+      "step": 2050
+    },
+    {
+      "epoch": 8.24,
+      "grad_norm": 0.7724836468696594,
+      "learning_rate": 9.183037464140804e-06,
+      "loss": 0.10006082057952881,
+      "step": 2060
+    },
+    {
+      "epoch": 8.28,
+      "grad_norm": 1.0587371587753296,
+      "learning_rate": 8.783807425793721e-06,
+      "loss": 0.11560235023498536,
+      "step": 2070
+    },
+    {
+      "epoch": 8.32,
+      "grad_norm": 0.8337858319282532,
+      "learning_rate": 8.392612565854375e-06,
+      "loss": 0.10931503772735596,
+      "step": 2080
+    },
+    {
+      "epoch": 8.36,
+      "grad_norm": 0.805338978767395,
+      "learning_rate": 8.009529148535855e-06,
+      "loss": 0.10900030136108399,
+      "step": 2090
+    },
+    {
+      "epoch": 8.4,
+      "grad_norm": 0.7612441182136536,
+      "learning_rate": 7.63463185670939e-06,
+      "loss": 0.1069128155708313,
+      "step": 2100
+    },
+    {
+      "epoch": 8.4,
+      "eval_loss": 0.6247864961624146,
+      "eval_runtime": 18.281,
+      "eval_samples_per_second": 21.881,
+      "eval_steps_per_second": 3.665,
+      "step": 2100
+    },
+    {
+      "epoch": 8.44,
+      "grad_norm": 0.8081948757171631,
+      "learning_rate": 7.267993777344856e-06,
+      "loss": 0.09856721758842468,
+      "step": 2110
+    },
+    {
+      "epoch": 8.48,
+      "grad_norm": 0.7861329913139343,
+      "learning_rate": 6.909686387262254e-06,
+      "loss": 0.10609345436096192,
+      "step": 2120
+    },
+    {
+      "epoch": 8.52,
+      "grad_norm": 0.7145861387252808,
+      "learning_rate": 6.559779539197231e-06,
+      "loss": 0.105103600025177,
+      "step": 2130
+    },
+    {
+      "epoch": 8.56,
+      "grad_norm": 0.7359808683395386,
+      "learning_rate": 6.21834144818314e-06,
+      "loss": 0.10853493213653564,
+      "step": 2140
+    },
+    {
+      "epoch": 8.6,
+      "grad_norm": 0.8519245982170105,
+      "learning_rate": 5.885438678252342e-06,
+      "loss": 0.11464111804962158,
+      "step": 2150
+    },
+    {
+      "epoch": 8.64,
+      "grad_norm": 0.8307661414146423,
+      "learning_rate": 5.5611361294594325e-06,
+      "loss": 0.10765299797058106,
+      "step": 2160
+    },
+    {
+      "epoch": 8.68,
+      "grad_norm": 0.8340169787406921,
+      "learning_rate": 5.245497025228874e-06,
+      "loss": 0.10699164867401123,
+      "step": 2170
+    },
+    {
+      "epoch": 8.72,
+      "grad_norm": 0.7895165085792542,
+      "learning_rate": 4.938582900029437e-06,
+      "loss": 0.10728691816329956,
+      "step": 2180
+    },
+    {
+      "epoch": 8.76,
+      "grad_norm": 0.7967789769172668,
+      "learning_rate": 4.640453587377957e-06,
+      "loss": 0.11177785396575927,
+      "step": 2190
+    },
+    {
+      "epoch": 8.8,
+      "grad_norm": 0.8613453507423401,
+      "learning_rate": 4.351167208174639e-06,
+      "loss": 0.11041848659515381,
+      "step": 2200
+    },
+    {
+      "epoch": 8.8,
+      "eval_loss": 0.6235533356666565,
+      "eval_runtime": 19.0901,
+      "eval_samples_per_second": 20.953,
+      "eval_steps_per_second": 3.51,
+      "step": 2200
+    },
+    {
+      "epoch": 8.84,
+      "grad_norm": 0.6587359309196472,
+      "learning_rate": 4.0707801593723e-06,
+      "loss": 0.1085782766342163,
+      "step": 2210
+    },
+    {
+      "epoch": 8.88,
+      "grad_norm": 0.7126621603965759,
+      "learning_rate": 3.799347102981665e-06,
+      "loss": 0.11138873100280762,
+      "step": 2220
+    },
+    {
+      "epoch": 8.92,
+      "grad_norm": 0.7560760974884033,
+      "learning_rate": 3.536920955414885e-06,
+      "loss": 0.10770895481109619,
+      "step": 2230
+    },
+    {
+      "epoch": 8.96,
+      "grad_norm": 0.95421302318573,
+      "learning_rate": 3.2835528771693992e-06,
+      "loss": 0.11167995929718018,
+      "step": 2240
+    },
+    {
+      "epoch": 9.0,
+      "grad_norm": 0.9774760007858276,
+      "learning_rate": 3.039292262854088e-06,
+      "loss": 0.11738998889923095,
+      "step": 2250
+    },
+    {
+      "epoch": 9.04,
+      "grad_norm": 0.7680178880691528,
+      "learning_rate": 2.804186731559677e-06,
+      "loss": 0.10072145462036133,
+      "step": 2260
+    },
+    {
+      "epoch": 9.08,
+      "grad_norm": 0.8222008943557739,
+      "learning_rate": 2.5782821175753422e-06,
+      "loss": 0.09228388667106628,
+      "step": 2270
+    },
+    {
+      "epoch": 9.12,
+      "grad_norm": 0.8610215783119202,
+      "learning_rate": 2.361622461453178e-06,
+      "loss": 0.09626876711845397,
+      "step": 2280
+    },
+    {
+      "epoch": 9.16,
+      "grad_norm": 0.7807718515396118,
+      "learning_rate": 2.154250001422431e-06,
+      "loss": 0.0960278868675232,
+      "step": 2290
+    },
+    {
+      "epoch": 9.2,
+      "grad_norm": 0.8036084175109863,
+      "learning_rate": 1.956205165155078e-06,
+      "loss": 0.0941778838634491,
+      "step": 2300
+    },
+    {
+      "epoch": 9.2,
+      "eval_loss": 0.6419874429702759,
+      "eval_runtime": 19.9334,
+      "eval_samples_per_second": 20.067,
+      "eval_steps_per_second": 3.361,
+      "step": 2300
+    },
+    {
+      "epoch": 9.24,
+      "grad_norm": 0.7480472326278687,
+      "learning_rate": 1.7675265618843362e-06,
+      "loss": 0.09725146293640137,
+      "step": 2310
+    },
+    {
+      "epoch": 9.28,
+      "grad_norm": 0.8559448719024658,
+      "learning_rate": 1.5882509748777808e-06,
+      "loss": 0.09353782534599304,
+      "step": 2320
+    },
+    {
+      "epoch": 9.32,
+      "grad_norm": 0.6416171193122864,
+      "learning_rate": 1.4184133542663014e-06,
+      "loss": 0.09848537445068359,
+      "step": 2330
+    },
+    {
+      "epoch": 9.36,
+      "grad_norm": 0.7388947606086731,
+      "learning_rate": 1.258046810230562e-06,
+      "loss": 0.10164464712142944,
+      "step": 2340
+    },
+    {
+      "epoch": 9.4,
+      "grad_norm": 0.8187626600265503,
+      "learning_rate": 1.1071826065460588e-06,
+      "loss": 0.0934177041053772,
+      "step": 2350
+    },
+    {
+      "epoch": 9.44,
+      "grad_norm": 0.865635871887207,
+      "learning_rate": 9.65850154488218e-07,
+      "loss": 0.1012031078338623,
+      "step": 2360
+    },
+    {
+      "epoch": 9.48,
+      "grad_norm": 0.8829763531684875,
+      "learning_rate": 8.340770070986214e-07,
+      "loss": 0.09371918439865112,
+      "step": 2370
+    },
+    {
+      "epoch": 9.52,
+      "grad_norm": 0.7734853625297546,
+      "learning_rate": 7.11888853813436e-07,
+      "loss": 0.09450345039367676,
+      "step": 2380
+    },
+    {
+      "epoch": 9.56,
+      "grad_norm": 0.7692961096763611,
+      "learning_rate": 5.993095154552431e-07,
+      "loss": 0.09499152898788452,
+      "step": 2390
+    },
+    {
+      "epoch": 9.6,
+      "grad_norm": 1.1678398847579956,
+      "learning_rate": 4.963609395891299e-07,
+      "loss": 0.10716021060943604,
+      "step": 2400
+    },
+    {
+      "epoch": 9.6,
+      "eval_loss": 0.6402375102043152,
+      "eval_runtime": 18.9858,
+      "eval_samples_per_second": 21.068,
+      "eval_steps_per_second": 3.529,
+      "step": 2400
+    },
+    {
+      "epoch": 9.64,
+      "grad_norm": 0.7258604764938354,
+      "learning_rate": 4.030631962439302e-07,
+      "loss": 0.09596163630485535,
+      "step": 2410
+    },
+    {
+      "epoch": 9.68,
+      "grad_norm": 0.8662357330322266,
+      "learning_rate": 3.1943447399958027e-07,
+      "loss": 0.09645589590072631,
+      "step": 2420
+    },
+    {
+      "epoch": 9.72,
+      "grad_norm": 0.8258174061775208,
+      "learning_rate": 2.4549107644117885e-07,
+      "loss": 0.09415926933288574,
+      "step": 2430
+    },
+    {
+      "epoch": 9.76,
+      "grad_norm": 0.911540150642395,
+      "learning_rate": 1.8124741898058462e-07,
+      "loss": 0.10026730298995971,
+      "step": 2440
+    },
+    {
+      "epoch": 9.8,
+      "grad_norm": 0.8336577415466309,
+      "learning_rate": 1.267160260461253e-07,
+      "loss": 0.09711679220199584,
+      "step": 2450
+    },
+    {
+      "epoch": 9.84,
+      "grad_norm": 0.7324675917625427,
+      "learning_rate": 8.190752864088436e-08,
+      "loss": 0.09345818758010864,
+      "step": 2460
+    },
+    {
+      "epoch": 9.88,
+      "grad_norm": 0.9261553287506104,
+      "learning_rate": 4.683066227023081e-08,
+      "loss": 0.102751624584198,
+      "step": 2470
+    },
+    {
+      "epoch": 9.92,
+      "grad_norm": 0.9403973817825317,
+      "learning_rate": 2.1492265238748366e-08,
+      "loss": 0.0988599717617035,
+      "step": 2480
+    },
+    {
+      "epoch": 9.96,
+      "grad_norm": 0.7062044739723206,
+      "learning_rate": 5.897277317157279e-09,
+      "loss": 0.09828301668167114,
+      "step": 2490
+    },
+    {
+      "epoch": 10.0,
+      "grad_norm": 0.7819132804870605,
+      "learning_rate": 4.873877924582715e-11,
+      "loss": 0.0937616467475891,
+      "step": 2500
+    },
+    {
+      "epoch": 10.0,
+      "eval_loss": 0.6409608721733093,
+      "eval_runtime": 17.8761,
+      "eval_samples_per_second": 22.376,
+      "eval_steps_per_second": 3.748,
+      "step": 2500
+    },
+    {
+      "epoch": 10.0,
+      "step": 2500,
+      "total_flos": 3.634151342457697e+19,
+      "train_loss": 0.2690703985452652,
+      "train_runtime": 10014.7733,
+      "train_samples_per_second": 5.991,
+      "train_steps_per_second": 0.25
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 2500,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 3.634151342457697e+19,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5228cdd5eb9358c1a0239811495149912109515f66a9f22386e040bdf16b1dd0
+size 5713