Instructions to use VisionXLab/FIRM-Video-8B-InternVL3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use VisionXLab/FIRM-Video-8B-InternVL3 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="VisionXLab/FIRM-Video-8B-InternVL3")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("VisionXLab/FIRM-Video-8B-InternVL3")
model = AutoModelForMultimodalLM.from_pretrained("VisionXLab/FIRM-Video-8B-InternVL3", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use VisionXLab/FIRM-Video-8B-InternVL3 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "VisionXLab/FIRM-Video-8B-InternVL3"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "VisionXLab/FIRM-Video-8B-InternVL3",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/VisionXLab/FIRM-Video-8B-InternVL3

SGLang

How to use VisionXLab/FIRM-Video-8B-InternVL3 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "VisionXLab/FIRM-Video-8B-InternVL3" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "VisionXLab/FIRM-Video-8B-InternVL3",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "VisionXLab/FIRM-Video-8B-InternVL3" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "VisionXLab/FIRM-Video-8B-InternVL3",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use VisionXLab/FIRM-Video-8B-InternVL3 with Docker Model Runner:
```
docker model run hf.co/VisionXLab/FIRM-Video-8B-InternVL3
```

FIRM-Video-8B-InternVL3 / trainer_state.json

zpy777

Upload folder using huggingface_hub

e9c2648 verified 13 days ago

Raw

History Blame Contribute Delete

103 kB

	{
	"best_global_step": null,
	"best_metric": null,
	"best_model_checkpoint": null,
	"epoch": 2.0,
	"eval_steps": 500,
	"global_step": 5504,
	"is_hyper_param_search": false,
	"is_local_process_zero": true,
	"is_world_process_zero": true,
	"log_history": [
	{
	"epoch": 0.003633720930232558,
	"grad_norm": 26.975276947021484,
	"learning_rate": 1.6304347826086958e-07,
	"loss": 2.3138734817504885,
	"step": 10
	},
	{
	"epoch": 0.007267441860465116,
	"grad_norm": 23.218563079833984,
	"learning_rate": 3.442028985507247e-07,
	"loss": 2.209037017822266,
	"step": 20
	},
	{
	"epoch": 0.010901162790697675,
	"grad_norm": 12.365008354187012,
	"learning_rate": 5.253623188405797e-07,
	"loss": 1.977761459350586,
	"step": 30
	},
	{
	"epoch": 0.014534883720930232,
	"grad_norm": 7.03855562210083,
	"learning_rate": 7.065217391304348e-07,
	"loss": 1.6725656509399414,
	"step": 40
	},
	{
	"epoch": 0.018168604651162792,
	"grad_norm": 5.794276237487793,
	"learning_rate": 8.8768115942029e-07,
	"loss": 1.4673602104187011,
	"step": 50
	},
	{
	"epoch": 0.02180232558139535,
	"grad_norm": 4.949296474456787,
	"learning_rate": 1.068840579710145e-06,
	"loss": 1.354590892791748,
	"step": 60
	},
	{
	"epoch": 0.025436046511627907,
	"grad_norm": 4.834152698516846,
	"learning_rate": 1.25e-06,
	"loss": 1.2498680114746095,
	"step": 70
	},
	{
	"epoch": 0.029069767441860465,
	"grad_norm": 4.502780437469482,
	"learning_rate": 1.4311594202898552e-06,
	"loss": 1.137081241607666,
	"step": 80
	},
	{
	"epoch": 0.032703488372093026,
	"grad_norm": 3.9782040119171143,
	"learning_rate": 1.6123188405797101e-06,
	"loss": 1.1200174331665038,
	"step": 90
	},
	{
	"epoch": 0.036337209302325583,
	"grad_norm": 4.458466529846191,
	"learning_rate": 1.7934782608695653e-06,
	"loss": 1.091425323486328,
	"step": 100
	},
	{
	"epoch": 0.03997093023255814,
	"grad_norm": 4.185495853424072,
	"learning_rate": 1.9746376811594204e-06,
	"loss": 1.0495548248291016,
	"step": 110
	},
	{
	"epoch": 0.0436046511627907,
	"grad_norm": 4.143903732299805,
	"learning_rate": 2.1557971014492757e-06,
	"loss": 1.012381362915039,
	"step": 120
	},
	{
	"epoch": 0.047238372093023256,
	"grad_norm": 4.220444202423096,
	"learning_rate": 2.3369565217391307e-06,
	"loss": 1.004845428466797,
	"step": 130
	},
	{
	"epoch": 0.050872093023255814,
	"grad_norm": 4.035683631896973,
	"learning_rate": 2.518115942028986e-06,
	"loss": 0.9986296653747558,
	"step": 140
	},
	{
	"epoch": 0.05450581395348837,
	"grad_norm": 4.283889293670654,
	"learning_rate": 2.6992753623188405e-06,
	"loss": 1.011301040649414,
	"step": 150
	},
	{
	"epoch": 0.05813953488372093,
	"grad_norm": 4.427198886871338,
	"learning_rate": 2.880434782608696e-06,
	"loss": 1.0010372161865235,
	"step": 160
	},
	{
	"epoch": 0.06177325581395349,
	"grad_norm": 4.012505054473877,
	"learning_rate": 3.061594202898551e-06,
	"loss": 0.9743292808532715,
	"step": 170
	},
	{
	"epoch": 0.06540697674418605,
	"grad_norm": 3.958740472793579,
	"learning_rate": 3.242753623188406e-06,
	"loss": 0.9683514595031738,
	"step": 180
	},
	{
	"epoch": 0.0690406976744186,
	"grad_norm": 3.877817153930664,
	"learning_rate": 3.4239130434782614e-06,
	"loss": 0.9519264221191406,
	"step": 190
	},
	{
	"epoch": 0.07267441860465117,
	"grad_norm": 4.2695841789245605,
	"learning_rate": 3.6050724637681163e-06,
	"loss": 0.9914299011230469,
	"step": 200
	},
	{
	"epoch": 0.07630813953488372,
	"grad_norm": 4.172298908233643,
	"learning_rate": 3.7862318840579717e-06,
	"loss": 0.9663248062133789,
	"step": 210
	},
	{
	"epoch": 0.07994186046511628,
	"grad_norm": 3.850515127182007,
	"learning_rate": 3.967391304347827e-06,
	"loss": 0.9332950592041016,
	"step": 220
	},
	{
	"epoch": 0.08357558139534883,
	"grad_norm": 4.158744812011719,
	"learning_rate": 4.148550724637682e-06,
	"loss": 0.9559157371520997,
	"step": 230
	},
	{
	"epoch": 0.0872093023255814,
	"grad_norm": 3.918304920196533,
	"learning_rate": 4.329710144927536e-06,
	"loss": 0.9728614807128906,
	"step": 240
	},
	{
	"epoch": 0.09084302325581395,
	"grad_norm": 3.7205450534820557,
	"learning_rate": 4.510869565217392e-06,
	"loss": 0.9517169952392578,
	"step": 250
	},
	{
	"epoch": 0.09447674418604651,
	"grad_norm": 4.09927225112915,
	"learning_rate": 4.692028985507246e-06,
	"loss": 0.924904727935791,
	"step": 260
	},
	{
	"epoch": 0.09811046511627906,
	"grad_norm": 3.5890684127807617,
	"learning_rate": 4.873188405797102e-06,
	"loss": 0.9166353225708008,
	"step": 270
	},
	{
	"epoch": 0.10174418604651163,
	"grad_norm": 3.7735586166381836,
	"learning_rate": 4.999995937615819e-06,
	"loss": 0.9599678993225098,
	"step": 280
	},
	{
	"epoch": 0.10537790697674419,
	"grad_norm": 4.090644836425781,
	"learning_rate": 4.999923717819873e-06,
	"loss": 0.9784866333007812,
	"step": 290
	},
	{
	"epoch": 0.10901162790697674,
	"grad_norm": 3.6386289596557617,
	"learning_rate": 4.999761225821655e-06,
	"loss": 0.9279603004455567,
	"step": 300
	},
	{
	"epoch": 0.11264534883720931,
	"grad_norm": 3.862521171569824,
	"learning_rate": 4.999508467488746e-06,
	"loss": 0.9072933197021484,
	"step": 310
	},
	{
	"epoch": 0.11627906976744186,
	"grad_norm": 3.708317756652832,
	"learning_rate": 4.999165451948247e-06,
	"loss": 0.9534824371337891,
	"step": 320
	},
	{
	"epoch": 0.11991279069767442,
	"grad_norm": 3.7813735008239746,
	"learning_rate": 4.998732191586442e-06,
	"loss": 0.9028387069702148,
	"step": 330
	},
	{
	"epoch": 0.12354651162790697,
	"grad_norm": 3.4658145904541016,
	"learning_rate": 4.998208702048356e-06,
	"loss": 0.9283824920654297,
	"step": 340
	},
	{
	"epoch": 0.12718023255813954,
	"grad_norm": 3.554983615875244,
	"learning_rate": 4.997595002237186e-06,
	"loss": 0.9344646453857421,
	"step": 350
	},
	{
	"epoch": 0.1308139534883721,
	"grad_norm": 3.5096824169158936,
	"learning_rate": 4.9968911143136225e-06,
	"loss": 0.8727184295654297,
	"step": 360
	},
	{
	"epoch": 0.13444767441860464,
	"grad_norm": 4.025424957275391,
	"learning_rate": 4.996097063695047e-06,
	"loss": 0.9354368209838867,
	"step": 370
	},
	{
	"epoch": 0.1380813953488372,
	"grad_norm": 3.9307773113250732,
	"learning_rate": 4.9952128790546115e-06,
	"loss": 0.9102288246154785,
	"step": 380
	},
	{
	"epoch": 0.14171511627906977,
	"grad_norm": 3.7353551387786865,
	"learning_rate": 4.9942385923202096e-06,
	"loss": 0.8842922210693359,
	"step": 390
	},
	{
	"epoch": 0.14534883720930233,
	"grad_norm": 3.4209423065185547,
	"learning_rate": 4.993174238673318e-06,
	"loss": 0.9148771286010742,
	"step": 400
	},
	{
	"epoch": 0.14898255813953487,
	"grad_norm": 3.5949199199676514,
	"learning_rate": 4.9920198565477295e-06,
	"loss": 0.9135640144348145,
	"step": 410
	},
	{
	"epoch": 0.15261627906976744,
	"grad_norm": 3.3128015995025635,
	"learning_rate": 4.990775487628162e-06,
	"loss": 0.8838287353515625,
	"step": 420
	},
	{
	"epoch": 0.15625,
	"grad_norm": 3.315894603729248,
	"learning_rate": 4.989441176848755e-06,
	"loss": 0.9229852676391601,
	"step": 430
	},
	{
	"epoch": 0.15988372093023256,
	"grad_norm": 3.24360728263855,
	"learning_rate": 4.988016972391448e-06,
	"loss": 0.8780896186828613,
	"step": 440
	},
	{
	"epoch": 0.16351744186046513,
	"grad_norm": 3.412320613861084,
	"learning_rate": 4.98650292568424e-06,
	"loss": 0.9195170402526855,
	"step": 450
	},
	{
	"epoch": 0.16715116279069767,
	"grad_norm": 3.335376739501953,
	"learning_rate": 4.984899091399331e-06,
	"loss": 0.897549057006836,
	"step": 460
	},
	{
	"epoch": 0.17078488372093023,
	"grad_norm": 3.4164929389953613,
	"learning_rate": 4.98320552745115e-06,
	"loss": 0.8903985977172851,
	"step": 470
	},
	{
	"epoch": 0.1744186046511628,
	"grad_norm": 3.1281023025512695,
	"learning_rate": 4.981422294994261e-06,
	"loss": 0.862525749206543,
	"step": 480
	},
	{
	"epoch": 0.17805232558139536,
	"grad_norm": 3.1380455493927,
	"learning_rate": 4.9795494584211575e-06,
	"loss": 0.8879472732543945,
	"step": 490
	},
	{
	"epoch": 0.1816860465116279,
	"grad_norm": 3.2967255115509033,
	"learning_rate": 4.9775870853599365e-06,
	"loss": 0.9352281570434571,
	"step": 500
	},
	{
	"epoch": 0.18531976744186046,
	"grad_norm": 3.2854440212249756,
	"learning_rate": 4.975535246671853e-06,
	"loss": 0.8670001029968262,
	"step": 510
	},
	{
	"epoch": 0.18895348837209303,
	"grad_norm": 3.250467300415039,
	"learning_rate": 4.97339401644877e-06,
	"loss": 0.8876986503601074,
	"step": 520
	},
	{
	"epoch": 0.1925872093023256,
	"grad_norm": 3.321596622467041,
	"learning_rate": 4.9711634720104714e-06,
	"loss": 0.9065715789794921,
	"step": 530
	},
	{
	"epoch": 0.19622093023255813,
	"grad_norm": 3.3532400131225586,
	"learning_rate": 4.968843693901879e-06,
	"loss": 0.9148740768432617,
	"step": 540
	},
	{
	"epoch": 0.1998546511627907,
	"grad_norm": 3.0000295639038086,
	"learning_rate": 4.96643476589014e-06,
	"loss": 0.8900104522705078,
	"step": 550
	},
	{
	"epoch": 0.20348837209302326,
	"grad_norm": 3.054938554763794,
	"learning_rate": 4.963936774961605e-06,
	"loss": 0.8990900993347168,
	"step": 560
	},
	{
	"epoch": 0.20712209302325582,
	"grad_norm": 3.737675666809082,
	"learning_rate": 4.961349811318681e-06,
	"loss": 0.8763975143432617,
	"step": 570
	},
	{
	"epoch": 0.21075581395348839,
	"grad_norm": 3.054027795791626,
	"learning_rate": 4.958673968376582e-06,
	"loss": 0.8707597732543946,
	"step": 580
	},
	{
	"epoch": 0.21438953488372092,
	"grad_norm": 3.250688314437866,
	"learning_rate": 4.955909342759949e-06,
	"loss": 0.8907978057861328,
	"step": 590
	},
	{
	"epoch": 0.2180232558139535,
	"grad_norm": 3.243490695953369,
	"learning_rate": 4.953056034299367e-06,
	"loss": 0.8893247604370117,
	"step": 600
	},
	{
	"epoch": 0.22165697674418605,
	"grad_norm": 3.19518780708313,
	"learning_rate": 4.9501141460277536e-06,
	"loss": 0.8762750625610352,
	"step": 610
	},
	{
	"epoch": 0.22529069767441862,
	"grad_norm": 3.316575050354004,
	"learning_rate": 4.9470837841766446e-06,
	"loss": 0.8776466369628906,
	"step": 620
	},
	{
	"epoch": 0.22892441860465115,
	"grad_norm": 3.0390732288360596,
	"learning_rate": 4.943965058172354e-06,
	"loss": 0.8640607833862305,
	"step": 630
	},
	{
	"epoch": 0.23255813953488372,
	"grad_norm": 3.0142030715942383,
	"learning_rate": 4.940758080632023e-06,
	"loss": 0.8418485641479492,
	"step": 640
	},
	{
	"epoch": 0.23619186046511628,
	"grad_norm": 3.2164595127105713,
	"learning_rate": 4.937462967359556e-06,
	"loss": 0.8713783264160156,
	"step": 650
	},
	{
	"epoch": 0.23982558139534885,
	"grad_norm": 3.233011484146118,
	"learning_rate": 4.934079837341436e-06,
	"loss": 0.8551886558532715,
	"step": 660
	},
	{
	"epoch": 0.24345930232558138,
	"grad_norm": 2.8924005031585693,
	"learning_rate": 4.93060881274243e-06,
	"loss": 0.8672443389892578,
	"step": 670
	},
	{
	"epoch": 0.24709302325581395,
	"grad_norm": 3.2440953254699707,
	"learning_rate": 4.927050018901176e-06,
	"loss": 0.8703802108764649,
	"step": 680
	},
	{
	"epoch": 0.2507267441860465,
	"grad_norm": 3.142500400543213,
	"learning_rate": 4.923403584325659e-06,
	"loss": 0.8739217758178711,
	"step": 690
	},
	{
	"epoch": 0.2543604651162791,
	"grad_norm": 3.116825580596924,
	"learning_rate": 4.919669640688567e-06,
	"loss": 0.848177146911621,
	"step": 700
	},
	{
	"epoch": 0.25799418604651164,
	"grad_norm": 2.9440503120422363,
	"learning_rate": 4.91584832282254e-06,
	"loss": 0.8814225196838379,
	"step": 710
	},
	{
	"epoch": 0.2616279069767442,
	"grad_norm": 3.111051559448242,
	"learning_rate": 4.911939768715302e-06,
	"loss": 0.8624879837036132,
	"step": 720
	},
	{
	"epoch": 0.26526162790697677,
	"grad_norm": 3.0284619331359863,
	"learning_rate": 4.907944119504674e-06,
	"loss": 0.8661399841308594,
	"step": 730
	},
	{
	"epoch": 0.2688953488372093,
	"grad_norm": 3.13392972946167,
	"learning_rate": 4.903861519473479e-06,
	"loss": 0.8655827522277832,
	"step": 740
	},
	{
	"epoch": 0.27252906976744184,
	"grad_norm": 3.3833975791931152,
	"learning_rate": 4.899692116044334e-06,
	"loss": 0.8533415794372559,
	"step": 750
	},
	{
	"epoch": 0.2761627906976744,
	"grad_norm": 3.6023480892181396,
	"learning_rate": 4.895436059774325e-06,
	"loss": 0.8765420913696289,
	"step": 760
	},
	{
	"epoch": 0.279796511627907,
	"grad_norm": 2.986733913421631,
	"learning_rate": 4.89109350434957e-06,
	"loss": 0.8727610588073731,
	"step": 770
	},
	{
	"epoch": 0.28343023255813954,
	"grad_norm": 3.0566656589508057,
	"learning_rate": 4.886664606579671e-06,
	"loss": 0.8449860572814941,
	"step": 780
	},
	{
	"epoch": 0.2870639534883721,
	"grad_norm": 3.382549524307251,
	"learning_rate": 4.88214952639205e-06,
	"loss": 0.8713051795959472,
	"step": 790
	},
	{
	"epoch": 0.29069767441860467,
	"grad_norm": 3.041335105895996,
	"learning_rate": 4.877548426826173e-06,
	"loss": 0.8494526863098144,
	"step": 800
	},
	{
	"epoch": 0.29433139534883723,
	"grad_norm": 2.935906410217285,
	"learning_rate": 4.872861474027667e-06,
	"loss": 0.8570301055908203,
	"step": 810
	},
	{
	"epoch": 0.29796511627906974,
	"grad_norm": 2.9906163215637207,
	"learning_rate": 4.868088837242318e-06,
	"loss": 0.8500057220458984,
	"step": 820
	},
	{
	"epoch": 0.3015988372093023,
	"grad_norm": 3.039278984069824,
	"learning_rate": 4.863230688809955e-06,
	"loss": 0.8411312103271484,
	"step": 830
	},
	{
	"epoch": 0.30523255813953487,
	"grad_norm": 3.0760066509246826,
	"learning_rate": 4.858287204158236e-06,
	"loss": 0.8711600303649902,
	"step": 840
	},
	{
	"epoch": 0.30886627906976744,
	"grad_norm": 3.267594814300537,
	"learning_rate": 4.853258561796307e-06,
	"loss": 0.8397054672241211,
	"step": 850
	},
	{
	"epoch": 0.3125,
	"grad_norm": 2.9856038093566895,
	"learning_rate": 4.848144943308355e-06,
	"loss": 0.8396913528442382,
	"step": 860
	},
	{
	"epoch": 0.31613372093023256,
	"grad_norm": 2.778498888015747,
	"learning_rate": 4.842946533347056e-06,
	"loss": 0.8382930755615234,
	"step": 870
	},
	{
	"epoch": 0.31976744186046513,
	"grad_norm": 3.0495941638946533,
	"learning_rate": 4.837663519626904e-06,
	"loss": 0.8362157821655274,
	"step": 880
	},
	{
	"epoch": 0.3234011627906977,
	"grad_norm": 3.0766844749450684,
	"learning_rate": 4.8322960929174325e-06,
	"loss": 0.8268529891967773,
	"step": 890
	},
	{
	"epoch": 0.32703488372093026,
	"grad_norm": 3.0890631675720215,
	"learning_rate": 4.826844447036329e-06,
	"loss": 0.8377861022949219,
	"step": 900
	},
	{
	"epoch": 0.33066860465116277,
	"grad_norm": 2.9986603260040283,
	"learning_rate": 4.82130877884243e-06,
	"loss": 0.8358505249023438,
	"step": 910
	},
	{
	"epoch": 0.33430232558139533,
	"grad_norm": 2.8856871128082275,
	"learning_rate": 4.8156892882286206e-06,
	"loss": 0.8571210861206054,
	"step": 920
	},
	{
	"epoch": 0.3379360465116279,
	"grad_norm": 3.0375850200653076,
	"learning_rate": 4.809986178114608e-06,
	"loss": 0.8279001235961914,
	"step": 930
	},
	{
	"epoch": 0.34156976744186046,
	"grad_norm": 2.7761526107788086,
	"learning_rate": 4.804199654439601e-06,
	"loss": 0.8522439956665039,
	"step": 940
	},
	{
	"epoch": 0.345203488372093,
	"grad_norm": 3.0612213611602783,
	"learning_rate": 4.798329926154868e-06,
	"loss": 0.843962574005127,
	"step": 950
	},
	{
	"epoch": 0.3488372093023256,
	"grad_norm": 3.3076977729797363,
	"learning_rate": 4.792377205216198e-06,
	"loss": 0.8473945617675781,
	"step": 960
	},
	{
	"epoch": 0.35247093023255816,
	"grad_norm": 2.7613766193389893,
	"learning_rate": 4.78634170657624e-06,
	"loss": 0.8338375091552734,
	"step": 970
	},
	{
	"epoch": 0.3561046511627907,
	"grad_norm": 2.9154269695281982,
	"learning_rate": 4.780223648176746e-06,
	"loss": 0.8274489402770996,
	"step": 980
	},
	{
	"epoch": 0.35973837209302323,
	"grad_norm": 2.8659121990203857,
	"learning_rate": 4.774023250940699e-06,
	"loss": 0.8512365341186523,
	"step": 990
	},
	{
	"epoch": 0.3633720930232558,
	"grad_norm": 2.993288516998291,
	"learning_rate": 4.767740738764337e-06,
	"loss": 0.8384992599487304,
	"step": 1000
	},
	{
	"epoch": 0.36700581395348836,
	"grad_norm": 2.8574283123016357,
	"learning_rate": 4.761376338509065e-06,
	"loss": 0.8402500152587891,
	"step": 1010
	},
	{
	"epoch": 0.3706395348837209,
	"grad_norm": 3.0336382389068604,
	"learning_rate": 4.754930279993266e-06,
	"loss": 0.8413254737854003,
	"step": 1020
	},
	{
	"epoch": 0.3742732558139535,
	"grad_norm": 2.7382609844207764,
	"learning_rate": 4.748402795984004e-06,
	"loss": 0.8280426025390625,
	"step": 1030
	},
	{
	"epoch": 0.37790697674418605,
	"grad_norm": 2.9250476360321045,
	"learning_rate": 4.741794122188611e-06,
	"loss": 0.8249115943908691,
	"step": 1040
	},
	{
	"epoch": 0.3815406976744186,
	"grad_norm": 2.767603874206543,
	"learning_rate": 4.735104497246185e-06,
	"loss": 0.8396032333374024,
	"step": 1050
	},
	{
	"epoch": 0.3851744186046512,
	"grad_norm": 2.853214740753174,
	"learning_rate": 4.728334162718967e-06,
	"loss": 0.8256677627563477,
	"step": 1060
	},
	{
	"epoch": 0.38880813953488375,
	"grad_norm": 2.9439098834991455,
	"learning_rate": 4.72148336308362e-06,
	"loss": 0.8689069747924805,
	"step": 1070
	},
	{
	"epoch": 0.39244186046511625,
	"grad_norm": 2.757331371307373,
	"learning_rate": 4.7145523457224005e-06,
	"loss": 0.8378227233886719,
	"step": 1080
	},
	{
	"epoch": 0.3960755813953488,
	"grad_norm": 2.884744882583618,
	"learning_rate": 4.707541360914225e-06,
	"loss": 0.8407644271850586,
	"step": 1090
	},
	{
	"epoch": 0.3997093023255814,
	"grad_norm": 2.771481513977051,
	"learning_rate": 4.700450661825635e-06,
	"loss": 0.8069744110107422,
	"step": 1100
	},
	{
	"epoch": 0.40334302325581395,
	"grad_norm": 2.8814430236816406,
	"learning_rate": 4.693280504501651e-06,
	"loss": 0.8310678482055665,
	"step": 1110
	},
	{
	"epoch": 0.4069767441860465,
	"grad_norm": 2.7411293983459473,
	"learning_rate": 4.686031147856531e-06,
	"loss": 0.8619399070739746,
	"step": 1120
	},
	{
	"epoch": 0.4106104651162791,
	"grad_norm": 2.7865939140319824,
	"learning_rate": 4.678702853664418e-06,
	"loss": 0.8180102348327637,
	"step": 1130
	},
	{
	"epoch": 0.41424418604651164,
	"grad_norm": 2.827937602996826,
	"learning_rate": 4.671295886549887e-06,
	"loss": 0.8340296745300293,
	"step": 1140
	},
	{
	"epoch": 0.4178779069767442,
	"grad_norm": 2.8981363773345947,
	"learning_rate": 4.6638105139783955e-06,
	"loss": 0.8189412117004394,
	"step": 1150
	},
	{
	"epoch": 0.42151162790697677,
	"grad_norm": 2.6089940071105957,
	"learning_rate": 4.656247006246617e-06,
	"loss": 0.8204338073730468,
	"step": 1160
	},
	{
	"epoch": 0.4251453488372093,
	"grad_norm": 2.7830424308776855,
	"learning_rate": 4.648605636472683e-06,
	"loss": 0.8322567939758301,
	"step": 1170
	},
	{
	"epoch": 0.42877906976744184,
	"grad_norm": 2.8253374099731445,
	"learning_rate": 4.640886680586326e-06,
	"loss": 0.7853239059448243,
	"step": 1180
	},
	{
	"epoch": 0.4324127906976744,
	"grad_norm": 2.7395074367523193,
	"learning_rate": 4.6330904173189105e-06,
	"loss": 0.8274772644042969,
	"step": 1190
	},
	{
	"epoch": 0.436046511627907,
	"grad_norm": 2.8378896713256836,
	"learning_rate": 4.625217128193366e-06,
	"loss": 0.8572607040405273,
	"step": 1200
	},
	{
	"epoch": 0.43968023255813954,
	"grad_norm": 2.8448286056518555,
	"learning_rate": 4.617267097514028e-06,
	"loss": 0.822886848449707,
	"step": 1210
	},
	{
	"epoch": 0.4433139534883721,
	"grad_norm": 2.8581748008728027,
	"learning_rate": 4.609240612356367e-06,
	"loss": 0.7989314556121826,
	"step": 1220
	},
	{
	"epoch": 0.44694767441860467,
	"grad_norm": 2.6748061180114746,
	"learning_rate": 4.6011379625566245e-06,
	"loss": 0.8521600723266601,
	"step": 1230
	},
	{
	"epoch": 0.45058139534883723,
	"grad_norm": 2.6777000427246094,
	"learning_rate": 4.5929594407013434e-06,
	"loss": 0.8159719467163086,
	"step": 1240
	},
	{
	"epoch": 0.45421511627906974,
	"grad_norm": 2.781126022338867,
	"learning_rate": 4.584705342116807e-06,
	"loss": 0.8263021469116211,
	"step": 1250
	},
	{
	"epoch": 0.4578488372093023,
	"grad_norm": 3.040696144104004,
	"learning_rate": 4.576375964858373e-06,
	"loss": 0.7871081352233886,
	"step": 1260
	},
	{
	"epoch": 0.46148255813953487,
	"grad_norm": 2.9158823490142822,
	"learning_rate": 4.567971609699709e-06,
	"loss": 0.828523826599121,
	"step": 1270
	},
	{
	"epoch": 0.46511627906976744,
	"grad_norm": 2.79797625541687,
	"learning_rate": 4.559492580121936e-06,
	"loss": 0.8283788681030273,
	"step": 1280
	},
	{
	"epoch": 0.46875,
	"grad_norm": 2.6865875720977783,
	"learning_rate": 4.550939182302666e-06,
	"loss": 0.8271783828735352,
	"step": 1290
	},
	{
	"epoch": 0.47238372093023256,
	"grad_norm": 2.9567787647247314,
	"learning_rate": 4.5423117251049455e-06,
	"loss": 0.8017131805419921,
	"step": 1300
	},
	{
	"epoch": 0.47601744186046513,
	"grad_norm": 2.9435689449310303,
	"learning_rate": 4.533610520066104e-06,
	"loss": 0.8426065444946289,
	"step": 1310
	},
	{
	"epoch": 0.4796511627906977,
	"grad_norm": 3.003634452819824,
	"learning_rate": 4.524835881386508e-06,
	"loss": 0.8319045066833496,
	"step": 1320
	},
	{
	"epoch": 0.48328488372093026,
	"grad_norm": 2.7824082374572754,
	"learning_rate": 4.515988125918206e-06,
	"loss": 0.8165316581726074,
	"step": 1330
	},
	{
	"epoch": 0.48691860465116277,
	"grad_norm": 2.6757547855377197,
	"learning_rate": 4.507067573153496e-06,
	"loss": 0.8275221824645996,
	"step": 1340
	},
	{
	"epoch": 0.49055232558139533,
	"grad_norm": 2.7333762645721436,
	"learning_rate": 4.498074545213386e-06,
	"loss": 0.7960930824279785,
	"step": 1350
	},
	{
	"epoch": 0.4941860465116279,
	"grad_norm": 2.8366189002990723,
	"learning_rate": 4.489009366835956e-06,
	"loss": 0.8204254150390625,
	"step": 1360
	},
	{
	"epoch": 0.49781976744186046,
	"grad_norm": 2.8708670139312744,
	"learning_rate": 4.479872365364644e-06,
	"loss": 0.7996141910552979,
	"step": 1370
	},
	{
	"epoch": 0.501453488372093,
	"grad_norm": 2.927445411682129,
	"learning_rate": 4.470663870736414e-06,
	"loss": 0.7884037017822265,
	"step": 1380
	},
	{
	"epoch": 0.5050872093023255,
	"grad_norm": 2.7692763805389404,
	"learning_rate": 4.461384215469847e-06,
	"loss": 0.8215972900390625,
	"step": 1390
	},
	{
	"epoch": 0.5087209302325582,
	"grad_norm": 2.714712619781494,
	"learning_rate": 4.452033734653136e-06,
	"loss": 0.8339604377746582,
	"step": 1400
	},
	{
	"epoch": 0.5123546511627907,
	"grad_norm": 2.8098700046539307,
	"learning_rate": 4.442612765931981e-06,
	"loss": 0.819193172454834,
	"step": 1410
	},
	{
	"epoch": 0.5159883720930233,
	"grad_norm": 2.9055378437042236,
	"learning_rate": 4.433121649497401e-06,
	"loss": 0.8303899765014648,
	"step": 1420
	},
	{
	"epoch": 0.5196220930232558,
	"grad_norm": 2.8050930500030518,
	"learning_rate": 4.423560728073444e-06,
	"loss": 0.8256185531616211,
	"step": 1430
	},
	{
	"epoch": 0.5232558139534884,
	"grad_norm": 2.776355266571045,
	"learning_rate": 4.413930346904819e-06,
	"loss": 0.7909624576568604,
	"step": 1440
	},
	{
	"epoch": 0.5268895348837209,
	"grad_norm": 2.799743175506592,
	"learning_rate": 4.404230853744424e-06,
	"loss": 0.8219989776611328,
	"step": 1450
	},
	{
	"epoch": 0.5305232558139535,
	"grad_norm": 2.6980018615722656,
	"learning_rate": 4.3944625988407896e-06,
	"loss": 0.8094476699829102,
	"step": 1460
	},
	{
	"epoch": 0.534156976744186,
	"grad_norm": 2.7240099906921387,
	"learning_rate": 4.384625934925431e-06,
	"loss": 0.7854260444641114,
	"step": 1470
	},
	{
	"epoch": 0.5377906976744186,
	"grad_norm": 2.8290176391601562,
	"learning_rate": 4.374721217200113e-06,
	"loss": 0.7866772174835205,
	"step": 1480
	},
	{
	"epoch": 0.5414244186046512,
	"grad_norm": 2.822491407394409,
	"learning_rate": 4.364748803324021e-06,
	"loss": 0.8133050918579101,
	"step": 1490
	},
	{
	"epoch": 0.5450581395348837,
	"grad_norm": 2.792834520339966,
	"learning_rate": 4.354709053400846e-06,
	"loss": 0.8320375442504883,
	"step": 1500
	},
	{
	"epoch": 0.5486918604651163,
	"grad_norm": 2.802419900894165,
	"learning_rate": 4.344602329965788e-06,
	"loss": 0.7835256576538085,
	"step": 1510
	},
	{
	"epoch": 0.5523255813953488,
	"grad_norm": 2.7649500370025635,
	"learning_rate": 4.334428997972454e-06,
	"loss": 0.8077914237976074,
	"step": 1520
	},
	{
	"epoch": 0.5559593023255814,
	"grad_norm": 2.6321909427642822,
	"learning_rate": 4.324189424779686e-06,
	"loss": 0.7951157093048096,
	"step": 1530
	},
	{
	"epoch": 0.559593023255814,
	"grad_norm": 2.90681529045105,
	"learning_rate": 4.313883980138296e-06,
	"loss": 0.7936089038848877,
	"step": 1540
	},
	{
	"epoch": 0.5632267441860465,
	"grad_norm": 2.783231258392334,
	"learning_rate": 4.303513036177711e-06,
	"loss": 0.8397782325744629,
	"step": 1550
	},
	{
	"epoch": 0.5668604651162791,
	"grad_norm": 2.735781669616699,
	"learning_rate": 4.293076967392539e-06,
	"loss": 0.7950201034545898,
	"step": 1560
	},
	{
	"epoch": 0.5704941860465116,
	"grad_norm": 2.7377381324768066,
	"learning_rate": 4.282576150629044e-06,
	"loss": 0.8493624687194824,
	"step": 1570
	},
	{
	"epoch": 0.5741279069767442,
	"grad_norm": 2.8512473106384277,
	"learning_rate": 4.272010965071538e-06,
	"loss": 0.8232376098632812,
	"step": 1580
	},
	{
	"epoch": 0.5777616279069767,
	"grad_norm": 2.664661407470703,
	"learning_rate": 4.2613817922286875e-06,
	"loss": 0.8027572631835938,
	"step": 1590
	},
	{
	"epoch": 0.5813953488372093,
	"grad_norm": 2.655877113342285,
	"learning_rate": 4.250689015919741e-06,
	"loss": 0.7969479560852051,
	"step": 1600
	},
	{
	"epoch": 0.5850290697674418,
	"grad_norm": 2.7822320461273193,
	"learning_rate": 4.239933022260668e-06,
	"loss": 0.83875732421875,
	"step": 1610
	},
	{
	"epoch": 0.5886627906976745,
	"grad_norm": 2.707385778427124,
	"learning_rate": 4.2291141996502115e-06,
	"loss": 0.7893309593200684,
	"step": 1620
	},
	{
	"epoch": 0.592296511627907,
	"grad_norm": 2.7490038871765137,
	"learning_rate": 4.21823293875587e-06,
	"loss": 0.8146431922912598,
	"step": 1630
	},
	{
	"epoch": 0.5959302325581395,
	"grad_norm": 2.7448737621307373,
	"learning_rate": 4.207289632499787e-06,
	"loss": 0.8016767501831055,
	"step": 1640
	},
	{
	"epoch": 0.5995639534883721,
	"grad_norm": 2.762751817703247,
	"learning_rate": 4.1962846760445606e-06,
	"loss": 0.7814556121826172,
	"step": 1650
	},
	{
	"epoch": 0.6031976744186046,
	"grad_norm": 2.8780152797698975,
	"learning_rate": 4.185218466778981e-06,
	"loss": 0.8208811759948731,
	"step": 1660
	},
	{
	"epoch": 0.6068313953488372,
	"grad_norm": 2.860222816467285,
	"learning_rate": 4.174091404303671e-06,
	"loss": 0.8105660438537597,
	"step": 1670
	},
	{
	"epoch": 0.6104651162790697,
	"grad_norm": 2.866760730743408,
	"learning_rate": 4.162903890416666e-06,
	"loss": 0.8159380912780761,
	"step": 1680
	},
	{
	"epoch": 0.6140988372093024,
	"grad_norm": 2.6297552585601807,
	"learning_rate": 4.151656329098899e-06,
	"loss": 0.7768171310424805,
	"step": 1690
	},
	{
	"epoch": 0.6177325581395349,
	"grad_norm": 2.73301100730896,
	"learning_rate": 4.1403491264996144e-06,
	"loss": 0.7993753910064697,
	"step": 1700
	},
	{
	"epoch": 0.6213662790697675,
	"grad_norm": 2.8257224559783936,
	"learning_rate": 4.1289826909217034e-06,
	"loss": 0.7895632266998291,
	"step": 1710
	},
	{
	"epoch": 0.625,
	"grad_norm": 2.745051860809326,
	"learning_rate": 4.117557432806958e-06,
	"loss": 0.8250743865966796,
	"step": 1720
	},
	{
	"epoch": 0.6286337209302325,
	"grad_norm": 2.557063102722168,
	"learning_rate": 4.106073764721252e-06,
	"loss": 0.8024697303771973,
	"step": 1730
	},
	{
	"epoch": 0.6322674418604651,
	"grad_norm": 2.8046607971191406,
	"learning_rate": 4.094532101339641e-06,
	"loss": 0.7836231231689453,
	"step": 1740
	},
	{
	"epoch": 0.6359011627906976,
	"grad_norm": 2.7214012145996094,
	"learning_rate": 4.08293285943139e-06,
	"loss": 0.8137659072875977,
	"step": 1750
	},
	{
	"epoch": 0.6395348837209303,
	"grad_norm": 2.6622800827026367,
	"learning_rate": 4.071276457844925e-06,
	"loss": 0.817494010925293,
	"step": 1760
	},
	{
	"epoch": 0.6431686046511628,
	"grad_norm": 2.906050443649292,
	"learning_rate": 4.059563317492705e-06,
	"loss": 0.7947888851165772,
	"step": 1770
	},
	{
	"epoch": 0.6468023255813954,
	"grad_norm": 2.7529661655426025,
	"learning_rate": 4.0477938613360265e-06,
	"loss": 0.7987423896789551,
	"step": 1780
	},
	{
	"epoch": 0.6504360465116279,
	"grad_norm": 2.8821957111358643,
	"learning_rate": 4.035968514369748e-06,
	"loss": 0.8163650512695313,
	"step": 1790
	},
	{
	"epoch": 0.6540697674418605,
	"grad_norm": 2.667881727218628,
	"learning_rate": 4.024087703606945e-06,
	"loss": 0.8074198722839355,
	"step": 1800
	},
	{
	"epoch": 0.657703488372093,
	"grad_norm": 2.6779751777648926,
	"learning_rate": 4.012151858063487e-06,
	"loss": 0.7849567413330079,
	"step": 1810
	},
	{
	"epoch": 0.6613372093023255,
	"grad_norm": 2.7572410106658936,
	"learning_rate": 4.0001614087425515e-06,
	"loss": 0.8169970512390137,
	"step": 1820
	},
	{
	"epoch": 0.6649709302325582,
	"grad_norm": 2.729264736175537,
	"learning_rate": 3.988116788619055e-06,
	"loss": 0.7919125080108642,
	"step": 1830
	},
	{
	"epoch": 0.6686046511627907,
	"grad_norm": 2.716473340988159,
	"learning_rate": 3.976018432624025e-06,
	"loss": 0.8070649147033692,
	"step": 1840
	},
	{
	"epoch": 0.6722383720930233,
	"grad_norm": 2.732853651046753,
	"learning_rate": 3.963866777628885e-06,
	"loss": 0.7902655601501465,
	"step": 1850
	},
	{
	"epoch": 0.6758720930232558,
	"grad_norm": 2.834977865219116,
	"learning_rate": 3.95166226242969e-06,
	"loss": 0.7799157619476318,
	"step": 1860
	},
	{
	"epoch": 0.6795058139534884,
	"grad_norm": 2.7858405113220215,
	"learning_rate": 3.939405327731268e-06,
	"loss": 0.8153091430664062,
	"step": 1870
	},
	{
	"epoch": 0.6831395348837209,
	"grad_norm": 2.7912445068359375,
	"learning_rate": 3.927096416131324e-06,
	"loss": 0.7729724884033203,
	"step": 1880
	},
	{
	"epoch": 0.6867732558139535,
	"grad_norm": 2.8135998249053955,
	"learning_rate": 3.914735972104442e-06,
	"loss": 0.798687744140625,
	"step": 1890
	},
	{
	"epoch": 0.690406976744186,
	"grad_norm": 2.7605013847351074,
	"learning_rate": 3.902324441986041e-06,
	"loss": 0.8077116012573242,
	"step": 1900
	},
	{
	"epoch": 0.6940406976744186,
	"grad_norm": 2.720306158065796,
	"learning_rate": 3.889862273956261e-06,
	"loss": 0.7985315799713135,
	"step": 1910
	},
	{
	"epoch": 0.6976744186046512,
	"grad_norm": 2.7768609523773193,
	"learning_rate": 3.877349918023772e-06,
	"loss": 0.7771415710449219,
	"step": 1920
	},
	{
	"epoch": 0.7013081395348837,
	"grad_norm": 2.8074111938476562,
	"learning_rate": 3.864787826009533e-06,
	"loss": 0.7958961486816406,
	"step": 1930
	},
	{
	"epoch": 0.7049418604651163,
	"grad_norm": 2.90981388092041,
	"learning_rate": 3.852176451530468e-06,
	"loss": 0.7736949920654297,
	"step": 1940
	},
	{
	"epoch": 0.7085755813953488,
	"grad_norm": 2.7982535362243652,
	"learning_rate": 3.839516249983093e-06,
	"loss": 0.751534366607666,
	"step": 1950
	},
	{
	"epoch": 0.7122093023255814,
	"grad_norm": 2.6651105880737305,
	"learning_rate": 3.826807678527067e-06,
	"loss": 0.8033027648925781,
	"step": 1960
	},
	{
	"epoch": 0.715843023255814,
	"grad_norm": 2.684173107147217,
	"learning_rate": 3.814051196068684e-06,
	"loss": 0.7902488708496094,
	"step": 1970
	},
	{
	"epoch": 0.7194767441860465,
	"grad_norm": 2.5975940227508545,
	"learning_rate": 3.8012472632443065e-06,
	"loss": 0.7830523490905762,
	"step": 1980
	},
	{
	"epoch": 0.7231104651162791,
	"grad_norm": 3.0428762435913086,
	"learning_rate": 3.7883963424037286e-06,
	"loss": 0.7818472385406494,
	"step": 1990
	},
	{
	"epoch": 0.7267441860465116,
	"grad_norm": 2.6959846019744873,
	"learning_rate": 3.775498897593478e-06,
	"loss": 0.8127883911132813,
	"step": 2000
	},
	{
	"epoch": 0.7303779069767442,
	"grad_norm": 2.730274200439453,
	"learning_rate": 3.762555394540066e-06,
	"loss": 0.7654043674468994,
	"step": 2010
	},
	{
	"epoch": 0.7340116279069767,
	"grad_norm": 2.6885862350463867,
	"learning_rate": 3.7495663006331633e-06,
	"loss": 0.7917684555053711,
	"step": 2020
	},
	{
	"epoch": 0.7376453488372093,
	"grad_norm": 2.7472004890441895,
	"learning_rate": 3.7365320849087262e-06,
	"loss": 0.7998331546783447,
	"step": 2030
	},
	{
	"epoch": 0.7412790697674418,
	"grad_norm": 2.767969846725464,
	"learning_rate": 3.7234532180320605e-06,
	"loss": 0.7952752590179444,
	"step": 2040
	},
	{
	"epoch": 0.7449127906976745,
	"grad_norm": 2.6142473220825195,
	"learning_rate": 3.710330172280821e-06,
	"loss": 0.8077480316162109,
	"step": 2050
	},
	{
	"epoch": 0.748546511627907,
	"grad_norm": 2.7542755603790283,
	"learning_rate": 3.6971634215279664e-06,
	"loss": 0.7876755237579346,
	"step": 2060
	},
	{
	"epoch": 0.7521802325581395,
	"grad_norm": 2.6669540405273438,
	"learning_rate": 3.683953441224636e-06,
	"loss": 0.7639244079589844,
	"step": 2070
	},
	{
	"epoch": 0.7558139534883721,
	"grad_norm": 2.7806105613708496,
	"learning_rate": 3.6707007083829913e-06,
	"loss": 0.8007266998291016,
	"step": 2080
	},
	{
	"epoch": 0.7594476744186046,
	"grad_norm": 2.7667555809020996,
	"learning_rate": 3.6574057015589877e-06,
	"loss": 0.7894309520721435,
	"step": 2090
	},
	{
	"epoch": 0.7630813953488372,
	"grad_norm": 2.709882974624634,
	"learning_rate": 3.6440689008350904e-06,
	"loss": 0.7834427356719971,
	"step": 2100
	},
	{
	"epoch": 0.7667151162790697,
	"grad_norm": 2.6989541053771973,
	"learning_rate": 3.630690787802944e-06,
	"loss": 0.8200445175170898,
	"step": 2110
	},
	{
	"epoch": 0.7703488372093024,
	"grad_norm": 2.769083261489868,
	"learning_rate": 3.6172718455459783e-06,
	"loss": 0.7756868362426758,
	"step": 2120
	},
	{
	"epoch": 0.7739825581395349,
	"grad_norm": 2.8316144943237305,
	"learning_rate": 3.6038125586219663e-06,
	"loss": 0.7860795974731445,
	"step": 2130
	},
	{
	"epoch": 0.7776162790697675,
	"grad_norm": 2.8543038368225098,
	"learning_rate": 3.5903134130455285e-06,
	"loss": 0.7748101234436036,
	"step": 2140
	},
	{
	"epoch": 0.78125,
	"grad_norm": 3.5973920822143555,
	"learning_rate": 3.5767748962705756e-06,
	"loss": 0.7883114814758301,
	"step": 2150
	},
	{
	"epoch": 0.7848837209302325,
	"grad_norm": 2.770141124725342,
	"learning_rate": 3.563197497172717e-06,
	"loss": 0.8006394386291504,
	"step": 2160
	},
	{
	"epoch": 0.7885174418604651,
	"grad_norm": 2.8230247497558594,
	"learning_rate": 3.5495817060315975e-06,
	"loss": 0.7962004661560058,
	"step": 2170
	},
	{
	"epoch": 0.7921511627906976,
	"grad_norm": 2.885927677154541,
	"learning_rate": 3.535928014513201e-06,
	"loss": 0.7753909587860107,
	"step": 2180
	},
	{
	"epoch": 0.7957848837209303,
	"grad_norm": 2.7065513134002686,
	"learning_rate": 3.5222369156520927e-06,
	"loss": 0.7820100784301758,
	"step": 2190
	},
	{
	"epoch": 0.7994186046511628,
	"grad_norm": 2.6883857250213623,
	"learning_rate": 3.5085089038336157e-06,
	"loss": 0.7736979007720948,
	"step": 2200
	},
	{
	"epoch": 0.8030523255813954,
	"grad_norm": 2.665839195251465,
	"learning_rate": 3.49474447477604e-06,
	"loss": 0.7771726131439209,
	"step": 2210
	},
	{
	"epoch": 0.8066860465116279,
	"grad_norm": 2.775423765182495,
	"learning_rate": 3.4809441255126598e-06,
	"loss": 0.7675837993621826,
	"step": 2220
	},
	{
	"epoch": 0.8103197674418605,
	"grad_norm": 2.7208096981048584,
	"learning_rate": 3.4671083543738505e-06,
	"loss": 0.7966558456420898,
	"step": 2230
	},
	{
	"epoch": 0.813953488372093,
	"grad_norm": 2.8475911617279053,
	"learning_rate": 3.453237660969068e-06,
	"loss": 0.772910213470459,
	"step": 2240
	},
	{
	"epoch": 0.8175872093023255,
	"grad_norm": 2.780776023864746,
	"learning_rate": 3.439332546168812e-06,
	"loss": 0.7731334209442139,
	"step": 2250
	},
	{
	"epoch": 0.8212209302325582,
	"grad_norm": 2.746427536010742,
	"learning_rate": 3.425393512086538e-06,
	"loss": 0.7510773658752441,
	"step": 2260
	},
	{
	"epoch": 0.8248546511627907,
	"grad_norm": 2.705579996109009,
	"learning_rate": 3.4114210620605248e-06,
	"loss": 0.7795310974121094,
	"step": 2270
	},
	{
	"epoch": 0.8284883720930233,
	"grad_norm": 2.866590738296509,
	"learning_rate": 3.3974157006357013e-06,
	"loss": 0.7611992835998536,
	"step": 2280
	},
	{
	"epoch": 0.8321220930232558,
	"grad_norm": 2.670555591583252,
	"learning_rate": 3.383377933545429e-06,
	"loss": 0.808956527709961,
	"step": 2290
	},
	{
	"epoch": 0.8357558139534884,
	"grad_norm": 2.7718398571014404,
	"learning_rate": 3.3693082676932326e-06,
	"loss": 0.7825285434722901,
	"step": 2300
	},
	{
	"epoch": 0.8393895348837209,
	"grad_norm": 2.710367441177368,
	"learning_rate": 3.3552072111345047e-06,
	"loss": 0.8079341888427735,
	"step": 2310
	},
	{
	"epoch": 0.8430232558139535,
	"grad_norm": 2.7639403343200684,
	"learning_rate": 3.341075273058153e-06,
	"loss": 0.7904543876647949,
	"step": 2320
	},
	{
	"epoch": 0.846656976744186,
	"grad_norm": 2.618574380874634,
	"learning_rate": 3.3269129637682178e-06,
	"loss": 0.79985671043396,
	"step": 2330
	},
	{
	"epoch": 0.8502906976744186,
	"grad_norm": 2.6449429988861084,
	"learning_rate": 3.3127207946654434e-06,
	"loss": 0.7599962234497071,
	"step": 2340
	},
	{
	"epoch": 0.8539244186046512,
	"grad_norm": 2.722358226776123,
	"learning_rate": 3.29849927822881e-06,
	"loss": 0.7602685451507568,
	"step": 2350
	},
	{
	"epoch": 0.8575581395348837,
	"grad_norm": 2.695909023284912,
	"learning_rate": 3.2842489279970314e-06,
	"loss": 0.7823375701904297,
	"step": 2360
	},
	{
	"epoch": 0.8611918604651163,
	"grad_norm": 2.84134840965271,
	"learning_rate": 3.2699702585500072e-06,
	"loss": 0.8109968185424805,
	"step": 2370
	},
	{
	"epoch": 0.8648255813953488,
	"grad_norm": 2.819700241088867,
	"learning_rate": 3.2556637854902456e-06,
	"loss": 0.7682777404785156,
	"step": 2380
	},
	{
	"epoch": 0.8684593023255814,
	"grad_norm": 2.7188804149627686,
	"learning_rate": 3.2413300254242417e-06,
	"loss": 0.7601710796356201,
	"step": 2390
	},
	{
	"epoch": 0.872093023255814,
	"grad_norm": 2.7124922275543213,
	"learning_rate": 3.2269694959438236e-06,
	"loss": 0.7514790534973145,
	"step": 2400
	},
	{
	"epoch": 0.8757267441860465,
	"grad_norm": 2.796386241912842,
	"learning_rate": 3.212582715607463e-06,
	"loss": 0.8110128402709961,
	"step": 2410
	},
	{
	"epoch": 0.8793604651162791,
	"grad_norm": 2.6567811965942383,
	"learning_rate": 3.1981702039215483e-06,
	"loss": 0.7716230392456055,
	"step": 2420
	},
	{
	"epoch": 0.8829941860465116,
	"grad_norm": 2.7610013484954834,
	"learning_rate": 3.183732481321629e-06,
	"loss": 0.7623504638671875,
	"step": 2430
	},
	{
	"epoch": 0.8866279069767442,
	"grad_norm": 2.900007963180542,
	"learning_rate": 3.1692700691536184e-06,
	"loss": 0.777961540222168,
	"step": 2440
	},
	{
	"epoch": 0.8902616279069767,
	"grad_norm": 2.6529910564422607,
	"learning_rate": 3.154783489654969e-06,
	"loss": 0.7695446491241456,
	"step": 2450
	},
	{
	"epoch": 0.8938953488372093,
	"grad_norm": 2.8663878440856934,
	"learning_rate": 3.1402732659358187e-06,
	"loss": 0.7680397033691406,
	"step": 2460
	},
	{
	"epoch": 0.8975290697674418,
	"grad_norm": 2.582566022872925,
	"learning_rate": 3.1257399219600926e-06,
	"loss": 0.7840737342834473,
	"step": 2470
	},
	{
	"epoch": 0.9011627906976745,
	"grad_norm": 2.6422550678253174,
	"learning_rate": 3.1111839825265933e-06,
	"loss": 0.7447628974914551,
	"step": 2480
	},
	{
	"epoch": 0.904796511627907,
	"grad_norm": 2.8522748947143555,
	"learning_rate": 3.0966059732500396e-06,
	"loss": 0.7944939136505127,
	"step": 2490
	},
	{
	"epoch": 0.9084302325581395,
	"grad_norm": 2.84637188911438,
	"learning_rate": 3.0820064205420967e-06,
	"loss": 0.7697494506835938,
	"step": 2500
	},
	{
	"epoch": 0.9120639534883721,
	"grad_norm": 2.6845076084136963,
	"learning_rate": 3.0673858515923603e-06,
	"loss": 0.7460027694702148,
	"step": 2510
	},
	{
	"epoch": 0.9156976744186046,
	"grad_norm": 2.8455183506011963,
	"learning_rate": 3.052744794349323e-06,
	"loss": 0.7743155479431152,
	"step": 2520
	},
	{
	"epoch": 0.9193313953488372,
	"grad_norm": 2.6090633869171143,
	"learning_rate": 3.0380837775013072e-06,
	"loss": 0.7766456604003906,
	"step": 2530
	},
	{
	"epoch": 0.9229651162790697,
	"grad_norm": 2.987428903579712,
	"learning_rate": 3.0234033304573797e-06,
	"loss": 0.7843479633331298,
	"step": 2540
	},
	{
	"epoch": 0.9265988372093024,
	"grad_norm": 2.794109582901001,
	"learning_rate": 3.0087039833282293e-06,
	"loss": 0.7880901336669922,
	"step": 2550
	},
	{
	"epoch": 0.9302325581395349,
	"grad_norm": 2.7973132133483887,
	"learning_rate": 2.9939862669070274e-06,
	"loss": 0.7407669067382813,
	"step": 2560
	},
	{
	"epoch": 0.9338662790697675,
	"grad_norm": 2.6931514739990234,
	"learning_rate": 2.9792507126502595e-06,
	"loss": 0.7737750053405762,
	"step": 2570
	},
	{
	"epoch": 0.9375,
	"grad_norm": 2.7191874980926514,
	"learning_rate": 2.964497852658535e-06,
	"loss": 0.7814858436584473,
	"step": 2580
	},
	{
	"epoch": 0.9411337209302325,
	"grad_norm": 2.7388243675231934,
	"learning_rate": 2.9497282196573733e-06,
	"loss": 0.7539449214935303,
	"step": 2590
	},
	{
	"epoch": 0.9447674418604651,
	"grad_norm": 2.594496250152588,
	"learning_rate": 2.934942346977967e-06,
	"loss": 0.7599133491516114,
	"step": 2600
	},
	{
	"epoch": 0.9484011627906976,
	"grad_norm": 2.6111090183258057,
	"learning_rate": 2.9201407685379242e-06,
	"loss": 0.7895750999450684,
	"step": 2610
	},
	{
	"epoch": 0.9520348837209303,
	"grad_norm": 2.7242159843444824,
	"learning_rate": 2.905324018821985e-06,
	"loss": 0.7981988430023194,
	"step": 2620
	},
	{
	"epoch": 0.9556686046511628,
	"grad_norm": 2.847475290298462,
	"learning_rate": 2.890492632862728e-06,
	"loss": 0.7751128196716308,
	"step": 2630
	},
	{
	"epoch": 0.9593023255813954,
	"grad_norm": 2.7806103229522705,
	"learning_rate": 2.875647146221241e-06,
	"loss": 0.7592604637145997,
	"step": 2640
	},
	{
	"epoch": 0.9629360465116279,
	"grad_norm": 3.4675211906433105,
	"learning_rate": 2.860788094967793e-06,
	"loss": 0.7890193939208985,
	"step": 2650
	},
	{
	"epoch": 0.9665697674418605,
	"grad_norm": 2.7082395553588867,
	"learning_rate": 2.845916015662467e-06,
	"loss": 0.7717397212982178,
	"step": 2660
	},
	{
	"epoch": 0.970203488372093,
	"grad_norm": 2.694770574569702,
	"learning_rate": 2.8310314453357902e-06,
	"loss": 0.7812085151672363,
	"step": 2670
	},
	{
	"epoch": 0.9738372093023255,
	"grad_norm": 2.8651087284088135,
	"learning_rate": 2.816134921469339e-06,
	"loss": 0.794677734375,
	"step": 2680
	},
	{
	"epoch": 0.9774709302325582,
	"grad_norm": 2.712221384048462,
	"learning_rate": 2.8012269819763337e-06,
	"loss": 0.7712698459625245,
	"step": 2690
	},
	{
	"epoch": 0.9811046511627907,
	"grad_norm": 2.781355381011963,
	"learning_rate": 2.7863081651822122e-06,
	"loss": 0.7705037593841553,
	"step": 2700
	},
	{
	"epoch": 0.9847383720930233,
	"grad_norm": 2.675858974456787,
	"learning_rate": 2.7713790098051906e-06,
	"loss": 0.7476750373840332,
	"step": 2710
	},
	{
	"epoch": 0.9883720930232558,
	"grad_norm": 2.681861639022827,
	"learning_rate": 2.7564400549368127e-06,
	"loss": 0.7630274295806885,
	"step": 2720
	},
	{
	"epoch": 0.9920058139534884,
	"grad_norm": 2.920302629470825,
	"learning_rate": 2.74149184002248e-06,
	"loss": 0.7798184871673584,
	"step": 2730
	},
	{
	"epoch": 0.9956395348837209,
	"grad_norm": 2.8120718002319336,
	"learning_rate": 2.726534904841974e-06,
	"loss": 0.781275749206543,
	"step": 2740
	},
	{
	"epoch": 0.9992732558139535,
	"grad_norm": 2.6712262630462646,
	"learning_rate": 2.7115697894899675e-06,
	"loss": 0.7707889556884766,
	"step": 2750
	},
	{
	"epoch": 1.002906976744186,
	"grad_norm": 2.6090970039367676,
	"learning_rate": 2.696597034356517e-06,
	"loss": 0.6963653564453125,
	"step": 2760
	},
	{
	"epoch": 1.0065406976744187,
	"grad_norm": 2.7611663341522217,
	"learning_rate": 2.6816171801075507e-06,
	"loss": 0.6414544105529785,
	"step": 2770
	},
	{
	"epoch": 1.010174418604651,
	"grad_norm": 2.7799036502838135,
	"learning_rate": 2.6666307676653485e-06,
	"loss": 0.6255013465881347,
	"step": 2780
	},
	{
	"epoch": 1.0138081395348837,
	"grad_norm": 2.7116854190826416,
	"learning_rate": 2.651638338189003e-06,
	"loss": 0.6427628517150878,
	"step": 2790
	},
	{
	"epoch": 1.0174418604651163,
	"grad_norm": 2.8874623775482178,
	"learning_rate": 2.6366404330548847e-06,
	"loss": 0.6351820945739746,
	"step": 2800
	},
	{
	"epoch": 1.021075581395349,
	"grad_norm": 2.93046498298645,
	"learning_rate": 2.6216375938370887e-06,
	"loss": 0.6491835117340088,
	"step": 2810
	},
	{
	"epoch": 1.0247093023255813,
	"grad_norm": 2.82824969291687,
	"learning_rate": 2.6066303622878802e-06,
	"loss": 0.6247024536132812,
	"step": 2820
	},
	{
	"epoch": 1.028343023255814,
	"grad_norm": 2.7935800552368164,
	"learning_rate": 2.591619280318131e-06,
	"loss": 0.6559598922729493,
	"step": 2830
	},
	{
	"epoch": 1.0319767441860466,
	"grad_norm": 2.8328874111175537,
	"learning_rate": 2.5766048899777524e-06,
	"loss": 0.6576976776123047,
	"step": 2840
	},
	{
	"epoch": 1.035610465116279,
	"grad_norm": 2.6769254207611084,
	"learning_rate": 2.561587733436119e-06,
	"loss": 0.6029273986816406,
	"step": 2850
	},
	{
	"epoch": 1.0392441860465116,
	"grad_norm": 2.838825225830078,
	"learning_rate": 2.5465683529624953e-06,
	"loss": 0.6157190322875976,
	"step": 2860
	},
	{
	"epoch": 1.0428779069767442,
	"grad_norm": 2.6545779705047607,
	"learning_rate": 2.5315472909064502e-06,
	"loss": 0.633302640914917,
	"step": 2870
	},
	{
	"epoch": 1.0465116279069768,
	"grad_norm": 2.840080976486206,
	"learning_rate": 2.5165250896782763e-06,
	"loss": 0.6440884113311768,
	"step": 2880
	},
	{
	"epoch": 1.0501453488372092,
	"grad_norm": 2.7946488857269287,
	"learning_rate": 2.5015022917293983e-06,
	"loss": 0.6195040702819824,
	"step": 2890
	},
	{
	"epoch": 1.0537790697674418,
	"grad_norm": 2.7816784381866455,
	"learning_rate": 2.486479439532793e-06,
	"loss": 0.6428169250488281,
	"step": 2900
	},
	{
	"epoch": 1.0574127906976745,
	"grad_norm": 2.7808825969696045,
	"learning_rate": 2.4714570755633915e-06,
	"loss": 0.6089080810546875,
	"step": 2910
	},
	{
	"epoch": 1.0610465116279069,
	"grad_norm": 2.9815218448638916,
	"learning_rate": 2.456435742278499e-06,
	"loss": 0.6232911109924316,
	"step": 2920
	},
	{
	"epoch": 1.0646802325581395,
	"grad_norm": 2.8019156455993652,
	"learning_rate": 2.441415982098201e-06,
	"loss": 0.644099760055542,
	"step": 2930
	},
	{
	"epoch": 1.068313953488372,
	"grad_norm": 2.777388095855713,
	"learning_rate": 2.4263983373857773e-06,
	"loss": 0.6139822483062745,
	"step": 2940
	},
	{
	"epoch": 1.0719476744186047,
	"grad_norm": 2.5681188106536865,
	"learning_rate": 2.4113833504281187e-06,
	"loss": 0.6084999084472656,
	"step": 2950
	},
	{
	"epoch": 1.0755813953488371,
	"grad_norm": 2.812208652496338,
	"learning_rate": 2.396371563416146e-06,
	"loss": 0.6468010902404785,
	"step": 2960
	},
	{
	"epoch": 1.0792151162790697,
	"grad_norm": 2.7401721477508545,
	"learning_rate": 2.3813635184252285e-06,
	"loss": 0.6259693622589111,
	"step": 2970
	},
	{
	"epoch": 1.0828488372093024,
	"grad_norm": 2.7435302734375,
	"learning_rate": 2.3663597573956113e-06,
	"loss": 0.6502357482910156,
	"step": 2980
	},
	{
	"epoch": 1.086482558139535,
	"grad_norm": 2.9690780639648438,
	"learning_rate": 2.3513608221128477e-06,
	"loss": 0.6185351848602295,
	"step": 2990
	},
	{
	"epoch": 1.0901162790697674,
	"grad_norm": 2.996502161026001,
	"learning_rate": 2.336367254188229e-06,
	"loss": 0.6417867183685303,
	"step": 3000
	},
	{
	"epoch": 1.09375,
	"grad_norm": 2.8609607219696045,
	"learning_rate": 2.3213795950392355e-06,
	"loss": 0.6164373397827149,
	"step": 3010
	},
	{
	"epoch": 1.0973837209302326,
	"grad_norm": 2.8891489505767822,
	"learning_rate": 2.3063983858699808e-06,
	"loss": 0.6243292808532714,
	"step": 3020
	},
	{
	"epoch": 1.101017441860465,
	"grad_norm": 2.7142832279205322,
	"learning_rate": 2.291424167651668e-06,
	"loss": 0.6159169197082519,
	"step": 3030
	},
	{
	"epoch": 1.1046511627906976,
	"grad_norm": 2.84442400932312,
	"learning_rate": 2.2764574811030597e-06,
	"loss": 0.6410219192504882,
	"step": 3040
	},
	{
	"epoch": 1.1082848837209303,
	"grad_norm": 2.816093921661377,
	"learning_rate": 2.2614988666709464e-06,
	"loss": 0.6166658401489258,
	"step": 3050
	},
	{
	"epoch": 1.1119186046511629,
	"grad_norm": 2.8112494945526123,
	"learning_rate": 2.24654886451064e-06,
	"loss": 0.6123160362243653,
	"step": 3060
	},
	{
	"epoch": 1.1155523255813953,
	"grad_norm": 2.8347339630126953,
	"learning_rate": 2.2316080144664577e-06,
	"loss": 0.6261919498443603,
	"step": 3070
	},
	{
	"epoch": 1.119186046511628,
	"grad_norm": 3.0708513259887695,
	"learning_rate": 2.216676856052237e-06,
	"loss": 0.6543515205383301,
	"step": 3080
	},
	{
	"epoch": 1.1228197674418605,
	"grad_norm": 2.624223470687866,
	"learning_rate": 2.2017559284318517e-06,
	"loss": 0.647164249420166,
	"step": 3090
	},
	{
	"epoch": 1.1264534883720931,
	"grad_norm": 2.8848209381103516,
	"learning_rate": 2.18684577039974e-06,
	"loss": 0.6185880661010742,
	"step": 3100
	},
	{
	"epoch": 1.1300872093023255,
	"grad_norm": 2.8103585243225098,
	"learning_rate": 2.171946920361449e-06,
	"loss": 0.6510479927062989,
	"step": 3110
	},
	{
	"epoch": 1.1337209302325582,
	"grad_norm": 2.793445348739624,
	"learning_rate": 2.157059916314199e-06,
	"loss": 0.6501373291015625,
	"step": 3120
	},
	{
	"epoch": 1.1373546511627908,
	"grad_norm": 2.8822362422943115,
	"learning_rate": 2.142185295827448e-06,
	"loss": 0.6411583900451661,
	"step": 3130
	},
	{
	"epoch": 1.1409883720930232,
	"grad_norm": 2.8682944774627686,
	"learning_rate": 2.1273235960234844e-06,
	"loss": 0.616573715209961,
	"step": 3140
	},
	{
	"epoch": 1.1446220930232558,
	"grad_norm": 2.8865904808044434,
	"learning_rate": 2.1124753535580308e-06,
	"loss": 0.6400615215301514,
	"step": 3150
	},
	{
	"epoch": 1.1482558139534884,
	"grad_norm": 2.9084205627441406,
	"learning_rate": 2.0976411046008683e-06,
	"loss": 0.6371217727661133,
	"step": 3160
	},
	{
	"epoch": 1.151889534883721,
	"grad_norm": 2.7648379802703857,
	"learning_rate": 2.0828213848164693e-06,
	"loss": 0.6138511180877686,
	"step": 3170
	},
	{
	"epoch": 1.1555232558139534,
	"grad_norm": 2.8824777603149414,
	"learning_rate": 2.0680167293446585e-06,
	"loss": 0.6189014434814453,
	"step": 3180
	},
	{
	"epoch": 1.159156976744186,
	"grad_norm": 2.6267261505126953,
	"learning_rate": 2.05322767278129e-06,
	"loss": 0.6297481060028076,
	"step": 3190
	},
	{
	"epoch": 1.1627906976744187,
	"grad_norm": 2.7822842597961426,
	"learning_rate": 2.0384547491589416e-06,
	"loss": 0.6191432476043701,
	"step": 3200
	},
	{
	"epoch": 1.166424418604651,
	"grad_norm": 6.852417469024658,
	"learning_rate": 2.023698491927629e-06,
	"loss": 0.619715404510498,
	"step": 3210
	},
	{
	"epoch": 1.1700581395348837,
	"grad_norm": 2.771573781967163,
	"learning_rate": 2.008959433935547e-06,
	"loss": 0.6091080665588379,
	"step": 3220
	},
	{
	"epoch": 1.1736918604651163,
	"grad_norm": 2.977626085281372,
	"learning_rate": 1.9942381074098248e-06,
	"loss": 0.6197806358337402,
	"step": 3230
	},
	{
	"epoch": 1.177325581395349,
	"grad_norm": 2.740381956100464,
	"learning_rate": 1.9795350439373085e-06,
	"loss": 0.6083920001983643,
	"step": 3240
	},
	{
	"epoch": 1.1809593023255813,
	"grad_norm": 2.862666606903076,
	"learning_rate": 1.9648507744453657e-06,
	"loss": 0.6217282295227051,
	"step": 3250
	},
	{
	"epoch": 1.184593023255814,
	"grad_norm": 2.7649455070495605,
	"learning_rate": 1.9501858291827157e-06,
	"loss": 0.637911319732666,
	"step": 3260
	},
	{
	"epoch": 1.1882267441860466,
	"grad_norm": 2.8812625408172607,
	"learning_rate": 1.935540737700278e-06,
	"loss": 0.6026611804962159,
	"step": 3270
	},
	{
	"epoch": 1.191860465116279,
	"grad_norm": 2.8181381225585938,
	"learning_rate": 1.9209160288320517e-06,
	"loss": 0.6029317378997803,
	"step": 3280
	},
	{
	"epoch": 1.1954941860465116,
	"grad_norm": 2.935403823852539,
	"learning_rate": 1.906312230676024e-06,
	"loss": 0.645781135559082,
	"step": 3290
	},
	{
	"epoch": 1.1991279069767442,
	"grad_norm": 2.8169732093811035,
	"learning_rate": 1.891729870575092e-06,
	"loss": 0.6284616470336915,
	"step": 3300
	},
	{
	"epoch": 1.2027616279069768,
	"grad_norm": 2.8880012035369873,
	"learning_rate": 1.8771694750980277e-06,
	"loss": 0.6369534492492676,
	"step": 3310
	},
	{
	"epoch": 1.2063953488372092,
	"grad_norm": 2.939326286315918,
	"learning_rate": 1.8626315700204606e-06,
	"loss": 0.6241387367248535,
	"step": 3320
	},
	{
	"epoch": 1.2100290697674418,
	"grad_norm": 2.9609262943267822,
	"learning_rate": 1.848116680305892e-06,
	"loss": 0.6426369190216065,
	"step": 3330
	},
	{
	"epoch": 1.2136627906976745,
	"grad_norm": 2.8644490242004395,
	"learning_rate": 1.833625330086739e-06,
	"loss": 0.6207024574279785,
	"step": 3340
	},
	{
	"epoch": 1.2172965116279069,
	"grad_norm": 2.843106508255005,
	"learning_rate": 1.8191580426454064e-06,
	"loss": 0.6336426734924316,
	"step": 3350
	},
	{
	"epoch": 1.2209302325581395,
	"grad_norm": 3.1315815448760986,
	"learning_rate": 1.8047153403953948e-06,
	"loss": 0.6345052719116211,
	"step": 3360
	},
	{
	"epoch": 1.224563953488372,
	"grad_norm": 2.949122905731201,
	"learning_rate": 1.7902977448624334e-06,
	"loss": 0.6444235801696777,
	"step": 3370
	},
	{
	"epoch": 1.2281976744186047,
	"grad_norm": 2.835570812225342,
	"learning_rate": 1.7759057766656468e-06,
	"loss": 0.6098015785217286,
	"step": 3380
	},
	{
	"epoch": 1.2318313953488371,
	"grad_norm": 2.9221482276916504,
	"learning_rate": 1.7615399554987588e-06,
	"loss": 0.6236421585083007,
	"step": 3390
	},
	{
	"epoch": 1.2354651162790697,
	"grad_norm": 2.8423030376434326,
	"learning_rate": 1.7472008001113244e-06,
	"loss": 0.6142833709716797,
	"step": 3400
	},
	{
	"epoch": 1.2390988372093024,
	"grad_norm": 2.8265531063079834,
	"learning_rate": 1.7328888282899956e-06,
	"loss": 0.6144848346710206,
	"step": 3410
	},
	{
	"epoch": 1.2427325581395348,
	"grad_norm": 2.9081876277923584,
	"learning_rate": 1.7186045568398274e-06,
	"loss": 0.6280595779418945,
	"step": 3420
	},
	{
	"epoch": 1.2463662790697674,
	"grad_norm": 3.0004329681396484,
	"learning_rate": 1.7043485015656164e-06,
	"loss": 0.6482262134552002,
	"step": 3430
	},
	{
	"epoch": 1.25,
	"grad_norm": 2.7952747344970703,
	"learning_rate": 1.6901211772532717e-06,
	"loss": 0.6031205654144287,
	"step": 3440
	},
	{
	"epoch": 1.2536337209302326,
	"grad_norm": 3.007476329803467,
	"learning_rate": 1.675923097651228e-06,
	"loss": 0.648679256439209,
	"step": 3450
	},
	{
	"epoch": 1.2572674418604652,
	"grad_norm": 2.8375749588012695,
	"learning_rate": 1.6617547754518964e-06,
	"loss": 0.6462221622467041,
	"step": 3460
	},
	{
	"epoch": 1.2609011627906976,
	"grad_norm": 2.735497236251831,
	"learning_rate": 1.6476167222731448e-06,
	"loss": 0.6184795379638672,
	"step": 3470
	},
	{
	"epoch": 1.2645348837209303,
	"grad_norm": 3.0197677612304688,
	"learning_rate": 1.6335094486398295e-06,
	"loss": 0.6254825115203857,
	"step": 3480
	},
	{
	"epoch": 1.2681686046511627,
	"grad_norm": 2.9058151245117188,
	"learning_rate": 1.6194334639653588e-06,
	"loss": 0.6179644584655761,
	"step": 3490
	},
	{
	"epoch": 1.2718023255813953,
	"grad_norm": 2.8195924758911133,
	"learning_rate": 1.6053892765332957e-06,
	"loss": 0.6161976814270019,
	"step": 3500
	},
	{
	"epoch": 1.275436046511628,
	"grad_norm": 2.803330421447754,
	"learning_rate": 1.5913773934790062e-06,
	"loss": 0.6237835884094238,
	"step": 3510
	},
	{
	"epoch": 1.2790697674418605,
	"grad_norm": 2.789128303527832,
	"learning_rate": 1.577398320771344e-06,
	"loss": 0.6449883460998536,
	"step": 3520
	},
	{
	"epoch": 1.2827034883720931,
	"grad_norm": 2.9578816890716553,
	"learning_rate": 1.5634525631943844e-06,
	"loss": 0.6117482662200928,
	"step": 3530
	},
	{
	"epoch": 1.2863372093023255,
	"grad_norm": 2.8928158283233643,
	"learning_rate": 1.5495406243291925e-06,
	"loss": 0.6140883445739747,
	"step": 3540
	},
	{
	"epoch": 1.2899709302325582,
	"grad_norm": 2.741936206817627,
	"learning_rate": 1.5356630065356395e-06,
	"loss": 0.6419960498809815,
	"step": 3550
	},
	{
	"epoch": 1.2936046511627908,
	"grad_norm": 2.890014171600342,
	"learning_rate": 1.5218202109342655e-06,
	"loss": 0.5909524440765381,
	"step": 3560
	},
	{
	"epoch": 1.2972383720930232,
	"grad_norm": 2.7969582080841064,
	"learning_rate": 1.508012737388181e-06,
	"loss": 0.5948357582092285,
	"step": 3570
	},
	{
	"epoch": 1.3008720930232558,
	"grad_norm": 2.8763949871063232,
	"learning_rate": 1.4942410844850144e-06,
	"loss": 0.6192843914031982,
	"step": 3580
	},
	{
	"epoch": 1.3045058139534884,
	"grad_norm": 2.978353977203369,
	"learning_rate": 1.4805057495189178e-06,
	"loss": 0.614830493927002,
	"step": 3590
	},
	{
	"epoch": 1.308139534883721,
	"grad_norm": 2.9594194889068604,
	"learning_rate": 1.4668072284725974e-06,
	"loss": 0.6267194271087646,
	"step": 3600
	},
	{
	"epoch": 1.3117732558139534,
	"grad_norm": 2.777967691421509,
	"learning_rate": 1.4531460159994125e-06,
	"loss": 0.6032927513122559,
	"step": 3610
	},
	{
	"epoch": 1.315406976744186,
	"grad_norm": 2.823629856109619,
	"learning_rate": 1.4395226054055084e-06,
	"loss": 0.6197312355041504,
	"step": 3620
	},
	{
	"epoch": 1.3190406976744187,
	"grad_norm": 2.972522735595703,
	"learning_rate": 1.4259374886320064e-06,
	"loss": 0.6175290107727051,
	"step": 3630
	},
	{
	"epoch": 1.322674418604651,
	"grad_norm": 3.864912271499634,
	"learning_rate": 1.4123911562372411e-06,
	"loss": 0.5972983360290527,
	"step": 3640
	},
	{
	"epoch": 1.3263081395348837,
	"grad_norm": 2.642355442047119,
	"learning_rate": 1.398884097379038e-06,
	"loss": 0.6334139823913574,
	"step": 3650
	},
	{
	"epoch": 1.3299418604651163,
	"grad_norm": 2.843428134918213,
	"learning_rate": 1.3854167997970607e-06,
	"loss": 0.6349598407745362,
	"step": 3660
	},
	{
	"epoch": 1.333575581395349,
	"grad_norm": 2.9344217777252197,
	"learning_rate": 1.3719897497951934e-06,
	"loss": 0.6205593585968018,
	"step": 3670
	},
	{
	"epoch": 1.3372093023255813,
	"grad_norm": 2.964674949645996,
	"learning_rate": 1.3586034322239777e-06,
	"loss": 0.6372932910919189,
	"step": 3680
	},
	{
	"epoch": 1.340843023255814,
	"grad_norm": 2.9677414894104004,
	"learning_rate": 1.3452583304631118e-06,
	"loss": 0.5919643402099609,
	"step": 3690
	},
	{
	"epoch": 1.3444767441860466,
	"grad_norm": 2.970325469970703,
	"learning_rate": 1.3319549264039874e-06,
	"loss": 0.6072146415710449,
	"step": 3700
	},
	{
	"epoch": 1.348110465116279,
	"grad_norm": 2.8650684356689453,
	"learning_rate": 1.318693700432298e-06,
	"loss": 0.6300556182861328,
	"step": 3710
	},
	{
	"epoch": 1.3517441860465116,
	"grad_norm": 3.107119083404541,
	"learning_rate": 1.305475131410682e-06,
	"loss": 0.6029188156127929,
	"step": 3720
	},
	{
	"epoch": 1.3553779069767442,
	"grad_norm": 2.9079434871673584,
	"learning_rate": 1.2922996966614407e-06,
	"loss": 0.6184733390808106,
	"step": 3730
	},
	{
	"epoch": 1.3590116279069768,
	"grad_norm": 2.755070924758911,
	"learning_rate": 1.2791678719492931e-06,
	"loss": 0.6277907371520997,
	"step": 3740
	},
	{
	"epoch": 1.3626453488372092,
	"grad_norm": 3.044586658477783,
	"learning_rate": 1.2660801314642038e-06,
	"loss": 0.6256103515625,
	"step": 3750
	},
	{
	"epoch": 1.3662790697674418,
	"grad_norm": 2.9525163173675537,
	"learning_rate": 1.2530369478042567e-06,
	"loss": 0.6190742015838623,
	"step": 3760
	},
	{
	"epoch": 1.3699127906976745,
	"grad_norm": 2.899778366088867,
	"learning_rate": 1.2400387919585876e-06,
	"loss": 0.6287983894348145,
	"step": 3770
	},
	{
	"epoch": 1.3735465116279069,
	"grad_norm": 2.8771963119506836,
	"learning_rate": 1.2270861332903774e-06,
	"loss": 0.6238520622253418,
	"step": 3780
	},
	{
	"epoch": 1.3771802325581395,
	"grad_norm": 2.7630653381347656,
	"learning_rate": 1.2141794395199112e-06,
	"loss": 0.6069451332092285,
	"step": 3790
	},
	{
	"epoch": 1.380813953488372,
	"grad_norm": 2.847613573074341,
	"learning_rate": 1.2013191767076756e-06,
	"loss": 0.6269810676574707,
	"step": 3800
	},
	{
	"epoch": 1.3844476744186047,
	"grad_norm": 2.879697799682617,
	"learning_rate": 1.1885058092375385e-06,
	"loss": 0.6266268730163574,
	"step": 3810
	},
	{
	"epoch": 1.3880813953488373,
	"grad_norm": 2.9201645851135254,
	"learning_rate": 1.1757397997999778e-06,
	"loss": 0.6043568611145019,
	"step": 3820
	},
	{
	"epoch": 1.3917151162790697,
	"grad_norm": 3.0528650283813477,
	"learning_rate": 1.1630216093753756e-06,
	"loss": 0.6078732490539551,
	"step": 3830
	},
	{
	"epoch": 1.3953488372093024,
	"grad_norm": 2.8742434978485107,
	"learning_rate": 1.1503516972173675e-06,
	"loss": 0.6376693248748779,
	"step": 3840
	},
	{
	"epoch": 1.3989825581395348,
	"grad_norm": 2.934420585632324,
	"learning_rate": 1.1377305208362613e-06,
	"loss": 0.594962215423584,
	"step": 3850
	},
	{
	"epoch": 1.4026162790697674,
	"grad_norm": 2.923924684524536,
	"learning_rate": 1.1251585359825193e-06,
	"loss": 0.6222631454467773,
	"step": 3860
	},
	{
	"epoch": 1.40625,
	"grad_norm": 2.8981270790100098,
	"learning_rate": 1.1126361966302983e-06,
	"loss": 0.6129872322082519,
	"step": 3870
	},
	{
	"epoch": 1.4098837209302326,
	"grad_norm": 2.969998598098755,
	"learning_rate": 1.1001639549610546e-06,
	"loss": 0.6390999794006348,
	"step": 3880
	},
	{
	"epoch": 1.4135174418604652,
	"grad_norm": 2.8989439010620117,
	"learning_rate": 1.0877422613472196e-06,
	"loss": 0.5968554496765137,
	"step": 3890
	},
	{
	"epoch": 1.4171511627906976,
	"grad_norm": 2.928755044937134,
	"learning_rate": 1.075371564335937e-06,
	"loss": 0.6382357597351074,
	"step": 3900
	},
	{
	"epoch": 1.4207848837209303,
	"grad_norm": 2.8408379554748535,
	"learning_rate": 1.0630523106328605e-06,
	"loss": 0.6284653186798096,
	"step": 3910
	},
	{
	"epoch": 1.4244186046511627,
	"grad_norm": 3.0078012943267822,
	"learning_rate": 1.0507849450860277e-06,
	"loss": 0.6212825775146484,
	"step": 3920
	},
	{
	"epoch": 1.4280523255813953,
	"grad_norm": 2.981374979019165,
	"learning_rate": 1.0385699106697983e-06,
	"loss": 0.6222080230712891,
	"step": 3930
	},
	{
	"epoch": 1.431686046511628,
	"grad_norm": 2.9854984283447266,
	"learning_rate": 1.026407648468854e-06,
	"loss": 0.6164888858795166,
	"step": 3940
	},
	{
	"epoch": 1.4353197674418605,
	"grad_norm": 2.661363124847412,
	"learning_rate": 1.0142985976622718e-06,
	"loss": 0.6039540767669678,
	"step": 3950
	},
	{
	"epoch": 1.4389534883720931,
	"grad_norm": 3.1405701637268066,
	"learning_rate": 1.0022431955076686e-06,
	"loss": 0.6292922973632813,
	"step": 3960
	},
	{
	"epoch": 1.4425872093023255,
	"grad_norm": 2.916566848754883,
	"learning_rate": 9.902418773254064e-07,
	"loss": 0.6347232341766358,
	"step": 3970
	},
	{
	"epoch": 1.4462209302325582,
	"grad_norm": 2.7745325565338135,
	"learning_rate": 9.782950764828788e-07,
	"loss": 0.6333943367004394,
	"step": 3980
	},
	{
	"epoch": 1.4498546511627908,
	"grad_norm": 2.7898178100585938,
	"learning_rate": 9.664032243788565e-07,
	"loss": 0.6126363754272461,
	"step": 3990
	},
	{
	"epoch": 1.4534883720930232,
	"grad_norm": 3.018996238708496,
	"learning_rate": 9.545667504279147e-07,
	"loss": 0.6129265785217285,
	"step": 4000
	},
	{
	"epoch": 1.4571220930232558,
	"grad_norm": 2.880807638168335,
	"learning_rate": 9.427860820449236e-07,
	"loss": 0.6031558990478516,
	"step": 4010
	},
	{
	"epoch": 1.4607558139534884,
	"grad_norm": 2.9041407108306885,
	"learning_rate": 9.310616446296139e-07,
	"loss": 0.6169293880462646,
	"step": 4020
	},
	{
	"epoch": 1.464389534883721,
	"grad_norm": 2.7830722332000732,
	"learning_rate": 9.193938615512193e-07,
	"loss": 0.6310886383056641,
	"step": 4030
	},
	{
	"epoch": 1.4680232558139534,
	"grad_norm": 2.8672258853912354,
	"learning_rate": 9.077831541331842e-07,
	"loss": 0.6097919464111328,
	"step": 4040
	},
	{
	"epoch": 1.471656976744186,
	"grad_norm": 3.0425376892089844,
	"learning_rate": 8.962299416379533e-07,
	"loss": 0.6108634948730469,
	"step": 4050
	},
	{
	"epoch": 1.4752906976744187,
	"grad_norm": 3.1145591735839844,
	"learning_rate": 8.847346412518312e-07,
	"loss": 0.6188015937805176,
	"step": 4060
	},
	{
	"epoch": 1.478924418604651,
	"grad_norm": 2.893653631210327,
	"learning_rate": 8.732976680699153e-07,
	"loss": 0.6100796699523926,
	"step": 4070
	},
	{
	"epoch": 1.4825581395348837,
	"grad_norm": 2.7886803150177,
	"learning_rate": 8.619194350811094e-07,
	"loss": 0.6053628921508789,
	"step": 4080
	},
	{
	"epoch": 1.4861918604651163,
	"grad_norm": 3.087308168411255,
	"learning_rate": 8.506003531532103e-07,
	"loss": 0.6025334358215332,
	"step": 4090
	},
	{
	"epoch": 1.489825581395349,
	"grad_norm": 2.927269220352173,
	"learning_rate": 8.393408310180722e-07,
	"loss": 0.6248285293579101,
	"step": 4100
	},
	{
	"epoch": 1.4934593023255813,
	"grad_norm": 2.8830156326293945,
	"learning_rate": 8.281412752568435e-07,
	"loss": 0.601197338104248,
	"step": 4110
	},
	{
	"epoch": 1.497093023255814,
	"grad_norm": 2.9723641872406006,
	"learning_rate": 8.170020902852896e-07,
	"loss": 0.6211593627929688,
	"step": 4120
	},
	{
	"epoch": 1.5007267441860463,
	"grad_norm": 2.9226901531219482,
	"learning_rate": 8.059236783391885e-07,
	"loss": 0.6135400295257568,
	"step": 4130
	},
	{
	"epoch": 1.504360465116279,
	"grad_norm": 2.7875003814697266,
	"learning_rate": 7.949064394598033e-07,
	"loss": 0.6013899326324463,
	"step": 4140
	},
	{
	"epoch": 1.5079941860465116,
	"grad_norm": 2.9814796447753906,
	"learning_rate": 7.839507714794389e-07,
	"loss": 0.5883074283599854,
	"step": 4150
	},
	{
	"epoch": 1.5116279069767442,
	"grad_norm": 2.743788242340088,
	"learning_rate": 7.73057070007078e-07,
	"loss": 0.6389053344726563,
	"step": 4160
	},
	{
	"epoch": 1.5152616279069768,
	"grad_norm": 2.8575990200042725,
	"learning_rate": 7.622257284140927e-07,
	"loss": 0.6024910926818847,
	"step": 4170
	},
	{
	"epoch": 1.5188953488372094,
	"grad_norm": 2.818049907684326,
	"learning_rate": 7.514571378200408e-07,
	"loss": 0.6210941314697266,
	"step": 4180
	},
	{
	"epoch": 1.5225290697674418,
	"grad_norm": 2.9408257007598877,
	"learning_rate": 7.407516870785405e-07,
	"loss": 0.6397895812988281,
	"step": 4190
	},
	{
	"epoch": 1.5261627906976745,
	"grad_norm": 3.026486396789551,
	"learning_rate": 7.301097627632367e-07,
	"loss": 0.607673454284668,
	"step": 4200
	},
	{
	"epoch": 1.5297965116279069,
	"grad_norm": 2.9318480491638184,
	"learning_rate": 7.195317491538308e-07,
	"loss": 0.5777746677398682,
	"step": 4210
	},
	{
	"epoch": 1.5334302325581395,
	"grad_norm": 2.8912055492401123,
	"learning_rate": 7.090180282222109e-07,
	"loss": 0.6106159210205078,
	"step": 4220
	},
	{
	"epoch": 1.537063953488372,
	"grad_norm": 2.863584280014038,
	"learning_rate": 6.985689796186593e-07,
	"loss": 0.6015161037445068,
	"step": 4230
	},
	{
	"epoch": 1.5406976744186047,
	"grad_norm": 2.8779637813568115,
	"learning_rate": 6.881849806581405e-07,
	"loss": 0.5897069931030273,
	"step": 4240
	},
	{
	"epoch": 1.5443313953488373,
	"grad_norm": 2.8327910900115967,
	"learning_rate": 6.778664063066773e-07,
	"loss": 0.5938574790954589,
	"step": 4250
	},
	{
	"epoch": 1.5479651162790697,
	"grad_norm": 2.9977948665618896,
	"learning_rate": 6.676136291678118e-07,
	"loss": 0.583928394317627,
	"step": 4260
	},
	{
	"epoch": 1.5515988372093024,
	"grad_norm": 3.029289722442627,
	"learning_rate": 6.574270194691485e-07,
	"loss": 0.6218618392944336,
	"step": 4270
	},
	{
	"epoch": 1.5552325581395348,
	"grad_norm": 2.750098943710327,
	"learning_rate": 6.473069450489888e-07,
	"loss": 0.6067587375640869,
	"step": 4280
	},
	{
	"epoch": 1.5588662790697674,
	"grad_norm": 3.033493757247925,
	"learning_rate": 6.37253771343044e-07,
	"loss": 0.6148815631866456,
	"step": 4290
	},
	{
	"epoch": 1.5625,
	"grad_norm": 2.889744758605957,
	"learning_rate": 6.272678613712437e-07,
	"loss": 0.6183005332946777,
	"step": 4300
	},
	{
	"epoch": 1.5661337209302326,
	"grad_norm": 3.086649179458618,
	"learning_rate": 6.173495757246251e-07,
	"loss": 0.5992136001586914,
	"step": 4310
	},
	{
	"epoch": 1.5697674418604652,
	"grad_norm": 2.90238618850708,
	"learning_rate": 6.074992725523102e-07,
	"loss": 0.6303570747375489,
	"step": 4320
	},
	{
	"epoch": 1.5734011627906976,
	"grad_norm": 2.995821475982666,
	"learning_rate": 5.977173075485778e-07,
	"loss": 0.5939804077148437,
	"step": 4330
	},
	{
	"epoch": 1.5770348837209303,
	"grad_norm": 2.9475557804107666,
	"learning_rate": 5.880040339400142e-07,
	"loss": 0.6166177749633789,
	"step": 4340
	},
	{
	"epoch": 1.5806686046511627,
	"grad_norm": 3.1026580333709717,
	"learning_rate": 5.783598024727618e-07,
	"loss": 0.6242012500762939,
	"step": 4350
	},
	{
	"epoch": 1.5843023255813953,
	"grad_norm": 2.9524173736572266,
	"learning_rate": 5.687849613998534e-07,
	"loss": 0.6128497123718262,
	"step": 4360
	},
	{
	"epoch": 1.587936046511628,
	"grad_norm": 3.0528416633605957,
	"learning_rate": 5.592798564686339e-07,
	"loss": 0.6200154304504395,
	"step": 4370
	},
	{
	"epoch": 1.5915697674418605,
	"grad_norm": 3.027564525604248,
	"learning_rate": 5.498448309082777e-07,
	"loss": 0.6254747390747071,
	"step": 4380
	},
	{
	"epoch": 1.5952034883720931,
	"grad_norm": 2.8900911808013916,
	"learning_rate": 5.40480225417395e-07,
	"loss": 0.6331692695617676,
	"step": 4390
	},
	{
	"epoch": 1.5988372093023255,
	"grad_norm": 2.9777119159698486,
	"learning_rate": 5.311863781517296e-07,
	"loss": 0.5979508399963379,
	"step": 4400
	},
	{
	"epoch": 1.6024709302325582,
	"grad_norm": 2.78714919090271,
	"learning_rate": 5.219636247119448e-07,
	"loss": 0.6151116371154786,
	"step": 4410
	},
	{
	"epoch": 1.6061046511627906,
	"grad_norm": 2.840519666671753,
	"learning_rate": 5.128122981315068e-07,
	"loss": 0.6057453155517578,
	"step": 4420
	},
	{
	"epoch": 1.6097383720930232,
	"grad_norm": 2.6660914421081543,
	"learning_rate": 5.03732728864663e-07,
	"loss": 0.6372000694274902,
	"step": 4430
	},
	{
	"epoch": 1.6133720930232558,
	"grad_norm": 2.752408504486084,
	"learning_rate": 4.947252447745013e-07,
	"loss": 0.6181222438812256,
	"step": 4440
	},
	{
	"epoch": 1.6170058139534884,
	"grad_norm": 2.874816417694092,
	"learning_rate": 4.857901711211161e-07,
	"loss": 0.6298707962036133,
	"step": 4450
	},
	{
	"epoch": 1.620639534883721,
	"grad_norm": 2.794626474380493,
	"learning_rate": 4.769278305498632e-07,
	"loss": 0.6108518600463867,
	"step": 4460
	},
	{
	"epoch": 1.6242732558139537,
	"grad_norm": 2.8354408740997314,
	"learning_rate": 4.681385430797086e-07,
	"loss": 0.6353400230407715,
	"step": 4470
	},
	{
	"epoch": 1.627906976744186,
	"grad_norm": 2.9009554386138916,
	"learning_rate": 4.594226260916701e-07,
	"loss": 0.6042340278625489,
	"step": 4480
	},
	{
	"epoch": 1.6315406976744184,
	"grad_norm": 2.8754899501800537,
	"learning_rate": 4.507803943173592e-07,
	"loss": 0.6066765308380127,
	"step": 4490
	},
	{
	"epoch": 1.635174418604651,
	"grad_norm": 2.9461255073547363,
	"learning_rate": 4.422121598276161e-07,
	"loss": 0.6181964874267578,
	"step": 4500
	},
	{
	"epoch": 1.6388081395348837,
	"grad_norm": 2.949796438217163,
	"learning_rate": 4.3371823202124156e-07,
	"loss": 0.602027177810669,
	"step": 4510
	},
	{
	"epoch": 1.6424418604651163,
	"grad_norm": 2.9036405086517334,
	"learning_rate": 4.2529891761382e-07,
	"loss": 0.5935526847839355,
	"step": 4520
	},
	{
	"epoch": 1.646075581395349,
	"grad_norm": 2.956812620162964,
	"learning_rate": 4.1695452062665067e-07,
	"loss": 0.6340709686279297,
	"step": 4530
	},
	{
	"epoch": 1.6497093023255816,
	"grad_norm": 2.9781763553619385,
	"learning_rate": 4.0868534237576476e-07,
	"loss": 0.6047008514404297,
	"step": 4540
	},
	{
	"epoch": 1.653343023255814,
	"grad_norm": 2.7890918254852295,
	"learning_rate": 4.0049168146104487e-07,
	"loss": 0.6058784484863281,
	"step": 4550
	},
	{
	"epoch": 1.6569767441860463,
	"grad_norm": 2.9034600257873535,
	"learning_rate": 3.923738337554467e-07,
	"loss": 0.6007349967956543,
	"step": 4560
	},
	{
	"epoch": 1.660610465116279,
	"grad_norm": 2.934140205383301,
	"learning_rate": 3.843320923943089e-07,
	"loss": 0.5974663734436035,
	"step": 4570
	},
	{
	"epoch": 1.6642441860465116,
	"grad_norm": 2.7771620750427246,
	"learning_rate": 3.7636674776477507e-07,
	"loss": 0.6011982440948487,
	"step": 4580
	},
	{
	"epoch": 1.6678779069767442,
	"grad_norm": 2.8858895301818848,
	"learning_rate": 3.6847808749530005e-07,
	"loss": 0.6234261512756347,
	"step": 4590
	},
	{
	"epoch": 1.6715116279069768,
	"grad_norm": 2.909925937652588,
	"learning_rate": 3.6066639644527195e-07,
	"loss": 0.6237231254577636,
	"step": 4600
	},
	{
	"epoch": 1.6751453488372094,
	"grad_norm": 2.912471055984497,
	"learning_rate": 3.529319566947184e-07,
	"loss": 0.6141771793365478,
	"step": 4610
	},
	{
	"epoch": 1.6787790697674418,
	"grad_norm": 2.846323013305664,
	"learning_rate": 3.4527504753412517e-07,
	"loss": 0.6363746643066406,
	"step": 4620
	},
	{
	"epoch": 1.6824127906976745,
	"grad_norm": 2.9305002689361572,
	"learning_rate": 3.3769594545435103e-07,
	"loss": 0.5971851348876953,
	"step": 4630
	},
	{
	"epoch": 1.6860465116279069,
	"grad_norm": 2.954364776611328,
	"learning_rate": 3.301949241366398e-07,
	"loss": 0.5849172592163085,
	"step": 4640
	},
	{
	"epoch": 1.6896802325581395,
	"grad_norm": 2.9237112998962402,
	"learning_rate": 3.227722544427428e-07,
	"loss": 0.5754762649536133,
	"step": 4650
	},
	{
	"epoch": 1.693313953488372,
	"grad_norm": 2.8475701808929443,
	"learning_rate": 3.154282044051335e-07,
	"loss": 0.570661211013794,
	"step": 4660
	},
	{
	"epoch": 1.6969476744186047,
	"grad_norm": 3.0029714107513428,
	"learning_rate": 3.081630392173329e-07,
	"loss": 0.6232757091522216,
	"step": 4670
	},
	{
	"epoch": 1.7005813953488373,
	"grad_norm": 3.1450560092926025,
	"learning_rate": 3.009770212243293e-07,
	"loss": 0.6245338439941406,
	"step": 4680
	},
	{
	"epoch": 1.7042151162790697,
	"grad_norm": 2.9455108642578125,
	"learning_rate": 2.93870409913109e-07,
	"loss": 0.6269196510314942,
	"step": 4690
	},
	{
	"epoch": 1.7078488372093024,
	"grad_norm": 2.832939386367798,
	"learning_rate": 2.8684346190328393e-07,
	"loss": 0.6224913597106934,
	"step": 4700
	},
	{
	"epoch": 1.7114825581395348,
	"grad_norm": 2.767312526702881,
	"learning_rate": 2.798964309378255e-07,
	"loss": 0.6045539379119873,
	"step": 4710
	},
	{
	"epoch": 1.7151162790697674,
	"grad_norm": 2.9070537090301514,
	"learning_rate": 2.7302956787390006e-07,
	"loss": 0.6086003303527832,
	"step": 4720
	},
	{
	"epoch": 1.71875,
	"grad_norm": 2.816274642944336,
	"learning_rate": 2.662431206738167e-07,
	"loss": 0.6211147308349609,
	"step": 4730
	},
	{
	"epoch": 1.7223837209302326,
	"grad_norm": 2.8480284214019775,
	"learning_rate": 2.595373343960658e-07,
	"loss": 0.6072659492492676,
	"step": 4740
	},
	{
	"epoch": 1.7260174418604652,
	"grad_norm": 2.88503098487854,
	"learning_rate": 2.5291245118647304e-07,
	"loss": 0.6151592254638671,
	"step": 4750
	},
	{
	"epoch": 1.7296511627906976,
	"grad_norm": 3.040351629257202,
	"learning_rate": 2.4636871026945757e-07,
	"loss": 0.5649903297424317,
	"step": 4760
	},
	{
	"epoch": 1.7332848837209303,
	"grad_norm": 3.088498830795288,
	"learning_rate": 2.399063479393912e-07,
	"loss": 0.6478461265563965,
	"step": 4770
	},
	{
	"epoch": 1.7369186046511627,
	"grad_norm": 2.9244418144226074,
	"learning_rate": 2.335255975520656e-07,
	"loss": 0.6145503997802735,
	"step": 4780
	},
	{
	"epoch": 1.7405523255813953,
	"grad_norm": 2.8953909873962402,
	"learning_rate": 2.272266895162664e-07,
	"loss": 0.596315574645996,
	"step": 4790
	},
	{
	"epoch": 1.744186046511628,
	"grad_norm": 3.0355396270751953,
	"learning_rate": 2.2100985128545498e-07,
	"loss": 0.6153911590576172,
	"step": 4800
	},
	{
	"epoch": 1.7478197674418605,
	"grad_norm": 2.813180685043335,
	"learning_rate": 2.1487530734955242e-07,
	"loss": 0.6165101051330566,
	"step": 4810
	},
	{
	"epoch": 1.7514534883720931,
	"grad_norm": 2.9147789478302,
	"learning_rate": 2.0882327922683354e-07,
	"loss": 0.591987133026123,
	"step": 4820
	},
	{
	"epoch": 1.7550872093023255,
	"grad_norm": 2.9788706302642822,
	"learning_rate": 2.0285398545592999e-07,
	"loss": 0.6493700981140137,
	"step": 4830
	},
	{
	"epoch": 1.7587209302325582,
	"grad_norm": 2.7692644596099854,
	"learning_rate": 1.969676415879368e-07,
	"loss": 0.621481990814209,
	"step": 4840
	},
	{
	"epoch": 1.7623546511627906,
	"grad_norm": 2.873155355453491,
	"learning_rate": 1.9116446017862917e-07,
	"loss": 0.6019724845886231,
	"step": 4850
	},
	{
	"epoch": 1.7659883720930232,
	"grad_norm": 2.8992841243743896,
	"learning_rate": 1.8544465078078693e-07,
	"loss": 0.6230618953704834,
	"step": 4860
	},
	{
	"epoch": 1.7696220930232558,
	"grad_norm": 3.053983211517334,
	"learning_rate": 1.798084199366293e-07,
	"loss": 0.5850800514221192,
	"step": 4870
	},
	{
	"epoch": 1.7732558139534884,
	"grad_norm": 2.929288387298584,
	"learning_rate": 1.7425597117035537e-07,
	"loss": 0.597361946105957,
	"step": 4880
	},
	{
	"epoch": 1.776889534883721,
	"grad_norm": 2.917435884475708,
	"learning_rate": 1.6878750498079273e-07,
	"loss": 0.5993693351745606,
	"step": 4890
	},
	{
	"epoch": 1.7805232558139537,
	"grad_norm": 3.0222082138061523,
	"learning_rate": 1.6340321883416222e-07,
	"loss": 0.6234514236450195,
	"step": 4900
	},
	{
	"epoch": 1.784156976744186,
	"grad_norm": 3.007539749145508,
	"learning_rate": 1.5810330715694266e-07,
	"loss": 0.5682544708251953,
	"step": 4910
	},
	{
	"epoch": 1.7877906976744184,
	"grad_norm": 2.8434793949127197,
	"learning_rate": 1.5288796132885393e-07,
	"loss": 0.5935199737548829,
	"step": 4920
	},
	{
	"epoch": 1.791424418604651,
	"grad_norm": 2.786289691925049,
	"learning_rate": 1.4775736967594379e-07,
	"loss": 0.5871915817260742,
	"step": 4930
	},
	{
	"epoch": 1.7950581395348837,
	"grad_norm": 2.984264850616455,
	"learning_rate": 1.4271171746378764e-07,
	"loss": 0.6204954147338867,
	"step": 4940
	},
	{
	"epoch": 1.7986918604651163,
	"grad_norm": 2.8578007221221924,
	"learning_rate": 1.3775118689080048e-07,
	"loss": 0.601482343673706,
	"step": 4950
	},
	{
	"epoch": 1.802325581395349,
	"grad_norm": 2.9476382732391357,
	"learning_rate": 1.3287595708165457e-07,
	"loss": 0.6299497127532959,
	"step": 4960
	},
	{
	"epoch": 1.8059593023255816,
	"grad_norm": 2.946737051010132,
	"learning_rate": 1.2808620408081536e-07,
	"loss": 0.6067971229553223,
	"step": 4970
	},
	{
	"epoch": 1.809593023255814,
	"grad_norm": 2.9323055744171143,
	"learning_rate": 1.2338210084617996e-07,
	"loss": 0.6230051040649414,
	"step": 4980
	},
	{
	"epoch": 1.8132267441860463,
	"grad_norm": 3.0436692237854004,
	"learning_rate": 1.1876381724283515e-07,
	"loss": 0.6139970302581788,
	"step": 4990
	},
	{
	"epoch": 1.816860465116279,
	"grad_norm": 2.9199883937835693,
	"learning_rate": 1.142315200369229e-07,
	"loss": 0.6280738830566406,
	"step": 5000
	},
	{
	"epoch": 1.8204941860465116,
	"grad_norm": 2.92120361328125,
	"learning_rate": 1.0978537288961616e-07,
	"loss": 0.5834029197692872,
	"step": 5010
	},
	{
	"epoch": 1.8241279069767442,
	"grad_norm": 3.087002754211426,
	"learning_rate": 1.0542553635121134e-07,
	"loss": 0.6059079647064209,
	"step": 5020
	},
	{
	"epoch": 1.8277616279069768,
	"grad_norm": 2.9495768547058105,
	"learning_rate": 1.0115216785533149e-07,
	"loss": 0.6083863258361817,
	"step": 5030
	},
	{
	"epoch": 1.8313953488372094,
	"grad_norm": 2.7307910919189453,
	"learning_rate": 9.696542171323902e-08,
	"loss": 0.5967412948608398,
	"step": 5040
	},
	{
	"epoch": 1.8350290697674418,
	"grad_norm": 2.919420003890991,
	"learning_rate": 9.286544910826395e-08,
	"loss": 0.6107944488525391,
	"step": 5050
	},
	{
	"epoch": 1.8386627906976745,
	"grad_norm": 3.077995538711548,
	"learning_rate": 8.885239809034718e-08,
	"loss": 0.6117441654205322,
	"step": 5060
	},
	{
	"epoch": 1.8422965116279069,
	"grad_norm": 2.8363027572631836,
	"learning_rate": 8.492641357069148e-08,
	"loss": 0.5940214633941651,
	"step": 5070
	},
	{
	"epoch": 1.8459302325581395,
	"grad_norm": 2.935504913330078,
	"learning_rate": 8.10876373165298e-08,
	"loss": 0.5961288452148438,
	"step": 5080
	},
	{
	"epoch": 1.849563953488372,
	"grad_norm": 2.998866081237793,
	"learning_rate": 7.733620794600604e-08,
	"loss": 0.6110012054443359,
	"step": 5090
	},
	{
	"epoch": 1.8531976744186047,
	"grad_norm": 3.069027900695801,
	"learning_rate": 7.367226092317076e-08,
	"loss": 0.5856106281280518,
	"step": 5100
	},
	{
	"epoch": 1.8568313953488373,
	"grad_norm": 2.948878049850464,
	"learning_rate": 7.009592855308722e-08,
	"loss": 0.6236933708190918,
	"step": 5110
	},
	{
	"epoch": 1.8604651162790697,
	"grad_norm": 2.9042975902557373,
	"learning_rate": 6.66073399770556e-08,
	"loss": 0.5904771327972412,
	"step": 5120
	},
	{
	"epoch": 1.8640988372093024,
	"grad_norm": 3.013841390609741,
	"learning_rate": 6.320662116794912e-08,
	"loss": 0.610911750793457,
	"step": 5130
	},
	{
	"epoch": 1.8677325581395348,
	"grad_norm": 2.9105255603790283,
	"learning_rate": 5.989389492566527e-08,
	"loss": 0.5993082046508789,
	"step": 5140
	},
	{
	"epoch": 1.8713662790697674,
	"grad_norm": 2.9587931632995605,
	"learning_rate": 5.66692808726918e-08,
	"loss": 0.6032283306121826,
	"step": 5150
	},
	{
	"epoch": 1.875,
	"grad_norm": 2.8542747497558594,
	"learning_rate": 5.3532895449785735e-08,
	"loss": 0.6127349376678467,
	"step": 5160
	},
	{
	"epoch": 1.8786337209302326,
	"grad_norm": 2.926576614379883,
	"learning_rate": 5.0484851911770664e-08,
	"loss": 0.5979854583740234,
	"step": 5170
	},
	{
	"epoch": 1.8822674418604652,
	"grad_norm": 2.8693761825561523,
	"learning_rate": 4.7525260323446897e-08,
	"loss": 0.5823795318603515,
	"step": 5180
	},
	{
	"epoch": 1.8859011627906976,
	"grad_norm": 2.8152878284454346,
	"learning_rate": 4.4654227555614716e-08,
	"loss": 0.5919192790985107,
	"step": 5190
	},
	{
	"epoch": 1.8895348837209303,
	"grad_norm": 2.8572585582733154,
	"learning_rate": 4.187185728121823e-08,
	"loss": 0.588810396194458,
	"step": 5200
	},
	{
	"epoch": 1.8931686046511627,
	"grad_norm": 2.9437289237976074,
	"learning_rate": 3.9178249971599523e-08,
	"loss": 0.6189647674560547,
	"step": 5210
	},
	{
	"epoch": 1.8968023255813953,
	"grad_norm": 2.8459627628326416,
	"learning_rate": 3.657350289287265e-08,
	"loss": 0.6121656894683838,
	"step": 5220
	},
	{
	"epoch": 1.900436046511628,
	"grad_norm": 3.0236427783966064,
	"learning_rate": 3.4057710102409215e-08,
	"loss": 0.6005516529083252,
	"step": 5230
	},
	{
	"epoch": 1.9040697674418605,
	"grad_norm": 3.0145411491394043,
	"learning_rate": 3.163096244544361e-08,
	"loss": 0.6206085205078125,
	"step": 5240
	},
	{
	"epoch": 1.9077034883720931,
	"grad_norm": 2.7869176864624023,
	"learning_rate": 2.929334755179175e-08,
	"loss": 0.6017412185668946,
	"step": 5250
	},
	{
	"epoch": 1.9113372093023255,
	"grad_norm": 2.7438442707061768,
	"learning_rate": 2.7044949832686908e-08,
	"loss": 0.611904239654541,
	"step": 5260
	},
	{
	"epoch": 1.9149709302325582,
	"grad_norm": 2.8678460121154785,
	"learning_rate": 2.4885850477732178e-08,
	"loss": 0.6094479084014892,
	"step": 5270
	},
	{
	"epoch": 1.9186046511627906,
	"grad_norm": 2.9781479835510254,
	"learning_rate": 2.281612745196782e-08,
	"loss": 0.6219481945037841,
	"step": 5280
	},
	{
	"epoch": 1.9222383720930232,
	"grad_norm": 3.0608043670654297,
	"learning_rate": 2.083585549305628e-08,
	"loss": 0.5890862464904785,
	"step": 5290
	},
	{
	"epoch": 1.9258720930232558,
	"grad_norm": 3.0372581481933594,
	"learning_rate": 1.8945106108584634e-08,
	"loss": 0.6225212574005127,
	"step": 5300
	},
	{
	"epoch": 1.9295058139534884,
	"grad_norm": 3.0238840579986572,
	"learning_rate": 1.7143947573479704e-08,
	"loss": 0.6276484489440918,
	"step": 5310
	},
	{
	"epoch": 1.933139534883721,
	"grad_norm": 2.9078598022460938,
	"learning_rate": 1.543244492754531e-08,
	"loss": 0.5980105400085449,
	"step": 5320
	},
	{
	"epoch": 1.9367732558139537,
	"grad_norm": 2.8863484859466553,
	"learning_rate": 1.3810659973112206e-08,
	"loss": 0.6007534980773925,
	"step": 5330
	},
	{
	"epoch": 1.940406976744186,
	"grad_norm": 2.7025346755981445,
	"learning_rate": 1.227865127280764e-08,
	"loss": 0.5999039649963379,
	"step": 5340
	},
	{
	"epoch": 1.9440406976744184,
	"grad_norm": 2.943773031234741,
	"learning_rate": 1.0836474147438991e-08,
	"loss": 0.614153242111206,
	"step": 5350
	},
	{
	"epoch": 1.947674418604651,
	"grad_norm": 2.920337438583374,
	"learning_rate": 9.484180673997312e-09,
	"loss": 0.6102373123168945,
	"step": 5360
	},
	{
	"epoch": 1.9513081395348837,
	"grad_norm": 2.9423022270202637,
	"learning_rate": 8.22181968377661e-09,
	"loss": 0.5994505882263184,
	"step": 5370
	},
	{
	"epoch": 1.9549418604651163,
	"grad_norm": 2.965574264526367,
	"learning_rate": 7.0494367606105396e-09,
	"loss": 0.5969588279724121,
	"step": 5380
	},
	{
	"epoch": 1.958575581395349,
	"grad_norm": 2.9373302459716797,
	"learning_rate": 5.967074239226212e-09,
	"loss": 0.6102699279785156,
	"step": 5390
	},
	{
	"epoch": 1.9622093023255816,
	"grad_norm": 3.050691604614258,
	"learning_rate": 4.974771203715145e-09,
	"loss": 0.607047176361084,
	"step": 5400
	},
	{
	"epoch": 1.965843023255814,
	"grad_norm": 2.996528387069702,
	"learning_rate": 4.07256348612356e-09,
	"loss": 0.6196224212646484,
	"step": 5410
	},
	{
	"epoch": 1.9694767441860463,
	"grad_norm": 2.9037375450134277,
	"learning_rate": 3.260483665155911e-09,
	"loss": 0.5886441230773926,
	"step": 5420
	},
	{
	"epoch": 1.973110465116279,
	"grad_norm": 2.8899309635162354,
	"learning_rate": 2.538561065000833e-09,
	"loss": 0.6017139434814454,
	"step": 5430
	},
	{
	"epoch": 1.9767441860465116,
	"grad_norm": 2.9255728721618652,
	"learning_rate": 1.906821754270871e-09,
	"loss": 0.6277134895324707,
	"step": 5440
	},
	{
	"epoch": 1.9803779069767442,
	"grad_norm": 3.010143995285034,
	"learning_rate": 1.365288545061849e-09,
	"loss": 0.6320951461791993,
	"step": 5450
	},
	{
	"epoch": 1.9840116279069768,
	"grad_norm": 3.0056440830230713,
	"learning_rate": 9.139809921282494e-10,
	"loss": 0.5893740177154541,
	"step": 5460
	},
	{
	"epoch": 1.9876453488372094,
	"grad_norm": 2.968355894088745,
	"learning_rate": 5.529153921779439e-10,
	"loss": 0.6041777610778809,
	"step": 5470
	},
	{
	"epoch": 1.9912790697674418,
	"grad_norm": 2.8749659061431885,
	"learning_rate": 2.821047832837764e-10,
	"loss": 0.5875132083892822,
	"step": 5480
	},
	{
	"epoch": 1.9949127906976745,
	"grad_norm": 2.902491331100464,
	"learning_rate": 1.0155894441227221e-10,
	"loss": 0.5986234664916992,
	"step": 5490
	},
	{
	"epoch": 1.9985465116279069,
	"grad_norm": 2.9591474533081055,
	"learning_rate": 1.1284395070310095e-11,
	"loss": 0.6058923721313476,
	"step": 5500
	},
	{
	"epoch": 2.0,
	"step": 5504,
	"total_flos": 2.0144853494769123e+19,
	"train_loss": 0.736562816166254,
	"train_runtime": 18605.1295,
	"train_samples_per_second": 9.464,
	"train_steps_per_second": 0.296
	}
	],
	"logging_steps": 10,
	"max_steps": 5504,
	"num_input_tokens_seen": 0,
	"num_train_epochs": 2,
	"save_steps": 500,
	"stateful_callbacks": {
	"TrainerControl": {
	"args": {
	"should_epoch_stop": false,
	"should_evaluate": false,
	"should_log": false,
	"should_save": true,
	"should_training_stop": true
	},
	"attributes": {}
	}
	},
	"total_flos": 2.0144853494769123e+19,
	"train_batch_size": 2,
	"trial_name": null,
	"trial_params": null
	}