README.md

---
language:
- en
datasets:
- squad
model:
- facebook/data2vec-text-base
---
<h1>data2vec squad</h1>
This is a testing fine tuned data2vec model in the squad dataset, any improvements and suggestions are welcome!

<h3>Intended use</h3>
Question Answering

<h3>Training results</h3>
	<table>
		<thead>
			<tr>
				<th>Epoch</th>
				<th>Training Loss</th>
				<th>Validation Loss</th>
			</tr>
		</thead>
		<tbody>
			<tr>
				<td>1</td>
				<td><span style="font-family: Roboto, Noto, sans-serif; font-size: 14px; font-style: normal; font-weight: 400; text-align: right;">1.015800</span><br></td>
				<td><span style="font-family: Roboto, Noto, sans-serif; font-size: 14px; font-style: normal; font-weight: 400; text-align: right;">0.997690</span><br></td>
			</tr>
			<tr>
				<td>2</td>
				<td><span style="font-family: Roboto, Noto, sans-serif; font-size: 14px; font-style: normal; font-weight: 400; text-align: right;">0.804400</span></td>
				<td><span style="font-family: Roboto, Noto, sans-serif; font-size: 14px; font-style: normal; font-weight: 400; text-align: right;">0.950322</span><br></td>
			</tr>
		</tbody>
</table>
<h3>Hyperparameters</h3>
<ul>
    <li>evaluation_strategy="epoch"</li>
    <li>learning_rate=2e-5</li>
    <li>per_device_train_batch_size=15</li>
    <li>per_device_eval_batch_size=15</li>
    <li>num_train_epochs=2</li>
    <li>weight_decay=0.01</li>
</ul>

<h3>Frameworks and libraries used:</h3>
<ul>
  <li>transformers</li>
  <li>datasets</li>
  <li>evaluate</li>
</ul>