abhi11nav

/

codebert-gpt2-commitgen

like 1

Just had some brief experimentation and so far it looks like CodeTrans, even using T5-small outperforms this model. Simple example being the following

diff --git a/docs/index.md b/docs/index.md
index c1750d5..7859157 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -78,7 +78,7 @@ id: home
 <hr class="home-divider" />
 <section class="home-bottom-section">
   <div class="buttons-unit">
-    <a href="/getting-started.html" class="button">Get Started</a>
-    <a href="/download.html" class="button">Download React v{{site.react_version}}</a>
+    <a href="docs/getting-started.html" class="button">Get Started</a>
+    <a href="downloads.html" class="button">Download React v{{site.react_version}}</a>
   </div>
 </section>

CodeTrans yields "Fix broken link to version" while GPT2-commitgen yields "Fix typo in README". I used https://huggingface.co/SEBIS/code_trans_t5_small_commit_generation_transfer_learning_finetune.

I think this is likely due to CodeTrans being a fine-tuned model, having seen a lot of code. I'm still a bit surprised as GPT2-commitgen is larger and uses a better decoder.