this was fun though! maybe still useful for something but i think that finetuning gpt2 is not quite as performant as fine tuning some of these other modern models.
this "AutoCoder" model is supposedly better than gpt4 at coding... https://huggingface.co/Bin12345/AutoCoder