As we entered June, the Maeumjigi team completed the major developments of the 1st semester Capstone project and focused entirely on building safety nets and preparing for the final presentation.
In the final week of May (27th-30th), we dedicated our efforts to wrapping up the results of the highly anticipated 4th Fine-Tuning (fine_tuning_4) and solidifying the system's framework.
On May 21st and 22nd, the Maeumjigi Capstone team embarked on massive data generation and preparation for the 4th Fine-Tuning (fine_tuning_4).
Over the two days of May 19th and 20th, the automation of evaluation for the 3rd fine-tuning (fine_tuning_3) and the enhancement of backend/frontend features for building the actual app service proceeded in parallel.
I must correct my previous assumption that "May 13th was the last of Maeumjigi's May development"! On May 17th, a massive wave of backend integration testing and report drafting hit us.
On May 13th, after fixing the hardcoding bugs discovered yesterday, we finally resumed proper inference testing for the 2nd round fine-tuned models.
The 2nd round of fine-tuning (fine_tuning_2) parallel training, which was ambitiously started to correct the LoRA hyperparameter mistake (alpha/rank setup error), finally ended on May 12th!
May 8th was the day we were building our capstone design mid-term presentation (PPT) based on the fine-tuning results. However, while happily filling in the slides, I discovered a massive, chilling mistake.
May 7th was the day we synthesized the fine-tuning results of our 5 models (Llama3.3-70B, DSR1-70B, HCX-Omni-8B, Qwen35-9B, HCX-Text-1.5B) and proceeded with full-scale visualization and local environment setup.
On May 6th, we finally escaped the long swamp of fine-tuning! After several nights of work and hyperparameter tuning, the training for mid-size models like HCX-Omni-8B and Qwen35-9B was successfully completed.
After experiencing the OOM (Out of Memory) hell during the fine-tuning of massive 70B models, on May 4th, we attempted a bold change in strategy. By drastically reducing the batch size, we barely managed to push the 70B models (Llama3.3, DeepSeek-R1) to checkpoint-1200 and 1300, but the training time was absurdly long, resulting in terrible efficiency.
On May 3, having gained enough experience with the 8B models, our team finally challenged the ultimate kings of performance: fine-tuning massive 70B (70 billion parameters) models. We attempted to train both Llama 3.3 70B and DeepSeek-R1 70B, but as expected, we hit a massive hardware resource wall.
On April 9, using the Colab notebook we ambitiously completed yesterday, we finally attempted the first training (fine-tuning) of our artificial intelligence model (Llama 3.1). However, reality was not that easy, and we had to face numerous errors. It was also a tough day as we had to simultaneously create the presentation materials for the 6th week of the Capstone Design project.
April 8th was the day we experienced the biggest crisis and a 'laugh-and-cry' happening while working on the Maeumjigi project. It was the Claude API monthly limit exceeded & bill bomb incident.
This is the record of April 1, when we began processing a massive amount of conversation data into a format suitable for training the Llama 3.1 model, which will become the core engine of the Maeumjigi project.