Skip to main content

[Maeumjigi Capstone Dev Log #15] Visualizing Fine-Tuning Results of 5 Models & Local Environment Setup

· 2 min read
Jihwan Park (jaybee)
Jihwan Park (jaybee)
AI & Robotics Engineer

Cover Image

May 7th was the day we synthesized the fine-tuning results of our 5 models (Llama3.3-70B, DSR1-70B, HCX-Omni-8B, Qwen35-9B, HCX-Text-1.5B) and proceeded with full-scale visualization and local environment setup.

Visualization and Local Setup for Model Comparison​

To comfortably review the result comparison code (which was previously running on the heavy server) on my personal laptop, I created a new notebook, 31_comparison_local.ipynb.

"Can't 30_comparison.ipynb run on my laptop too?" "Please tell me how to install pandas, matplotlib, seaborn, and bert_score on the currently selected kernel."

While setting up the local Jupyter kernel in VSCode and installing the necessary libraries, I faced frequent Kernel Crashes and annoying errors like ModuleNotFoundError: No module named 'ft_utils'. However, I resolved them step-by-step by fixing folder paths. I also successfully recovered the missing ft_metrics.json files for some models (like DSR1-70B) using a script.

5-Model Comprehensive Report and Graph Visualization​

The most important task was visualizing the performance of these 5 models so they could be compared at a glance. Together with the AI, we established a plan named 5MODELS_ORGANIZATION_PLAN_present.md and began systematically organizing the data.

"What is the difference between the faint line and the thick line? Bring the training loss and eval loss for each model and visualize them as graphs." "Remove the thick line and make the faint line clearer and save it."

I removed the unnecessary trend lines (thick lines) from the matplotlib graphs generated by the AI, finally extracting 03_training_loss_all.png and 04_eval_loss_all.png which neatly gathered the Training and Eval Loss curves of all models in one place. We also perfectly documented graphs analyzing the LoRA Rank Decomposition for each model, as well as a GPU memory usage chart (02_gpu_memory.png).

Graph Visualization

Now, based on these graphs, we can beautifully decorate our capstone design mid-term presentation materials for next week. Which of these 5 models will ultimately be selected as the best quality 'Maeumjigi' chatbot?