Skip to main content

23 posts tagged with "devlog"

View All Tags

[Maeumjigi Capstone Dev Log #13] Limits of 70B Training and the Pivot to Mid-Size Models (HCX, Qwen)

· 2 min read
Jihwan Park (jaybee)
AI & Robotics Engineer

Cover Image

After experiencing the OOM (Out of Memory) hell during the fine-tuning of massive 70B models, on May 4th, we attempted a bold change in strategy. By drastically reducing the batch size, we barely managed to push the 70B models (Llama3.3, DeepSeek-R1) to checkpoint-1200 and 1300, but the training time was absurdly long, resulting in terrible efficiency.

[Maeumjigi Capstone Dev Log #12] 70B Massive Model Fine-Tuning and GPU Memory (OOM) Struggle

· 2 min read
Jihwan Park (jaybee)
AI & Robotics Engineer

Cover Image

On May 3, having gained enough experience with the 8B models, our team finally challenged the ultimate kings of performance: fine-tuning massive 70B (70 billion parameters) models. We attempted to train both Llama 3.3 70B and DeepSeek-R1 70B, but as expected, we hit a massive hardware resource wall.

[Maeumjigi Capstone Dev Log #10] Lifting Application Programming Interface (API) Limits and Resuming Data Synthesis

· 2 min read
Jihwan Park (jaybee)
AI & Robotics Engineer

Cover Image

April 14 is the day we resumed the synthetic data (v5) generation work, which had been temporarily halted since the bill bomb incident. We increased the API billing limit (lifted the restriction) and ran the system, but once again, it didn't go smoothly.

[Maeumjigi Capstone Dev Log #9] Llama 3.1 Fine-Tuning Error Struggle and Week 6 Mid-term Presentation

· 3 min read
Jihwan Park (jaybee)
AI & Robotics Engineer

Cover Image

On April 9, using the Colab notebook we ambitiously completed yesterday, we finally attempted the first training (fine-tuning) of our artificial intelligence model (Llama 3.1). However, reality was not that easy, and we had to face numerous errors. It was also a tough day as we had to simultaneously create the presentation materials for the 6th week of the Capstone Design project.

[Maeumjigi Capstone Dev Log #7] Synthetic Data (v5) Generation Optimization and Frontend Integration

· 2 min read
Jihwan Park (jaybee)
AI & Robotics Engineer

Cover Image

April 7 was the day we attempted integration by running the frontend code built by a teammate in a local environment, and also the day we launched a massive script using Artificial Intelligence (AI) to 'synthesize' 60,000 new counseling data points to fill our dataset gaps.

[Maeumjigi Capstone Dev Log #5] Purifying 100,000 Emotion Data Points and the Pressure of Application Programming Interface (API) Costs

· 2 min read
Jihwan Park (jaybee)
AI & Robotics Engineer

Cover Image

This is the record of April 3, the day we began in earnest the task of refining the training conversation data to feed into the deep learning model (Llama 3.1). It was also the day we ran a massive pipeline that makes the Artificial Intelligence (AI) read the text and judge the emotion on its own, going beyond simple data merging.

[Maeumjigi Capstone Dev Log #3] Alarm App Build Test and Fine-Tuning Preprocessing Kickoff

· 2 min read
Jihwan Park (jaybee)
AI & Robotics Engineer

Cover Image

This is the record of March 31, when we conducted the Android build test of the MND Convention reservation alarm app, which was the initial direction of the Maeumjigi project, while simultaneously starting the Llama 3.1 fine-tuning data preprocessing, hinting at a full transition into an AI-based project.

[Maeumjigi Capstone Dev Log #2] Initial Alarm App Planning and Testing

· 2 min read
Jihwan Park (jaybee)
AI & Robotics Engineer

Cover Image

This is the record of March 30, when we officially started developing the 'MND Convention Reservation Alarm' application, which was the initial idea for the Maeumjigi project. Based on the coding rules we set last time, we established a detailed project plan and laid the foundation for Test-Driven Development (TDD).