AISLMFine-TuningNLP2026View on GitHub Read Article Series View Hugging Face Model

Fine-Tuning SLMs: Gemma 3 for Astronomy

A specialized training pipeline for fine-tuning Google's Gemma 3 (1B) model on astronomy MCQ datasets using Parameter-Efficient Fine-Tuning (PEFT).

Project Overview

This project explores the capacity of Small Language Models (SLMs) to handle domain-specific reasoning. By fine-tuning Gemma 3 using LoRA, the system transforms general-purpose weights into an astronomy expert capable of answering complex multiple-choice questions while maintaining a low computational footprint.

Key Features

Implemented Parameter-Efficient Fine-Tuning (PEFT) using LoRA to reduce GPU memory overhead

Comparative analysis of model capacity between Gemma-3-270M and 1B parameters

Custom data pipeline for formatting raw astronomy MCQs into optimized prompt-answer pairs

Optimized for resource-constrained environments like Google Colab (T4 GPU)

Successfully addressed underfitting issues by scaling model capacity while maintaining training efficiency

Integrated with Hugging Face Transformers for seamless model deployment and sharing

Technology Stack

Backend

PythonPyTorchTransformersPEFT/LoRA

Tools & Platforms

Hugging FaceGoogle Colab (T4 GPU)Gemma 3

Back to Projects