Rahul Murugan

Rahul
Murugan

- Manhattan, New York

Hi, I’m Rahul, a master’s student at Columbia University, passionate about advancing AI through large language models (LLMs) and data-driven innovation. My work focuses on developing LLM benchmarks and evaluation frameworks, enabling rigorous assessment of reasoning and performance in complex, real-world scenarios. With hands-on experience in retrieval-augmented generation pipelines and scalable AI systems, I aim to bridge cutting-edge research with impactful applications.

Education

Columbia University - New York, USA (2024 – Present)

Master of Science in Electrical Engineering (Specializing in Data-Driven Analysis and Computation)

CGPA: 3.73

Relevant Coursework: Generative AI and Modern Deep Learning, Applied Machine Learning, Neural Networks and Deep Learning, Analysis of Algorithms, Data-Driven methods in Finance

Shiv Nadar University - Chennai, India (2020 – 2024)

Bachelor of Engineering in Electrical Engineering

CGPA: 3.63

Work Experience

Graduate Research Assistant, Columbia University Department of Computer Science - New York, USA

Project Lead for the CS Advising Assistant initiative, responsible for designing and implementing an advanced chatbot system to address diverse student inquiries. Focused on optimizing the Retrieval-Augmented Generation (RAG) pipeline to enhance both accuracy and operational efficiency.

Machine learning Intern, Spark Minda Technical Center - Bangalore, India

Developed a high-precision Passenger Counting System utilizing YOLOv8 computer vision algorithms, optimized for real-time detection and tracking, and customized for deployment on Raspberry Pi (CM4) to improve public transport efficiency.

AI Research Intern, Indian Institute of Technology (IIT) Madras - Chennai, India

Created a Retrieval-Augmented Generation (RAG) pipeline for Pfizer, enabling an efficient question-answering system and streamlining data extraction from large-scale patent text files.

Robert Bosch Centre of Data Science and Artificial Intelligence – IIT Madras - Chennai, India

Automated data mining and extraction from sustainability reports using advanced NLP techniques, integrating GPT-4 API and LangChain framework to validate a Climate Action tool, and built a machine learning model to analyze India’s energy consumption patterns and enhance decision-making processes.

Research & Projects

EQUATOR: A Deterministic Framework for Evaluating LLM Reasoning with Open-Ended Questions (Project Link)

EQUATOR presents a deterministic evaluation framework designed to assess the reasoning capabilities of Large Language Models (LLMs) on open-ended questions. The framework leverages automated evaluation pipelines powered by smaller LLMs like LLaMA 3.2B, ensuring scalable and efficient testing.
To broaden accessibility, plans were initiated to develop a collaborative website for sharing evaluations, benchmarks, and insights, aiming to support researchers and practitioners in advancing the reliability and performance of LLMs. This project establishes a foundation for transparent and standardized evaluation practices in the AI community.

Enhanced Momentum with Momentum Transformers (Project Link)

Momentum Transformer extending its application from futures and equity indices to equities, leveraging a hybrid architecture that combines attention mechanisms with LSTMs to capture long-term dependencies and adapt to evolving market conditions. The model successfully demonstrated its capability to enhance performance in dynamic trading environments while providing interpretable and intelligent decision-making insights.

Audio Classification of Normal Traffic and Siren Sounds: Spectrogram Encoding and Raw Audio Feature Extraction Approaches (Project Link)

AI-powered adaptive traffic system to classify emergency vehicle sirens from general road noise using audio data transformed into acoustic features and spectrograms. The system achieved exceptional accuracy, with fine-tuned ResNet18 correctly identifying all emergency and road noise samples, demonstrating its potential to enhance urban traffic management and emergency response efficiency.

Fine Tuning and Evaluating Large Language Models for HTML Code Generation (Project Link)

Fine-tuned the Falcon 7b language model to generate HTML code from natural language prompts, optimizing memory usage and computational efficiency through advanced techniques like BitsAndBytes quantization. Leveraged LoRA (Low-Rank Adaptation) for fine-tuning, employing gradient checkpointing and mixed-precision training to enhance the model's ability to translate natural language into accurate HTML code. The model's effectiveness was evaluated through exact match comparisons, ensuring high-quality output.

Sentiment Analysis using VADER – Price returns vs Sentiment (Project Link)

Developed an end-to-end data pipeline to analyze stock market sentiment by web scraping over 100 daily news articles from Finviz using requests and BeautifulSoup. Integrated the VADER tool to visualize sentiment trends across multiple trading days and utilized the yFinance API to automate historical price returns extraction, uncovering a strong correlation between sentiment shifts and stock price movements.

Publications

EQUATOR: A Deterministic Framework for Evaluating LLM Reasoning with Open-Ended Questions

arxiv · 31 Dec 2024

Authors: Raymond Bernard, Shaina Raza (PhD), Subhabrata Das (PhD), Rahul Murugan
Year: 2024
DOI: https://doi.org/10.48550/arXiv.2501.00257