Please use this identifier to cite or link to this item: http://dspace.dtu.ac.in:8080/jspui/handle/repository/21671
Title: REINFORCEMENT LEARNING FOR WEIGHT INITIALIZATION
Authors: YAMAN
SHONDHANI, DEEPANSHU
Keywords: REINFORCEMENT LEARNING
WEIGHT INITIALIZATION
DEEP NEURAL NETWORKS
ACTIVATION FUNCTIONS
Issue Date: May-2025
Series/Report no.: TD-7876;
Abstract: Effective neural network weight initialization is crucial for successful training, yet standard methods often rely on assumptions violated by modern architectures and advanced activation functions like Swish. This dissertation details a study investigating the feasibility of using Reinforcement Learning (RL) to tune a scaling factor for He initialization when employing Swish activations. An RL agent explored different scaling factors, evaluating them by training a small convolutional neural network on CIFAR-10 for a 7-epoch proxy task. Over 200 episodes, the RL agent demonstrated learning, converging towards a specific range of scaling factors that optimized the 7-epoch validation accuracy. This preliminary investigation highlights the functionality of the RL framework for initialization tuning and underscores the importance of evaluating the fidelity of short-term proxy tasks in predicting longer-term training performance, informing subsequent research into more complex symbolic initialization discovery.
URI: http://dspace.dtu.ac.in:8080/jspui/handle/repository/21671
Appears in Collections:M Sc Applied Maths

Files in This Item:
File Description SizeFormat 
YAMAN & DEEPANSHU m.sC..pdf1.06 MBAdobe PDFView/Open


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.