Please use this identifier to cite or link to this item:
http://dspace.dtu.ac.in:8080/jspui/handle/repository/21671
Title: | REINFORCEMENT LEARNING FOR WEIGHT INITIALIZATION |
Authors: | YAMAN SHONDHANI, DEEPANSHU |
Keywords: | REINFORCEMENT LEARNING WEIGHT INITIALIZATION DEEP NEURAL NETWORKS ACTIVATION FUNCTIONS |
Issue Date: | May-2025 |
Series/Report no.: | TD-7876; |
Abstract: | Effective neural network weight initialization is crucial for successful training, yet standard methods often rely on assumptions violated by modern architectures and advanced activation functions like Swish. This dissertation details a study investigating the feasibility of using Reinforcement Learning (RL) to tune a scaling factor for He initialization when employing Swish activations. An RL agent explored different scaling factors, evaluating them by training a small convolutional neural network on CIFAR-10 for a 7-epoch proxy task. Over 200 episodes, the RL agent demonstrated learning, converging towards a specific range of scaling factors that optimized the 7-epoch validation accuracy. This preliminary investigation highlights the functionality of the RL framework for initialization tuning and underscores the importance of evaluating the fidelity of short-term proxy tasks in predicting longer-term training performance, informing subsequent research into more complex symbolic initialization discovery. |
URI: | http://dspace.dtu.ac.in:8080/jspui/handle/repository/21671 |
Appears in Collections: | M Sc Applied Maths |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
YAMAN & DEEPANSHU m.sC..pdf | 1.06 MB | Adobe PDF | View/Open |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.