SUMMARIZING VIDEOS WITH ATTENTION BASED NETWORK

Abstract

In this project, I explored a new and more efficient way to summarize videos by focusing on the most important moments—what we call keyshots. Instead of relying on the usual complex models like bi-directional LSTMs with attention, which are not only difficult to implement but also require a lot of computational resources, I took a different route. I designed a simpler model based on a soft self-attention mechanism that’s much easier to work with and faster to train. What makes this approach stand out is that it processes the entire video sequence in just one forward and one backward pass during training. That means it’s not only lightweight but also well-suited for real-world applications where speed and efficiency matter. The self-attention mechanism allows the model to understand the importance of each frame in the context of the whole video—without needing any complex recurrence. I tested this method on two popular video summarization datasets, TvSum and SumMe, and was excited to see that it outperformed many of the existing state-of-the-art techniques. This showed me that a simpler, more streamlined approach can still deliver powerful results. It was a rewarding experience to challenge the norm and come up with a solution that’s both practical and effective.