A Comparative Study of Transformer-based and Hybrid Deep Learning Models for Long Document Summarization of academic research papers

Abstract

Automatic summarization of long scientific texts has been established as an essential task within the domain of Natural Language Processing(NLP), aiming to reduce information overload and facilitate knowledge acquisition from complex academic documents. Despite its importance, fact-based conventional systems of Automatic Text Summarization have mostly failed in ensuring coherence on the document level and keeping factual correctness. To address these limitations, further recent progress in deep learning has turned more intelligent models toward the equilibrium structural fidelity with fluency. This paper presents a comparative study of two state-of-the-art summarization techniques: (1) hybrid deep learning, which combines Bidirectional LSTM-based sentence classification with a hierarchical attention-driven encoder-decoder for abstractive summarization, and (2) fine-tuned Transformer-based architectures, specifically T5-base and T5-large models. The first method emphasizes structural awareness and hierarchical processing to preserve document-level semantics and mitigate issues such as repetition and out-of-vocabulary (OOV) tokens. In contrast, the Transformer-based models leverage large-scale pretraining with self-attention mechanisms for producing fluent summaries that are richly filled with context. Both methods were evaluated on two benchmark datasets: arXiv and PubMed. The hybrid model achieved ROUGE-F1 scores of (46.7, 19.4, 35.4) and (47.0, 21.3, 39.7), respectively, while the T5-large model outperformed it with scores of (55.8, 33.8, 47.9) and (54.9, 32.0, 48.7). These results show that while Transformer models perform better in abstraction and fluency, the hybrid model has fact control ability that is much interpretable and aligns with document structure. This comparison gives important insight into the trade-off between structure-aware hybrid frameworks and large-scale generative models in academic summarization.