Optimizing Training Time in Deep Learning Models Using Distributed Computing Techniques

Abstract

Training deep learning models is often a time-consuming process, especially when hardware resources are tight. This bottleneck slows down experimentation and increases development costs. The primary goal of this study was to investigate whether distributed computing could cut down on training time while keeping model accuracy intact. To test this, we trained a Convolutional Neural Network (CNN) on the CIFAR-10 dataset using two different configurations: a standard single-device setup and a distributed synchronous setup powered by TensorFlow’s MirroredStrategy. To ensure a fair comparison, both trials used the exact same architecture, hyperparameters, and preprocessing. The results were clear: distributed training reduced the total time by about 19.5%, with validation accuracy remaining nearly identical to the single-device approach. These findings suggest that distributed training is a practical way to accelerate deep learning workflows without sacrificing performance, serving as an efficient solution even in environments with limited hardware.