With the full popularity of China's railwayization process, it has brought about the problem of the management ability of railway traffic safety. Railway traffic safety emergency management capabilities are low. When an accident occurs, clearer data cannot be obtained in the first time to have a general understanding of the accident. Therefore, the problem of organizing rescue has always plagued relevant railway workers. This study aims to study the improvement of railway traffic emergency management based on image recognition technology in the context of big data. To this end, this study proposes image recognition technology based on deep learning, and through the relayout of the railway traffic emergency management system, so that the railway traffic problems can be dealt within time as soon as they occur, and designed an experiment to explore the ability of image recognition. The results of the experiment show that the efficiency of the improved railway traffic emergency management system has increased by 27%, and the recognition capability has increased by 64%. It can very well help current railway workers to carry out emergency management for railway traffic safety.