Multimodal covid network: Multimodal bespoke convolutional neural network architectures for COVID‐19 detection from chest X‐ray's and computerized tomography scans

Abstract

AI‐based tools were developed in the existing works, which focused on one type of image data; either CXR's or computerized tomography (CT) scans for COVID‐19 prediction. There is a need for an AI‐based tool that predicts COVID‐19 detection from chest images such as Chest X‐ray (CXR) and CT scans given as inputs. This research gap is considered the core objective of the proposed work. In the proposed work, multimodal CNN architectures were developed based on the parameters and hyperparameters of neural networks. Nine experiments evaluate optimizers, learning rates, and the number of epochs. Based on the experimental results, suitable parameters are fixed for multimodal architecture development for COVID‐19 detection. We have constructed a bespoke convolutional neural network (CNN) architecture named multimodal covid network (MMCOVID‐NET) by varying the number of layers from two to seven, which can predict covid or normal images from both CXR's and CT scans. In the proposed work, we have experimented by constructing 24 models for COVID‐19 prediction. Among them, four models named MMCOVID‐NET‐I, MMCOVID‐NET‐II, MMCOVID‐NET‐III, and MMCOVID‐NET‐IV performed well by producing an accuracy of 100%. We obtained these results from a small dataset. So we repeated these experiments in a larger dataset. We inferred that MMCOVID‐NET‐III outperformed all the state‐of‐the‐art methods by producing an accuracy of 99.75%. The experiments carried out in this work conclude that the parameters and hyperparameters play a vital role in increasing or decreasing the model's performance.