Deep learning such as deep neural networks has revolutionized the computer vision area. Deep learning-based algorithms have surpassed conventional algorithms in terms of performance by a significant margin. This paper reviews our works in the application of deep convolutional neural networks to target recognition and terrain classification using the SAR image. A convolutional neural network is employed to automatically extract a hierarchic feature representation from the data, based on which the target recognition and terrain classification can be conducted. Experimental results on the MSTAR benchmark dataset reveal that deep convolutional network could achieve a state-of-the-art classification accuracy of 99% for the 10-class task. For a polarimetric SAR image classification, we propose complex-valued convolutional neural networks for complex SAR images. This algorithm achieved a state-of-the-art accuracy of 95% for the 15-class task on the Flevoland benchmark dataset.