This paper presents a temporal model of video for video retrieval systems. Time is the dimension that plays an essential role in the perception of video document content. The modelling of the different temporal characteristics of the video content is however a complicated problem. Our approach to this problem has consisted of distinguishing two different dimensions of time for video documents: video time and story time. The semantic structure proposed based on these two temporal dimensions, provides the user with a new modality of content description which is close to the semantic perception of the video documents. We present here how using the conceptual graphs formalism, the details of the temporal model can be represented in query, document and matching function of the retrieval process.