With the development of the open source community, the code is often copied, spread, and evolved in multiple software systems, which brings uncertainty and risk to the software system (e.g., bug propagation and copyright infringement). Therefore, it is important to conduct code clone detection to discover similar code pairs. Many approaches have been proposed to detect code clones where token-based tools can scale to big code. However, due to the lack of program details, they cannot handle more complicated code clones, \ie semantic code clones. In this paper, we introduce \emph{CC2Vec}, a novel code encoding method designed to swiftly identify simple code clones while also enhancing the capability for semantic code clone detection. To retain the program details between tokens, \emph{CC2Vec} divides them into different categories (\ie typed tokens) according to the syntactic types and then applies two self-attention mechanism layers to encode them. To resist changes in the code structure of semantic code clones, \emph{CC2Vec} performs contrastive learning to reduce the differences introduced by different code implementations. We evaluate \emph{CC2Vec} on two widely used datasets (\ie BigCloneBench and Google Code Jam) and the results report that our method can effectively detect simple code clones. In addition, \emph{CC2Vec} not only attains comparable performance to widely used semantic code clone detection systems such as \emph{ASTNN}, \emph{SCDetector}, and \emph{FCCA} by simply fine-tuning, but also significantly surpasses these methods in both detection efficiency.