Safety accidents occur frequently in the construction industry, and unsafe behavior of workers is one of the main reasons for the occurrence of safety accidents. In recent years, the rapid development of emerging technologies has provided effective support for the automatic identification of unsafe behaviors. In particular, computer vision technology does not require equipment to be attached to workers. It has little impact on its operation and can process a large amount of image data in a timely and fast manner. Therefore, it is more suitable for the construction site environment. This study starts from the hoisting operation on the construction site of prefabricated building projects and introduces relevant machine learning methods. Early warning analysis on the safety risk of the tower crane is carried out to complete the hoisting operation of prefabricated components. To suspend dangerous tasks in time according to the warning results, relevant measures are taken and potential risks are eliminated to prevent accidents.