Road traffic accident is one of the major causes of death in Bangladesh. In this article, we propose a method that uses road side video data to learn the traffic pattern and uses vision based techniques to track and determine various kinetic features of vehicles. Finally, the proposed method detects anomaly (possibility of accident) and accidents on the road. The proposed method shows approximately 85% accuracy level in detecting special situations.