In recent years, the resurgence of neural network architectures and its application in reinforcement learning and computer vision have allowed more complicated algorithms to develop. Addressing the need for a better transportation system, we want to explore an autonomous system via creating a self-driving car that is capable of lane following, traffic sign/pedestrian detection, collision avoidance, and path planning. This not only allows us to understand the intricacy of handling data by assuring the generalization of the model, it also furthers our knowledge of hardware optimization. Therefore, the project requires a close collaboration between data science, deep reinforcement learning, control systems, and hardware design. Such collaboration will be a testament of what we have achieved while pursuing engineering disciplines. Using our implementations of lane line detection, traffic sign detection, pedestrian detection, etc., we hope to apply our models to a real car that is capable of safely transporting passengers in a real-world traffic setting.