I am using Jetson reinforcement for my quadcopter reinforcement in simulation. Maybe it will help you. because you can create AI agents that can learn from the interactive environment, gather experience, and system of reward with deep RL. You can also use end-to-end neural networks that translate raw pixels into action as per your need and use that RL-trained agent to complete complex tasks.
The best thing is you can easily transfer RL-agent which simulated in the simulator to real-world robots. and We are using multiple Nano's easily to perform the complex tasks of navigation and co-ordination of Our Quad-copter.
Webinar of Deep RL on Jetson
Google group to get help for Deep RL on jetson nano
Hope it helps you.