Deep learning (DL) has led to near or better than human performance in image classification or object/speech recognition. DL is now providing new tools to address autonomous robotic manipulation and navigation challenges. One of the fundamental capabilities necessary for robotic manipulation is the ability to reorient objects within the hand. In this paper, we describe an approach using Deep Reinforcement Learning (DRL) techniques to learn a policy to perform in-hand manipulation directly from raw image pixels. This paper presents an overview of the working prototype, the description of the algorithms and a working prototype using the Modular Prosthetic Limb (MPL) in a Gazebo simulation.