A variety of magnetic devices can be manipulated remotely using a single permanent “actuator” magnet positioned in space by a robotic manipulator. This paper describes the spherical-actuator-magnet manipulator (SAMM), which is designed to replace or augment the singularity-prone spherical wrist used by prior permanent-magnet manipulation systems. The SAMM uses three omniwheels to enable holonomic control of the heading of its magnet's dipole and to enable its magnet to be rotated continuously about any axis of rotation. The SAMM performs closed-loop control of its dipole's heading using field measurements obtained from Hall-effect sensors as feedback, combined with modeled dynamics, using an extended Kalman filter. We describe the operation and construction of the SAMM, develop and characterize controllers for the SAMM's spherical magnet, and demonstrate remote actuation of an untethered magnetic device in a lumen using the SAMM.