This review article provides an overview of Artificial Intelligence governance in biomedicine. The article discusses the current limitations of AI governance as well as possible paths to address and eliminate these limitations. An effective, clearly-defined regulatory system is necessary to promote the growth of medical AI in a safe, secure, and user-centric way.