“The problem is that I’m the president of the United States, I’m not the emperor of the United States." Obama said