Democrats in Congress are pushing for his resignation, but so far most GOP members have remained quiet.