"cleared direct [fix name]" is a change to your routing, not the clearance limit. You should technically proceed direct to the airport from the fix that you were just cleared to (unless your cleared route contained additional fixes beyond the IAF, which would be pretty unusual). This is not likely what the controller would be expecting, though, practically speaking.
You would only enter a hold at the fix if the clearance was "cleared to the [fix name]." If that was the controller's intention, however, it would almost certainly your holding instructions, too, because if he's doing that, he's likely holding the airspace for another departure/arrival in a non-radar environment (one in, one out).