Police were called to a hotel on Liverpool Street, Darlinghurst, at about 4.30pm yesterday after reports of a Nazi symbol being displayed.
A short time later, a 51-year-old man and a 44-year-old man were arrested.
Police also seized the flag and several electronic devices.
The men have been charged with displaying a Nazi symbol.
They were refused bail and will appear in court today.
