Emergency services were called to a beach off Boneo Road at 9.45am and carried out a search of the ocean.
The Victoria Police Air Wing found a man unresponsive in the water.
The man, who has not been publicly identified, was brought back to shore but could not be revived.
A report will be prepared for the information of the coroner.
The stretch of Victorian coastline is notoriously dangerous and has been the site of several tragic deaths in the last decade.
It is understood the 53-year-old woman was among a group of three people washed off shore by a wave.
