Five people were injured, Mayor Eric Adams said.
The three-storey building fell around 4pm local time (6am AEST) a few blocks from City Hall and the Brooklyn Bridge.
It is less than a kilometre from the New York Stock Exchange.
Bystander video shows cars hanging precariously from a buckled upper deck of the building.
“It felt like an earthquake,” Liam Gaeta, a student at nearby Pace University, said.
Read Related Also: Pink Opens Up About Questions She Wished She Could Have Asked Late Father
He said he heard “a large noise and a big rumbling, and then we all got evacuated”.
Other Pace students described hearing screams and seeing cars falling in the building when it caved in.
The Fire Department said there were reports of people trapped and that searches were being conducted to make sure everyone was accounted for.
At one point, firefighters were pulled out because of concerns about the building’s stability, the department said.