Economists are forecasting further pain for mortgage-holders, with rate rises likely next month and September.
The RBA has also indicated the pause on rate rises will not be long-lasting.
Albanese claimed things were improving for many Australians.
“The fact that interest rates were paused yesterday at 4.1 is a good thing,” he said on Today.
“You have a two-speed economy. Some people are doing it really tough.
“Then you have positive signs at the same time.”
He pointed to “record” jobs growth of 465,000 in his government’s first year, along with a decrease in inflation.
Read Related Also: Miss World Australia finalist Tirah Ciampa from Tasmania opens up about her pancreatic cancer battle and the symptoms she missed
“It peaked at the end of last year,” he said.
He also cited the budget surplus of $2.4 billion.
Revealed: Worst 10 suburbs for mortgage arrears across Australia
“I get that people are doing it tough. For me this isn’t about politics, this is about assisting people,” Albanese said.
“I know what it’s like to do it tough. And I get that people out there are under pressure.”
He claimed the government’s measures in childcare assistance, tripling the Medibank bulk-billing incentive, fee-free TAFE, and energy price relief were helping to put downward pressure on inflation.