An enthralling victory in Adelaide means the hosts move level with Japan and the Saudis at the top of Group B on 16 points.
The top two teams get a direct ticket to Russia next year, with the third-placed nation having to go through two play-off rounds if they want to reach the tournament.
Tomi Juric, who also scored in the away qualifier against the Saudis, gave Australia the lead after a dreadful mistake from goalkeeper Yasser Al-Mosailem.
Salem Al Dawsari equalised for the visitors, only for Juric to head Ange Postecoglou's men back in front.
Mohammed Al Sahlawi then struck to ensure a thrilling first half ended level, but Celtic's Rogic produced a superb strike to earn a vital victory.
Japan hold a game in hand - against Iraq on Tuesday - on Australia and Saudi Arabia, who both have just two matches left to play.
Al-Mosailem had already had a lucky escape when he had a clearance charged down in the early stages, before his shocking error in the seventh minute gave the hosts the lead.
The keeper slipped as he kicked the ball out from his hands, with Juric impressively taking one touch and firing into an empty net from 25 yards before the shot-stopper could recover.
But Saudi Arabia hit back after 23 minutes when Al Dawsari drove forward and exchanged a slick one-two with Al Sahlawi to cut through Australia's defence before firing low beyond Mat Ryan.
They were only level for 13 minutes as Australia struck again, Juric restoring their lead with a precise header into the far corner from eight yards after Mathew Leckie's superb run and cross down the right created the opportunity.
Taiseer Al Jassam had a goal correctly disallowed for offside as Saudi Arabia pushed for another equaliser.
And they did strike back two minutes into first-half stoppage time, Yahia Al Shehri sending a long pass into the penalty area which was headed across goal by Al Jassam, allowing Al Sahlawi to race on and power in a finish from close range.
A thrilling clash continued with Australia taking the lead for a third time after 64 minutes, Rogic collecting Aaron Mooy's pass and unleashing brilliant strike from 25 yards that flew past Al-Mosailem into the top corner.
Al Sahlawi saw a shot saved by Ryan at the other end, before Australia fumed when they had a penalty claim turned down after Leckie was barged to the ground by Salman Al Faraj as he raced through on goal.
Juric had a chance for a hat-trick goal that would have sealed victory, but he could only fire over a golden opportunity presented to him by substitute Aziz Behich.
That did not cost Australia, though, Ryan not having a major save to make in the final stages as Postecoglou's men celebrated a vital triumph.
Copyright © 2021 Goal.com All rights reserved.