Berwick and Spartans must do jt all again after the sides drew 1-1 in Edinburgh, the reigning Lowland League champions denying the Scottish Division Two outfit with an injury time equaliser.
Paul Willis gave Rangers the lead after only four minutes, but Ali MacKinnon forced a replay with a 92nd minute leveller.
Spartans had already beaten Clyde and Morton in previous rounds of the competition and were looking to complete a hat-trick over league opposition.
They were also trying to become the first East of Scotland (Lowland) League side since 1948 to reach the quarter-finals of the cup, and with a decent crowd at Ainslie Park the stage was set for a thrilling encounter.
Spartans gave debuts to two new signings -Eddie Malone and Paul Sludden, whilst Berwick handed recent acquisition Blair Henderson his first start, and he pulled a shot wide after only two minutes.
It took Rangers only four minutes to open the scoring when a Lee Currie corner was cleared, but only into the path of Willis, who from 20 yards out smashed the ball into the roof of the net to give Rangers an early lead.
And as the visitors continued to set the early pace, Darren Lavery headed over from another corner, Stevie Notman had a header saved by goalkeeper Kevin Swain, whilst Willis shot over as Rangers tried to hit the home side on the break down the left.
As Spartans overcame their early nerves the game evened out a little although Berwick looked the more comfortable of the two sides.
It wasn’t until the 25th minute that Swain was called back into action again, saving low from a Currie effort outside the box, whilst Spartans had to wait until the 33rd minute before they registered their first effort on goal, a shot from Sludden being saved by Billy Bald.
As they grew in confidence Iain Thomson shot low from outside the box, and again the effort was saved by Bald in the bottom corner, whilst Sludden shot wide from an acute angle on the right.
In first half injury time Lavery had a shot saved by Swain and at the break it remained 0-1.
Early in the second half a shot from Notman was blocked and a Lavery effort, from a one on one situation with the goalkeeper, was saved by Swain as Rangers started in a positive manner.
Henderson also headed wide from a corner and had a header saved by Swain, before Sludden missed from close range for Spartans following a quick break, unluckily pushing the ball wide as he slid in.
The Edinburgh side continued to pose a threat and after 65 minutes substitute Errol Douglas, just three minutes after being introduced, forced Bald into another save. Alan Brown then shot narrowly wide as the contingent of away fans prepared for a nervy final 15 minutes.
They had every reason to be as Ali MacKinnon had a shot saved by Bald, and it was MacKinnon who was to emerge the hosts’ saviour, grabbing a sensational 92nd minute equaliser with a shot into the bottom left hand corner.
Berwick fans also had their hearts in their mouths a minute later when substitute Jack Beacher shot narrowly wide.
Spartans: Swain, Nixon, Malone, MacKinnon, Cennerazzo, Beesley, Thomson (Brown 46), Motion (Douglas 62), Stevenson, Bremner, Sludden (Beacher 80).
Berwick: Bald, Jacobs, Young, Fairbairn, Drummond, Notman, Currie, Gray (Miller 71), Lavery, Henderson (Russell 81), Willis.
Referee: Don Robertson.