Berwick went down to a 2-1 defeat at Stirling on Saturday, and in so doing slipped to the bottom of the Scottish Second Division.
East Stirling won 3-0 away at Queen’s Park and as a result leapfrogged Rangers in the basement stakes.
The Shielfield Park outfit, who have not won a league game since Boxing Day, will now be hoping to lift themselves back off the bottom when they face Annan in a re-arranged fixture at home on Tuesday.
Berwick took an early lead through striker Blair Henderson, but two goals in the space of three minutes saw Albion hit back to take the win.
First off all Ross Smith equalised with a header, before Sean Dickson fired home from the penalty spot after Berwick defender Jordan McGregor was sent off for a last man challenge.
After Callum Crane had a shot from outside the box saved, Henderson put Rangers in front after nine minutes with a shot into the bottom corner after winning a challenge and then riding a tackle.
But on 13 minutes it was all square again when Ross headed in a cross from Mark Lamont.
The decisive point in the game came after 15 minutes when Darren Smith drew a foul from McGregor, who was shown a straight red card, and on 16 minutes Dickson netted from the spot.
It was an uphill struggle for the visitors after this, although they tried to redeem themselves with Finn Graham (twice), Michael McKenna (twice) and Henderson all going close, with goalkeeper Smith saving on each occasion.
At the other end Dickson and Phil Johnston both shot wide, with Steven Doris also going close just before half-time.
In the second half Stirling started strongly and they created four chances in the first four minutes, but failed to convert with any of them.
Doris then shot wide and Dickson had a header saved by Kevin Walker before Berwick tried to force their way back into contention, notably through Henderson and Graham.
Smith and Ross McMillan then both had efforts saved by Walker whilst Lewis McLear pulled a shot wide.
Substitute Darren Lavery and Stevie Notman both had efforts saved late in the game as Rangers tried to salvage a point, but it wasn’t enough as they dropped to the foot of the table.
Stirling: Smith, Johnston, R Smith (McMillan 67), Buist, Forsyth, Dickson (Mazel 78), McLear, Robertson, Lamont (Olanrewaju 76), Doris, D Smith.
Berwick: K Walker Beveridge Crane Notman McGregor Wilson Graham (Lavery 74), Stevenson (McNeil 68), McKenna Henderson (Trialist 79), Martin.
Referee: Steven Kirkland.