Goals by Chris Caddon and Mark McGuigan saw Rovers move eight points clear at the head of Scottish League two, whilst Rangers, who have now slipped to ninth place, remain five points behind fourth with only four games remaining.
Berwick went into the game unbeaten in their previous four matches, but desperate for a win to maintain their play-off hopes.
The only change from the previous fixtures was the inclusion of David Gold, at the expense of Ross Gray in midfield.
Paul Willis had the first attempt on goal after nine minutes, when he shot from outside the box was saved by Parry.
After 18 minutes, Blair Hemderson, who had netted four times in the previous four matches, headed narrowly over and then Lee Currie had a shot blocked.
A minute later Kerr Young shot over the bar and as Berwick continued to create the majority of chances Gold had a shot saved by Parry after half an hour.
A minute after that Michael Andrews was forced into making his first save of the game as Mark McGuigan tested him from outside the box.
And as Rovers tried to force their way back into contention’ Andrews had to save again to deny Michael Dunlop from a header.
In the second half Rovers started brightly and Andrews had to save again, this time from Ross Davidson, but after 62 minutes the Berwick keeper was beaten when Albion full back Caddon fired a right foot shot into the net from outside the box after Edward Ferns created the opening.
New signing Chris Dillon was introduced by Rangers to try and give them some fire power up front, but Albion continued to press and after Davidson had a shot saved by Andrews, Rovers went 2-0 up after 71 minutes when McGuigan headed home from inside the six yard box following a cross by Marc McKenzie.
Berwick made a double substitution to try and turn things around, before Willis shot over. A minute later, at the other end, Caddon pulled a shot wide, but at the end of the day Rangers were unable to claw anything back .
Albion: Parry, Caddon, Dunlop, Dunlop, Turnbull, Ferns (Mullins 64), Fisher, Young (McCluskey 83), McKenzie (Love 86), McGuigan, Davidson.
Berwick: Andrews, Jacobs, Young (Martyniuk 77), Fairbairn, Maxwell, Gold (Gray 77), Wilkie’ Willis, Currie, Russell (Dillon 68), Henderson”
Referee: David Lowe.