Berwick recorded their first win in eight league games when they beat Albion Rovers at Shielfield on Saturday.
A goal at the start and the end of the first half put Rangers in the driving seat, but they made it difficult for themselves when they scored an own goal midway through the second.
Berwick got off to the perfect start when Damien Gielty fired them in front with his first goal of the season after just 33 seconds of play.
Lee Currie swung in a corner from the right and as it was cleared to Gielty on the edge of the box he shot home first time through a crowd of players.
Berwick’s best chances after that came from set pieces, two Currie free kicks causing problems in the visitors’ defence.
Scott Mercer shot over under pressure from Berwick keeper Paul Grant after Albion’s best player, Mark McGuigan, broke down the right.
But Rangers gave themselves some breathing space when they added a second after 41 minutes when they were awarded a penalty for a foul on Ross Gray in the box.
Skipper Currie had his original spot kick saved by Rovers’ keeper Neil Parry, who turned his shot onto a post, but the Berwick midfielder was quickest to react to the rebound and headed into the net as he followed up.
Rangers had unexpectedly dropped leading strike pair Darren Lavery and Scott Dalziel for the match - they were both on the bench.
Josh Morris and Kenny O’Brien were asked to lead the line and they made themselves busy, although both visibly tired as the second half progressed and were eventually replaced by their more experienced counterparts.
At the start of the second half O’Brien glanced a header wide from a Gielty cross and Hoskins fired over after the ball broke to him on the edge of the box.
The second half almost became a battle of wills as both sides battled for supremacy. Berwick had a two-goal advantage and a third would almost certainly have killed off any hopes of an Albion fight-back.
But after 69 minutes the visitors were handed an unexpected lifeline when full-back Dean Hoskins inadvertently scored an unfortunate own goal with an attempted back-pass from 20 yards out which beat the bewildered Grant in the bottom corner.
The introduction of Lavery and Dalziel gave Rangers fresh impetus, but they were unable to add a third goal and in view of the fact they had only taken three points out of a possible 21, the players were naturally pleased to hear the final whistle.
Berwick Rangers: Grant, Jacobs, Hoskins, Dunlop, Tulloch, Gielty, Currie, Notman, Morris (Lavery 73), O’Brien (Dalziel 83), Gray
Subs: Lavery, Dalziel, Janczyk, Ronald, Carse, O’Connor.
Booked: Gray 85 foul.
Albion Rovers: Parry, Reid, R Dunlop, Cusack, Donnelly (Tiffney 86), M Dunlop, Mercer (Flood 46), Shankland (Chaplain 46), McGuigan, Dallas, Phillips. Subs: McGinley, Flood, Tiffney, Chaplain, Russell, Maguire
Booked: M Dunlop 43 foul; Donnelly 53 foul.
Referee: Colin Steven.