Coldstream dropped to second bottom of the East of Scotland Premier Division after getting off to a bad start in Edinburgh.
But after a poor first-half performance, Stevie Gibbard’s injury-hit team showed spirit and character to mount a comeback attempt against ten men before being hit with a late sucker-punch.
Civil Service Strollers moved up to third and are now unbeaten in their last six matches after racing into a two-goal lead at Edinburgh College Sportsground.
Ross McNeil’s side went ahead when striker Kieran McGachie found space to fire home from inside the box and Chris Moffat doubled the lead inside 25 minutes by heading home from a corner.
Coldstream defender Michael Allan had to be taken off with a suspected broken nose to compound a miserable first half for the visitors.
The Streamers pushed up early in the second half but were caught when a through-ball allowed McGachie to outpace Phil Johnson and slot past goalkeeper Mark Walters.
But the visitors refused to cave in. When Craig Heugh was upended in front of goal half an hour from time, Connor Kane was ordered off and John Crawford scored from the spot for Coldstream.
Coldstream turned the screw, but Strollers grabbed a fourth on the break just before full time from Chris Tobin.
“The first 25 minutes was sublime,” said McNeil.
Coldstream: Walters, Powling, Allan (Johnson 33), Ainslie, S Simpson, Rutherford (Gibbard 76), Hebdon, Crawford, J Simpson, Heugh.