It took 93 minutes of constant attacking before Ethan Ebanks-Landell netted United's winner on a tense night at Bramall Lane.
For the second game running the opposition finished with nine men and Bury looked like holding on until their former loanee pounced to score.
Wilder made two enforced changes. Coutts was fit after picking up an injury on Saturday but Freeman and Duffy were ruled out. O'Connell and Scougall returned to the starting line-up. There was a reshape with Basham moving into the right wing back role.
The referee was the highlight of the opening ten minutes. He called a foul throw against Leigh after previously warning him and was then very expressive when indicating that he wasn't awarding United a penalty when Scougall and Barnett clashed.
The Blades were dominating possession early on. O'Connell played a number of cross field balls to Basham and Coutts and Fleck were seeing plenty of action.
Sharp prodded over after Lafferty tapped back in Basham's cross and a long-range effort from Coutts was gathered by Williams at the second attempt, whilst a second penalty appeal was turned down when the ball hit Kay's arm.
Bury were getting white shirts behind the ball whenever possible, making it difficult for United, but on 27 minutes Lafferty headed over when unmarked from Sharp's cross.
More United chances followed, with two coming from O'Connell crosses - Basham headed wide and a glancing flick from Scougall touched the top of the crossbar.
On 39 minutes a good move involving O'Connell, Fleck and Scougall resulted in Done shooting at Williams, whilst the last highlight of the half was a red card for former Blades Academy youngster Mellis, who lunged in at Done and gave referee Joyce little option.
For the start of the second period, Clarke replaced Scougall but the first chance of the half fell to Bury and Leigh's shot from distance was close but drifted wide.
United were quickly back on the front foot. A scramble saw Kay head clear when Basham was looking dangerous and a Coutts corner was headed over by O'Connell. Done also headed over a Fleck cross, whilst a defender was sacrificed for Chapman just after the hour.
Chapman almost made an instant impact, from his cross Clarke's header was cleared off the line by Leigh. The pressure continued and a third penalty appeal was denied when Fleck and Vaughan clashed.
Lavery was introduced for the final 15 minutes and Clarke headed home a Coutts cross on 78 minutes but referee Joyce adjudged that the striker had fouled.
On 85 minutes Leigh was booked for a foul on Coutts, it was his second yellow after timewasting moments earlier, as Bury dropped to nine men.
Bury now had everyone behind the ball, protecting their point, but Fleck fired over on 86 minutes and the crosses continued to rain in. Basham missed a header from a Coutts cross and the ball hit Kay and landed in the goalkeeper's hands.
As six minutes of injury-time was announced all three of the subs combined to go the closest yet for United. From Lavery's cross, Clarke battled and Chapman's shot beat Williams but Kay miraculously cleared off the line.
Kay, however, could not stop United's winner in the third minute of added time. Lafferty and Lavery combined and EBANKS-LANDELL sidefooted into the corner, prompting wild celebrations from the players, and Wilder, who headed down the South Stand touchline.
United comfortable played out the three remaining minutes to clock up a seventh straight home win.
Blades: Moore, Fleck, O'Connell, Basham, Scougall (Clarke 45), Sharp, Done (Lavery 75), Wright (Chapman 63), Coutts, Ebanks-Landell, Lafferty.
Unused: Hussey, Lavery, Ramsdale, Brown, Whiteman.
Bury: Williams, Jones, Leigh, Kay, Mayor, Pope (Etuhu 45), Vaughan, Mellis, Soares, Burgess (Tutte 60), Barnett (Danns 77).
Unused: Rachubka, Hope, Miller, Bedeau.