Spanish midfielder Rodri scored a late winner for Manchester City after poor individual performances from the league title defenders.
City had more ball possession and tried to create chances throughout the first half. But Sheffield United held on with their dogged defense.
Erling Haaland had an opportunity to score for his side but he played the ball against the post.
In the second half, Haaland made amends, heading in a cross from Jack Grealish into the back of the net to put the Blues up.
Sheffield United equalized in the 85th minute when the hosts capitalized on a defensive error from Kyle Walker who failed to clear the ball and Jayden Bogle fired a superb shot away from Ederson.
City manager Pep Guardiola was not on the touchline at Bramall Lane having undergone back surgery. His side were looking to share points in their third game, but Rodri bailed them out with a stunning strike into the top corner in the 88th minute.
Sheffield United have not gotten any points since they were promoted to the Premier League side. Manchester City are currently on top of the table with 9 points from 3 games.