Kenny McLean, Todd Cantwell and the prolific Teemu Pukki took advantage of a ragged City defence without the injured Aymeric Laporte for a remarkable upset despite Sergio Aguero and Rodri pulling goals back for the visitors.
Defeat for City sees Pep Guardiola's men already fall five points behind European champions Liverpool – who have a perfect record from their first five games – in the title race.
"We are absolutely delighted, it is a great day for us, for the whole club and for the supporters," said Norwich manager Daniel Farke, who had to name two goalkeepers on his substitute bench just to make up the numbers due to an injury crisis.
"I can’t praise enough the spirit and mentality of my players because it was an unbelievably complicated week.
"We worked on a different approach because we thought we have to change a bit our base formation for this game, but after all these injuries, we turned back because I didn’t want to annoy the lads too much or confuse them."
City's decision not to strengthen at centre-back over the summer despite former captain Vincent Kompany's departure to become player-manager at Anderlecht could cost Guardiola's side the title with Laporte sidelined by a serious knee injury until at least January.
"We have to learn from this and carry on," said Guardiola.
"We did not have urgency in the final third that we normally have. In football you can't always avoid mistakes.
"I don't know how many shots we had or how many they had, but football is about goals and about what you do in the boxes."
The City manager fielded his only two fit centre-backs John Stones and Nicolas Otamendi, but they struggled to cope and gifted Norwich back their two-goal lead at the start of the second-half after Aguero had reduced the arrears.
Guardiola's decision to rest Kevin de Bruyne will also be questioned as the visitors started slowly and were punished when McLean rose highest from a corner to head home the opening goal.
It was 2-0 just before the half hour mark as Kyle Walker played Pukki onside and the Finn unselfishly squared for Cantwell to tap into an empty net.
"We were a bit scared of the quality they have, but from the beginning you could see we do have a chance and from that we built our belief," said Pukki.
City looked to have a route back into the game just before half-time when Bernardo Silva's cross picked out Aguero to continue his record of scoring in every game so far this season in the Premier League.
But the visitors' momentum was halted just five minutes after the restart.
Stones needlessly played Otamendi into trouble and the Argentine was robbed by Emiliano Buendia, who fed Pukki to prod home his sixth goal of the season.
Guardiola responded by throwing on De Bruyne and Gabriel Jesus, but it was not until Rodri drilled into the bottom corner two minutes from time that City looked like mounting a late comeback.
Aguero, Jesus and Raheem Sterling all fired too close to Tim Krul in the dying seconds as Norwich held out for a famous victory.