Pirates reliever Brendan Donnelly (2-0) picked up the win with just one-third inning pitched in the eighth.
Brewers lefty Randy Wolf yielded only two runs over eight innings, scattering 10 hits while walking one and striking out four batters. His eight-inning outing marked the most for any Brewers starter through 20 games this season.
"I felt a lot better than giving up 10 hits," Wolf said. "I felt like I was able to throw pretty much all my pitches where I wanted to today. ... My fastball command was pretty good and my curveball, which has been so-so this year, was better today."
Wolf left with the game tied at 2, and thanks to a go-ahead home run by Prince Fielder to lead off the bottom half of the eighth, the Brewers' starter was in line for his second win over the Pirates in as many starts.
When it was over, however, Wolf had to settle for his second no-decision of the season.
More importantly, the Brewers took yet another loss with Hoffman on the mound.
"It punches the team in the gut. They battled very hard tonight to put us in position to win, similar to the other games I let get away," Hoffman said. "That has a tough effect on a ballclub. You come in here [to the clubhouse] and you can feel the mood. Randy pitched his tail off and he deserved to be on top as the winning pitcher tonight.
"As a whole, we lost the ballgame, and there's really nobody else you can point that finger to other than myself. Individually, that's the difficult part. Individually, you feel responsible for letting that get away."
Leading off the ninth, Hoffman gave up a home run to Pirates shortstop Ronny Cedeno on an 0-1 fastball that he drove out to left-center field. Hoffman answered by striking out Andrew McCutchen on five pitches, but the inning was far from over.
Andy LaRoche followed with a single for his fourth hit of the ballgame and Lastings Milledge doubled to left. Hoffman intentionally walked Garrett Jones, bringing up catcher Ryan Doumit, who crushed a 2-0 fastball to right for a game-winning grand slam.
With that, the Pirates went from staring down their 23rd consecutive loss at Miller Park to celebrating a stunning come-from-behind victory.
"Hitting is contagious. When the first two guys get on and they hit the balls hard, it gave everybody else confidence," Doumit said of the ninth. "I can't say enough about this win today."
The home runs in the ninth were the fourth and fifth allowed by Hoffman on the season in just eight innings of work.
Last season, Hoffman surrendered just two long balls in 54 innings.
Four of the five home runs have been hit on fastballs, while Hoffman -- who is 3-for-6 on the year in save opportunities -- has thrown noticeably fewer changeups in 2010 than in past years.
"It is a concern; we played a solid ballgame," manager Ken Macha said. "Guys played hard, we get the big hit from Prince. ... But those last three outs of the game are ... the most critical thing."
Fielder's home run, his second of the year, came off lefty Javier Lopez and gave the Brewers a 3-2 lead after battling with the Pirates in a rare pitchers' duel between the two clubs.
Right-hander Jeff Karstens pitched 6 2/3 innings for the Pirates, giving up just two runs on six hits while walking four and striking out four. Karstens was especially tough on righties in the game, allowing them to reach base just twice.
With the win, their first in Milwaukee since May 3, 2007, the Pirates snapped a seven-game losing streak overall and a 22-game skid at Miller Park.
"We haven't won here in Milwaukee in three years. Everyone knows that," Doumit said. "Once Fielder hit that go-ahead home run, it would have been so easy for us to roll over. We didn't give up. This one feels good."