Unless Bogut suddenly averages 25 points, 10 boards and 2 blocks in his prime, everyone is going to see him as a disappointment somehow. He was drafted on need, and might not turn out to be the best player from his draft class, but he was a smart pick at the time, as the Buck definitely needed a young talented Centre to build around for the future. Just because he's not dominating the league, it doesn't mean he wasn't the right choice for the Bucks that year. Not every #1 pick can turn out to be a superstar, but considering Bogut's impressive college credentials coming in, he had (and still has) as much chance as anyone in that draft class.
You can say something like "But they could have had Andrew Bynum!", and despite reportedly being a fast learner, he was still relatively young and an unknown commodity at the time. He was even considered to be a questionable pick for the Lakers late in the lottery. He's obviously turned out well, but who knows how he would have gone without Kareem and Phil Jackson's coaching and influence. I doubt that a 17-year-old Bynum being drafted #1 would have turned out well. The pressure of being the #1 pick, combined with learning the NBA game, and an average head coach in Terry Stotts, as well as being only 17, all would have combined to probably turn him into Kwame Brown or Michael Olowokandi 2.0. That is, no confidence in himself, and never turning into the player he should have become.
And there was no point in chosing Deron Williams, Chris Paul or Raymond Felton, because the Bucks already had Mo Williams and T.J. Ford at the PG spot. Marvin Williams would have been a bit of a head scratcher as well, because despite turning into a pretty good player, he really hasn't set the world on fire (possibly because of Atlanta's constant wealth of forwards), and he wasn't really what was desperately needed by the Bucks at the time.
If you're going purely by the PER (Player Efficiency Rating) stat over their careers (which is a decent indicator of a player's overall effectiveness in the minutes they play) then the 2005 Draft ranks like this so far:
#4 Chris Paul - 25.0
#30 David Lee - 18.1
#10 Andrew Bynum - 17.3
#3 Deron Williams - 17.1
#40 Monta Ellis - 16.3
#1 Andrew Bogut - 16.1
#17 Danny Granger - 15.8
#7 Charlie Villenueva 15.8
#45 Louis Williams - 15.8
#9 Ike Diogu - 15.7
#19 Hakim Warrick - 15.6
#26 Jason Maxiell - 15.5
#21 Nate Robinson - 14.9
#49 Andray Blatche - 14.6
#37 Ronny Turiaf - 14.5
#50 Ryan Gomes - 14.0
#33 Brandon Bass - 14.0
#8 Channing Frye - 13.9
#5 Raymond Felton 13.7
#22 Jarrett Jack - 13.2
#27 Linas Kleiza - 13.1
#23 Francisco Garcia - 12.9
#14 Rashard McCants - 12.6
#24 Luther Head - 12.6
#38 Travis Deiner - 12.5
#34 C.J. Miles - 12.0
#31 Salim Stoudamire - 11.8 (Out of the league)
#18 Gerald Green - 11.5
#16 Joey Graham - 11.3
#6 Martell Webster - 11.2
#25 Johan Petro - 10.6
#48 Mickael Gelebale - 10.0 (Out of the league)
#32 Daniel Ewing - 8.3 (Out of the league)
#15 Antoine Wright - 7.0
#53 Oriene Greene - 6.0 (Out of the league)
Anyone who hasn't played less than 100 games and/or 1,000 minutes isn't included there, because most of their their numbers tend to be a little bit unrealistically inflated. There's probably only a few players of interest below those benchmarks anyway:
#56 Amir Johnson - 17.5
#13 Sean May - 16.0
#55 Lawrence Roberts - 13.5
#36 Ersan Ilyatsova - 12.1
#20 Julius Hodge - 10.7
And comparing Bogut to #1 draft picks of the last 10 years, he's decent, but not spectacular:
2003. LeBron James - 25.7
2002. Yao Ming - 23.2
1999. Elton Brand - 22.4
2004. Dwight Howard - 20.7
2007. Greg Oden - 16.5
2005. Andrew Bogut - 16.1
2008. Derrick Rose - 16.1
2000. Kenyon Martin - 15.9
2001. Kwame Brown - 12.9
2006. Andrea Bargnani - 11.7
People can judge for themselves by the numbers, but I think for what was available at the time (in terms of player ability and knowledge of how they'd pan out), and for what the Bucks needed, Bogut was the right pick.