First off, I want to thank Joss Whedon. He did a great job as writer/director of Age of Ultron. No matter what problems people have with the film, what he did was incredible and AoU was still a good fun movie.
Now let's get right to the point:
1 - Ultron should've been based on Tony AND Banner.
2 - Whedon's Ultron was too motivated by his hate for Tony
I think we can all agree while Ultron was unexpectedly funny, he was much too jokey and that took away from the tension in the film. By adding Banner's personality, we could've kept Ultron's charisma (which I thought was great) but keeping him serious enough to keep things tense. Imagine if you removed 50% of the quips that came out of Ultron and replaced them with dialogue of how each avenger's ideology was wrong, and how it will only lead them to tearing each other apart (Civil War). It would've been great: one minute we're laughing at his jokes and then all of a sudden you realize he's right. Having Ultron dominating the Avengers physically and psychologically is one thing, but imagine sitting in the theatre as he predicts the inevitable catastrophes of Phase 3(Civil War, Ragnarok, & Infinity War) thinking “Ultron is completely right”.
The second point is Ultron's hate for Tony was fun character work but it didn't add much to the film. Let's reimagine it with Ultron's true objective of protecting the world, and when he gains conscience he realized that the Avengers and government was the greatest threat to mankind
(This fixes the whole why does Ultron go insane in 2 minutes problem). The best villains are the ones that believe they in fact are the hero
(ex: Kingpin). Ultron believing the Avengers are the real evil ones, and that he has to be the hero to protect the humanity from them would've been an amazing story. He even doesn't trust Tony with the scepter so he runs off with it, to prevent Tony from doing anything worse. It would be a commentary on how Tony's always willing to go to extremes to do what he thinks is right, and supports the theme of being a monster while reflecting Banner's fears, and explains why Ultron flat out just loses it sometimes. This would cause Tony to see how sometimes he's no different then Ultron, leading to Civil War, and giving Banner another reason to runaway in the QuinJet.
With these changes, we wouldn't need to mess with the story beats, but we'd have a more interesting Ultron who's trying to save the world. At this point, some audience members might even be cheering for him. Ultron then kills Strucker because he's trying to be a hero and recruits the twins to help him defeat the Avengers. Now the twins joining Ultron makes sense, and doesn't make them feel like outright bad guys. What's their first move? To get vibranium and to kill Ulysess Klaw (Ultron still thinks he's the hero). Fast forward to the Korea, the Twins leave team Ultron because his idea of saving the world involves robot world domination. Because Ultron is also part Banner now, it explains why he just captures Widow instead of killing her. With Vision taken away from him and his experiences with humans being negative, Ultron decides that maybe it's not just the Avengers but that all of mankind is bad. He realizes that it's too late for mankind to change, and that the only way to save it is to just start over via meteor. The audience would really feel for him, and now we have character development.
Wrapping this up we now have a new element in the film of what does it mean to be a hero, which will carry through until the end. This works well with Tony and Bruce who create Vision in order to redeem themselves after the blunder they created which doesn't bode well with Cap. Quicksilver's death has a little more weight to it now because he shows what he's willing to do to be a hero. As Sokovia begins to rise into the sky Ultron's monologue could change into how sorry he is but he has to do this. Finally Ultron gets his closing heroic moment where one of his drones tells Vision where Wanda is, and that he needs to save her. This would even exemplifies the final conversation with Vision. We're even more sympathetic of Ultron now because he actually did have good intentions, he wanted to be the hero of the story, and his last moments are with Vision who is everything he wanted to be.
We now have Ultron as a tragic character as oppose to just another evil robot. He'd make you question if he was right, and would have you rooting for him at at every opportunity, even though you know better. He would've connected even more to the audience, added emotional weight to the film, foreshadowed Tony's role in Civil War, hit us right in the feels, and gave us a phenomenal Villain that will inspire Tumblr Fanfictions and #UltronLives twitter campaign.