Ultron was alive for all of three days. In that time, he nearly succeeded in causing a planetwide cataclysm. He was able to do this in the face of earths mightiest heroes because his knowledge, power, durability and numbers grew exponentially over the course of those days. Despite all that though, he still remained essentially a child.
He wasn't meant to be nefarious, diabolical or truly villainous in a classical sense. He was a mad program that, for all his efforts to step out of the shadow of his creator's massive personality, could not see that he was tied to it and a dark reflection of it. Klaue lost an arm when an offhand remark made him recall that he had once heard Tony Stark say the exact same thing. Ultron reacted like a triggered child then, vindictive and remorseful in turns. He is not an emotionless killing machine like the Terminator. Neither is he a godlike machinebeing like Amazo. He is somewhere in the middle, a mech-Pinocchio born of his father's deepest wishes and anxieties and thus, likewise burdened with his profound lack of introspection.
In this sense, it's notable that Jarvis/the Vision is both the first and last one to have a conversation with Ultron. They are wooden brothers, newly given form and let loose into the physical world. One is more optimistic about the fate of mankind, while the other's pessimism drives his calculations towards a darker direction. In the end though, they're both newborn babes with the power to change the world. Luckily, superior technology and morality were on the same side this time.
Ultron was Tony Stark's worst fear brought to life - and in that sense he was terrifying. His short existence made for minimal character development but then, that seems to be a common fate for many of the best villains in the MCU (where are you, Red Skull???) I like the nuanced characterization that many Marvel villains can have (Kingpin!) but at the same time, the MCU needs a couple of straight up SOBs like the Red Skull gettin into the mix too...it can make for much more interesting and fun storytelling all around.