There is pretty good evidence that humans have been conquering and often exterminating each other for more or less ever. It's likely that our pre-human ancestors behaved similarly.
By that grim standard, empire building has mostly been a less brutal alternative, though often not a whole lot less brutal. According to Niall Ferguson, the former colonies collectively asked in the UN that they be paid an idemnity of 777 trillion dollars for the harm they suffered in conquest. Well, they shouldn't hold their breath. One problem, of course, is that most of the perps and victims have already passed from the world, and the survivors, in many cases, are descended from both. More to the point, we are all of us survivors of those who stole, murdered, and otherwise competed viciously for the privilege of bringing us into the world.
The peoples of the former colonies are not the people that the conquerors found. They have been transformed and usually racially mixed. That is even more true in a cultural sense than a genetic one. The political institutions of, say, India, or Egypt, owe less to those of their ancestors of 400 years ago than they do to those of Britain. They expelled their Western rulers but kept huge portions of the culture those rulers brought. Of course that trend is even more pronounced in the settler colonies like the US, Australia, and Latin America, where the emigrants form a large proportion of the population.
Representative government, Western television and cell phones are evidence that the colonies got something they wanted out of the arrangement.