There are a couple opinions bounced around in the article and in the comments. I have a slightly different view that I didn't find in either.
I believe that Dormin is an ancient evil that was sealed by members of the village that Wander belonged to. They split up Dormin and placed parts of him, or his spirit, in the varying colossi to act as guardians of this ancient evil. They then sealed off the land, and kept the one thing that could release Dormin from the Colossi, the sword.
Now, I believe that the elders kept details about this land secret, so that noone would get curious and possible upset what they had achieved. But, Wander hears that some powerful ancient being resides there, possibly a rumor, or someone just knew part of the truth, and he believes that this spirit can bring the girl back. He then somehow finds out that the sword is the key to getting in there, steals the sword, gets in, and the rest is history.
I think it's entirely possible that Wander believed the colossi to be evil. After all, the only person who can talk to him is the one thing that can bring his loved one back, and the colossi can't verbally defend themselves and explain. The best they can do is try to kill him. I do, however, believe that the colossi are good, just protective. After all, they should be protective.
So that's what I believe.
