If you are looking at this from an Abramic (Judeo/Christian/Islamic) point of view, then yes, it is idolatry, because the concept of idolatry itself is an abramic concept, as is the concept of "worship" in this sense. But this is taking a concept out of one context and trying to apply it to a different context.
if I, as a Buddhist, brought a Buddhist point of view to a Christian concept, I might ask, "isn't belief in God just an extension of the idea of an existent 'self'?" and from a Buddhist point oif view the answer would be yes, but from a Christian point of view the answer might be no.
So, if you mix these different contexts the answer is going to be confusing.
The regarding of idols as actual beings, or as objects of devotion depends on the notion that the beings represented in these images, statues and so forth have an independently functioning reality to them. For example, Catholics pray before a statue of Mary because they believe Mary exists outside of the mind of the worshiper and exercises a kind of conscious awareness that hears the prayers of the worshiper.
In Buddhism, all Buddhas and so forth represented as images are in fact ultimately aspects (or, in Vajrayana Buddhism, manifestations), of one's own true mind, the awakened state free of confusion. In discussing whether one regards them as 'real' or not, one must also ask to what degree one sees oneself as 'real'. In buddhism, seeing yourself as real (as Christians regard themselves as "real') might be considered a type of idol worship.
To say that the deities and various celestial beings in Buddhism are real or not is actually beside the point.
Are they any more "real" than the person who "worships" them?
Profile Picture: "The Foaming Monk"
The Chinese characters are Fo (buddha) and Ming (bright). The image is of a student of Buddhism, who, imagining himself to be a monk, and not understanding the true meaning of the words takes the sound of the words literally. Likewise, People on web forums sometime seem to be foaming at the mouth. Original painting by P.Volker /used by permission.