The Principle of Worshipping God Alone in the Old Testament

We are used to thinking of pagan gods as simply not existing. Is that how the israelites would have understood matters? Actually know and to understand why we need a little more background on the old testament world view. Remember the image of the divine court in heaven, with god ruling everything as the supreme being but with various angelic courtiers of different ranks under him. So it didn't matter what country an israelite was in, he was still supposed to worship only the lord.