Just because Israel got into trouble and lost the Ark of the Covenant, that doesn't mean the Philistines weren't in trouble for taking it. There are many other cases like this in the Bible... where there was not a good side versus a bad side... there were just 2 bad sides. In this setting both groups displeased God. The Jews AND the Philistines found themselves in opposition to Him. And, it didn't take too long for the Philistines to start suffering calamitous consequences.
Throughout human history men have been worshipping anything and everything except the true God of heaven. In this chapter we find a microcosm of why such idolatry is ridiculous. When the Philistines put the Ark of the Covenant into their temple for their god Dagon, 2 nights in a row God miraculously caused the graven image of Dagon to fall on its face before the Ark. But, that wasn't all that God did to show His disapproval. God caused all the men who lived in the region around the temple of Dagon to become very ill with a physical ailment. When the Ark was sent to other areas in Philistia (Gath and then Ekron), the judgment of God became increasingly severe against them all. What they expected to be a good luck charm of sorts, for them became a horrific curse.
Someone might look at this whole situation in retrospect and ask why the Philistines got into so much trouble. After all, they didn't know any better, right? And, they were truly being used by God to expose the evils of the Hebrews. So, shouldn't they have been given a pass? The answer is: evidently not! Not for them, and not for us. When we head out to "set someone else straight" we would do well to remember that we are obligated to avoid becoming crooked ourselves in the process. God may use us to help or even to discipline another one of His children, but if we are not careful we will earn for ourselves an equally hard spiritual spanking in the process.