SDC Building takes no damage in Combat Phase....

So maybe I am 100% wrong here and I don't understand the mechanics of the new settlement system, but during Combat Phase, isn't everything on the island vulnerable to attack? For whatever reason this 2x2 wood structure took no damage from ship cannons and the fire attack from the swivel.