It's illegal to drive along the border wall if you are a US citizen

While spending time at the border, I had to ask permission to speak with illegal aliens and permission to be on the road next to the wall. It is clearly marked that only authorized personnel can be on the "Federal property"