Well let's see ... initially there was "fire" ... then we added "matches". At some point, I came up with the idea of giving experience for building these fires. So it became a skill.
As for training speed, it seems to be one of the fastest skills to level ... assuming you can afford matches. That is it's real purpose: what us developers call a "money sink". Essentially, every day more and more money is introduced into the game ... without a way to get that money out of the game, inflation gets out of control. In other words, players need something to buy with all the gold they collect.
We have nearly the exact same amount of gold in-game today as we did when I introduced the Arson skill. So it seems to be doing it's job. At least up to a certain point, the highest level players still have stockpiles of gold they may never be able to spend (I'm working on that problem).
There is no way to turn it off, other than to not build fires. You should see less notifications as you level the skill, as each level requires more experience.