I imagine that it would be something involving the post.asp script where it checks to see if there are characters that need to be converted to images. You would need to count every time it finds one, and check it against the limit before letting it update the tables.
I'll play with this one a little, as we're having this problem on our boards right now.