Theoretically such fines could be imposed since there are laws and acts in place which would empower KZN Wildlife to do that. I've never heard of it happening though.
There used to be a system whereby people from Bergwatch would check up on certain caves after hikers had been there. If there was a mess, the hikers would be contacted (details taken from the rescue register), and asked to return and clean the cave. I've also seen a "name and shame" system some years back, whereby pictures of the mess would be posted on the notice boards, along with the names of those who made it. If I remember correctly, I've even seen this in connection with Xeni Cave before.
I have no idea how effective this was. Obviously it required active policing and manpower, and unfortunately the authorities seem to have their hands full chasing the smugglers these days. Though, I think a short list of popular caves which are easy to walk to, could be checked on regularly by patrols. I will look into the matter a little more and see what systems are in place (other than the systematic evaluation of trails and caves already taking place).
A lot can be also said about concerned hikers, such as we are, taking ownership and doing whatever we can about the problem. This includes things such cleaning up caves ourselves, reporting it to the authorities (and following it up), talking to other hikers about it, and even confronting culprits at times.