There must be nutrients in the water that are supporting them,as the tank ages they will stabilize I expect.
Different rock seems to spawn different small things ,when Ive had different tanks there have been as I said tiny worms,your sponges, I even have the odd few of those bannana algaes growing from nowhere after um....4years or so.
If they bother you remove them,but they will come back to the degree the tank can feed ,where it is right now in its development.