I don't know why anyone would want to annoy a worm, but apparently lemon balm does the trick. I had some less-than-perfect leaves that I didn't use in making my lemon balm tea, so I fed them to the worms. Rather, I put the leaves in their bin; feeding was out of the question. The next time I checked, all the worms were huddled on the side of the bin furthest from the leaves. They didn't seem particularly unhappy, but they didn't return to the other side until a few days after I removed the offending foliage.