Why do you want to dig them up? Are you sure they're overcrowding? If so, I would do it in September when the leaves are turning yellow. Or do you just want to move them to a different location? If that's the case I think you can do it now as long as you don't disturb the bulbs too much, but I think September might still be the preferred time, which is when they're more dormant.
You better leave Lily alone. She's has been planted here for a reason & is here to stay. She's not disturbing any of us bulbs. So to answer your question; there is no right time to remove Lily. ツ
ohhhhhh! Then you should dig them up right before you move, and try to leave the soil around them intact :)
I want to dig them up cuz I moved and wanted to take them with me