I believe its another fertility reference. Easter in the Christian sense is about related to new life (death and resurrection of Jesus), and also the Easter celebrations common now come from Pagan fertility festivals. Eggs are symbols of new life, and rabbits are rather prolific breeders.
Haithem is right. Eggs and bunnies are old pagan symbols. When Christianity spread in Europe, the priests adopted the rites and symbols of the old religions. Because it was way easier to make people turn to the new religion while letting them keep their old rites.
I'm muslim..but I think .Easter bunny comes from pagan times,it was a symbol of fertility and the rebirth of nature following winter.
opps, please ignore the "about" in the first line! I should not answer things at 1am, I cant type properly!