In the "Brihadaranyaka Upanishad" it is said that it is not the physical person who is attractive, but it is the Atman residing in that person which attracts us. It is that which provides all our delights. Love for someone, the delight experience in that love, both come from the source we call God.