I too sleep with my crystals,
I have Nuummite, Preseli, Blue Stone and two large Clear , Quartz Pebbles.

I have as yet experienced no adverse effects, my dreams have become clearer though.