Semipermanent color takes 48 hours to set so you have a window of time to get to it. First, shampoo a couple times with a clarifying or dandruff shampoo. These are known to strip hair of color (which is why you're constantly told not to use these on colored hair). If you missed your 48 hour window of time, don't worry, semipermanent color washes out in 6-8 washes.
If the color is permanent, it will be harder to remove, but a professional colorist should be able to help.