A lot depends on what you consider a supercar.
Assuming you mean a 200+ MPH car that handles the cheapest would be a Ford Mustang Shelby GT500. A little higher up would be a Chevrolet Corvette ZR1 and the Dodge Viper SRT.
By raw performance I think the GT500, ZR1, and SRT all qualify as supercars but to me none of these cares seem exotic. Part of owning a supercar is not seeing another one in the parking lot. To me the cheapest exotic supercar would be an Audi R8. If I were in the market for a car in the six figures I think I'd go for a McLaren MP4-12c though. You won't see an R8 or MP4-12c every day though I have seen them parked at the posh Dallas Northpark Mall before.
If you just want to go fast what you need is a sleeper car because exotics attract too much attention from law enforcement. A fast car that doesn't call a lot of attention to itself is a Subaru Impreza WRX STi. Not exactly a supercar but still faster than 99% of the cars on the road and the car is kinda' ordinary looking and even ugly so every cop on the beat won't be speed gunning for you. It is also cheap enough that you have some money left over for a few go-fast upgrades.
The cheapest supercar I have been willing to spend my money on was the Bugatti Veyron Supersport.
However I also ordered a little Lamborghini Aventador to potter around in while the Bugatti was in for servicing.