I agree the neighborhood near the Coliseum is way too rough. (There are actually nice parts of Oakland, but not near there.) But more importantly, it's a horrible football stadium. It was a great baseball park before 'Mount Davis' was built in centerfield to accommodate the Raiders. Now it stinks for baseball or football.
Honestly, I can't think of a single suitable venue in the Bay Area. Oakland's out, Candlestick Park is a dump, and Memorial Stadium at Cal is waaaay too flat. The new stadium being built at Stanford looks nice, but the capacity isn't big enough (50,000).