Mobile Edge Computing (MEC) is delivering a rich portfolio of computation services to resource-constrained mobile devices, enabling ultra-low latency and location-awareness for the emerging mobile applications. However, the vulnerability of this new paradigm to potential security and privacy issues prevents mobile users from fully embracing its advantage. While various defensive strategies have been proposed to secure the connection between the end devices and edge servers, an equally important issue, the serverside risk is still under-investigated for most edge computing systems. To handle these server-side risks, a Risk-aware Computation Offloading (RCO) policy is proposed in this paper to distribute computation tasks safely among geographically distributed edge sites under server-side attacks. RCO takes into account the strategic behaviors of the potential attackers in the edge system and finds an appropriate balance between risk management and service delay reduction. The Bayesian Stackelberg game is employed to formulate the RCO problem, which describes an appropriate relation between the edge system (as a defender) and the attacker. In particular, the Bayesian Stackelberg game captures the uncertainty of attacker's behavior and enables RCO to work even when the edge system does not know precisely the attacker that it is playing against. To facilitate the derivation of Stackelberg equilibria, two pruning rules, Heuristic Pruning (HP) and Branch-and-Bound (BaB), are proposed. HP prunes by analyzing the user demand distribution and attack type, and BaB prunes by obtaining the tight upper/lower bound of edge system utility with assist of disjunctive programming and Bender's cut. Extensive simulations show that the proposed algorithm helps improve the scalability and efficiency of risk-aware computation offloading.