Security systems using brain signals or Electroencephalography (EEG), is an emerging field of research. Brain signal characteristics such as chaotic nature and uniqueness, make it an appropriate information source to be used in security systems. In this paper, E-BIAS, a pervasive EEG-based security system with both identification and authentication functionalities is developed. The main challenges are: 1) accuracy, 2) timeliness, 3) energy efficiency, 4) usability, and 5) robustness. Therefore, we apply Machine Learning (ML) algorithms with low training times, multi-tier distributed computing architecture, and commercial single channel dry electrode wireless EEG headsets to respectively overcome the first four challenges. With only two minutes of training time and a simple rest task, the authentication and identification performance reaches 95% and 80%, respectively on 10 subjects. We finally test the robustness of our EEG-based seamless security system against three types of attacks: a) brain impersonation, b) database hacking, and c) communication snooping and discuss the system configurations which can avoid data leakage.