Federated learning (FL) enables an effective and private distributed learning process. However, it is vulnerable against several types of attacks, such as Byzantine behaviors. The first purpose of this work is to demonstrate mathematically that traditional arithmetic-averaging model-combining approach will ultimately diverge to an unstable solution in the presence of Byzantine agents. This article also proposes a low-complexity, decentralized Byzantine resilient training mechanism. The proposed technique identifies and isolates hostile nodes rather than just mitigating their impact on the global model. In addition, the suggested approach may be used alone or in conjunction with other protection techniques to provide an additional layer of security in the event of misdetection. The suggested solution is decentralized, allowing all participating nodes to jointly identify harmful individuals using a novel cross-check mechanism. To prevent biased assessments, the identification procedure is done blindly and is incorporated into the regular training process. A smart activation mechanism based on flag activation is also proposed to reduce the network overhead. Finally, general mathematical proofs combined with extensive experimental results applied in a healthcare electrocardiogram (ECG) monitoring scenario show that the proposed techniques are very efficient at accurately predicting heart problems.