Background: Cardiovascular diseases are recognized generally to be among the number one illness causing death across the globe. Predicting heart disease using a computer-aided technique makes it easier for medical practitioners to diagnose and thereby saving lives and reducing costs. Feature selection has become an essential component for developing Machine learning models. It chooses the most relevant features from the available dataset, thereby shortening the training period, making the model easier to train, improving generalization and decreasing overfitting without necessarily compromising the system’s accuracy. Aim: The purpose of this work is to design and build an optimal model for the prediction of heart diseases, especially at an early stage by considering certain features that are most relevant for the prediction without compromising the system’s accuracy. Method: The Cleveland UCI dataset with 303 instances were used in training the model and the findings showthat selectKBest is an effective tool in improving the prediction of heart diseases. The performance metrics Accuracy, Sensitivity, Precision were measured. Results: the study found that when hybridizing k-Nearest Neighbor Bagging, Decision Tree Bagging, Gradient Boosting generated the highest accuracy of 90%, 85% and 88% respectively.