Big data analysis poses an obstacle to machine learning, researchers, policymakers, and the
development sectors. Advanced high-throughput technologies produce a large amount of data in a
parallel and efficient manner. These datasets are characterized under high dimensions. The
researchers are confronted with a significant obstacle in such cases. Also, in the case of the high
dimensional dataset, there may exist outliers. This study proposes two approaches to select an
optimal set of features in the presence of outliers and aims to minimize the error rate. The proposed
methods uses different robust measures of scale such as Confidential, Confidential with a robust measure of location median. Initially, a minimum subset of
variables is selected using the greedy search technique. Furthermore, a robust Fisher score is
calculated for the remaining variables which are arranged in decreasing order of magnitude. The
minimum subset and the variables selected by the robust Fisher score are then merged. To deal
with redundancy in the selected variables, Ridge and Least Absolute Shrinkage Selection Operator
(LASSO) are used. The proposed method is evaluated empirically on publically available datasets
and the results are compared with well-known variable selection techniques on the basis of
classification error rates using 𝑘-Nearest Neighbors (𝑘-NN), Support Vector Machine (SVM), and
Random Forest (RF) Classifiers.
