Face recognition is normally used in automated surveillance, individual identification, and database searches for specific faces. Face detection, representation, and matching are the different stages of the face recognition process. The face detection starts from the query image, and then features are retrieved using a face recognition algorithm in the next stage. Matching the query face with the database is the final stage. However, face recognition algorithms perform low in unrestricted environments such as those with variations in an individual's lighting, posture, and facial expressions. This paper proposes a face recognition system designed to address these challenges using Convolutional Neural Networks (CNNs), Local Binary Pattern (LBP) histograms, and Histogram of Oriented Gradients (HOG). Initially, face detection from the input image is accomplished using the Viola-Jones technique. The feature space is created through fusing the features that were extracted using CNN, HOG, and the LBP histogram. SVM and KNN classifiers are used to assess the classification ability for various HOG cell sizes