Human gender plays an imperative role as social construct and an essential form of an individual’s personality. Gender classification uses the face of a person in each image to recognize the gender (male or female) of the given person. There are two types of input face images (standard or non-standard). It is highly reflected in social communication, forensic science, surveillance, and target marketing. Gender recognition previously depended only on standard face images. Using the term “standard” means that the image was taken in a standard light without any background variations and without any cropped parts. This type of image facilitates solving gender recognition problems. However, if some variations happened to these images, the result would not be accurate. But most real-life images are not captured in a suitable manner. These images are called non-standard images, as they have a lot of variations, like illumination and head pose. The image may also have a lot of faces, where one of them may wear sunglasses or other accessories. Using this type of image will affect the accuracy of gender recognition approaches. Nowadays, selfie images appear as they are non-standard images. People take selfie images of themselves. Selfie images are very complex, as some parts of the images are cropped and damaged. This paper proposes a (CNNGA) technique to increase the performance of gender recognition from unconstrained and selfie images. This technique relies on merging deep learning approach with a genetic algorithm. This proposed technique achieves 90.2% accuracy in recognizing gender from the selfie dataset. The experiments use various challenge datasets, which are widely adopted in the scientific community like LFW, Data Hub, FERET, and Caltech-web Faces.