Autonomous and intelligent systems are increasingly deployed in complex, real-world environments characterized by stochastic dynamics, partial observability, delayed feedback, and continual change, where classical model-based control strategies often struggle due to their reliance on accurate system identification, fixed assumptions, and limited scalability. In response to these challenges, Reinforcement Learning (RL) has emerged as a compelling control paradigm that enables agents to autonomously learn optimal or near-optimal control policies directly through interaction with their environment, leveraging reward-driven feedback rather than explicit system models. This article surveys and synthesizes reinforcement learning-based control mechanisms with a particular emphasis on actor-critic architectures and deep reinforcement learning approaches for continuous control, which have proven especially effective in high-dimensional and nonlinear domains. Drawing on foundational and influential studies published between 2000 and 2021, the discussion examines how RL frameworks facilitate adaptive decision-making, online policy improvement, and robust control under uncertainty, while also addressing critical issues related to convergence, stability, safety, and sample efficiency. Representative applications in robotics, autonomous navigation, and intelligent cyber-physical systems are highlighted to demonstrate practical impact, and publicly available architectural diagrams are integrated to clearly illustrate core learning loops, policy-value interactions, and control workflows, providing a cohesive and accessible reference for researchers and practitioners designing next-generation intelligent autonomous controllers.