This Article Presents a New Optimization Method for Complex Power Distribution Networks (PDNs) with Irregular Shapes and Multilayer Structures using Deep Reinforcement Learning (DRL), Which Has Not Been Considered Before. a Fast Boundary Integration Method is Applied to Compute the Impedance Matrix of a PDN Structure. Subsequently, a New DRL Algorithm based on Proximal Policy Optimization (PPO) is Proposed to Optimize the Decoupling Capacitor (Decap) Placement by Minimizing the Number of Decaps While Satisfying the Desired Target Impedance. in the Proposed Approach, the PDN Structure Information is Encoded into Matrices and Serves as the Input of the DRL Algorithm, Which Increases the Flexibility of the Method to Be Extended and Generalized to Different PDN Configurations. Also, the Output of the Algorithm Determines the Selection of Decap Types and Locations Collaboratively, Making It Easier to Find the Optimal Solution in a Huge Search Space. the Proposed Method is Compared with the State-Of-The-Art Approaches and Shows Consistent Advantages in Reducing the Number of Decaps in Different Testing Cases.