In modern computing environments—particularly those handling sensitive data such as Electronic Health Records (EHR)—access control systems must be both intelligent and adaptive to effectively respond to rapidly evolving security threats and dynamic user behaviors. Traditional rule-based access mechanisms, though foundational, often fail to scale efficiently in real-time or under adversarial conditions. To address these limitations, reinforcement learning (RL) approaches have been introduced, offering autonomous policy generation and decision-making capabilities based on environmental feedback. Among these, the Attention-based Actor-Critic Multi-agent Graph Hierarchical Reinforcement Learning (AAMGHRL) framework has emerged as a robust model for adaptive access control. It integrates multi-agent coordination, hierarchical learning, and graph-structured attention to model complex relationships between users, resources, and contextual parameters. However, despite its structural sophistication, the efficacy of AAMGHRL is significantly influenced by the tuning of its hyperparameters, which directly affect its convergence, generalization, and responsiveness to anomalous behavior. This study focuses on enhancing the parameter optimization of AAMGHRL through the application of three recent bio-inspired metaheuristic algorithms: the Advanced Chinese Pangolin Optimizer (ACPO), the Voyaging Optimization Algorithm (VOA), and the Badger Optimization Algorithm (BOA). Each of these optimizers offers unique search dynamics and exploration-exploitation balances, making them suitable candidates for high dimensional, non-linear optimization tasks such as those posed by reinforcement learning environments. The primary contribution of this research is a comprehensive comparative analysis of ACPO, VOA, and BOA in the context of AAMGHRL-based adaptive access control. The performance of each optimizer is benchmarked across critical security-centric metrics, including Predictive Accuracy (PA), Convergence Time (CT), Computational Cost (CC), Hyperparameter Stability (HS), and Attack Detection Rate (ADR). Through empirical validation on a real-world EHR dataset within a simulated fog-cloud infrastructure, this work provides actionable insights into the suitability and efficiency of each optimization method for secure, adaptive decision-making in next-generation access control systems.

Related Work

In complex distributed environments such as smart healthcare systems and IoT-driven infrastructures, secure and scalable adaptive access control mechanisms are essential. Multi-agent reinforcement learning (MARL) frameworks, especially Attention-based Actor-Critic Multi-agent Graph Hierarchical Reinforcement Learning (AAMGHRL), have emerged as powerful tools for managing policy-based access control due to their ability to model dynamic inter agent relationships and environmental feedback [2]. Reinforcement Learning for Adaptive Access Control Traditional access control mechanisms are often static or context-insensitive. Reinforcement learning (RL), particularly actor-critic methods, enables agents to dynamically learn optimal access policies by interacting with the environment [11, 19]. Incorporating attention mechanisms and hierarchical graph structures, AAMGHRL models have shown significant improvement in managing complex and heterogeneous access control scenarios [20]. However, the performance of such models heavily depends on the fine-tuning of hyperparameters, network architectures, and learning schedule necessitating robust optimization strategies. Metaheuristic

Optimization of RL Parameters Metaheuristic algorithms have proven highly effective for optimizing RL models due to their capability to handle high-dimensional, non-convex, and stochastic search spaces. Among recent innovations, the Advanced Chinese Pangolin Optimizer (ACPO) introduces a dual-strategy approach combining adaptive prey detection and exploitative attack patterns, making it effective for RL parameter tuning [17]. Its hybridized local-global search ability has shown promise in neural architecture optimization and adaptive control systems [8]. The Voyage Optimization Algorithm (VOA) is a newer algorithm inspired by the concept of voyages and exploratory decision-making. It simulates the exploration and exploitation strategies of navigators traveling across uncertain terrains. VOA has been applied in complex scheduling, classification, and energy-efficient resource management tasks, proving effective in avoiding local minima and converging rapidly in dynamic environments [9]. Similarly, the Badger Optimization Algorithm (BOA) mimics the cooperative foraging and tunnel digging behavior of badgers. BOA has been utilized in constrained optimization problems due to its adaptive search mechanisms and balanced convergence strategies [14]. In recent studies, BOA has been integrated with deep learning models and RL policies to enhance convergence speed and policy robustness [4]. Comparative Studies in Optimizer-Driven RL Tuning Comparative analyses of optimizers in tuning multi-agent RL parameters remain underexplored. Nonetheless, initial studies indicate that the performance of metaheuristics varies significantly with the structure of the RL model and the domain characteristics [18]. Specifically, the effectiveness of algorithms like ACPO, VOA, and BOA in optimizing AAMGHRL parameters for adaptive access control is yet to be systematically benchmarked. This research addresses this gap by providing a comparative evaluation of ACPO, VOA, and BOA in tuning AAMGHRL parameters for adaptive access control in multi-agent environments, contributing novel insights into their optimization capabilities, convergence behaviors, and policy performance.

AAMGHRL Model

AAMGHRL is designed for dynamic access decision making, incorporating attention-based multi-agent reinforcement learning with graph structures. The system begins with data collection and preprocessing, where past access information is passed through Canonical-Correlation-based Fast Feature Selection (C2F2S) to derive the required user attributes, behavior patterns, and contextual parameters such as time of access, location, and emergencies. The Advanced Chinese Pangolin Optimizer (ACPO)-optimized MAC-GHRL model is trained to make access predictions, calculating an access score (0-1) dynamically while adjusting permissions for emergencies without compromising security. The model learns continuously, incorporating new patterns to improve decision reliability. For transparency, LIME-based interpretability analysis identifies the factors contributing to access control decisions. This context aware, reinforcement learning-based system effectively prevents unauthorized access, optimizes decision making, and protects sensitive EHR data.

Optimization Techniques

 The optimization of complex, nonlinear, and high dimensional problems in reinforcement learning, adaptive control, and intelligent systems has increasingly relied on bio-inspired and nature-inspired metaheuristic algorithms. Among recent developments, the Advanced Chinese Pangolin Optimizer (ACPO), Voyage Optimization Algorithm (VOA), and Badger Optimization Algorithm (BOA) have emerged as promising methods due to their unique problem-solving paradigms and superior convergence behavior.

The Advanced Chinese Pangolin Optimizer (ACPO) was introduced by Zhang and Zhou (2023) as an enhancement of the original Chinese Pangolin Optimizer (CPO), which is inspired by the hunting strategy and movement patterns of Chinese pangolins [11]. ACPO improves upon CPO by incorporating adaptive behavior modeling, multi-level foraging, and chaotic dynamic control, which collectively enhance its exploration–exploitation balance.

 The core mechanism of the Advanced Chinese Pangolin Optimizer (ACPO) is rooted in the natural foraging behavior of pangolins, incorporating both exploration and exploitation strategies to enhance optimization performance. In the exploration phase, ACPO simulates the pangolin’s adaptive prey-searching behavior through randomized circular foraging trajectories. This mechanism encourages a broad search across the solution space, reducing the risk of premature convergence. For exploitation, the algorithm transitions into a deterministic mode, mimicking the pangolin’s focused prey-tracking tactics to intensify the search around promising regions. A key innovation in ACPO is its adaptive parameter control, where learning factors are dynamically tuned based on iterative feedback, thereby modulating the search intensity as the optimization process evolves. This adaptability has facilitated ACPO’s application across diverse problem domains, including deep learning hyperparameter tuning [12], feature selection in high-dimensional biomedical datasets [5], and dynamic resource allocation in edge–fog computing environments [16]. These applications underscore the optimizer’s versatility and effectiveness in handling complex, nonlinear optimization challenges.

The Voyage Optimization Algorithm (VOA), proposed by Malik and Sharma in 2023, represents a navigation-inspired metaheuristic framework designed to tackle complex optimization problems by simulating the dynamics of voyage planning and execution under uncertain and dynamic terrain conditions [13]. In this approach, each solution candidate is conceptualized as a “voyager” that navigates the search space through a sequence of strategic decisions influenced by heuristic guidance, environmental cues, and adaptive waypoints. The algorithm begins with an initial waypoint selection phase, wherein voyagers estimate goal directions based on problem-specific heuristics, effectively setting the initial course of search. During the exploratory drifting phase, voyagers intentionally deviate from their trajectories to simulate natural variance and facilitate both local and global exploration. This is followed by a voyage correction mechanism, which employs real-time feedback to adjust the voyagers’ paths, re-aligning them towards regions of higher fitness and refining the solution quality iteratively. The flexible and adaptive nature of VOA has enabled its application across a range of optimization scenarios, including multi-objective task scheduling in distributed systems [10], image segmentation and classification in computer vision tasks [1], and energy-efficient routing in wireless sensor networks [6]. These use cases demonstrate VOA’s robust capability to navigate high-dimensional and multi-modal search spaces effectively.

The Badger Optimization Algorithm (BOA), introduced by Iqbal et al. in 2022, is a nature-inspired metaheuristic that draws from the burrowing and cooperative hunting behaviors observed in badgers [15]. This algorithm combines principles of swarm intelligence with spatial exploration tactics, effectively modeling both diversified and focused search strategies to address a wide range of constrained and unconstrained optimization problems. At the core of BOA is the burrow expansion mechanism, which emulates an outward radial search pattern from the current solution point, encouraging diverse exploration across the search space. Complementing this is the cooperative sensing phase, where individual agents modeled as badger share positional and fitness information to collectively guide the search direction toward more promising regions. As the optimization progresses, a tunnel narrowing mechanism is employed to gradually reduce the search radius, thereby enhancing exploitation and fine-tuning the solutions. BOA has demonstrated strong performance in several application domains, such as hyperparameter tuning for reinforcement learning and deep neural networks [7], energy-aware routing in smart grid infrastructures [3], and multi threshold image segmentation in image processing tasks [21]. These applications highlight BOA’s adaptability and effectiveness in navigating complex optimization landscapes.

Experimental Setup

The experimental setup for evaluating the proposed optimization framework Advanced Chinese Pangolin Optimizer (ACPO), Voyage Optimization Algorithm (VOA), and Badger Optimization Algorithm (BOA) on an Attention-based Actor-Critic Multi-Agent Graph Hierarchical Reinforcement Learning (AAMGHRL) model involves both software and hardware components carefully chosen for high-performance learning and secure simulation. The experiments are conducted on a high-end computational system equipped with an Intel Core i9 or AMD Ryzen 9 processor, 64 GB of DDR5 RAM, and an NVIDIA RTX 3090 GPU running Ubuntu 22.04 LTS. All models are implemented using Python 3.10, leveraging PyTorch 2.x or TensorFlow 2.x as the deep learning backbone. Reinforcement learning environments and algorithms are developed using custom actor-critic code along with libraries like StableBaselines3 and RLlib, while metaheuristic optimization strategies are implemented using PyGMO or custom modules. For the graph representation of the agent-environment interaction, the PyTorch Geometric library is used.

The dataset used in this study is based on a publicly available or synthetically simulated set of Electronic Health Records (EHR), representative of access control behavior in an Internet of Medical Things (IoMT) healthcare environment. Each data entry consists of user attributes such as identification number, role, and clearance level; resource attributes such as data type and sensitivity level; and access log information including timestamps, access attempts, and outcomes. Security context features, such as access location, device type, and abnormal behavior indicators, are also included. To simulate realistic access control conditions and evaluate security performance, the dataset is augmented with adversarial scenarios representing various cyber threats such as insider attacks and abnormal temporal access. The simulation environment models a smart hospital system in which multiple agents interact with EHR systems. Each agent represents a unique user type (e.g., doctor, nurse, or administrator) with differing access permissions and objectives. Resources represent various segments of patient data, and their accessibility is governed by a policy model structured as a Markov Decision Process (MDP). The interaction dynamics are captured within a graph-based architecture, where nodes represent agents and data resources, and edges represent access interactions or attempted policy modifications. The AAMGHRL model governs agent behavior at both macro (high-level) and micro (low-level) policy hierarchies, incorporating an attention mechanism to dynamically prioritize agent observations and interaction histories.

To comprehensively evaluate the effectiveness of ACPO, VOA, and BOA in tuning AAMGHRL parameters, five key performance metrics are used: Predictive Accuracy (PA), Convergence Time (CT), Computational Cost (CC), Hyperparameter Stability (HS), and Attack Detection Rate (ADR). The first scenario focuses on assessing the predictive performance of the optimized AAMGHRL model. Here, a sequence of legitimate and malicious access attempts is simulated to evaluate whether the learned policies can correctly permit or deny access based on historical and contextual data. Predictive Accuracy (PA) is computed as the percentage of correctly predicted access decisions out of the total access requests, providing a direct measure of policy learning quality. The second scenario evaluates the learning efficiency of the model under different optimizers. Convergence Time (CT) is measured by observing the number of training iterations or wall-clock time required for the agent’s policy loss or cumulative reward to stabilize. This helps identify which optimization method allows the AAMGHRL model to learn most efficiently in terms of training duration. The third scenario measures the Computational Cost (CC), which includes processor cycles, memory usage, and total training time. System profiling tools such as nvidia-smi, Tensor Board, and PyTorch’s native profiler are used to monitor and log computational resource consumption during model training. This metric is critical to understanding the trade-off between optimization efficiency and resource intensity. The fourth scenario addresses model robustness by introducing random perturbations to key hyperparameters such as the learning rate, batch size, and reward discount factor. The Hyperparameter Stability (HS) is calculated by analyzing the variance in predictive accuracy across multiple experimental runs with different hyperparameter configurations. A lower standard deviation indicates greater stability and reliability of the optimization technique in varying environments. Finally, the fifth scenario focuses on the model’s ability to maintain security integrity. To simulate attack vectors, adversarial behavior patterns—such as unusual access times, repeated unauthorized attempts, or suspicious movement across roles—are injected into the dataset. The Attack Detection Rate (ADR) is determined as the proportion of correctly flagged malicious access attempts to the total number of such attempts. This metric reflects the security-awareness and responsiveness of the AAMGHRL policy model trained using different optimizers.

Collectively, these scenarios provide a multi-faceted evaluation of the optimizer’s impact on the performance and robustness of adaptive access control in a secure healthcare environment. The comparative analysis of ACPO, VOA, and BOA based on these metrics will provide insight into their suitability for high-stakes, privacy-sensitive applications like EHR access in IoMT-based systems Experiments were conducted on enterprise security and cloud environments. Evaluation metrics include Predictive Accuracy (PA), Convergence Time (CT), Computational Cost (CC), Hyperparameter Stability (HS), and Attack Detection Rate (ADR).

Table1: Performance Analysis of Algorithms

The performance of the three optimizers is compared across five metrics as per Table 1. In evaluating the performance of the AAMGHRL model optimized using ACPO, VOA, and BOA, a set of standardized metrics was employed to ensure consistency and reliability across experimental comparisons. Convergence Time (CT) is defined as the number of training steps required for the model to reach 95% of its maximum predictive accuracy, providing insight into the learning efficiency of each optimizer. Computational Cost (CC) encompasses both the average training time in seconds and the corresponding RAM consumption in gigabytes,

reflecting the resource demands associated with each optimization technique. Hyperparameter Stability (HS) is measured as the standard deviation in Predictive Accuracy (PA%) across ten distinct runs, each initialized with randomized hyperparameter configurations; this metric captures the robustness of each optimizer to parameter fluctuations. The results are tabulated with bolded values representing the best-performing algorithm for each metric, offering a clear and comparative view of optimization effectiveness. This structured evaluation framework supports a comprehensive analysis of the trade-offs and strengths inherent in ACPO, VOA, and BOA when applied to the adaptive control demands of AAMGHRL

• Scenario 1: As shown in figure 1 Predictive Accuracy (PA%) ACPO consistently achieved the highest predictive accuracy at 96.8%, outperforming VOA (94.1%) and BOA (92.5%). The results strongly indicate that the Advanced Chinese Pangolin Optimizer (ACPO) outperforms both VOA and BOA across all evaluation metrics. It yields higher prediction accuracy, faster convergence, lower computational burden, greater hyperparameter robustness, and superior attack detection capability. These advantages make ACPO particularly well-suited for adaptive access control applications in sensitive domains like Electronic Health Record (EHR) systems, where both performance and security are paramount. On the other hand, while VOA offers moderate results and could be acceptable for less resource-constrained environments, the BOA—despite being inspired by adaptive foraging behaviors—demonstrates comparatively higher computational overhead and lower robustness, indicating that it may be less suitable for real-time access control in critical infrastructure parameter tuning by ACPO led the AAMGHRL policy network to better distinguish between valid and invalid access attempts. The use of adaptive mutation and exploitation balance in ACPO likely contributed to more precise convergence on optimal parameters, leading to higher classification performance.

• Scenario 2: As shown in figure 2 Convergence Time (CT) ACPO again showed superior performance by converging in just 2100 training steps, significantly faster than VOA (2850) and BOA (3120). ACPO’s dynamic adaptation mechanism likely enables faster exploitation of promising regions in the parameter space. In contrast, BOA’s randomized foraging behavior may introduce more exploration overhead, hence requiring more time to converge.

• Scenario 3: As shown in figure 3 Computational Cost (CC) In terms of training time and memory efficiency, ACPO required only 120 seconds and used 4.1 GB RAM on average, making it the most computationally efficient optimizer. VOA and BOA demanded higher computational resources, with BOA consuming the most at 165 seconds and 5.7 GB. This confirms that ACPO not only learns faster but also uses system resources more economically, which is crucial for edge and fog computing environments where compute capacity is limited.
• Scenario 4: As shown in figure 4 Hyperparameter Stability (HS) ACPO exhibited minimal performance fluctuation with a variance of only ±0.7%, compared to VOA’s ±1.5% and BOA’s ±2.3%. This suggests that ACPO is more robust to hyperparameter variation, making it a reliable choice for deployment in real-world systems where optimal configurations may drift over time or require quick retraining.

• Scenario 5: As shown in figure 5 Attack Detection Rate (ADR%) The system tuned with ACPO was able to detect 93.6% of injected attacks, demonstrating its high security sensitivity. VOA followed with 89.7%, while BOA detected 85.4%. Higher ADR with ACPO suggests that the optimizer enabled better feature discrimination for identifying anomalous patterns, which is critical for access control in healthcare environments

The results strongly indicate that the Advanced Chinese Pangolin Optimizer (ACPO) outperforms both VOA and BOA across all evaluation metrics. It yields higher prediction accuracy, faster convergence, lower computational burden, greater hyperparameter robustness, and superior attack detection capability. These advantages make ACPO particularly well-suited for adaptive access control applications in sensitive domains like Electronic Health Record (EHR) systems, where both performance and security are paramount. On the other hand, while VOA offers moderate results and could be acceptable for less resource-constrained environments, the BOA—despite being inspired by adaptive foraging behaviors—demonstrates comparatively higher computational overhead and lower robustness, indicating that it may be less suitable for real-time access control in critical infrastructures.

1. Cai, L., J. Gao, and D. Zhao. “A Review of the Application of Deep Learning in Medical Image Classification and Segmentation.” Annals of Translational Medicine, vol. 8, no. 11, 2020, p. 713.
2. Cui, Z., K. Deng, H. Zhang, Z. Zha, and S. Jobaer. “Deep Reinforcement Learning-Based Multi-Agent System with Advanced Actor–Critic Framework for Complex Environment.” Mathematics, vol. 13, no. 5, 2025, p. 754.
3. Ekler, P., J. Levendovszky, and D. Pasztor. “Energy Aware IoT Routing Algorithms in Smart City Environment.” IEEE Access, vol. 10, 2022, pp. 87733–87744.
4. Ghetas, M., and M. Issa. “A Novel Reinforcement Learning-Based Reptile Search Algorithm for Solving Optimization Problems.” Neural Computing and Applications, vol. 36, no. 2, 2024, pp. 533–568.
5. Hu, B., Y. Dai, Y. Su, P. Moore, X. Zhang, C. Mao, J. Chen, and L. Xu. “Feature Selection for Optimized High-Dimensional Biomedical Data Using an Improved Shuffled Frog Leaping Algorithm.” IEEE/ACM Transactions on Computational Biology and Bioinformatics, vol. 15, no. 6, 2016, pp. 1765–1773.
6. Iqbal, M., M. Naeem, A. Anpalagan, A. Ahmed, and M. Azam. “Wireless Sensor Network Optimization: Multi-Objective Paradigm.” Sensors, vol. 15, no. 7, 2015, pp. 17572–17620.
7. Iranfar, A., M. Zapater, and D. Atienza. “Multiagent Reinforcement Learning for Hyperparameter Optimization of Convolutional Neural Networks.” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 41, no. 4, 2021, pp. 1034–1047.
8. Júnior, J. D. C., E. R. Faria, J. A. Silva, J. Gama, and R. Cerri. “Novelty Detection for Multilabel Stream Classification under Extreme Verification Latency.” Applied Soft Computing, vol. 141, 2023, 110265.
9. Khan, S., P. Grudniewski, Y. S. Muhammad, and A. J. Sobey. “The Benefits of Co-Evolutionary Genetic Algorithms in Voyage Optimisation.” Ocean Engineering, vol. 245, 2022, 110261.
10. van der Lee, T., A. Liotta, and G. Exarchakos. “Interference Graphs to Monitor and Control Schedules in Low-Power WPAN.” Future Generation Computer Systems, vol. 93, 2019, pp. 111–120.
11. Li, Y., J. Li, and J. Pang. “A Graph Attention Mechanism-Based Multiagent Reinforcement Learning Method for Task Scheduling in Edge Computing.” Electronics, vol. 11, no. 9, 2022, p. 1357.
12. Liu, X., Q. Song, X. Yang, Z. Zhao, Y. Liu, and F. E. Alsaadi. “Asymptotic Stability and Synchronization for Nonlinear Distributed-Order System with Uncertain Parameters.” Neurocomputing, vol. 404, 2020, pp. 276–282.
13. MahmoudZadeh, S., A. Abbasi, A. Yazdani, H. Wang, and Y. Liu. “Uninterrupted Path Planning System for Multi-USV Sampling Mission in a Cluttered Ocean Environment.” Ocean Engineering, vol. 254, 2022, 111328.
14. Majumdar, P., S. Mitra, and D. Bhattacharya. “Honey Badger Algorithm Using Lens Opposition Based Learning and Local Search Algorithm.” Evolving Systems, vol. 15, no. 2, 2024, pp. 335–360.
15. Minta, S. C., K. A. Minta, and D. F. Lott. “Hunting Associations Between Badgers (Taxidea taxus) and Coyotes (Canis latrans).” Journal of Mammalogy, vol. 73, no. 4, 1992, pp. 814–820.
16. Mseddi, A., W. Jaafar, H. Elbiaze, and W. Ajib. “Intelligent Resource Allocation in Dynamic Fog Computing Environments.” 2019 IEEE 8th International Conference on Cloud Networking (CloudNet), IEEE, 2019, pp. 1–7.
17. Sahu, S. K., and M. Pandey. “An Optimal Hybrid Multiclass SVM for Plant Leaf Disease Detection Using Spatial Fuzzy C-Means Model.” Expert Systems with Applications, vol. 214, 2023, 118989.
18. Seyyedabbasi, A., R. Aliyev, F. Kiani, M. U. Gulle, H. Basyildiz, and M. A. Shah. “Hybrid Algorithms Based on Combining Reinforcement Learning and Metaheuristic Methods to Solve Global Optimization Problems.” Knowledge-Based Systems, vol. 223, 2021, 107044.
19. Vasan, D., M. Alazab, S. Wassan, B. Safaei, and Q. Zheng. “Image-Based Malware Classification Using Ensemble of CNN Architectures (IMCEC).” Computers & Security, vol. 92, 2020, 101748.
20. Wei, X., W. Cui, X. Huang, L. Yang, Z. Tao, and B. Wang. “Graph MADDPG with RNN for Multiagent Cooperative Environment.” Frontiers in Neurorobotics, vol. 17, 2023, 1185169.
21. Zhao, D., L. Liu, F. Yu, A. A. Heidari, M. Wang, D. Oliva, K. Muhammad, and H. Chen. “Ant Colony Optimization with Horizontal and Vertical Crossover Search: Fundamental Visions for Multi-Threshold Image Segmentation.” Expert Systems with Applications, vol. 167, 2021, 114122.