A Deep Reinforcement Learning Method for Service Composition in Cyber-Physical-Social Systems

Cyber-Physical-Social Systems(CPSS), oriented towards services, view various resources from physical, network, and social domains as service components. Through orchestrated composition techniques, these resources from the three domains are integrated to achieve personalized system integration. Despite the effective support of existing service-oriented architectures for service composition, accomplishing large-scale service composition within the highly dynamic CPSS environment remains a considerable challenge. To address these challenges more effectively, this paper proposes a CPPS service composition method based on Proximal Policy Optimization (PPO). This approach involves iteratively training and learning the optimal composition strategy through Deep Reinforcement Learning (DRL). The introduced learning algorithm designs the action space of PPO as a variable action set for each decision state. Moreover, it enhances and refines the reward function based on CPSS scenarios, aiming to attain service compositions that meet constraints while optimizing the Quality of Service (QoS). Experimental comparisons with other DRL methods and heuristic algorithms demonstrate that our proposed approach exhibits superior reliability, adaptability, and scalability. It outperforms alternative methods in addressing large-scale dynamic service composition challenges within the CPSS context.