What is the main way of using ns3-ai to develop DQN algorithms?

Hello everyone,

I am currently researching the topic of using ML algorithms for Rate Adaptation (RAA) in wireless networks, specifically 802.11ax. My research objective is to demonstrate the capability and effectiveness of ML algorithms (in this case DQN) and how they are able to surpass conventional algorithms like Minstrel or Ideal in throughput.

To do this research, I am using Network Simulator 3 (NS3) as the main training and testing environment for the DQN algorithm. Then, to solve the problem of communicating between Python and NS3 (written in C++), I am using the ns3-ai module, which utilizes a shared memory pool approach to this situation.

The main problem I am encountering is the lack of implementation code anywhere on the internet on this specific topic. I have gone through a few research papers like “ns3-ai: Rate Control for Wireless LAN by Deep Q-Network.pdf&iconf=SISA&year=2022&vol=69&number=SS1-6&lang=E)”, “NS3-AI: Enable Applying Artificial Intelligence to Network Simulation in ns-3“, and “Rate Adaptation with Q-Learning in CSMA/CA Wireless Networks“. While they did give me valuable insights, they never go in depth about how exactly the code was written.

So, my questions is, how would I go upon using DQN in NS3 and ns3-ai? How would I observe the states and perform actions in NS3? Given that it is not the typical gymnasium environment where the environment kind of “accepts” actions.

Here is what I have tried.

I have tried to understand/reverse-engineer the sample code provided I have read through their tutorial, but it only describes a simple scenario of sending data back and forth between Python and C++, disregarding the NS3 simulation environment. The creator says that there is skeleton code but I can’t find it.

I would highly appreciate any help I get!

submitted by /u/SkullNighter
[link] [comments]

Leave a Reply

The Future Is A.I. !
To top