Reinforcement Learning Competition

• Operating System: Ubuntu 18.04.4 LTS (4 vCPUs, 2.5 GHz, No GPU, 2 GiB memory)
• Languages: Python3 (ver. 3.6.9)
• Code Size + Library: < 30MB (players must use the available library of the coding language or embed library into your source code so it can be run on the server.)
• Tensorflow 1.14.0 and 2.2.0
• Keras 2.3.1
• Numpy 1.18.4
• Pandas 0.15
• PyTorch 1.5.0
• joblib 0.16.0
• ray 0.8.6 (ray[rllib], ray[tune])
• requests 2.24.0
• semver 2.10.2 
• tf-agents 0.3.0 (0.5.0 on tensorflow 2.2.0)
• Pyqlearning v1.2.4
• Mushroom-RL v1.4.0
• gym 0.17.2
• opencv-python 4.2.0.34
• prettytable 0.7.2
• yacs 0.1.7

♦ SOURCE CODE STRUCTURE:

• The strucure is as below: 
  
  
  Where:

• Your project needs 2 folders src and build
• Folder src includes source code and file build.sh: When file build.sh is run, source codes which are compiled will be sent to folder build. Note that a source code is successfully built if there is no message (including warning) returns when executing file build. Run command is python3.
• Folder build needs file run.sh
• Your project will be run when calling file run.sh
• How to upload source code:
  
• When your client connects successfully to server and gets at least 50 points, it is considered satisfactory and the status is Verified
• Compress the project into a .zip file with any name and upload the file to system
• After uploading the code, the system will return one of the following status: Building, Build Failed, Verifying, Failed, Verified
  
  • Build Failed, Failed: the system will allow users to download the log file.
  • Verifying, Failed, Verified: users will be able to view the game play screen.

♦ COMMUNICATION BETWEEN SERVER AND PLAYERS:

Server and player will communicate via socket. Socket information (host, port) will be transfer through input parameters when file run.sh

• Messages between player and server:
  
  • When player connects to server successfully, server will send a message to player with the following content:
    

    { 
        "playerId": int,
        “posx”: int,
        “posy”: int,
        “energy”: int,
        "gameinfo": {
            "numberOfPlayers": int,
            "width": int,
            "height": int,
            "steps": int,
            "golds":[
                {
                    "posx": int,
                    "posy": int,
                    "amount": int
                }
            ],
            "obstacles":[
                {
                    "type": int, //0: land, 1: forest, 2: trap, 3: swamp
                    "posx": int,
                    "posy": int,
                    "value": int
                }
            ]
        }
    }

Where:
♦ Player information includes:
- playerId: the player's ID
- (posx, posy): the initial initialized coordinates
- energy: initial initialized energy

♦ Game information includes:
- numberOfPlayers: the number of players participating in the game
- (width, height): size of the map
- steps: the maximum number of steps (turns) in the game 
- golds: information of gold mines including coordinates (posx, posy)and initial amount of gold (amount)
- obstacles: information of obstacles along the way including coordinates (posx, posy), type of obstacles (type), and value of obstacles (value <= 0, players will loss (-value) energy if they move into the cell). If obstacle's type is forest (type = 1), value is 0.

Each type of obstacles is represented as different integers as follows: 0: land, 1: forest, 2: trap, 3: swamp

• Each turn, the player sends one of the following commands: "0", "1", "2", "3", "4", "5". The commands represent the following actions:
  

  "0"	go left
  "1"	go right
  "2"	go up
  "3"	go down
  "4"	take a rest
  "5"	dig for gold
  except the above commands	eliminated

• After sending actions to server, the server responds to the new status of the map as follows:

  {
          "players":[
              {
                  "playerId": int,
                  "posx": int,
                  "posy": int,
                  "score": int,
                  "energy": int,
                  "status": int //0: PLAYING, 1: ELIMINATED_WENT_OUT_MAP, 2: ELIMINATED_OUT_OF_ENERGY, 3: ELIMINATED_INVALID_ACTION, 4: STOP_EMPTY_GOLD, 5: STOP_END_STEP
                  "lastAction": int, //0: go left, 1: go right, 2: go up, 3: go down, 4: free, 5: craft, 6: eliminated (in case, action = null)
              }
          ],
          "golds":[
              {
                  "posx": int,
                  "posy": int,
                  "amount": int
              }
          ] ,
          "changedObstacles":[
              {
                  "posx": int,
                  "posy": int,
                  "type": int,
                  "value": int
              }
          ] 
  }
  

The status of all players in the match players includes:
- playerId: the player's ID
- (posx, posy): the current coordinates of the player
- score: player's score (amount of gold mined)
- energy: the remaining energy of the player
- lastAction: player's action at previous step (state = 6 means player is eliminated)
- status: player's status (playing, eliminated or stop - end game)

golds: the current information of golds on the map

changedObstacles: list of obstacles which have been changed by previous action (gold -> plan, trap -> plan, or new value for swamp)

Value of lastActionis one of following:

Value of statusis one of following:

• If player is eliminated, server will send last message to player and then disconnet the player.
• When the game is over, the server will disconnect the players.
• How to calculate the coordinates of the map:
  
  • (0,0)is the top left point of the map.
  • (posx, posy) is calculated from(0,0) with posx is the posx-thth column and posyis the posy-th row on the map.

♦ NUMBER OF SUBMISSIONS:

Teams are allowed to update unlimited submissions on the sever system, but can only play maximum 20 turns with the Organizer's Bot.

The turn with the Bot will be automatically activated after the code is verified successfully. Each turn has 5 matches.